KV-Cache-Serving-Risk-Dataset-Open-Weight-LLMs

This dataset simulates how GPU memory and decode-step latency scale for realistic open-weight LLM serving deployments. Each deployment fixes a model architecture (approximate published configs for Gemma 2B/7B, Gemma 2 9B/27B, Llama 3 8B/70B, and Mistral 7B), a GPU (T4 through H100-80GB, public datasheet memory and bandwidth), a weight precision (fp16/int8/int4), and a KV-cache allocator strategy (naive contiguous vs. paged), then sweeps batch size (1-64) and context length (512-131072 tokens). Every number is computed from deterministic formulas cited to ten sources: Attention Is All You Need, Scaling Laws for Neural Language Models, Chinchilla, MQA, GQA, FlashAttention, PagedAttention/vLLM, and the Gemma, Gemma 2, and Llama 3 technical reports. No real cluster telemetry is used - every value is reproducible from public model configs and public GPU specs.