Weights
parameters × effective bytes per weight8.03B × 0.60 bytes
4.49 GiB
AI Wiki tools
LLM VRAM and RAM calculator
Check whether a local language model fits your hardware: weights, KV cache, runtime reserve, and per-GPU split, with every formula shown.
No API key · Capacity estimate, not a benchmark
Add models in the comparison finder to carry a shortlist between tools.
Model and hardware
Calculated in your browser. Settings are included in the page URL and shared between tools: anyone with that URL can see them, and opening it sends the URL to AI Wiki. Do not enter confidential workload details.
Planning estimate
Llama 3.1 8B at Q4_K_M
8,192 tokens · 1 concurrent sequence
Estimated total allocation
6.69 GiB
Weights + maximum KV cache + a visible runtime planning reserve
WeightsKV cacheRuntime reserve
- Model weights
- 4.49 GiB
- 0.60 effective bytes/parameter
- KV cache
- 1.00 GiB
- 1.00 GiB per sequence
- Runtime reserve
- 1.20 GiB
- Workspace and allocation cushion
- Ideal split per GPU
- 6.69 GiB
- Total divided evenly across 1 GPU
Capacity checks
"Tight" means less than 15% of capacity remains. Real runtimes can need more.
Full GPU
Fits with reserve
6.69 GiB required per GPU
17.3 GiB headroom
RAM only
Fits with reserve
6.69 GiB required in system memory
25.3 GiB headroom
VRAM + RAM
Fits with reserve
About 32 of 32 layers fit on GPU (approximation); not a speed estimate
49.3 GiB headroom
Transparent methodology
How this estimate is calculated
All values use binary GiB (1,073,741,824 bytes). Quantized bytes per weight include scales and metadata; actual file formats vary.
KV cache
concurrency × sum over layer groups of (layers × cached tokens × values per token × bytes per value); fixed-state layers add their state instead1 × (32 × 8,192 × 2×8×128 × 2.00 B)
1.00 GiB
K and V are cached separately (2 × KV heads × head size values per token) unless a group stores one shared latent. Window and chunked layers cache at most their window; the KV precision selector sets bytes per value.
Runtime reserve
max(1 GiB, 10% of weights + 0.25 GiB/sequence + 0.5 GiB/GPU)10% of weights + 1 sequence(s) + 1 GPU(s)
1.20 GiB
Important limitations
- This is a capacity estimate, not a performance benchmark or a guarantee that a particular runtime will load the checkpoint.
- KV cache assumes every concurrent sequence reaches the selected context length. Prefix caching, paged attention, and runtime-specific cache layouts can change it substantially. Sliding windows, chunked attention, latent KV and linear attention are modeled for presets from their published configs, or when you describe the layers in custom mode.
- Presets are convenience approximations. Fine-tunes, conversions, multimodal adapters, and checkpoint revisions can have different tensors or architecture settings.
- RAM and VRAM are not always freely additive. CPU offload can be much slower, some tensors must stay together, and loaders may temporarily map or copy weights in both places.
- Leave extra capacity for the OS, display, runtime, drivers, and long-lived fragmentation. A result marked "tight" deserves a smaller context, lower concurrency, or more memory.
Reading the result
Memory capacity is only the first constraint
A model fitting in memory does not establish usable generation speed. Memory bandwidth, compute throughput, prompt processing, CPU offload, GPU interconnects, runtime kernels, and quantization format can all change performance. Treat this calculator as a first-pass capacity plan before testing the exact checkpoint and inference engine.
Weights are persistent
Every stored parameter consumes memory. Mixture-of-experts models still need all expert weights loaded even when each token activates only a subset, though the experts can live in system RAM.
KV cache grows with use
Longer contexts and more simultaneous sequences can turn a model that fits at idle into one that runs out of memory under load.
Headroom matters
Drivers, graph capture, temporary buffers, adapters, and fragmentation vary by runtime. The calculator flags less than 15% free capacity as tight.
Methodology source and scope
The weight-memory method follows the same core idea documented by the official Hugging Face Accelerate model memory estimator: estimate tensor storage from model configuration and dtype. Hugging Face explicitly notes that loading memory is not the full inference requirement, so this calculator separately adds KV cache and a visible runtime reserve. Exact runtime measurements remain authoritative.
Presets checked 2026-10-25against each model's Hugging Face config.json and safetensors metadata; quantization factors come from llama.cpp's block definitions and measured GGUF files. They are approximations, not vendor guarantees or hardware recommendations.
Common questions
LLM memory calculator FAQ
How much VRAM does an LLM need?
The largest component is usually model weights: total parameters multiplied by the effective bytes stored per parameter. KV cache then grows with layers, KV heads, attention head size, context length, cache precision, and the number of concurrent sequences. The runtime also needs workspace and allocation headroom.
Why is a 4-bit model estimated above 0.5 bytes per parameter?
Four raw bits equal 0.5 bytes, but quantized files also store scales, block metadata, alignment, and sometimes higher-precision tensors. Whole Q4_K_M GGUF files measure about 0.60 bytes per parameter, so the calculator uses that rather than pretending every 4-bit format has the same size. Q8_0 and Q6_K use llama.cpp's exact block sizes (1.0625 and 0.82 bytes), and a custom bits-per-weight field covers other formats.
Why do hybrid and sliding-window models need so much less KV cache?
Only layers that attend to the whole context grow their cache with every token. Sliding-window and chunked layers keep at most their window, latent-attention (MLA) layers cache one small shared vector instead of separate keys and values, and linear-attention or Mamba layers keep a fixed-size state. The presets read each layer type from the model's config.json, and custom mode lets you describe the same layer groups.
Can a 24 GB GPU run a large mixture-of-experts model?
Often, by keeping the routed experts in system RAM and the rest on the GPU, as llama.cpp does with --cpu-moe or --n-cpu-moe. The calculator shows that split for MoE presets: attention, shared experts and the KV cache on the GPU, expert weights in RAM. It checks capacity only; experts in RAM run at system-memory speed.
Can system RAM make up for missing VRAM?
Some runtimes can offload model layers to system RAM, so combined capacity may be enough to load a model. That does not mean it will be fast: CPU offload, memory bandwidth, interconnect speed, tensor placement, and backend support can dominate performance.
How should Apple Silicon unified memory be entered?
Do not count the same shared pool as both VRAM and RAM. Picking a Mac, DGX Spark or Ryzen AI Max system fills the pool in as system RAM and sets GPU count to zero; you can also do that by hand. Then lower the RAM field to the memory actually available to the inference process, leaving room for the OS, applications, and the display.