LLM VRAM Calculator
Choose the memory on each GPU, your card count, and a quantization level. The calculator estimates which open-weight models fit, then shows the minimum memory and card count for each result.
Your hardware
The default is one 24GB GPU at Q4, a common local-inference setup.
Total available memory: 24GB
Best default for fitting capable models on consumer hardware.
8 models fit
Models for 1× 24GB at 4-bit
Highest-ranked fit: Gemma 4 31B
| Model | Public score | Estimated minimum | Cards needed | Reference deployment |
|---|---|---|---|---|
| Gemma 4 31B31B dense | 60.19 | 20.4GB17.7GB weights, plus 15% allowance | 1× 24GB3.7GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Qwen3.6-27B27B dense | 52.79 | 17.7GB15.4GB weights, plus 15% allowance | 1× 24GB6.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Gemma 4 12B12B dense | 46.24 | 7.9GB6.9GB weights, plus 15% allowance | 1× 24GB16.1GB spare across your setup | No matching catalog config yet |
| GPT-OSS 20B20B MoE · 20B active | 41.91 | 13.2GB11.4GB weights, plus 15% allowance | 1× 24GB10.8GB spare across your setup | No matching catalog config yet |
| ZAYA1-8B8B dense | 33.4 | 5.3GB4.6GB weights, plus 15% allowance | 1× 24GB18.7GB spare across your setup | No matching catalog config yet |
| Phi-4-Reasoning 15B15B dense | — | 9.9GB8.6GB weights, plus 15% allowance | 1× 24GB14.1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Mellum2-12B-A2.5B-Thinking12B MoE · 2.5B active | — | 7.9GB6.9GB weights, plus 15% allowance | 1× 24GB16.1GB spare across your setup | No matching catalog config yet |
| Ornith-1.0-9B9B dense | — | 5.9GB5.2GB weights, plus 15% allowance | 1× 24GB18.1GB spare across your setup | No matching catalog config yet |
What the result does not promise
A memory fit does not guarantee a specific token speed or context length. The estimate covers weights and a fixed runtime allowance; the KV cache grows with context and concurrent requests. MoE rows use total parameters for memory even when far fewer parameters are active per token.
LLM VRAM calculator FAQ
How much VRAM does an LLM need?
An LLM needs enough memory for every model weight, plus runtime overhead and the KV cache. At Q4, a useful starting estimate is about 0.57GB per billion parameters before overhead. This calculator adds 15% for runtime use; longer contexts, larger batches, and some inference engines need more.
Can I run a 70B LLM on 24GB VRAM?
Not as a full 70B model on one 24GB card under this estimate. A 70B model needs about 45.9GB at Q4 after the calculator’s runtime allowance, so two 24GB cards are the practical minimum. CPU or system-memory offloading can work, but it usually reduces generation speed.
Do MoE active parameters reduce VRAM requirements?
Active parameters reduce the work performed for each generated token, but they do not remove the other experts from memory. VRAM sizing therefore uses total parameters, not active parameters. A 120B MoE with 12B active can compute like a smaller model while still storing roughly 120B parameters.
Is Apple unified memory the same as GPU VRAM?
Unified memory can hold model weights much like VRAM, so the same capacity estimate is a useful first check. It is not a performance equivalence: memory bandwidth, inference software, quantization support, and the share reserved by macOS all matter. Leave extra headroom when sizing a Mac for local inference.