LLM VRAM Calculator
Choose the memory on each GPU, your card count, and a quantization level. The calculator estimates which open-weight models fit, then shows the minimum memory and card count for each result.
Your hardware
The default is one 24GB GPU at Q4, a common local-inference setup.
Total available memory: 24GB
Best default for fitting capable models on consumer hardware.
36 models fit
Models for 1× 24GB at 4-bit
Highest-ranked fit: Sage-A1.5
| Model | Public score | Estimated minimum | Cards needed | Reference deployment |
|---|---|---|---|---|
| Sage-A1.535.95B MoE · 3B active | 62.82 | 23.6GB20.5GB weights, plus 15% allowance | 1× 24GB0.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Qwen3.8-27B27B dense | 59.69 | 17.7GB15.4GB weights, plus 15% allowance | 1× 24GB6.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Agents-A135B MoE · 3B active | 54.43 | 23GB20GB weights, plus 15% allowance | 1× 24GB1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Qwen3.6-27B27B dense | 48.53 | 17.7GB15.4GB weights, plus 15% allowance | 1× 24GB6.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Gemma 4 26B A4B25.2B MoE · 3.8B active | 45.68 | 16.6GB14.4GB weights, plus 15% allowance | 1× 24GB7.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Nemotron 3.5 Bolt 74B-A9B74B MoE · 9B active | 45.28 | 24GBCatalog placement; formula gives 48.6GB | 1× 24GB0GB spare across your setup | 1× NVIDIA H100 (80GB) · 80GB totalCheck monthly cost → |
| Muse Glimmer 30B29.6B dense | 41.2 | 19.5GB16.9GB weights, plus 15% allowance | 1× 24GB4.5GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Spark-X2.6-4B4.11B dense | 39.96 | 2.7GB2.4GB weights, plus 15% allowance | 1× 24GB21.3GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| Gemma 4 31B31B dense | 39.89 | 20.4GB17.7GB weights, plus 15% allowance | 1× 24GB3.7GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| LFM2.5-24B-A2B24B MoE · 2.45B active | 37.06 | 15.8GB13.7GB weights, plus 15% allowance | 1× 24GB8.2GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Gemma 4 12B12B dense | 36 | 7.9GB6.9GB weights, plus 15% allowance | 1× 24GB16.1GB spare across your setup | No matching catalog config yet |
| ZAYA1-8B8B dense | 34.61 | 5.3GB4.6GB weights, plus 15% allowance | 1× 24GB18.7GB spare across your setup | No matching catalog config yet |
| GPT-OSS 20B20B MoE · 20B active | 33.29 | 13.2GB11.4GB weights, plus 15% allowance | 1× 24GB10.8GB spare across your setup | No matching catalog config yet |
| Granite 4.2 8B8B dense | 32.8 | 5.3GB4.6GB weights, plus 15% allowance | 1× 24GB18.7GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| Sarvam 30B32B MoE · 2.4B active | 28.65 | 21GB18.3GB weights, plus 15% allowance | 1× 24GB3GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Flint39.79B MoE · 2.5B active | — | 24GBCatalog placement; formula gives 26.1GB | 1× 24GB0GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Clef-Omni35.3B MoE · 3B active | — | 23.2GB20.2GB weights, plus 15% allowance | 1× 24GB0.9GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| BTL-435B MoE · 3B active | — | 23GB20GB weights, plus 15% allowance | 1× 24GB1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Holo3.1-35B-A3B35B MoE · 3B active | — | 23GB20GB weights, plus 15% allowance | 1× 24GB1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| OriOn-Qwen-32B33B dense | — | 21.7GB18.9GB weights, plus 15% allowance | 1× 24GB2.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Jais-3-32B32B dense | — | 21GB18.3GB weights, plus 15% allowance | 1× 24GB3GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| HyperClova X Think 32B32B dense | — | 21GB18.3GB weights, plus 15% allowance | 1× 24GB3GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Hy-Memory-30B-A3B30.06B MoE · 3B active | — | 19.8GB17.2GB weights, plus 15% allowance | 1× 24GB4.2GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Clef27.4B dense | — | 18GB15.7GB weights, plus 15% allowance | 1× 24GB6GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Fara1.5-27B27B dense | — | 17.7GB15.4GB weights, plus 15% allowance | 1× 24GB6.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| OrcaCyber-Nano-1.027B dense | — | 17.7GB15.4GB weights, plus 15% allowance | 1× 24GB6.4GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Phi-4-Reasoning 15B15B dense | — | 9.9GB8.6GB weights, plus 15% allowance | 1× 24GB14.1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Mellum2.1-12B-A2.5B-Thinking12B MoE · 2.5B active | — | 7.9GB6.9GB weights, plus 15% allowance | 1× 24GB16.1GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Mellum2-12B-A2.5B-Thinking12B MoE · 2.5B active | — | 7.9GB6.9GB weights, plus 15% allowance | 1× 24GB16.1GB spare across your setup | No matching catalog config yet |
| Clef-Flash9.4B dense | — | 6.2GB5.4GB weights, plus 15% allowance | 1× 24GB17.8GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| Ornith-1.0-9B9B dense | — | 5.9GB5.2GB weights, plus 15% allowance | 1× 24GB18.1GB spare across your setup | No matching catalog config yet |
| Nemotron 3.5 Bolt E4B8B dense | — | 5.3GB4.6GB weights, plus 15% allowance | 1× 24GB18.7GB spare across your setup | 1× NVIDIA RTX 4090 (24GB) · 24GB totalCheck monthly cost → |
| Pharia-1-LLM-7B-control7.04B dense | — | 4.7GB4.1GB weights, plus 15% allowance | 1× 24GB19.3GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| Kanana-2 3B Instruct3.18B dense | — | 2.1GB1.9GB weights, plus 15% allowance | 1× 24GB21.9GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| MiniCPM5-2B2.52B dense | — | 1.7GB1.5GB weights, plus 15% allowance | 1× 24GB22.3GB spare across your setup | No matching catalog config yetCheck monthly cost → |
| North Micro Vision Instruct2.5B dense | — | 1.7GB1.5GB weights, plus 15% allowance | 1× 24GB22.3GB spare across your setup | No matching catalog config yetCheck monthly cost → |
What the result does not promise
A memory fit does not guarantee a specific token speed or context length. The estimate covers weights and a fixed runtime allowance; the KV cache grows with context and concurrent requests. MoE rows use total parameters for memory even when far fewer parameters are active per token.
Questions
How much VRAM does an LLM need?
An LLM needs enough memory for every model weight, plus runtime overhead and the KV cache. At Q4, a useful starting estimate is about 0.57GB per billion parameters before overhead. This calculator adds 15% for runtime use; longer contexts, larger batches, and some inference engines need more.
Can I run a 70B LLM on 24GB VRAM?
Not as a full 70B model on one 24GB card under this estimate. A 70B model needs about 45.9GB at Q4 after the calculator’s runtime allowance, so two 24GB cards are the practical minimum. CPU or system-memory offloading can work, but it usually reduces generation speed.
Do MoE active parameters reduce VRAM requirements?
Active parameters reduce the work performed for each generated token, but they do not remove the other experts from memory. VRAM sizing therefore uses total parameters, not active parameters. A 120B MoE with 12B active can compute like a smaller model while still storing roughly 120B parameters.
Is Apple unified memory the same as GPU VRAM?
Unified memory can hold model weights much like VRAM, so the same capacity estimate is a useful first check. It is not a performance equivalence: memory bandwidth, inference software, quantization support, and the share reserved by macOS all matter. Leave extra headroom when sizing a Mac for local inference.
Next checks
A model that fits your GPUs can still be weak at your job. The AI model selector shortlists by task first. Tell it the work must stay on your own hardware and it keeps the list to open weights.