BenchLM recommendation
Best Local LLMs (July 2026): Ranked by VRAM Tier & Task
Kimi K2.6 is the best local LLM right now, scoring 74 on BenchLM with an 80.2 on SWE-bench Verified — which also makes it the top local coding pick. The value pick is Qwen3.6-27B (65), which runs on a single 24GB RTX 4090 at Q4 and still posts a 77.2 SWE-bench Verified.
A "local LLM" here means an open-weight model — one whose weights you can download and run on hardware you control, whether that is a gaming PC with an RTX 4090, a multi-GPU server, or a Mac with a large pool of unified memory. Proprietary models like GPT-5.4 or Claude are excluded: you can only reach them through an API, so they can never be local.
Every score on this page is the model's BenchLM display score, the same provisional weighted average used across our leaderboard, so local models are measured on exactly the same scale as the frontier APIs they replace. Hardware placements come from BenchLM's self-host catalog, which maps each model to a reference GPU configuration at 4-bit (Q4) quantization. We group the rankings by VRAM tier because that — not benchmark scores — is usually the binding constraint when you run models locally.
Best local LLMs for 8–16GB VRAM
On an 8–16GB card (RTX 4060 Ti, 4070-class, or older 2080 Ti/3080 hardware), you are limited to models of roughly 8–20B parameters at Q4. None of these small models are in BenchLM's self-host calculator catalog yet, so the VRAM figures below are standard Q4 estimates from parameter count. The standout is JetBrains' Mellum2-12B-A2.5B-Thinking (59): a sparse MoE with only 2.5B active parameters, it is fast on modest hardware and scores within six points of models that need a 24GB card. Zyphra's ZAYA1-8B (55) is the best pure 8B option.
| Model | BenchLM score | Params / Q4 size | Context |
|---|---|---|---|
| Mellum2-12B-A2.5B-ThinkingJetBrains | 59 | 12B MoE (2.5B active), ~7GB at Q4 | 128K |
| ZAYA1-8BZyphra | 55 | 8B dense, ~5GB at Q4 | 131K |
| Gemma 4 12BGoogle | 52 | 12B dense, ~7GB at Q4 | 256K |
| Ternary Bonsai 8BPrism ML | 52 | 8B ternary, under 4GB | 64K |
| Ornith-1.0-9BDeepReinforce AI | 48 | 9B dense, ~6GB at Q4 | 256K |
| GPT-OSS 20BOpenAI | 18 | 20B MoE, ~12GB at Q4 | 128K |
Best local LLMs for 16–24GB VRAM
A single 24GB card — the RTX 4090 is the reference GPU in our self-host catalog — is the sweet spot for local inference in July 2026. Qwen3.6-27B (65) is the clear pick: a dense 27B that quantizes to Q4 on one RTX 4090, with a 262K context window and a 77.2 SWE-bench Verified that embarrasses much larger models. Google's Gemma 4 31B (61) is the multimodal alternative on the same hardware. Both outscore several models that need ten times the VRAM.
| Model | BenchLM score | Params / quant | Q4 config | Context |
|---|---|---|---|---|
| Qwen3.6-27BAlibaba | 65 | 27B dense | Q4 on 1× NVIDIA RTX 4090 (24GB) (24GB) | 262K |
| Gemma 4 31BGoogle | 61 | 31B dense | Q4 on 1× NVIDIA RTX 4090 (24GB) (24GB) | 256K |
Best local LLMs for 40GB+ VRAM and multi-GPU rigs
Past 40GB you move from "gaming PC" to dual-consumer-GPU builds (2× RTX 4090 gives 48GB) and then to datacenter hardware. This is where the frontier open-weight MoE models live. Kimi K2.6 (74) — a 1T-parameter MoE with 32B active and native INT4 quantization — is the best local model on BenchLM, period, but it needs 8× H100. GLM-5.1 (68, MIT license) is close behind on the same footprint, while Kimi K2.5 (63) is the strongest option at the 4× A100 level.
| Model | BenchLM score | Params / quant | Q4 reference config | Context |
|---|---|---|---|---|
| Kimi K2.6Moonshot AI | 74 | 1000B MoE (32B active) | Q4 on 8× NVIDIA H100 (80GB) (640GB) | 256K |
| GLM-5.1Z.AI | 68 | 744B MoE (40B active) | Q4 on 8× NVIDIA H100 (80GB) (640GB) | 203K |
| Kimi K2.5Moonshot AI | 63 | 120B dense | Q4 on 4× NVIDIA A100 (80GB) (320GB) | 256K |
| Mistral Small 4Mistral | 49 | 119B MoE (22B active) | Q4 on 2× NVIDIA RTX 5090 (32GB GDDR7) (64GB) | 256K |
| Qwen2.5-72BAlibaba | 47 | 72B dense | Q4 on 2× NVIDIA RTX 4090 (24GB) (48GB) | 128K |
| DeepSeek V3DeepSeek | 38 | 671B MoE (37B active) | Q4 on 8× NVIDIA H100 (80GB) (640GB) | 128K |
| DeepSeek-R1DeepSeek | 33 | 671B MoE (37B active) | Q4 on 8× NVIDIA H100 (80GB) (640GB) | 128K |
| Llama 4 ScoutMeta | 24 | 109B MoE (17B active) | Q4 on 2× NVIDIA RTX 4090 (24GB) (48GB) | 10M |
| Kimi K2.7 CodeMoonshot AI | 21 | 1000B MoE (32B active) | Q4 on 8× NVIDIA H100 (80GB) (640GB) | 256K |
| Llama 4 MaverickMeta | 19 | 400B MoE (17B active) | Q4 on 2× NVIDIA A100 (80GB) (160GB) | 1M |
Best local LLMs for Apple Silicon (Mac)
Apple Silicon plays by different rules: unified memory replaces VRAM, so a Mac Studio M3 Ultra with 512GB (the one Apple machine in our GPU catalog, at $7,999) can hold model weights no consumer GPU rig can. It is the only desktop that fits GLM-5.1's 744B parameters at Q4 — though Kimi K2.6's full trillion still does not fit. For ordinary 32–64GB MacBooks and Minis, Qwen3.6-27B and Gemma 4 31B are the same standout picks as on a 24GB GPU. Memory-size notes below are Q4 estimates from parameter count.
| Model | BenchLM score | Params / quant | Unified memory needed | Context |
|---|---|---|---|---|
| GLM-5.1Z.AI | 68 | 744B MoE (40B active) | ~420GB at Q4 — 512GB Mac Studio M3 Ultra only | 203K |
| Qwen3.6-27BAlibaba | 65 | 27B dense | ~15GB at Q4 — 32GB unified memory | 262K |
| Kimi K2.5Moonshot AI | 63 | 120B dense | ~68GB at Q4 — 96GB+ unified memory | 256K |
| Gemma 4 31BGoogle | 61 | 31B dense | ~18GB at Q4 — 32GB unified memory | 256K |
| Qwen2.5-72BAlibaba | 47 | 72B dense | ~40GB at Q4 — 64GB+ unified memory | 128K |
Best GPU for running local LLMs
For most people the answer is a used or new RTX 4090 (24GB): at roughly $1,800 to buy or a $0.49/hr median rental, it runs every model in our 16–24GB tier and is the reference consumer GPU in BenchLM's self-host catalog. The RTX 5090 (32GB)(~$3,500) adds headroom for bigger quants and longer contexts, and a pair of them covers Mistral Small 4's 64GB Q4 footprint.
Stepping up, the inference-optimized NVIDIA L40S (48GB) (~$1.20/hr rented) is the best cost-per-token choice for 30B-class models, while the A100 80GB (~$1.49/hr) is the budget datacenter workhorse — four of them run Kimi K2.5 at Q4. Frontier MoE models like Kimi K2.6 and GLM-5.1 want 8× H100 80GB (~$2.60/hr each), which is firmly rental territory for most teams. The wildcard is the Mac Studio M3 Ultra (512GB unified): purchase-only at $7,999, but it is the cheapest single machine that holds 400B+ parameter weights.
| GPU | Memory | Median rental | Purchase |
|---|---|---|---|
| NVIDIA RTX 4090 (24GB) | 24GB | $0.49/hr | $1,800 |
| NVIDIA RTX 5090 (32GB GDDR7) | 32GB | $0.89/hr | $3,500 |
| NVIDIA L40S (48GB) | 48GB | $1.20/hr | $9,500 |
| NVIDIA A100 (80GB) | 80GB | $1.49/hr | $12,000 |
| NVIDIA H100 (80GB) | 80GB | $2.60/hr | $28,000 |
| Mac Studio M3 Ultra (512GB unified) | 512GB unified | — | $7,999 |
Want exact monthly costs for a specific model and GPU? Use the self-host cost calculator.
Best local LLM for coding
Kimi K2.6is the best local coding model on BenchLM's data: 80.2 on SWE-bench Verified, 89.6 on LiveCodeBench, and 66.7 on Terminal-Bench 2 — numbers that compete with proprietary coding APIs. Moonshot also ships a purpose-built Kimi K2.7 Code variant on the same 1T MoE architecture, though its benchmark coverage is still sparse on our leaderboard.
If you do not have a multi-GPU rig, Qwen3.6-27B is the best coding model per gigabyte of VRAM: 77.2 SWE-bench Verified and 83.9 LiveCodeBench from a 27B dense model on a single RTX 4090. Kimi K2.5 (76.8 SWE-bench Verified) sits between the two on a 4× A100 setup. All three keep 256K contexts, enough for serious repository-scale work.
How this ranking works
Models qualify for this page when they are Open Weightin BenchLM's catalog and have a published display score. Hardware placements come from our self-host catalog, which pairs each model with a reference Q4 GPU configuration; the 8–16GB and Apple Silicon tiers extend that with standard Q4 sizing estimates (roughly 0.55–0.6GB per billion parameters) where no calculator entry exists yet. Within each tier, models are sorted strictly by BenchLM score — the same provisional weighted average behind the overall leaderboard, which weights agentic, coding, and reasoning benchmarks most heavily. GPU prices are median rental and purchase rates from our GPU catalog, snapshotted from RunPod, Lambda, AWS, and CoreWeave public pricing.
Local LLM FAQ
What is the best local LLM right now?
Kimi K2.6 is the highest-scoring open-weight model you can self-host, at 74 on BenchLM's July 2026 leaderboard. It is a 1T-parameter MoE (32B active) with native INT4 quantization, but it needs an 8× H100 node. If "local" means a single machine you own, Qwen3.6-27B (65) on one 24GB RTX 4090 is the best realistic answer, with GLM-5.1 (68) as the top MIT-licensed frontier option.
What is the best local LLM for coding?
Kimi K2.6 leads local coding with 80.2 on SWE-bench Verified and 89.6 on LiveCodeBench — the best sourced coding numbers of any open-weight model we track. On consumer hardware, Qwen3.6-27B is remarkably close at 77.2 SWE-bench Verified despite fitting a single RTX 4090 at Q4. Moonshot's dedicated Kimi K2.7 Code is also worth watching as its benchmark coverage fills in.
What GPU do I need to run a local LLM?
A 24GB card like the RTX 4090 (~$1,800, or $0.49/hr rented) runs the best small models — Qwen3.6-27B and Gemma 4 31B — at Q4. Two 4090s (48GB) handle 72B-class dense models. Frontier open-weight MoEs need datacenter GPUs: 4× A100 80GB for Kimi K2.5, 8× H100 for Kimi K2.6 or GLM-5.1. A Mac with 32GB+ unified memory is a valid GPU-free alternative for the small tier.
Are local LLMs as good as ChatGPT or Claude?
Not quite, but the gap is smaller than most people assume. The proprietary frontier — Claude Fable 5 (91), Claude Mythos 5 (89), Gemini 3.1 Pro (88) — still leads BenchLM's leaderboard, while the best local model, Kimi K2.6, scores 74. For coding specifically the gap nearly closes: 80.2 SWE-bench Verified is competitive with proprietary APIs, and you keep full data control and zero per-token costs.
Local LLM updates
Get notified when a new open-weight release changes the local rankings or drops the hardware bar.
Free. No spam. Unsubscribe anytime.