Best Local LLMs (October 2026): Benchmarks, Memory & Workload
As of October 2026, Qwen3.6-27B (47.95) is the capability-first starting point in this source-checked three-model shortlist. GPT-OSS 20B (33.51) has the clearest 16GB provider claim, while Gemma 4 12B (36.59) adds text, image, and audio input. These scores summarize general published capability evidence. They are not tests of a local quantization, runtime, computer, context length, or prompt.
The right local model is the strongest candidate that your runtime supports and your complete workload can hold in memory. Downloadable weights are only the first condition. License terms, quantization, context, KV cache, runtime buffers, concurrency, and CPU offload can all change the fit.
These three candidates cover coding and agent tasks, smaller multimodal workloads, and text reasoning. Other open-weight models may fit your task; this shortlist does not rank every downloadable model.
Primary model and runtime facts checked September 30, 2026.
Compare the source-checked local LLM shortlist
The score column comes from the same public projection used by the overall leaderboard. It compares published general capability evidence. It does not rank local throughput, quantized output quality, or the experience on your machine.
| Model | Public capability score | Published model facts | Local fit evidence | Workload starting point |
|---|---|---|---|---|
| Qwen3.6-27BAlibaba · Apache 2.0Official model card | 47.95 | 27B language model plus vision encoder; 262K native context | No minimum claimed. Qwen's full-context serving examples use tensor parallelism across eight GPUs and advise reducing context after an out-of-memory error. | Capability-first candidate for coding, agent work, and multimodal promptsLoses when: A smaller, text-only model is a better starting point when memory or runtime support is tight. |
| Gemma 4 12BGoogle · Apache 2.0Official model card | 36.59 | 11.95B Unified model; text, image, and audio input; 256K context | No minimum claimed. A weights-file size is not a runtime-memory requirement, and the exact artifact still has to match the runtime. | Smaller multimodal candidate for document, image, and audio-assisted workLoses when: GPT-OSS has a clearer provider memory claim for a text-only 16GB target. |
| GPT-OSS 20BOpenAI · Apache 2.0Official model card | 33.51 | 21B total / 3.6B active MoE; text-only; 128K context | OpenAI says its MXFP4 representation runs within 16GB of memory. That is a provider claim for that representation, not a promise for every runtime or context. | Text reasoning, function calling, and structured-output candidateLoses when: It is not the pick when the workload needs image or audio input. |
Fit the model to the memory you actually have
VRAM is memory attached to a discrete GPU. Unified memory is shared by the GPU, CPU, operating system, and other applications on Apple Silicon. System RAM can hold CPU-resident weights or an offloaded portion of a model, but offload changes the performance profile. Adding those labels together does not produce a universal minimum.
A 27B model at exactly four bits per parameter is 13.5GB of language weights in decimal units. That is weights-only arithmetic, not a buying recommendation. It excludes quantization metadata, non-quantized tensors, the vision component, runtime buffers, KV cache, context growth, concurrency, and operating-system memory.
Active parameters describe the part of a mixture-of-experts model used for a token. They can help explain compute, but all resident experts still need storage and memory treatment. A 32B-active model is not automatically a 32B-weight deployment.
Practical guidance by hardware tier
8-16GB VRAM or unified memory
This shortlist makes no 8GB recommendation. GPT-OSS 20B is the clearest 16GB candidate because OpenAI says the MXFP4 representation runs within 16GB. That does not establish a fit below 16GB or with every runtime. A different conversion, longer context, parallel requests, or extra buffers can exceed the provider figure.
24GB-class discrete GPUs
Qwen3.6-27B is worth evaluating when your runtime supports a suitable quantization, but the model name does not prove a full 262K-context fit. Qwen's own maximum-context examples use eight GPUs. Start with the context your task needs, then inspect actual allocation and offload before increasing it.
Apple Silicon and larger shared-memory systems
Apple unified memory can hold models that exceed one discrete GPU's VRAM, and llama.cpp supports Metal and CPU-GPU hybrid inference. Reserve memory for macOS and other applications, verify the exact model format, and treat CPU offload as part of the tested configuration rather than invisible extra capacity.
Multi-GPU servers
Several cards do not behave like one larger card by default. The runtime must support the model, quantization, and parallel layout you intend to use. Follow the model owner's deployment example, then test the actual context and concurrency instead of deriving a requirement from parameter count alone.
Best local LLM for coding
Qwen3.6-27B is the first coding candidate to test in this shortlist. Its public capability score is 47.95, and its official card publishes coding-agent results. Those results used specified provider scaffolds. They do not measure your quantized checkpoint inside your editor or agent loop.
Run the same repository task against Qwen3.6-27B and the smaller candidate that fits comfortably. Record completion, test pass rate, edit reversions, peak memory, prompt tokens, generated tokens, and elapsed time. The coding ranking supplies a capability shortlist; your repository check decides the local deployment.
Check runtime, memory, context, then workload
| Step | Verify | Reject the candidate when |
|---|---|---|
| 1. Runtime and license | Exact model identifier, quantization support, chat format, tools, modality support, and license terms. | The runtime substitutes another artifact or cannot reproduce the required interface. |
| 2. Memory | Peak VRAM, unified memory, and system RAM after the model loads. Record any CPU offload. | The workload leaves no operating headroom or relies on an unplanned offload path. |
| 3. Context and concurrency | Configured context, KV-cache use, prompt length, batch size, and simultaneous requests. | The advertised maximum fits only after shrinking the real workload. |
| 4. Workload check | Use representative prompts and score task completion, factual or constraint failures, latency, and resource use. | A smaller model completes more of the actual work within the same resource budget. |
Ollama documents that a larger configured context increases memory use and exposes the actual processor split and context through ollama ps. Use the LLM VRAM calculator for a planning estimate, then verify the running process before calling the fit proven.
Downloaded weights do not make inference free
Self-hosted weights can remove a vendor token fee. The machine, electricity or rental, storage, maintenance, monitoring, and operator time remain. Compare those inputs with the self-host cost calculator. Do not infer operating cost from a catalog token-price field.
How this shortlist works
Each row needs an open-weight public model entry and a primary model card that supports the architecture, context, access, and license statements shown here. Current ranking-eligible rows show their public leaderboard score; other shortlisted rows show unavailable instead of falling back to stored or provisional catalog values. Hardware statements remain provider claims or planning checks until a matching local configuration is tested.
Questions
What is the best local LLM right now?
Qwen3.6-27B is the capability-first candidate in this source-checked three-model shortlist, with a current public score of 47.95. It is not a universal winner. GPT-OSS 20B has a clearer 16GB provider claim, while Gemma 4 12B covers text, image, and audio input in a smaller model.
What is the best local LLM for coding?
Start the local coding comparison with Qwen3.6-27B, then test it against the smaller model that fits with headroom. Its 47.95 score is general published capability evidence, not a quantized repository test. Use the same task, tools, context, and acceptance checks for each candidate before choosing.
How much memory does a local LLM need?
There is no universal number from parameter count alone. The exact weights, quantization metadata, runtime buffers, KV cache, context length, concurrency, multimodal components, operating system, and CPU offload all matter. Measure VRAM, unified memory, and system RAM on the configuration you intend to keep running.
Can a Mac run a local LLM?
Yes, when the runtime supports the model format and the full workload fits in shared unified memory. llama.cpp supports Apple Silicon through Metal and can split work across CPU and GPU. Leave room for macOS and applications, record offload, and test the context you will actually use.
Are local LLMs free to run?
Downloadable weights may have no vendor token charge, subject to their license and usage terms. Running them still consumes hardware, electricity or rental, storage, maintenance, monitoring, and operator time. Compare total workload cost instead of translating a zero API-price field into free inference.
Local LLM updates
Get notified when a new open-weight release changes the local rankings or drops the hardware bar.
Read a sample issueJoin 5,500+ readers.
One email each week. Unsubscribe anytime.