LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (LiveCodeBench)
We show this table for reference; we do not rank on it.
Solar Pro 4 has the highest reported LiveCodeBench score at 87.8%, ahead of DeepSeek V3.1 (Reasoning) (74.8%) and Grok 3 Mini (74.8%) among 13 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.
A continuously updated coding benchmark built from newly collected LeetCode, AtCoder, and Codeforces problems. Fresh problem windows reduce one contamination path, but results still need a release and setup check.
Benchmark score on LiveCodeBench — October 29, 2026
We compile the LiveCodeBench rows from provider self-reports and secondary reports. Solar Pro 4 leads the table at 87.8%, followed by DeepSeek V3.1 (Reasoning) (74.8%) and Grok 3 Mini (74.8%). We do not use these results to rank models overall.
Solar Pro 4
Upstage
DeepSeek V3.1 (Reasoning)
DeepSeek
Grok 3 Mini
xAI
13 modelsCodingCurrentDisplay onlyUpdated October 29, 2026
API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.
Benchmark score table (13 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|
| 1 | Solar Pro 4Upstage | 87.8% | Not reported | Closed | $0.3 / $1.2 | 512K | Proprietary |
| 2 | DeepSeek V3.1 (Reasoning)DeepSeek | 74.8% | Not reported | Open | Not recorded | 128K | Not recorded |
| 3 | Grok 3 MinixAI | 74.8% | Not reported | Closed | $0.3 / $0.5 | 128K | Proprietary |
| 4 | DeepSeek-R1-0528DeepSeek | 73.3% | Not reported | Open | Not recorded | 128K | Not recorded |
| 5 | Mercury 2Inception | 67.3% | Not reported | Closed | $0.25 / $0.75 | 128K | Proprietary |
| 6 | Nemotron Ultra 253BNVIDIA | 66.3% | Not reported | Open | Not recorded | 32K | Not recorded |
| 7 | MiniMax M1 80kMiniMax | 65% | Not reported | Closed | Not recorded | 80K | Proprietary |
| 8 | Grok 3 [Beta]xAI | 57% | Not reported | Closed | Not recorded | 128K | Proprietary |
| 9 | DeepSeek V3.1DeepSeek | 56.4% | Not reported | Open | Not recorded | 128K | Not recorded |
| 10 | Phi-4-reasoningMicrosoft | 53.8% | Not reported | Open | Not recorded | 32K | Not recorded |
| 11 | Phi-4-reasoning-plusMicrosoft | 53.1% | Not reported | Open | Not recorded | 32K | Not recorded |
| 12 | Qwen2.5-MaxAlibaba | 38.7% | Not reported | Closed | Not recorded | Unknown | Proprietary |
| 13 | DeepSeek V3DeepSeek | 37.6% | Not reported | Open | $0.27 / $1.1 | 128K | MIT |
How to read this leaderboard
Prior editorial review by Glevd · 2026-07-15
Variant check: Catalog input/output prices and context limits describe current model access. They do not record the benchmark run’s token use, retries, or actual cost. A model’s weight-access label is separate from its license terms.
Use LiveCodeBench as a competitive-programming signal after matching the release, date window, task scenario, pass@k metric, sampling count, temperature, and execution policy. Explicit v5, v6, and Pass@1-COT rows now have separate lanes; remaining generic rows can still use different or incompletely labeled windows.
Operator receipt: 13 sourced rows are currently displayable on this page; the leading published row is Solar Pro 4 at 87.8%.
Honest limit: LiveCodeBench does not test work inside an existing repository. Release and date-window choice, the pruned versus original task set, generation settings, timeouts, and known autograding issues can affect a score. This legacy lane is a sourced result ledger, not one controlled BenchLM rerun.
Among the reported LiveCodeBench rows, Solar Pro 4 is first at 87.8%. The third row is 13.0 points behind. The broader top-10 range is 34.0 points, so the table still separates the published systems.
13 models have been evaluated on LiveCodeBench. The benchmark falls in the Coding category. LiveCodeBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About LiveCodeBench
Year
2024
Tasks
Continuously updated contest problems
Format
Competitive-programming evaluation
Difficulty
Competitive programming level
The official suite evaluates code generation, code execution, test-output prediction, and self-repair on contest problems added over time. Its releases and selectable date windows let evaluators separate older and newer problem sets instead of treating the benchmark as one fixed test.
Freshness and provenance
Version
Rolling 2026 set
Refresh cadence
Rolling
Staleness state
Current
Question availability
Delayed public release
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does LiveCodeBench measure?
LiveCodeBench evaluates coding on newly collected contest problems from LeetCode, AtCoder, and Codeforces. Its suite covers code generation, execution, test-output prediction, and self-repair, although provider tables commonly report code-generation pass rates. It tests competitive-programming correctness, not work inside an existing software repository.
Are LiveCodeBench scores directly comparable?
Only when the release, date window, scenario, pass@k metric, sampling count, temperature, and execution policy match. The official benchmark can change as new problems arrive, while providers may report different slices. BenchLM preserves sourced published rows, so unmatched settings support a directional read, not a precise ranking.
Can LiveCodeBench pick the best coding agent?
No. A high score supports competitive-programming performance under the reported setup. It does not establish codebase navigation, issue interpretation, tool use, patch review, or regression safety. Pair LiveCodeBench with a repository benchmark such as SWE-bench Pro and a trial drawn from your own languages, libraries, and runtime constraints.
Compare top models on LiveCodeBench
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.