Benchmark profile
Evaluating Large Language Models Trained on Code (HumanEval)
A set of 164 handwritten Python function-generation problems. HumanEval is useful as a historical floor check, but BenchLM's current exact-source table is too small to support a broad frontier-coding verdict.
Data verifiedHow to read this leaderboard
Read this page as a source-coverage receipt for the exact rows BenchLM can currently display, not as a ranking of the best coding models. Use the calibrated coding leaderboard and repository benchmarks for a broader decision.
Operator receipt: 3 sourced rows are currently displayable on this page; the leading published row is DeepSeek V4 Pro Base at 76.8%.
Honest limit: Only exact, displayable rows appear. Missing frontier rows are missing evidence, not zero scores, and the two visible DeepSeek base-model rows do not establish an overall coding winner.
Benchmark score on HumanEval — July 20, 2026
BenchLM mirrors the published score view for HumanEval. DeepSeek V4 Pro Base leads the public snapshot at 76.8% , followed by Soofi S 30B-A3B (73.8%) and DeepSeek V4 Flash Base (69.5%). BenchLM does not use these results to rank models overall.
DeepSeek V4 Pro Base
DeepSeek
deepseek-v4-pro-base
Soofi S 30B-A3B
Soofi Project
soofi-s-30b-a3b
DeepSeek V4 Flash Base
DeepSeek
deepseek-v4-flash-base
Benchmark score table (3 models)
ScoreThe published HumanEval snapshot places DeepSeek V4 Pro Base first at 76.8%. The third row is 7.3 points behind. The broader top-10 range is 7.3 points, so many of the published results sit in a relatively narrow band.
3 models have been evaluated on HumanEval. The benchmark falls in the Coding category. This category carries a 20% weight in BenchLM.ai's overall scoring system. HumanEval is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About HumanEval
Year
2021
Tasks
164 problems
Format
Python function generation
Difficulty
Introductory to intermediate programming
HumanEval measures functional correctness for programs synthesized from docstrings. The tasks range from string manipulation to intermediate algorithms, but they do not test repository navigation, multi-file changes, tool use, or long-running agent work.
BenchLM freshness & provenance
Version
HumanEval
Refresh cadence
Static
Staleness state
Stale
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does HumanEval measure?
A set of 164 handwritten Python function-generation problems. HumanEval is useful as a historical floor check, but BenchLM's current exact-source table is too small to support a broad frontier-coding verdict.
Which model scores highest on HumanEval?
DeepSeek V4 Pro Base by DeepSeek currently leads with a score of 76.8% on HumanEval.
How many models are evaluated on HumanEval?
3 AI models have been evaluated on HumanEval on BenchLM.
Compare Top Models on HumanEval
Choose a model with this week’s evidence
Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.
One email each week. Unsubscribe anytime.