ZeroBench_main with Python (ZeroBench w/ Python)
We show this table for reference; we do not rank on it.
A Python-assisted ZeroBench_main evaluation reported as pass@5.
Benchmark score on ZeroBench w/ Python — September 27, 2026
We compile the ZeroBench w/ Python rows from provider self-reports. Qwen3.8 Max leads the table at 49.0%, followed by DeepSeek V4.1 Flash (49.0%) and Kimi K3 (41.0%). We do not use these results to rank models overall.
Qwen3.8 Max
Alibaba
DeepSeek V4.1 Flash
DeepSeek
Kimi K3
Moonshot AI
3 modelsMultimodal & GroundedCurrentDisplay onlyUpdated September 27, 2026
Benchmark score table (3 models)
ScoreAmong the reported ZeroBench w/ Python rows, Qwen3.8 Max is first at 49.0%. The third row is 8.0 points behind. The broader top-10 range is 8.0 points, so many of the published results sit in a relatively narrow band.
3 models have been evaluated on ZeroBench w/ Python. The benchmark falls in the Multimodal & Grounded category. ZeroBench w/ Python is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ZeroBench w/ Python
Year
2026
Tasks
Visual reasoning questions with Python
Format
Pass@5
Difficulty
Tool-augmented visual reasoning
Moonshot follows the official ZeroBench setting and runs it five times. BenchLM keeps this tool-assisted result separate from the non-Python ZeroBench key.
Freshness and provenance
Version
ZeroBench w/ Python 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ZeroBench w/ Python measure?
A Python-assisted ZeroBench_main evaluation reported as pass@5.
Which model scores highest on ZeroBench w/ Python?
Qwen3.8 Max by Alibaba currently leads with a score of 49.0% on ZeroBench w/ Python.
How many models are evaluated on ZeroBench w/ Python?
3 AI models have been evaluated on ZeroBench w/ Python on BenchLM.
Compare top models on ZeroBench w/ Python
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.