ZeroBench_main with Python (ZeroBench w/ Python)
We show this table for reference; we do not rank on it.
A Python-assisted ZeroBench_main evaluation reported as pass@5.
Benchmark score on ZeroBench w/ Python — November 13, 2026
We compile the ZeroBench w/ Python rows from provider self-reports. Kimi K3.1 Pro leads the table at 63.0%, followed by Hy4 (55.4%) and DeepSeek V4.1 Flash 1017 (51.0%). We do not use these results to rank models overall.
Kimi K3.1 Pro
Moonshot AI
Reported by Moonshot AIHy4
Tencent
Reported by TencentDeepSeek V4.1 Flash 1017
DeepSeek
Reported by DeepSeek5 modelsMultimodal & GroundedCurrentDisplay onlyUpdated November 13, 2026
Benchmark score table (5 models)
All 5 results on this page were reported by the labs that built the models.
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AIReported by Moonshot AI | 63.0% | Reported by Moonshot AI | Not reported | Open |
| 2 | Hy4TencentReported by Tencent | 55.4% | Reported by Tencent | Not reported | Pending |
| 3 | DeepSeek V4.1 Flash 1017DeepSeekReported by DeepSeek | 51.0% | Reported by DeepSeek | Not reported | Open |
| 4 | Qwen3.8 MaxAlibabaReported by Alibaba | 49.0% | Reported by Alibaba | Not reported | Open |
| 5 | Kimi K3Moonshot AIReported by Moonshot AI | 41.0% | Reported by Moonshot AI | Not reported | Open |
- Lab-reported
- Benchmark-run
- Independent
Among the reported ZeroBench w/ Python rows, Kimi K3.1 Pro is first at 63.0%. The third row is 12.0 points behind. The broader top-10 range is 22.0 points, so the table still separates the published systems.
5 models have been evaluated on ZeroBench w/ Python. The benchmark falls in the Multimodal & Grounded category. ZeroBench w/ Python is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ZeroBench w/ Python
Year
2026
Tasks
Visual reasoning questions with Python
Format
Pass@5
Difficulty
Tool-augmented visual reasoning
Moonshot follows the official ZeroBench setting and runs it five times. BenchLM keeps this tool-assisted result separate from the non-Python ZeroBench key.
Freshness and provenance
Version
ZeroBench w/ Python 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ZeroBench w/ Python measure?
A Python-assisted ZeroBench_main evaluation reported as pass@5.
Which model scores highest on ZeroBench w/ Python?
Kimi K3.1 Pro by Moonshot AI currently leads with a score of 63.0% on ZeroBench w/ Python.
How many models are evaluated on ZeroBench w/ Python?
5 AI models have published results on ZeroBench w/ Python in the BenchLM catalog.
Compare top models on ZeroBench w/ Python
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.