PatchEval
We show this table for reference; we do not rank on it.
A vulnerability-repair benchmark that asks models to patch real security vulnerabilities in open-source code and checks the fix with tests.
Benchmark score on PatchEval — November 11, 2026
We compile the PatchEval rows from provider self-reports. Inkling 2 Small leads the table at 79.6%. We do not use these results to rank models overall.
1 modelCodingCurrentDisplay onlyUpdated November 11, 2026
Benchmark score table (1 model)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Inkling 2 SmallThinking Machines Lab | 79.6% | Not reported | Closed |
How to read this leaderboard
Operator receipt: 1 sourced row is currently displayable on this page; the leading published row is Inkling 2 Small at 79.6%.
Honest limit: Provider-run in the launch post; harness not published.
About PatchEval
Year
2026
Tasks
Real-world vulnerability patching tasks
Format
Patch success (%)
Difficulty
Security vulnerability repair
Thinking Machines reports PatchEval in the Inkling 2 Small launch (November 5, 2026): DeepSeek-V4.1-Flash-1017 80.4%, Inkling 2 Small 79.6%, Muse Spark Open 77.8%, Mistral Large 4 76.1%, GLM-5.3-Flash 74.8% and GLM 5.3 69.6%.
Freshness and provenance
Version
PatchEval 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does PatchEval measure?
A vulnerability-repair benchmark that asks models to patch real security vulnerabilities in open-source code and checks the fix with tests.
Which model scores highest on PatchEval?
Inkling 2 Small by Thinking Machines Lab currently leads with a score of 79.6% on PatchEval.
How many models are evaluated on PatchEval?
1 AI models have published results on PatchEval in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.