# PatchEval

> A vulnerability-repair benchmark that asks models to patch real security vulnerabilities in open-source code and checks the fix with tests.

Canonical page: https://benchlm.ai/benchmarks/patcheval

- Category: [Coding](/coding)
- Last updated: November 11, 2026

## About PatchEval

- Year: 2026
- Tasks: Real-world vulnerability patching tasks
- Format: Patch success (%)
- Difficulty: Security vulnerability repair
- Paper: [Introducing Inkling 2 Small](https://thinkingmachines.ai/news/inkling-2-small/)

Thinking Machines reports PatchEval in the Inkling 2 Small launch (November 5, 2026): DeepSeek-V4.1-Flash-1017 80.4%, Inkling 2 Small 79.6%, Muse Spark Open 77.8%, Mistral Large 4 76.1%, GLM-5.3-Flash 74.8% and GLM 5.3 69.6%.

PatchEval is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (1 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Inkling 2 Small](/models/inkling-2-small) | Thinking Machines Lab | 79.6% |

## FAQ

### What does PatchEval measure?

A vulnerability-repair benchmark that asks models to patch real security vulnerabilities in open-source code and checks the fix with tests.

### Which model scores highest on PatchEval?

Inkling 2 Small by Thinking Machines Lab currently leads with a score of 79.6% on PatchEval.

### How many models are evaluated on PatchEval?

1 AI models have published results on PatchEval in the BenchLM catalog.

### Does PatchEval affect BenchLM's overall score?

Not directly. PatchEval is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.