MaxxBench (mutated DeepSWE tasks) (MaxxBench)
We show this table for reference; we do not rank on it.
unLAP's MaxxBench takes 89 DeepSWE software engineering tasks and changes the instructions or starting code while keeping the core problem, to test whether coding agents adapt to new requirements or fall back on a memorized solution.
Benchmark score on MaxxBench — November 7, 2026
We compile the MaxxBench rows from benchmark-owner or independent runs. Claude Opus 5.5 leads the table at 76.4%, followed by GPT-6 Astra (71.9%) and GPT-5.6 Sol (69.1%). We do not use these results to rank models overall.
Claude Opus 5.5
Anthropic
GPT-6 Astra
OpenAI
GPT-5.6 Sol
OpenAI
14 modelsCodingCurrentDisplay onlyUpdated November 7, 2026
Benchmark score table (14 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 5.5Anthropic | 76.4% | Not reported | Closed |
| 2 | GPT-6 AstraOpenAI | 71.9% | Not reported | Closed |
| 3 | GPT-5.6 SolOpenAI | 69.1% | Not reported | Closed |
| 4 | Claude Sonnet 5.5Anthropic | 66.9% | Not reported | Closed |
| 5 | Claude Opus 5Anthropic | 63.5% | Not reported | Closed |
| 6 | Claude Fable 5Anthropic | 63.5% | Not reported | Closed |
| 7 | GLM-5.3Z.AI | 56.7% | Not reported | Open |
| 8 | DeepSeek V4.1 Flash 1017DeepSeek | 55.6% | Not reported | Open |
| 9 | GLM-5.3-FlashZ.AI | 55.1% | Not reported | Open |
| 10 | Gemini 3.8 FlashGoogle | 53.4% | Not reported | Closed |
| 11 | GPT-6 LunaOpenAI | 51.7% | Not reported | Closed |
| 12 | GPT-5.6 LunaOpenAI | 46.1% | Not reported | Closed |
| 13 | Muse Spark 1.2Meta | 38.8% | Not reported | Closed |
| 14 | DeepSeek V4 Flash 0731DeepSeek | 28.1% | Not reported | Open |
How to read this leaderboard
Higher is better. With 178 attempts, one extra pass moves the score by about 0.6 points, so models a few points apart are not reliably ordered (Claude Opus 5 and Claude Fable 5 tie at 113 passes). A large positive gap to DeepSWE means the model did much worse once the familiar task changed; Claude Opus 5.5 is the only model that scores higher on MaxxBench than on DeepSWE.
Operator receipt: 14 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5.5 at 76.4%.
Honest limit: One harness (mini-swe-agent) and one effort setting per model, so a score describes that configuration, not the model under every setup. The DeepSWE baselines behind the gap come from different publishers and settings, and the original and mutated task sets are not identical, so the gap is an estimate, not a contamination measurement. The paper is not out yet. BenchLM shows MaxxBench as display-only; it does not enter overall or category rankings.
Among the reported MaxxBench rows, Claude Opus 5.5 is first at 76.4%. The third row is 7.3 points behind. The broader top-10 range is 23.0 points, so the table still separates the published systems.
14 models have been evaluated on MaxxBench. The benchmark falls in the Coding category. We keep these results separate from the weighted global scoring system, so they remain source-specific evidence. MaxxBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About MaxxBench
Year
2026
Tasks
89 mutated DeepSWE feature-implementation tasks, two attempts per model
Format
Pass rate over 178 agent attempts, mini-swe-agent harness
Difficulty
Long-horizon software engineering under changed requirements
Each MaxxBench task starts from a DeepSWE feature-implementation task of medium length and changes what is asked (a renamed output, an extra constraint, a different interface) or the code the agent starts from, while the engineering problem stays the same. Reference patches grow by a median of 17 lines (3.5%). Every model runs in the same mini-swe-agent harness at the effort the source lists (max, xhigh or high), twice per task, and a run passes only if all required feature and regression tests pass in a separate grading container. The score is the share of the 178 attempts that pass. The leaderboard also shows an estimated benchmaxxing gap: the model's published DeepSWE score minus its MaxxBench score. Tasks, reference solutions, tests and per-run trajectories are published.
Freshness and provenance
Version
MaxxBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MaxxBench measure?
Whether a coding agent can still solve a familiar DeepSWE task once its requirements or starting code change. The core engineering problem stays the same, so a model that really understands it should adapt; a model tuned on the original task may ship a solution that no longer fits.
Which model leads MaxxBench?
Claude Opus 5.5 at max effort leads with 76.4% (136 of 178 attempts), ahead of GPT-6 Astra at xhigh (71.9%) and GPT-5.6 Sol at max (69.1%). Opus 5.5 is also the only model whose MaxxBench score is higher than its DeepSWE score.
What is the benchmaxxing estimate?
The MaxxBench leaderboard subtracts each model's MaxxBench score from its published DeepSWE score. A large gap, such as 25.2 points for DeepSeek V4 Flash or 21.1 for GPT-5.6 Luna, suggests the model does much better on the original tasks than on close variants. It is an estimate, because the baselines come from different sources and settings.
Compare top models on MaxxBench
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.