Skip to main content
BenchLM
Data

SWE-bench Pro

Claude Opus 5.5 leads the SWE-bench Pro leaderboard on BenchLM's October 2026 update with 89.9%, ahead of Claude Sonnet 5.5 (81.3%) and Claude Fable 5.1 (81.2%), across 85 models.

Data verified

A long-horizon repository benchmark built to test realistic software engineering work. Its scores need a task-quality and setup check before they support a coding-agent decision.

Top models on SWE-bench Pro — October 27, 2026

As of October 27, 2026, Claude Opus 5.5 leads the SWE-bench Pro leaderboard with 89.9% , followed by Claude Sonnet 5.5 (81.3%) and Claude Fable 5.1 (81.2%).

1Closed

Claude Opus 5.5

Anthropic

89.9%
Overall 86.66Context 1M
2Closed

Claude Sonnet 5.5

Anthropic

81.3%
Overall 82.25Context 1M
3Closed

Claude Fable 5.1

Anthropic

81.2%
Overall 83.22Context 1M

85 modelsCoding26% of Coding reference weightCurrentUpdated October 27, 2026

API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.

Leaderboard (85 models)

SWE-bench Pro leaderboard
RankModel / configurationScoreParameters (B)Open / closedAPI input / output ($/1M)ContextLicense
1Claude Opus 5.5Anthropic
89.9%
Not reportedClosed$4 / $201MProprietary
2Claude Sonnet 5.5Anthropic
81.3%
Not reportedClosed$2 / $101MProprietary
3Claude Fable 5.1Anthropic
81.2%
Not reportedClosed$10 / $501MProprietary
4Claude Mythos 5Anthropic
80.3%
Not reportedClosed$10 / $501M+Proprietary
5Claude Fable 5Anthropic
80%
Not reportedClosed$10 / $501M+Proprietary
6Claude Opus 5Anthropic
79.2%
Not reportedClosed$5 / $251MProprietary
7Sakana Fugu-UltraSakana AI
73.7%
Not reportedClosedNot recorded1MProprietary
8Claude Opus 4.8Anthropic
69.2%
Not reportedClosed$5 / $251MProprietary
9Ling 3.1 FlashInclusionAI
68.7%
Not reportedOpenNot recorded262KMIT
10Qwen3.8 MaxAlibaba
67.7%
Not reportedOpenNot recorded1MQwen3.8-Max License
11Beam 1.0 501B-A23BReflection AI
66.1%
Not reportedOpenNot recorded1MApache-2.0
12Beam 1.0 501B-A23B FP8Reflection AI
65.9%
Not reportedOpenNot recorded1MNot recorded
13Hy4 previewTencent
65.7%
Not reportedOpenNot recorded1MApache-2.0
14Beam 1.0 501B-A23B NVFP4Reflection AI
65.6%
Not reportedOpenNot recorded1MNot recorded
15Ornith-1.5-397BOrnith AI
65.1%
Not reportedOpenNot recorded262KMIT
16Grok 4.5xAI
64.7%
Not reportedClosed$2 / $6500KProprietary
17GPT-5.6 SolOpenAI
64.6%
Not reportedClosed$4 / $201.05MProprietary
18Claude Opus 4.7 (Adaptive)Anthropic
64.3%
Not reportedClosed$5 / $251MProprietary
19GPT-5.6 TerraOpenAI
63.4%
Not reportedClosed$2 / $121.05MProprietary
20Qwen3.8-Omni-FlashAlibaba
63.3%
Not reportedClosedNot recorded1MProprietary
21Claude Sonnet 5Anthropic
63.2%
Not reportedClosed$2 / $101MProprietary
22GPT-5.6 LunaOpenAI
62.7%
Not reportedClosed$0.2 / $1.21.05MProprietary
23Qwen3.8-Flash-NextAlibaba
62.5%
Not reportedOpenNot recorded262KQwen Community 1.0
24Ornith-1.0-397BDeepReinforce AI
62.2%
Not reportedOpenNot recorded256KMIT
25GLM-5.2Z.AI
62.1%
Not reportedOpen$1.4 / $4.41MMIT
26Qwen3.8-27BAlibaba
61.7%
Not reportedOpenNot recorded262KApache-2.0
27Muse Spark 1.1Meta
61.5%
Not reportedClosedNot recorded1MProprietary
28Sage-A1.5InternScience
61.2%
Not reportedOpenNot recorded262KApache-2.0
29dots3-note PreviewDots Studio
61%
Not reportedOpenNot recorded512KApache-2.0
30Qwen3.7 MaxAlibaba
60.6%
Not reportedClosedNot recorded1MProprietary
31Atria Dawn PreviewShanghai Artificial Intelligence Laboratory
59.6%
Not reportedOpenNot recorded256KMIT
32Ornith-1.5-35B-A3BOrnith AI
59.6%
Not reportedOpenNot recorded262KNot recorded
33Laguna S 2.1Poolside
59.4%
Not reportedOpen$0.1 / $0.21MOpenMDW-1.1
34MiniMax M3MiniMax
59%
Not reportedOpen$0.3 / $1.21MMiniMax Community License
35Sakana FuguSakana AI
59%
Not reportedClosedNot recorded1MProprietary
36GPT-5.5OpenAI
58.6%
Not reportedClosed$5 / $301MProprietary
37Kimi K2.6Moonshot AI
58.6%
Not reportedOpen$0.95 / $4256KModified MIT
38GLM-5.1Z.AI
58.4%
Not reportedOpen$1.4 / $4.4203KMIT
39GPT-5.4OpenAI
57.7%
Not reportedClosed$2.5 / $151.05MProprietary
40Qwen3.7 PlusAlibaba
57.6%
Not reportedClosedNot recorded1MProprietary
41Qwen 3.6 Max (preview)Alibaba
57.3%
Not reportedClosedNot recorded256KProprietary
42MiMo-V2.5-ProXiaomi
57.2%
Not reportedClosedNot recorded1MProprietary
43Claude Opus 4.5Anthropic
57.1%
Not reportedClosed$5 / $25200KProprietary
44GPT-5.3 CodexOpenAI
56.8%
Not reportedClosed$1.75 / $14400KProprietary
45Qwen3.6 PlusAlibaba
56.6%
Not reportedClosedNot recorded1MProprietary
46Ling 3.0 FlashInclusionAI
56.6%
Not reportedOpenNot recorded262KMIT
47Step 3.7 FlashStepFun
56.3%
Not reportedOpen$0.2 / $1.15256KApache-2.0
48MiniMax M2.7MiniMax
56.2%
Not reportedOpen$0.3 / $1.2200KMiniMax License
49MiMo-V2.5Xiaomi
56.1%
Not reportedClosedNot recorded1MProprietary
50Inkling-SmallThinking Machines Lab
55.9%
Not reportedOpen$0.58 / $1.441MNot recorded
51GPT-5.2OpenAI
55.6%
Not reportedClosed$1.75 / $14400KProprietary
52DeepSeek V4 Pro 0813DeepSeek
55.4%
Not reportedOpen$1.32 / $3.961MMIT
53Gemini 3.5 FlashGoogle
55.1%
Not reportedClosed$1.5 / $91MProprietary
54GLM-5Z.AI
55.1%
Not reportedOpen$1 / $3.2200KMIT
55DeepSeek V4 Pro (High)DeepSeek
54.4%
Not reportedOpenNot recorded1MNot recorded
56InklingThinking Machines Lab
54.3%
Not reportedOpen$1.87 / $4.681MApache-2.0
57Spark-X2.6-4BSparkLLM
54.3%
Not reportedOpenNot recorded1MApache-2.0
58Gemini 3.5 Flash-LiteGoogle
54.2%
Not reportedClosed$0.3 / $2.51MProprietary
59Qwen3.6-27BAlibaba
53.5%
Not reportedOpenNot recorded262KApache-2.0
60Claude Opus 4.6Anthropic
53.4%
Not reportedClosed$5 / $251MProprietary
61MAI-Thinking-1Microsoft
52.8%
Not reportedClosedNot recorded256KProprietary
62DeepSeek V4 Flash 0731DeepSeek
52.6%
Not reportedOpen$0.14 / $0.281MMIT
63Muse SparkMeta
52.4%
Not reportedClosedNot recorded262KProprietary
64DeepSeek V4 Flash (High)DeepSeek
52.3%
Not reportedOpenNot recorded1MNot recorded
65DeepSeek V4 ProDeepSeek
52.1%
Not reportedOpenNot recorded1MNot recorded
66Grok 4.20xAI
51.8%
Not reportedClosed$1.25 / $2.52MProprietary
67Nemotron 3.5 Super VL 120B A12BNVIDIA
51.8%
Not reportedOpenNot recorded1MOpenMDW-1.1
68Nemotron 3.5 Super VL 120B A12B NVFP4NVIDIA
51.3%
Not reportedOpenNot recorded1MNot recorded
69Muse Glimmer 30BMeta
51.2%
Not reportedOpenNot recorded131KApache-2.0
70Qwen3.5 397BAlibaba
50.9%
Not reportedOpen$0.6 / $3.6128KApache-2.0
71Kimi K2.5Moonshot AI
50.7%
Not reportedOpen$0.6 / $3256KModified MIT
72Ornith-1.0-35BDeepReinforce AI
50.4%
Not reportedOpenNot recorded256KNot recorded
73Qwen3.6-35B-A3BAlibaba
49.5%
Not reportedOpenNot recorded262KApache-2.0
74Laguna M.1Poolside
49.2%
Not reportedClosedNot recorded256KProprietary
75DeepSeek V4 FlashDeepSeek
49.1%
Not reportedOpenNot recorded1MNot recorded
76Laguna XS 2.1Poolside
47.6%
Not reportedOpen$0.1 / $0.2262KNot recorded
77Ornith-1.5-9BOrnith AI
47.5%
Not reportedOpenNot recorded262KNot recorded
78Laguna XS.2Poolside
46.3%
Not reportedOpenNot recorded256KNot recorded
79Ornith-1.0-9BDeepReinforce AI
42.9%
Not reportedOpenNot recorded256KNot recorded
80LongCat-Flash-Lite-SparseMeituan
40.6%
Not reportedOpenNot recorded1MMIT
81Granite 4.2 30BIBM
33.3%
Not reportedOpenNot recorded128KApache-2.0
82LLaDA2.2-flashInclusionAI
30.1%
Not reportedOpenNot recorded128KNot recorded
83Mellum2.1-12B-A2.5B-ThinkingJetBrains
28%
Not reportedOpenNot recorded128KApache-2.0
84Granite 4.2 8BIBM
19.1%
Not reportedOpenNot recorded128KApache-2.0
85MiniCPM5-2BOpenBMB
14.4%
Not reportedOpenNot recorded131KApache-2.0

How to read this leaderboard

Prior editorial review by Glevd · 2026-07-15

Variant check: Catalog input/output prices and context limits describe current model access. They do not record the benchmark run’s token use, retries, or actual cost. A model’s weight-access label is separate from its license terms.

Use SWE-bench Pro as evidence for long-horizon repository work only after matching the split, scaffold, tool budget, retry policy, and run count. The table preserves exact published rows from different providers, so a small score gap is directional unless the setups match.

Operator receipt: 85 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5.5 at 89.9%.

Honest limit: OpenAI's July 2026 audit estimated that about 30% of the 731-task public split is broken and retracted its earlier recommendation to adopt the benchmark. The current rows remain useful published receipts, but this page should not decide a coding-agent purchase by itself.

Calibrated coding rankingLiveCodeBench leaderboardHow benchmark confidence works
SWE-Bench Pro paperOfficial code and reproduction guideJuly 2026 task-quality audit

According to BenchLM.ai, Claude Opus 5.5 leads the SWE-bench Pro benchmark with a score of 89.9%, followed by Claude Sonnet 5.5 (81.3%) and Claude Fable 5.1 (81.2%). There is significant spread across the leaderboard, making this benchmark effective at differentiating model capabilities.

85 models have been evaluated on SWE-bench Pro. The benchmark falls in the Coding category. BenchAlign v5.8 gives SWE-bench Pro 26% of the Coding reference weight, so it moves the Coding leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.

About SWE-bench Pro

Year

2025

Tasks

1,865 repository problems

Format

Repository task completion

Difficulty

Long-horizon professional engineering

The authors assembled 1,865 problems from 41 repositories across public, held-out, and commercial splits. Agents receive a repository and issue, then produce a patch that must pass the evaluation tests without breaking existing behavior.

Teams weighing long-horizon repository work usually read this table next to DeepSWE scores, a second long-horizon benchmark built on original tasks.

SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

Freshness and provenance

Version

SWE-bench Pro 2025

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

Current

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does SWE-bench Pro measure?

SWE-bench Pro gives an agent a repository and issue description, then checks whether its patch passes new tests without breaking existing behavior. The full benchmark contains 1,865 problems from 41 repositories across public, held-out, and commercial splits, with work that can span multiple files and long execution horizons.

Are SWE-bench Pro scores directly comparable?

Only when the split and evaluation setup match. Public, held-out, and commercial tasks are different pools, while scaffold, tool budget, retry policy, and token budget can change pass rates. This page preserves exact published rows, but it does not pretend every provider ran the same harness.

Should SWE-bench Pro decide which coding agent to use?

No. The benchmark covers realistic repository work, but OpenAI's July 2026 audit estimated that about 30% of the public tasks are broken and retracted its earlier adoption recommendation. Use SWE-bench Pro alongside LiveCodeBench, other repository evaluations, and a workload-specific trial instead of treating one score as a procurement decision.

Compare top models on SWE-bench Pro

Claude Opus 5.5 vs Claude Sonnet 5.5Claude Sonnet 5.5 vs Claude Fable 5.1Claude Fable 5.1 vs Claude Mythos 5Claude Mythos 5 vs Claude Fable 5
Last updated: October 27, 2026 · BenchLM version SWE-bench Pro 2025

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 6,100+ subscribers.

One email each week. Unsubscribe anytime.