Skip to main content
Radar

Five or fewer confirmed AI changes, with original sources, on mornings when something changed.A free source-linked morning brief.

Start free brief
BenchLM

SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration? (SWE Refactor Bench)

Tests whether coding agents can complete long-horizon, whole-repository stack migrations while preserving the original program's behavior.

How to read this leaderboard

A nonzero composite means the run completed the requested migration and passed every frozen behavioral check. The score then rises with the number of six independent coding-agent verifiers that failed to find a behavioral difference.

Operator receipt: 26 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 47.0.

Honest limit: Each result belongs to a model, reasoning-effort level, agent client, and task budget. Treat the rows as complete-system configurations, not controlled measurements of model weights alone.

Official leaderboardTask catalogGitHub repository

How we show SWE Refactor Bench

We mirror the benchmark-owned SWE Refactor Bench configuration table from August 25, 2026 snapshot. Every one of the 26 configurations ran all 20 whole-repository migrations, for 520 graded runs in total.

The composite awards credit only after the submitted repository passes a migration audit and every frozen behavioral check. Six coding agents then search for executable counterexamples. We keep each reasoning-effort and client configuration separate, and the table stays display only because those choices are part of the result.

Snapshot

8 models26 configurations20 migrations520 graded runs28 acceptedDisplay only
SWE Refactor Bench leaderboardProject and methodologyTask catalogResearch paperGitHub repository

Composite score on SWE Refactor Bench — August 25, 2026 snapshot

We mirror the published composite score view for SWE Refactor Bench. Claude Opus 5 leads the public snapshot at 47.0, followed by Claude Opus 5 (34.5) and Claude Opus 5 (31.0). We do not use these results to rank models overall.

1Closed

Claude Opus 5

Anthropic

Claude Code · xhigh reasoning

$74.9 per task · 5 accepted · 6 broken · 1 blind

47.0
Overall 82.8
2Closed

Claude Opus 5

Anthropic

Claude Code · high reasoning

$55.7 per task · 4 accepted · 4 broken · 2 blind

34.5
Overall 82.8
3Closed

Claude Opus 5

Anthropic

Claude Code · max reasoning

$72.4 per task · 3 accepted · 4 broken · 2 blind

31.0
Overall 82.8
26 configurationsAgenticCurrentDisplay onlyUpdated August 25, 2026 snapshot

Composite score table (26 configurations)

Score
1
Claude Opus 5Anthropic · ClosedClaude Code · xhigh reasoning$74.9 per task · 5 accepted · 6 broken · 1 blind
47.0
2
Claude Opus 5Anthropic · ClosedClaude Code · high reasoning$55.7 per task · 4 accepted · 4 broken · 2 blind
34.5
3
Claude Opus 5Anthropic · ClosedClaude Code · max reasoning$72.4 per task · 3 accepted · 4 broken · 2 blind
31.0
4
GPT-5.6 SolOpenAI · ClosedCodex CLI · max reasoning$143.5 per task · 4 accepted · 3 broken · 0 blind
28.5
5
Claude Opus 5Anthropic · ClosedClaude Code · medium reasoning$38.4 per task · 2 accepted · 5 broken · 2 blind
28.5
6
Claude Opus 5Anthropic · ClosedClaude Code · low reasoning$34.8 per task · 2 accepted · 3 broken · 0 blind
20.5
7
Kimi K3Moonshot AI · ClosedClaude Code · max reasoning$28.9 per task · 2 accepted · 3 broken · 0 blind
19.5
8
GPT-5.6 SolOpenAI · ClosedCodex CLI · high reasoning$7.7 per task · 1 accepted · 4 broken · 0 blind
19.0
9
Claude Sonnet 5Anthropic · ClosedClaude Code · medium reasoning$11.9 per task · 1 accepted · 3 broken · 0 blind
15.0
10
GPT-5.6 LunaOpenAI · ClosedCodex CLI · max reasoning$2.8 per task · 0 accepted · 3 broken · 1 blind
10.5
11
Qwen3.8 MaxAlibaba · Open weightClaude Code · max reasoning$14.5 per task · 2 accepted · 0 broken · 4 blind
10.0
12
GPT-5.6 SolOpenAI · ClosedCodex CLI · xhigh reasoning$19.1 per task · 0 accepted · 3 broken · 1 blind
9.5
13
Claude Sonnet 5Anthropic · ClosedClaude Code · xhigh reasoning$27.0 per task · 1 accepted · 1 broken · 1 blind
9.0
14
Claude Sonnet 5Anthropic · ClosedClaude Code · max reasoning$27.5 per task · 1 accepted · 1 broken · 1 blind
8.5
15
DeepSeek V4 Flash 0731DeepSeek · ClosedClaude Code · max reasoning$4.3 per task · 0 accepted · 2 broken · 1 blind
7.0
16
GPT-5.6 SolOpenAI · ClosedCodex CLI · low reasoning$5.9 per task · 0 accepted · 2 broken · 0 blind
7.0
17
GPT-5.6 SolOpenAI · ClosedCodex CLI · medium reasoning$6.0 per task · 0 accepted · 2 broken · 3 blind
6.5
18
GLM-5.2Z.AI · Open weightClaude Code · max reasoning$17.5 per task · 0 accepted · 2 broken · 1 blind
6.5
19
Claude Sonnet 5Anthropic · ClosedClaude Code · high reasoning$24.6 per task · 0 accepted · 2 broken · 2 blind
6.0
20
GPT-5.6 LunaOpenAI · ClosedCodex CLI · xhigh reasoning$2.9 per task · 0 accepted · 2 broken · 2 blind
5.5
21
GPT-5.6 LunaOpenAI · ClosedCodex CLI · none reasoning$1.6 per task · 0 accepted · 1 broken · 1 blind
4.0
22
GPT-5.6 LunaOpenAI · ClosedCodex CLI · low reasoning$1.7 per task · 0 accepted · 1 broken · 1 blind
4.0
23
GPT-5.6 LunaOpenAI · ClosedCodex CLI · high reasoning$1.8 per task · 0 accepted · 1 broken · 2 blind
4.0
24
GPT-5.6 SolOpenAI · ClosedCodex CLI · none reasoning$2.9 per task · 0 accepted · 1 broken · 0 blind
4.0
25
Claude Sonnet 5Anthropic · ClosedClaude Code · low reasoning$4.4 per task · 0 accepted · 1 broken · 1 blind
4.0
26
GPT-5.6 LunaOpenAI · ClosedCodex CLI · medium reasoning$1.7 per task · 0 accepted · 0 broken · 1 blind
0.0

The published SWE Refactor Bench snapshot places Claude Opus 5 first at 47.0. The third row is 16.0 score units behind. The broader top-10 range is 36.5 score units, so the table still separates the published systems.

26 configurations have been evaluated on SWE Refactor Bench. The benchmark falls in the Agentic category. This category carries a 22% weight in BenchLM.ai's overall scoring system. SWE Refactor Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About SWE Refactor Bench

Year

2026

Tasks

20 whole-repository stack migrations

Format

Migration audit, frozen behavioral checks, and agentic verification

Difficulty

6- to 30-hour autonomous repository migrations

SWE Refactor Bench contains 20 repository-wide migrations across languages, frameworks, platforms, and build systems. A run receives credit only after the migration audit confirms that the old stack is gone, every frozen behavioral check passes, and six coding agents fail to find an executable counterexample. We mirror all 26 model, client, and reasoning-effort configurations as display-only evidence.

SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?Public benchmark source

BenchLM freshness & provenance

Version

SWE Refactor Bench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

20 public tasks with released score table and trajectories

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does SWE Refactor Bench measure?

Tests whether coding agents can complete long-horizon, whole-repository stack migrations while preserving the original program's behavior.

Which model leads the published SWE Refactor Bench snapshot?

Claude Opus 5 currently leads the published SWE Refactor Bench snapshot with 47.0 composite score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on SWE Refactor Bench?

The August 25, 2026 snapshot contains 26 configurations across 8 AI models.

Last updated: August 25, 2026 snapshot · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.