Skip to main content
benchlm.ai

Vals-hosted SWE-bench mirror (Vals SWE-bench mirror)

Vals AI hosted SWE-bench view for solving production software engineering tasks.

How BenchLM shows Vals SWE-bench mirror

BenchLM mirrors the public Vals AI Vals SWE-bench mirror leaderboard captured from https://www.vals.ai/benchmarks/swebench?suggested=open-weights-table and updated by Vals on June 17, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.

Vals SWE-bench mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.

64 Vals rows5 task viewspublic datasetTasks: Overall, 1-4 hours, 15 min - 1 hour, <15 min fix, >4 hoursDisplay only
Vals SWE-bench mirror on Vals AIVals methodologyVals home

Vals SWE-bench score on Vals SWE-bench mirror — June 17, 2026

BenchLM mirrors the published vals swe-bench score view for Vals SWE-bench mirror. Claude Fable 5 leads the public snapshot at 95.00% , followed by Claude Opus 4.8 (88.60%) and Claude Opus 4.8 Claude Code (85.80%). BenchLM does not use these results to rank models overall.

1

Claude Fable 5

Anthropic

anthropic/claude-fable-5

95.00%
Overall —
2

Claude Opus 4.8

Anthropic

anthropic/claude-opus-4-8

88.60%
Overall —
3

Claude Opus 4.8 Claude Code

Anthropic

anthropic/claude-opus-4-8-claude-code

85.80%
Overall —
64 modelsExternal benchmark mirrorsCurrentDisplay onlyUpdated June 17, 2026

The published Vals SWE-bench mirror snapshot is tightly clustered at the top: Claude Fable 5 sits at 95.00%, while the third row is only 9.20 points behind. The broader top-10 spread is 16.80 points, so the benchmark still separates strong models even when the leaders cluster.

64 models have been evaluated on Vals SWE-bench mirror. The benchmark falls in the External benchmark mirrors category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. Vals SWE-bench mirror is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Vals SWE-bench mirror

Year

2026

Tasks

Software engineering issue-resolution tasks

Format

Accuracy score

Difficulty

Production software engineering

BenchLM keeps this separate from its canonical SWE-bench Verified page so Vals-hosted results remain secondary context rather than source-of-record data.

Vals SWE-benchPublic benchmark source

BenchLM freshness & provenance

Version

Vals SWE-bench mirror 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Vals SWE-bench score table (64 models)

1
Claude Fable 5anthropic/claude-fable-5
95.00%
2
Claude Opus 4.8anthropic/claude-opus-4-8
88.60%
3
Claude Opus 4.8 Claude Codeanthropic/claude-opus-4-8-claude-code
85.80%
4
GLM 5.2zai/glm-5.2
82.80%
5
GPT-5.5openai/gpt-5.5
82.60%
6
Claude Opus 4.7anthropic/claude-opus-4-7
82.00%
7
Composer 2.5cursor/composer-2.5
79.60%
8
Gemini 3.5 Flashgoogle/gemini-3.5-flash
78.80%
9
Gemini 3.1 Pro Previewgoogle/gemini-3.1-pro-preview
78.80%
10
GPT-5.4openai/gpt-5.4-2026-03-05
78.20%
11
Claude Opus 4.6 Thinkinganthropic/claude-opus-4-6-thinking
78.20%
12
Kimi K2.7 Codekimi/kimi-k2.7-code
78.20%
13
GPT-5.3 Codexopenai/gpt-5.3-codex
78.00%
14
Claude Sonnet 4.6anthropic/claude-sonnet-4-6
77.40%
15
DeepSeek V4 Prodeepseek/deepseek-v4-pro
77.40%
16
GPT-5.5 Codexopenai/gpt-5.5-codex
76.40%
17
Claude Opus 4.5 20251101 Thinkinganthropic/claude-opus-4-5-20251101-thinking
76.40%
18
Gemini 3 Pro Previewgoogle/gemini-3-pro-preview
76.40%
19
GLM 5.1zai/glm-5.1
76.40%
20
GPT-5.5 Factoryopenai/gpt-5.5-factory
76.20%
21
Kimi K2.6kimi/kimi-k2.6
76.20%
22
GPT-5.2openai/gpt-5.2-2025-12-11
75.80%
23
Gemini 3 Flash Previewgoogle/gemini-3-flash-preview
75.00%
24
MiniMax M3minimax/MiniMax-M3
75.00%
25
MiniMax M2.1minimax/MiniMax-M2.1
74.80%
26
Muse Sparkmeta/muse_spark
74.40%
27
MiniMax M2.5minimax/MiniMax-M2.5
74.20%
28
Mimo V2.5 Proxiaomi/mimo-v2.5-pro
74.00%
29
MiniMax M2.7minimax/MiniMax-M2.7
73.80%
30
Qwen3.6 Plusalibaba/qwen3.6-plus
73.40%
31
GPT-5.4 Miniopenai/gpt-5.4-mini-2026-03-17
73.00%
32
Qwen3.6 Max Previewalibaba/qwen3.6-max-preview
72.80%
33
GPT-5.2 Codexopenai/gpt-5.2-codex
72.40%
34
Grok 4.20 0309 Reasoninggrok/grok-4.20-0309-reasoning
72.20%
35
Grok 4.3grok/grok-4.3
71.40%
36
GLM 5 Thinkingzai/glm-5-thinking
71.40%
37
Qwen3.5 Plus Thinkingalibaba/qwen3.5-plus-thinking
71.20%
38
Mimo V2.5xiaomi/mimo-v2.5
71.00%
39
Kimi K2.5 Thinkingkimi/kimi-k2.5-thinking
70.00%
40
Claude Sonnet 4.5 20250929 Thinkinganthropic/claude-sonnet-4-5-20250929-thinking
70.00%
41
Qwen3.6 27balibaba/qwen3.6-27b
70.00%
42
GPT-5.4 Nanoopenai/gpt-5.4-nano-2026-03-17
69.80%
43
GPT-5.1openai/gpt-5.1-2025-11-13
69.80%
44
GLM 4.7zai/glm-4.7
69.40%
45
GPT-5openai/gpt-5-2025-08-07
69.00%
46
Nemotron 3 Ultra 550b A55bnvidia/nemotron-3-ultra-550b-a55b
69.00%
47
Qwen3.7 Maxalibaba/qwen3.7-max
68.80%
48
DeepSeek V3p2 Thinkingfireworks/deepseek-v3p2-thinking
67.60%
49
Claude Haiku 4.5 20251001 Thinkinganthropic/claude-haiku-4-5-20251001-thinking
66.60%
50
Mistral Medium 3.5mistralai/mistral-medium-3.5
66.40%
51
Laguna M.1poolside/laguna-m.1
64.60%
52
Qwen3.5 Flashalibaba/qwen3.5-flash
64.40%
53
Laguna Xs.2poolside/laguna-xs.2
64.00%
54
Gemini 3.1 Flash Lite Previewgoogle/gemini-3.1-flash-lite-preview
62.80%
55
Devstral 2512mistralai/devstral-2512
62.80%
56
GPT-5 Miniopenai/gpt-5-mini-2025-08-07
60.80%
57
Kimi K2 Thinkingkimi/kimi-k2-thinking
60.20%
58
Grok 4 0709grok/grok-4-0709
57.80%
59
Gemini 2.5 Progoogle/gemini-2.5-pro
54.40%
60
Grok 4 Fast Reasoninggrok/grok-4-fast-reasoning
45.40%
61
Grok 4.1 Fast Reasoninggrok/grok-4-1-fast-reasoning
41.40%
62
Mistral Large 2512mistralai/mistral-large-2512
41.40%
63
GPT Oss 120bfireworks/gpt-oss-120b
33.60%
64
Command A 03 2025cohere/command-a-03-2025
7.80%

FAQ

What does Vals SWE-bench mirror measure?

Vals AI hosted SWE-bench view for solving production software engineering tasks.

Which model leads the published Vals SWE-bench mirror snapshot?

Claude Fable 5 currently leads the published Vals SWE-bench mirror snapshot with 95.00% vals swe-bench score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on Vals SWE-bench mirror?

64 AI models are included in BenchLM's mirrored Vals SWE-bench mirror snapshot, based on the public leaderboard captured on June 17, 2026.

Last updated: June 17, 2026 · mirrored from the public benchmark leaderboard

The AI models change fast. We track them for you.

For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.

Free. No spam. Unsubscribe anytime.