Benchmark profile
DeepSWE
A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
Data verifiedHow BenchLM shows DeepSWE
BenchLM mirrors the public DeepSWE leaderboard JSON from Datacurve. The snapshot shows the best available mini-swe-agent configuration per model, while preserving 50 underlying effort-level rows in the source metadata.
DeepSWE evaluates coding agents on 113 original, long-horizon software engineering tasks across 91 repositories and 5 languages, using isolated task environments and program-based verifiers.
DeepSWE is display only on BenchLM. The public rows combine a model, mini-swe-agent harness, and reasoning-effort setting, so BenchLM does not use these scores as weighted model-only ranking inputs.
Pass@1 on DeepSWE — July 25, 2026
BenchLM mirrors the published pass@1 view for DeepSWE. claude-opus-5[max] leads the public snapshot at 73.6% , followed by gpt-5-6-sol[max] (72.7%) and claude-fable-5[max] (69.7%). BenchLM does not use these results to rank models overall.
claude-opus-5[max]
Anthropic
mini_swe_agent_claude_opus_5_max
gpt-5-6-sol[max]
OpenAI
mini_swe_agent_gpt_5_6_sol_max
claude-fable-5[max]
Anthropic
mini_swe_agent_claude_fable_5_max
Pass@1 table (18 models)
ScoreThe published DeepSWE snapshot places claude-opus-5[max] first at 73.6%. The third row is 3.9 points behind. The broader top-10 range is 19.8 points, so the table still separates the published systems.
18 models have been evaluated on DeepSWE. The benchmark falls in the External benchmark mirrors category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. DeepSWE is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About DeepSWE
Year
2026
Tasks
113 software engineering tasks across 91 repositories and 5 languages
Format
Pass@1 with confidence interval, cost, time, and token metadata
Difficulty
Long-horizon software engineering
DeepSWE includes original tasks with isolated environments and program-based verifiers. BenchLM mirrors the public DeepSWE leaderboard JSON as display-only, using the best available mini-swe-agent configuration per model and preserving cost, time, token, and effort-level source metadata. Each row combines a model, agent harness, and reasoning-effort setting rather than a pure model-only benchmark score.
BenchLM freshness & provenance
Version
DeepSWE 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does DeepSWE measure?
A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
Which model leads the published DeepSWE snapshot?
claude-opus-5[max] currently leads the published DeepSWE snapshot with 73.6% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on DeepSWE?
18 AI models are included in BenchLM's mirrored DeepSWE snapshot, based on the public leaderboard captured on July 25, 2026.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.