Benchmark profile
DeepSWE
A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
Data verifiedHow BenchLM shows DeepSWE
BenchLM mirrors the public DeepSWE leaderboard JSON from Datacurve. The snapshot shows the best available mini-swe-agent configuration per model, while preserving 41 underlying effort-level rows in the source metadata.
DeepSWE evaluates coding agents on 113 original, long-horizon software engineering tasks across 91 repositories and 5 languages, using isolated task environments and program-based verifiers.
DeepSWE is display only on BenchLM. The public rows combine a model, mini-swe-agent harness, and reasoning-effort setting, so BenchLM does not use these scores as weighted model-only ranking inputs.
Pass@1 on DeepSWE — July 7, 2026
BenchLM mirrors the published pass@1 view for DeepSWE. gpt-5-6-sol[max] leads the public snapshot at 72.7% , followed by claude-fable-5[max] (69.7%) and gpt-5-6-terra[max] (69.6%). BenchLM does not use these results to rank models overall.
gpt-5-6-sol[max]
OpenAI
mini_swe_agent_gpt_5_6_sol_max
claude-fable-5[max]
Anthropic
mini_swe_agent_claude_fable_5_max
gpt-5-6-terra[max]
OpenAI
mini_swe_agent_gpt_5_6_terra_max
Pass@1 table (13 models)
ScoreThe published DeepSWE snapshot is tightly clustered at the top: gpt-5-6-sol[max] sits at 72.7%, while the third row is only 3.1 points behind. The broader top-10 spread is 35.3 points, so the benchmark still separates strong models even when the leaders cluster.
13 models have been evaluated on DeepSWE. The benchmark falls in the External benchmark mirrors category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. DeepSWE is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About DeepSWE
Year
2026
Tasks
113 software engineering tasks across 91 repositories and 5 languages
Format
Pass@1 with confidence interval, cost, time, and token metadata
Difficulty
Long-horizon software engineering
DeepSWE includes original tasks with isolated environments and program-based verifiers. BenchLM mirrors the public DeepSWE leaderboard JSON as display-only, using the best available mini-swe-agent configuration per model and preserving cost, time, token, and effort-level source metadata. Each row combines a model, agent harness, and reasoning-effort setting rather than a pure model-only benchmark score.
BenchLM freshness & provenance
Version
DeepSWE 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does DeepSWE measure?
A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
Which model leads the published DeepSWE snapshot?
gpt-5-6-sol[max] currently leads the published DeepSWE snapshot with 72.7% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on DeepSWE?
13 AI models are included in BenchLM's mirrored DeepSWE snapshot, based on the public leaderboard captured on July 7, 2026.
The AI models change fast. We track them for you.
A weekly brief for engineers and researchers covering new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.