SWE-Atlas Codebase QnA
A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.
Data verified 27 confirmed releases in the last 30 daysSee the free Radar BriefBenchmark score on SWE-Atlas Codebase QnA — September 2, 2026
We mirror the published score view for SWE-Atlas Codebase QnA. Muse Spark 1.3 leads the public snapshot at 59.4%. We do not use these results to rank models overall.
1 modelCodingCurrentDisplay onlyUpdated September 2, 2026
Benchmark score table (1 model)
ScoreAbout SWE-Atlas Codebase QnA
Year
2026
Tasks
124 codebase questions across 11 repositories
Format
Mean pass@1
Difficulty
Production codebase comprehension
The public QnA split contains 124 tasks across 11 repositories. Meta uses mini-swe-agent and task-specific rubric grading under Scale AI's leaderboard protocol; the provider-run row stays display-only.
BenchLM freshness & provenance
Version
SWE-Atlas Codebase QnA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does SWE-Atlas Codebase QnA measure?
A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.
Which model scores highest on SWE-Atlas Codebase QnA?
Muse Spark 1.3 by Meta currently leads with a score of 59.4% on SWE-Atlas Codebase QnA.
How many models are evaluated on SWE-Atlas Codebase QnA?
1 AI models have been evaluated on SWE-Atlas Codebase QnA on BenchLM.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.