SWE-Atlas Codebase QnA
We show this table for reference; we do not rank on it.
A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.
Benchmark score on SWE-Atlas Codebase QnA — October 17, 2026
We compile the SWE-Atlas Codebase QnA rows from provider self-reports. Ling 3.1 Flash leads the table at 64.5%, followed by Muse Spark 1.3 (59.4%) and Muse Spark Open (58.0%). We do not use these results to rank models overall.
Ling 3.1 Flash
InclusionAI
Muse Spark 1.3
Meta
Muse Spark Open
Meta
6 modelsCodingCurrentDisplay onlyUpdated October 17, 2026
Benchmark score table (6 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Ling 3.1 FlashInclusionAI | 64.5% | Not reported | Open |
| 2 | Muse Spark 1.3Meta | 59.4% | Not reported | Closed |
| 3 | Muse Spark OpenMeta | 58.0% | Not reported | Open |
| 4 | Beam 1.0 501B-A23B FP8Reflection AI | 32.2% | Not reported | Open |
| 5 | Beam 1.0 501B-A23BReflection AI | 31.8% | Not reported | Open |
| 6 | Beam 1.0 501B-A23B NVFP4Reflection AI | 31.4% | Not reported | Open |
Among the reported SWE-Atlas Codebase QnA rows, Ling 3.1 Flash is first at 64.5%. The third row is 6.5 points behind. The broader top-10 range is 33.1 points, so the table still separates the published systems.
6 models have been evaluated on SWE-Atlas Codebase QnA. The benchmark falls in the Coding category. SWE-Atlas Codebase QnA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About SWE-Atlas Codebase QnA
Year
2026
Tasks
124 codebase questions across 11 repositories
Format
Mean pass@1
Difficulty
Production codebase comprehension
The public QnA split contains 124 tasks across 11 repositories. Meta uses mini-swe-agent and task-specific rubric grading under Scale AI's leaderboard protocol; the provider-run row stays display-only.
Freshness and provenance
Version
SWE-Atlas Codebase QnA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-Atlas Codebase QnA measure?
A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.
Which model scores highest on SWE-Atlas Codebase QnA?
Ling 3.1 Flash by InclusionAI currently leads with a score of 64.5% on SWE-Atlas Codebase QnA.
How many models are evaluated on SWE-Atlas Codebase QnA?
6 AI models have published results on SWE-Atlas Codebase QnA in the BenchLM catalog.
Compare top models on SWE-Atlas Codebase QnA
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.