# SWE-Atlas Codebase QnA

> A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.

Canonical page: https://benchlm.ai/benchmarks/swe-atlas-codebase-qna

- Category: [Coding](/coding)
- Last updated: October 17, 2026

## About SWE-Atlas Codebase QnA

- Year: 2026
- Tasks: 124 codebase questions across 11 repositories
- Format: Mean pass@1
- Difficulty: Production codebase comprehension
- Paper: [Muse Spark 1.3 Evaluation Methodology](https://research.meta.ai/static/muse-spark-1-3-multimodal-evaluation-methodology)

The public QnA split contains 124 tasks across 11 repositories. Meta uses mini-swe-agent and task-specific rubric grading under Scale AI's leaderboard protocol; the provider-run row stays display-only.

SWE-Atlas Codebase QnA is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (6 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Ling 3.1 Flash](/models/ling-3-1-flash) | InclusionAI | 64.5% |
| 2 | [Muse Spark 1.3](/models/muse-spark-1-3) | Meta | 59.4% |
| 3 | [Muse Spark Open](/models/muse-spark-open) | Meta | 58.0% |
| 4 | [Beam 1.0 501B-A23B FP8](/models/reflection-beam-fp8) | Reflection AI | 32.2% |
| 5 | [Beam 1.0 501B-A23B](/models/reflection-beam) | Reflection AI | 31.8% |
| 6 | [Beam 1.0 501B-A23B NVFP4](/models/reflection-beam-nvfp4) | Reflection AI | 31.4% |

## FAQ

### What does SWE-Atlas Codebase QnA measure?

A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.

### Which model scores highest on SWE-Atlas Codebase QnA?

Ling 3.1 Flash by InclusionAI currently leads with a score of 64.5% on SWE-Atlas Codebase QnA.

### How many models are evaluated on SWE-Atlas Codebase QnA?

6 AI models have published results on SWE-Atlas Codebase QnA in the BenchLM catalog.

### Does SWE-Atlas Codebase QnA affect BenchLM's overall score?

Not directly. SWE-Atlas Codebase QnA is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Compare Top Models on SWE-Atlas Codebase QnA

- [Ling 3.1 Flash vs Muse Spark 1.3](/compare/ling-3-1-flash-vs-muse-spark-1-3)
- [Muse Spark 1.3 vs Muse Spark Open](/compare/muse-spark-1-3-vs-muse-spark-open)
- [Muse Spark Open vs Beam 1.0 501B-A23B FP8](/compare/muse-spark-open-vs-reflection-beam-fp8)
- [Beam 1.0 501B-A23B FP8 vs Beam 1.0 501B-A23B](/compare/reflection-beam-vs-reflection-beam-fp8)