A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.
BenchLM mirrors the published score view for NL2Repo. Claude Opus 4.5 leads the public snapshot at 43.2% , followed by GLM-5.1 (42.7%) and MiniMax M2.7 (39.8%). BenchLM does not use these results to rank models overall.
Claude Opus 4.5
Anthropic
GLM-5.1
Z.AI
MiniMax M2.7
MiniMax
The published NL2Repo snapshot is tightly clustered at the top: Claude Opus 4.5 sits at 43.2%, while the third row is only 3.4 points behind. The broader top-10 spread is 3.4 points, so many of the published scores sit in a relatively narrow band.
3 models have been evaluated on NL2Repo. The benchmark falls in the Coding category. This category carries a 20% weight in BenchLM.ai's overall scoring system. NL2Repo is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
Year
2026
Tasks
Natural language to repository tasks
Format
Repository understanding benchmark
Difficulty
System-level software comprehension
MiniMax cites NL2Repo as a system-level engineering benchmark that rewards deep understanding of complex repositories and their operational structure.
Version
NL2Repo 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.
Claude Opus 4.5 by Anthropic currently leads with a score of 43.2% on NL2Repo.
3 AI models have been evaluated on NL2Repo on BenchLM.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.