BabaIsBench (Baba Is You intro levels, pass@1) (BabaIsBench)
We show this table for reference; we do not rank on it.
Quesma's BabaIsBench has LLM agents play the puzzle game Baba Is You through a text interface, where a level is won by pushing word tiles around to rewrite its rules. The score here is the first-attempt pass rate on the eight intro levels.
Benchmark score on BabaIsBench — November 13, 2026
We compile the BabaIsBench rows from benchmark-owner or independent runs. Claude Opus 5 leads the table at 100%, followed by GPT-6 Astra (100%) and Claude Fable 5 (100%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
Run by BabaIsBenchGPT-6 Astra
OpenAI
Run by BabaIsBenchClaude Fable 5
Anthropic
Run by BabaIsBench32 modelsAgenticCurrentDisplay onlyUpdated November 13, 2026
Benchmark score table (32 models)
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Opus 5AnthropicRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 2 | GPT-6 AstraOpenAIRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 3 | Claude Fable 5AnthropicRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 4 | Claude Fable 5.1AnthropicRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 5 | GPT-5.6 SolOpenAIRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 6 | GPT-5.5OpenAIRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 7 | Gemini 3.8 FlashGoogleRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 8 | Gemini 3.7 FlashGoogleRun by BabaIsBench | 100% | Run by BabaIsBench | Not reported | Closed |
| 9 | Kimi K3Moonshot AIRun by BabaIsBench | 96% | Run by BabaIsBench | Not reported | Open |
| 10 | Claude Opus 4.8AnthropicRun by BabaIsBench | 96% | Run by BabaIsBench | Not reported | Closed |
| 11 | GLM-5.2Z.AIRun by BabaIsBench | 96% | Run by BabaIsBench | Not reported | Open |
| 12 | Grok 4.6xAIRun by BabaIsBench | 96% | Run by BabaIsBench | Not reported | Closed |
| 13 | GLM-5.3Z.AIRun by BabaIsBench | 96% | Run by BabaIsBench | Not reported | Open |
| 14 | Gemini 3.1 ProGoogleRun by BabaIsBench | 92% | Run by BabaIsBench | Not reported | Closed |
| 15 | GPT-5.6 TerraOpenAIRun by BabaIsBench | 88% | Run by BabaIsBench | Not reported | Closed |
| 16 | Gemini 3.5 FlashGoogleRun by BabaIsBench | 88% | Run by BabaIsBench | Not reported | Closed |
| 17 | Gemini 3.6 FlashGoogleRun by BabaIsBench | 88% | Run by BabaIsBench | Not reported | Closed |
| 18 | DeepSeek V4 Pro 0813DeepSeekRun by BabaIsBench | 75% | Run by BabaIsBench | Not reported | Open |
| 19 | Grok 4.5xAIRun by BabaIsBench | 75% | Run by BabaIsBench | Not reported | Closed |
| 20 | DeepSeek V4 Flash 0731DeepSeekRun by BabaIsBench | 75% | Run by BabaIsBench | Not reported | Open |
| 21 | DeepSeek V4 FlashDeepSeekRun by BabaIsBench | 75% | Run by BabaIsBench | Not reported | Open |
| 22 | Claude Sonnet 5AnthropicRun by BabaIsBench | 67% | Run by BabaIsBench | Not reported | Closed |
| 23 | GLM-5.3-FlashZ.AIRun by BabaIsBench | 67% | Run by BabaIsBench | Not reported | Open |
| 24 | GPT-5.6 LunaOpenAIRun by BabaIsBench | 63% | Run by BabaIsBench | Not reported | Closed |
| 25 | Qwen3.8 MaxAlibabaRun by BabaIsBench | 58% | Run by BabaIsBench | Not reported | Open |
| 26 | Hy3TencentRun by BabaIsBench | 54% | Run by BabaIsBench | Not reported | Open |
| 27 | MiniMax M3MiniMaxRun by BabaIsBench | 46% | Run by BabaIsBench | Not reported | Open |
| 28 | Muse Spark 1.3MetaRun by BabaIsBench | 46% | Run by BabaIsBench | Not reported | Closed |
| 29 | Qwen3.6-27BAlibabaRun by BabaIsBench | 29% | Run by BabaIsBench | Not reported | Open |
| 30 | DeepSeek V4 ProDeepSeekRun by BabaIsBench | 29% | Run by BabaIsBench | Not reported | Open |
| 31 | Qwen3.7 MaxAlibabaRun by BabaIsBench | 25% | Run by BabaIsBench | Not reported | Closed |
| 32 | Qwen3.8-27BAlibabaRun by BabaIsBench | 21% | Run by BabaIsBench | Not reported | Open |
- Lab-reported
- Benchmark-run
- Independent
How to read this leaderboard
Higher is better. Eight models solve every intro attempt, so the top of this score is saturated: it separates models that can play the game at all from those that can't, and says little about the leaders. With 24 attempts, one attempt is worth about four points. The Lake spreads the top out more (GPT-6 Astra and Claude Fable 5.1 solve all 15 levels, most others 6 to 14), and cost differs widely at the same pass rate: GPT-6 Astra's 24 intro attempts cost $5.33 and Claude Fable 5's cost $40.98.
Operator receipt: 32 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 100%.
Honest limit: Stage 0 is small (eight levels) and uses one harness. Stage 1 runs each model once, in different harnesses, so its results mix the model with its tooling and are not a like-for-like comparison. Baba Is You is a published game with public walkthroughs, so some solutions may be in training data even though prompts hide the game's name. We show it as display-only; it does not enter overall or category rankings.
Among the reported BabaIsBench rows, Claude Opus 5 is first at 100%. The third row is 0 points behind. The broader top-10 range is 4 points, so many of the published results sit in a relatively narrow band.
32 models have been evaluated on BabaIsBench. The benchmark falls in the Agentic category. We keep these results separate from the weighted global scoring system, so they remain source-specific evidence. BabaIsBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About BabaIsBench
Year
2026
Tasks
8 intro levels of Baba Is You, three attempts per level (Stage 0); 15 further levels run once (Stage 1)
Format
Agent plays through a text interface in the Harbor framework
Difficulty
Rule-rewriting puzzles that need planning over many moves
Levels and game logic run inside the Harbor framework, and the agent sees the board as text. Task prompts never name the game or the level. Stage 0, The Intro, has eight levels; every model gets three attempts per level in the Terminus-2 harness through OpenRouter, with a 60-minute limit per attempt. pass@1 is the share of those 24 attempts that solve the level, and pass@3 is the share of levels solved at least once. Stage 1, The Lake, adds 15 harder levels (two of them bonus levels), run once per model with a harness chosen for that model (Claude Code, Codex, kimi-cli or Terminus-2) and effectively no time limit. The leaderboard also lists turns, output tokens and total cost. We store Stage 0 pass@1 for all 32 models and keep each model's Lake result in its row note.
Freshness and provenance
Version
BabaIsBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does BabaIsBench measure?
Whether an LLM agent can win levels of Baba Is You, a puzzle game where you change the rules by moving word tiles. The agent plays through a text interface and is never told which game it is. The stored score is the first-attempt pass rate on eight intro levels.
Which model leads BabaIsBench?
Eight models pass all 24 intro attempts, including GPT-6 Astra, Claude Fable 5.1, Gemini 3.8 Flash and GPT-5.6 Sol. On the harder Lake stage, GPT-6 Astra and Claude Fable 5.1 solve all 15 levels; Astra does it in 50 minutes for $10.05, Fable 5.1 in two hours for $28.29.
Why do some models have a Lake result and others don't?
Quesma ran Stage 1, The Lake, for 19 of the 32 models, once each, with a harness picked per model such as Claude Code, Codex or Terminus-2. Because the harness changes from model to model, we keep the Lake result in each model's row note and store only the intro pass rate.
Compare top models on BabaIsBench
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.