# BabaIsBench (Baba Is You intro levels, pass@1) (BabaIsBench)

> Quesma's BabaIsBench has LLM agents play the puzzle game Baba Is You through a text interface, where a level is won by pushing word tiles around to rewrite its rules. The score here is the first-attempt pass rate on the eight intro levels.

Canonical page: https://benchlm.ai/benchmarks/babaisbench

- Category: [Agentic](/agentic)
- Last updated: November 13, 2026

## About BabaIsBench

- Year: 2026
- Tasks: 8 intro levels of Baba Is You, three attempts per level (Stage 0); 15 further levels run once (Stage 1)
- Format: Agent plays through a text interface in the Harbor framework
- Difficulty: Rule-rewriting puzzles that need planning over many moves
- Paper: [Baba Is Solved by Fable 5 and GPT-5.6 Sol, but at what cost? (Quesma)](https://quesma.com/blog/baba-is-bench/)

Levels and game logic run inside the Harbor framework, and the agent sees the board as text. Task prompts never name the game or the level. Stage 0, The Intro, has eight levels; every model gets three attempts per level in the Terminus-2 harness through OpenRouter, with a 60-minute limit per attempt. pass@1 is the share of those 24 attempts that solve the level, and pass@3 is the share of levels solved at least once. Stage 1, The Lake, adds 15 harder levels (two of them bonus levels), run once per model with a harness chosen for that model (Claude Code, Codex, kimi-cli or Terminus-2) and effectively no time limit. The leaderboard also lists turns, output tokens and total cost. We store Stage 0 pass@1 for all 32 models and keep each model's Lake result in its row note.

BabaIsBench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (30 models)

| Rank | Model | Creator | Score | Who ran it |
|------|-------|---------|-------|-----------|
| 1 | [GPT-6 Astra](/models/gpt-6-astra) | OpenAI | 100% | Run by BabaIsBench |
| 2 | [Claude Fable 5.1](/models/claude-fable-5-1) | Anthropic | 100% | Run by BabaIsBench |
| 3 | [Claude Fable 5](/models/claude-fable) | Anthropic | 100% | Run by BabaIsBench |
| 4 | [Claude Opus 5](/models/claude-opus-5) | Anthropic | 100% | Run by BabaIsBench |
| 5 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | OpenAI | 100% | Run by BabaIsBench |
| 6 | [GPT-5.5](/models/gpt-5-5) | OpenAI | 100% | Run by BabaIsBench |
| 7 | [Gemini 3.7 Flash](/models/gemini-3-7-flash) | Google | 100% | Run by BabaIsBench |
| 8 | [Gemini 3.8 Flash](/models/gemini-3-8-flash) | Google | 100% | Run by BabaIsBench |
| 9 | [Kimi K3](/models/kimi-k3) | Moonshot AI | 96% | Run by BabaIsBench |
| 10 | [Claude Opus 4.8](/models/claude-opus-4-8) | Anthropic | 96% | Run by BabaIsBench |
| 11 | [Grok 4.6](/models/grok-4-6) | xAI | 96% | Run by BabaIsBench |
| 12 | [GLM-5.3](/models/glm-5-3) | Z.AI | 96% | Run by BabaIsBench |
| 13 | [GLM-5.2](/models/glm-5-2) | Z.AI | 96% | Run by BabaIsBench |
| 14 | [Gemini 3.1 Pro](/models/gemini-3-1-pro) | Google | 92% | Run by BabaIsBench |
| 15 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | OpenAI | 88% | Run by BabaIsBench |
| 16 | [Gemini 3.6 Flash](/models/gemini-3-6-flash) | Google | 88% | Run by BabaIsBench |
| 17 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Google | 88% | Run by BabaIsBench |
| 18 | [Grok 4.5](/models/grok-4-5) | xAI | 75% | Run by BabaIsBench |
| 19 | [DeepSeek V4 Pro 0813](/models/deepseek-v4-pro-0813) | DeepSeek | 75% | Run by BabaIsBench |
| 20 | [DeepSeek V4 Flash 0731](/models/deepseek-v4-flash-0731) | DeepSeek | 75% | Run by BabaIsBench |
| 21 | [Claude Sonnet 5](/models/claude-sonnet-5) | Anthropic | 67% | Run by BabaIsBench |
| 22 | [GLM-5.3-Flash](/models/glm-5-3-flash) | Z.AI | 67% | Run by BabaIsBench |
| 23 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | OpenAI | 63% | Run by BabaIsBench |
| 24 | [Qwen3.8 Max](/models/qwen3-8-max) | Alibaba | 58% | Run by BabaIsBench |
| 25 | [Hy3](/models/hy3) | Tencent | 54% | Run by BabaIsBench |
| 26 | [MiniMax M3](/models/minimax-m3) | MiniMax | 46% | Run by BabaIsBench |
| 27 | [Muse Spark 1.3](/models/muse-spark-1-3) | Meta | 46% | Run by BabaIsBench |
| 28 | [Qwen3.6-27B](/models/qwen3-6-27b) | Alibaba | 29% | Run by BabaIsBench |
| 29 | [Qwen3.7 Max](/models/qwen3-7-max) | Alibaba | 25% | Run by BabaIsBench |
| 30 | [Qwen3.8-27B](/models/qwen3-8-27b) | Alibaba | 21% | Run by BabaIsBench |

## FAQ

### What does BabaIsBench measure?

Whether an LLM agent can win levels of Baba Is You, a puzzle game where you change the rules by moving word tiles. The agent plays through a text interface and is never told which game it is. The stored score is the first-attempt pass rate on eight intro levels.

### Which model leads BabaIsBench?

Eight models pass all 24 intro attempts, including GPT-6 Astra, Claude Fable 5.1, Gemini 3.8 Flash and GPT-5.6 Sol. On the harder Lake stage, GPT-6 Astra and Claude Fable 5.1 solve all 15 levels; Astra does it in 50 minutes for $10.05, Fable 5.1 in two hours for $28.29.

### Why do some models have a Lake result and others don't?

Quesma ran Stage 1, The Lake, for 19 of the 32 models, once each, with a harness picked per model such as Claude Code, Codex or Terminus-2. Because the harness changes from model to model, we keep the Lake result in each model's row note and store only the intro pass rate.

## Compare Top Models on BabaIsBench

- [GPT-6 Astra vs Claude Fable 5.1](/compare/claude-fable-5-1-vs-gpt-6-astra)
- [Claude Fable 5.1 vs Claude Fable 5](/compare/claude-fable-vs-claude-fable-5-1)
- [Claude Fable 5 vs Claude Opus 5](/compare/claude-fable-vs-claude-opus-5)
- [Claude Opus 5 vs GPT-5.6 Sol](/compare/claude-opus-5-vs-gpt-5-6-sol)