MazeBench: visual spatial reasoning in a 3D open world (MazeBench)
We show this table for reference; we do not rank on it.
Tests whether a coding agent can explore a 256-room 3D puzzle world and collect its 100 hidden gems, using box-pushing puzzles, camera rotation, and long-horizon planning.
Gems collected on MazeBench — October 15, 2026 snapshot
We mirror the published gems collected view for MazeBench. GPT-6 Astra leads the public snapshot at 23%, followed by GPT-6.1 Sol (17%) and GPT-6 Astra (16%). We do not use these results to rank models overall.
GPT-6 Astra
OpenAI
Codex · max reasoning
Tools · 113 rooms · Paused snapshot
GPT-6.1 Sol
OpenAI
Codex · max reasoning
Tools · 102 rooms · Paused snapshot
GPT-6 Astra
OpenAI
Codex · max reasoning
No tools · 104 rooms · Paused snapshot
62 published runsAgenticCurrentDisplay onlyUpdated October 15, 2026 snapshot
Gems collected table (62 published runs)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAICodex · max reasoningTools · 113 rooms · Paused snapshot | 23% | Not reported | Closed |
| 2 | GPT-6.1 SolOpenAICodex · max reasoningTools · 102 rooms · Paused snapshot | 17% | Not reported | Closed |
| 3 | GPT-6 AstraOpenAICodex · max reasoningNo tools · 104 rooms · Paused snapshot | 16% | Not reported | Closed |
| 4 | Claude Opus 5.5AnthropicClaude Code · max reasoningTools · 68 rooms · Stopped snapshot | 13% | Not reported | Closed |
| 5 | GPT-5.6 SolOpenAICodex · max reasoningTools · 67 rooms | 13% | Not reported | Closed |
| 6 | Gemini 4 ArgonGoogleAntigravity · high reasoningNo tools · 82 rooms · In-progress snapshot | 12% | Not reported | Closed |
| 7 | Claude Opus 5AnthropicClaude Code · max reasoningTools · 62 rooms | 12% | Not reported | Closed |
| 8 | Claude Fable 5AnthropicClaude Code · max reasoningTools · 43 rooms | 11% | Not reported | Closed |
| 9 | Claude Fable 5.1AnthropicClaude Code · max reasoningTools · 33 rooms | 10% | Not reported | Closed |
| 10 | GPT-6.1 SolOpenAICodex · max reasoningNo tools · 66 rooms · Paused snapshot | 9% | Not reported | Closed |
| 11 | Claude Opus 5.5AnthropicClaude Code · max reasoningNo tools · 50 rooms · In-progress snapshot | 9% | Not reported | Closed |
| 12 | Claude Sonnet 5.5AnthropicClaude Code · max reasoningTools · 49 rooms · Stopped snapshot | 8% | Not reported | Closed |
| 13 | GPT-6 SolOpenAICodex · max reasoningTools · 38 rooms · Paused snapshot | 7% | Not reported | Closed |
| 14 | GPT-5.6 TerraOpenAICodex · max reasoningTools · 12 rooms | 6% | Not reported | Closed |
| 15 | Gemini 3.8 FlashGoogleGoogle Antigravity · high reasoningTools · 12 rooms | 4% | Not reported | Closed |
| 16 | Claude Fable 5.1AnthropicClaude Code · max reasoningNo tools · 12 rooms · Paused snapshot | 2% | Not reported | Closed |
| 17 | Claude Fable 5AnthropicClaude Code · max reasoningNo tools · 10 rooms | 1% | Not reported | Closed |
| 18 | Muse Wonder FullMetaMuse Code · max reasoningNo tools · 8 rooms · In-progress snapshot | 1% | Not reported | Closed |
| 19 | Claude Sonnet 5.5AnthropicClaude Code · max reasoningNo tools · 8 rooms · Stopped snapshot | 1% | Not reported | Closed |
| 20 | Claude Opus 5AnthropicClaude Code · max reasoningNo tools · 8 rooms | 1% | Not reported | Closed |
| 21 | Claude Haiku 5.5AnthropicClaude Code · max reasoningTools · 7 rooms · In-progress snapshot | 1% | Not reported | Closed |
| 22 | GPT-6 SolOpenAICodex · max reasoningNo tools · 7 rooms · Paused snapshot | 1% | Not reported | Closed |
| 23 | Grok 4.7xAIGrok Build · xhigh reasoningNo tools · 7 rooms · Stopped snapshot | 1% | Not reported | Closed |
| 24 | Grok 4.6xAIGrok Build · xhigh reasoningTools · 6 rooms | 1% | Not reported | Closed |
| 25 | Claude Haiku 5.5AnthropicClaude Code · max reasoningNo tools · 5 rooms · Stopped snapshot | 1% | Not reported | Closed |
| 26 | Gemini 3.1 ProGoogleAntigravity · high reasoningTools · 5 rooms | 1% | Not reported | Closed |
| 27 | Gemini 3.7 FlashGoogleAntigravity · high reasoningTools · 4 rooms | 1% | Not reported | Closed |
| 28 | GPT-5.6 SolOpenAICodex · max reasoningNo tools · 4 rooms | 1% | Not reported | Closed |
| 29 | Kimi K3Moonshot AIKimi Code · max reasoningTools · 4 rooms | 1% | Not reported | Pending |
| 30 | GPT-5.5OpenAICodex · xhigh reasoningNo tools · 4 rooms | 0% | Not reported | Closed |
| 31 | Claude Opus 4.8AnthropicClaude Code · max reasoningNo tools · 4 rooms | 0% | Not reported | Closed |
| 32 | Muse WonderMetaMuse Code · max reasoningNo tools · 4 rooms · Stopped snapshot | 0% | Not reported | Closed |
| 33 | Muse Spark 1.3MetaMuse Code · max reasoningNo tools · 3 rooms · Paused snapshot | 0% | Not reported | Closed |
| 34 | GPT-6 LunaOpenAICodex · max reasoningNo tools · 2 rooms · Stopped snapshot | 0% | Not reported | Closed |
| 35 | Kimi K3Moonshot AIKimi Code · high reasoningNo tools · 2 rooms | 0% | Not reported | Pending |
| 36 | GPT-5.6 LunaOpenAICodex · max reasoningTools · 2 rooms | 0% | Not reported | Closed |
| 37 | GPT-5.6 TerraOpenAICodex · max reasoningNo tools · 2 rooms | 0% | Not reported | Closed |
| 38 | GLM-5.3-FlashZ.AIClaude Code · max reasoningTools · 2 rooms | 0% | Not reported | Open |
| 39 | Claude Sonnet 4.6AnthropicClaude Code · high reasoningNo tools · 2 rooms | 0% | Not reported | Closed |
| 40 | Gemini 3.6 FlashGoogleAntigravity · high reasoningTools · 2 rooms | 0% | Not reported | Closed |
| 41 | deepseek-v4-pro · Prime Agent high · ToolsDeepSeekPrime Agent · high reasoningTools · 2 rooms | 0% | Not reported | Unknown |
| 42 | GLM-5.3Z.AIPrime Agent · high reasoningTools · 2 rooms | 0% | Not reported | Open |
| 43 | GPT-5.4OpenAICodex · xhigh reasoningNo tools · 2 rooms | 0% | Not reported | Closed |
| 44 | Qwen3.8 MaxAlibabaPrime Agent · high reasoningTools · 2 rooms | 0% | Not reported | Open |
| 45 | Claude Sonnet 5AnthropicClaude Code · max reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 46 | Muse Spark 1.2MetaMuse Code · xhigh reasoningNo tools · 1 rooms · Paused snapshot | 0% | Not reported | Closed |
| 47 | Claude Sonnet 5AnthropicClaude Code · max reasoningTools · 1 rooms | 0% | Not reported | Closed |
| 48 | Muse Spark 1.3MetaMuse Code · ultra reasoningTools · 1 rooms | 0% | Not reported | Closed |
| 49 | Gemini 3.1 ProGoogleCodex · high reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 50 | GPT-5.6 LunaOpenAICodex · max reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 51 | GLM-5.2Z.AIPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Open |
| 52 | MiniMax M3MiniMaxPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Open |
| 53 | Gemini 3.5 FlashGooglePrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 54 | Claude Haiku 4.5AnthropicClaude CodeNo tools · 1 rooms | 0% | Not reported | Closed |
| 55 | deepseek-v4-pro · Prime high · No toolsDeepSeekPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Unknown |
| 56 | GPT-5.3-Codex-SparkOpenAICodex · xhigh reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 57 | deepseek-v4-flash · Prime high · No toolsDeepSeekPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Unknown |
| 58 | Grok 4.20xAIPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 59 | Nemotron 3 UltraNVIDIAPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Open |
| 60 | DeepSeek V4 Flash 0731DeepSeekPrime Agent · high reasoningTools · 1 rooms | 0% | Not reported | Open |
| 61 | MiMo-V2.5-ProXiaomiPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Closed |
| 62 | Llama 4 MaverickMetaPrime · high reasoningNo tools · 1 rooms | 0% | Not reported | Open |
How to read this leaderboard
One gem is one percent. Rooms visited is a second score, and the authors treat gems as the harder result because collecting one takes mastery of the puzzles, not just reaching its room. Python tools let an agent write a solver for the puzzles, so a Python-enabled score and a Python-disabled score measure different things. The authors treat a no-Python result as the stronger evidence of in-game planning.
Operator receipt: 62 sourced rows are currently displayable on this page; the leading published row is GPT-6 Astra at 23%.
Honest limit: Most rows are snapshots of one game, and several were paused, stopped at a provider usage limit, or still running when captured, so a score can rise. The authors revised levels and the engine while some runs were in progress. Each run lists its own gem total (71 to 100 across the mirrored runs), and the site scores every run out of 100. Harnesses differ across rows, each model has at most one run per setting, and cost figures are API-equivalent estimates that are missing for some runs. MazeBench is at version 0.7 and its final levels are still being built.
How we show MazeBench
MazeBench is a 3D puzzle world that agents play through their own coding-agent harness, such as Codex or Claude Code. A run scores one percent for each of the 100 hidden gems it collects across 256 rooms. We mirror every run on the leaderboard from the October 15, 2026 snapshot: 62 runs across 45 models, all in ASCII mode.
Each run had Python tools (25 runs) or played without them (37 runs). Python lets an agent write a solver, so the two settings measure different things and the table labels every row. A model's page shows its best published run and names the setting.
Many rows are snapshots of games that were paused, stopped at a provider usage limit, or still running when captured, so a score can rise. The authors revised levels and the engine while some runs were in progress. Each score belongs to one game played through one harness and reasoning setting, so the table is display only and does not enter model rankings.
Snapshot
The published MazeBench snapshot places GPT-6 Astra first at 23%. The third row is 7 points behind. The broader top-10 range is 14 points, so the table still separates the published systems.
62 published runs have been evaluated on MazeBench. The benchmark falls in the Agentic category. MazeBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About MazeBench
Year
2026
Tasks
100 hidden gems across 256 rooms in one open world
Format
Gems collected out of 100, with rooms visited as a second score
Difficulty
Long-horizon spatial planning, with single puzzles that can take over one hundred moves
MazeBench is a three-dimensional world of 256 rooms built around Sokoban-style box puzzles, with elevator tiles, toggleable walls, and icy slopes. The agent sees the world through a camera it can rotate to 20 angles, and 100 gems are hidden across the rooms. Agents play inside their own coding-agent harnesses, such as Codex and Claude Code, through an MCP server with eleven actions: four movement, four camera, undo, level reset, and teleport to a room already visited. Runs have no action limit and are stopped when the agent keeps circling without finding new board states. The leaderboard covers ASCII mode, where each tile is a 4x4 block of characters, with Python tools either enabled or disabled. The engine, level editor, and solution algorithms are open source.
Freshness and provenance
Version
MazeBench v0.7 (ASCII mode)
Refresh cadence
Rolling leaderboard
Staleness state
Current
Question availability
Open-source engine, level editor, and solution algorithms; run summaries published on the leaderboard
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MazeBench measure?
MazeBench measures whether an agent can plan through a three-dimensional puzzle world. The agent explores 256 rooms of box-pushing puzzles through a camera it can rotate, and scores one percent for each of the 100 hidden gems it collects. Rooms visited is a second score. The published runs use the text-based ASCII mode.
Why do some rows say Tools and others No tools?
Tools means the agent could run Python, which lets it write a solver for the puzzles. No tools means it had to plan inside the game. The same model can have one run in each setting, and the two scores are not interchangeable. A model's page shows its best published run and names the setting.
Why is MazeBench display only?
Each result is one game played through a specific harness, reasoning setting, and tool setting, and many are snapshots of runs that were paused, stopped at a usage limit, or still going. The scores are useful for comparing those exact runs, but they are not a model-only measurement, so they stay out of BenchLM's rankings.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.