# GameDevBench

> Evaluates coding agents on 333 multimodal game-development tasks in Godot, spanning 2D graphics, 3D graphics, user interfaces, and gameplay logic.

Canonical page: https://benchlm.ai/benchmarks/gamedevbench

- Category: [Coding](/coding)
- Last updated: ICML 2026 camera-ready results, updated September 09, 2026

## About GameDevBench

- Year: 2026
- Tasks: 333 tasks from 88 tutorials
- Format: Pass@1 on the full task set with 95% confidence intervals
- Difficulty: Multimodal game development in Godot 4.4.1
- Paper: [GameDevBench: Evaluating Agentic Capabilities Through Game Development](https://arxiv.org/abs/2602.11103)

GameDevBench derives 333 tasks from 88 web and video tutorials and runs them in Godot 4.4.1. The camera-ready leaderboard reports Pass@1 on the full task set using each model's best published agent harness and multimodal-feedback configuration, with 95% confidence intervals kept alongside the scores.

GameDevBench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (18 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [GPT-6 Astra](/models/gpt-6-astra) | Codex · Codex · high · 95% CI ±5 | OpenAI | 68.8% |
| 2 | [Claude Fable 5](/models/claude-fable) | Claude Code · Claude Code · xhigh · 95% CI ±5 | Anthropic | 67.3% |
| 3 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · Codex · xhigh · 95% CI ±5.2 | OpenAI | 63.7% |
| 4 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · Codex · high · 95% CI ±5.2 | OpenAI | 63.1% |
| 5 | [Muse Spark 1.2](/models/muse-spark-1-2) | Muse Code · Muse Code · high · 95% CI ±5.2 | Meta | 61.0% |
| 6 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · Codex · medium · 95% CI ±5.3 | OpenAI | 58.6% |
| 7 | [Kimi K3](/models/kimi-k3) | Kimi Code · Kimi Code · 95% CI ±5.3 | Moonshot AI | 58.0% |
| 8 | [Claude Opus 4.8](/models/claude-opus-4-8) | Claude Code · Claude Code · 95% CI ±5.3 | Anthropic | 55.9% |
| 9 | [GPT-5.5](/models/gpt-5-5) | Codex · Codex · 95% CI ±5.3 | OpenAI | 54.7% |
| 10 | [Gemini 3 Pro](/models/gemini-3-pro) | Gemini CLI · Gemini CLI · 95% CI ±5.4 | Google | 53.8% |
| 11 | [GPT-5.4](/models/gpt-5-4) | Codex · Codex · 95% CI ±5.4 | OpenAI | 52.0% |
| 12 | [Gemini 3 Flash](/models/gemini-3-flash) | Gemini CLI · Gemini CLI · 95% CI ±5.4 | Google | 46.8% |
| 13 | [GPT-5.4 mini](/models/gpt-5-4-mini) | Codex · Codex · 95% CI ±5.3 | OpenAI | 43.2% |
| 14 | [GLM-5.2](/models/glm-5-2) | OpenCode · OpenCode · 95% CI ±5.2 | Z.AI | 38.4% |
| 15 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Claude Code · Claude Code · 95% CI ±5.1 | Anthropic | 34.8% |
| 16 | [Kimi K2.5](/models/kimi-k2-5) | OpenHands · OpenHands · 95% CI ±4.4 | Moonshot AI | 20.7% |
| 17 | [Claude Haiku 4.5](/models/claude-haiku-4-5) | Claude Code · Claude Code · 95% CI ±4.2 | Anthropic | 18.6% |
| 18 | [Qwen3.5 397B](/models/qwen3-5-397b) | OpenHands · OpenHands · 95% CI ±2.4 | Alibaba | 5.4% |

## FAQ

### What does GameDevBench measure?

GameDevBench measures whether coding agents can complete 333 game-development tasks in Godot. Tasks cover 2D graphics, 3D graphics, user interfaces, and gameplay logic, requiring agents to work with code and visual assets such as shaders, sprites, animations, and scenes.

### Which configuration leads GameDevBench?

Claude Fable 5 at xhigh effort in Claude Code leads the August 13, 2026 camera-ready table at 67.3% Pass@1, with a reported 95% confidence interval of ±5.0 points. GPT-5.6 Sol at xhigh effort in Codex follows at 63.7% ±5.2.

### Does GameDevBench affect BenchLM rankings?

No. We keep GameDevBench display only because each row combines a model with an agent harness, reasoning setting, multimodal-feedback setup, and Godot environment. Those choices are part of the measured system, so the scores are not normalized base-model results.