# AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement (AI4AI-Bench)

> Tests whether coding agents can improve the training algorithm inside an existing AI research codebase, then survive a sealed training run and held-out evaluation.

Canonical page: https://benchlm.ai/benchmarks/ai4ai-bench

- Category: [Agentic](/agentic)
- Last updated: October 15, 2026 snapshot

## About AI4AI-Bench

- Year: 2026
- Tasks: 10 AI training-algorithm design tasks
- Format: Four-hour code rewrite followed by sealed training and held-out evaluation
- Difficulty: End-to-end AI research and algorithm design
- Paper: [AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement](https://arxiv.org/abs/2608.20318)

AI4AI-Bench freezes 10 research codebases spanning supervised fine-tuning, agentic reinforcement learning, distillation, reward modeling, preference optimization, diffusion reinforcement learning, machine unlearning, graph diffusion, model merging, and pruning. Agents receive four hours and one GPU to rewrite the method. A sealed pipeline then trains the submission from scratch for up to 12 hours and scores it on a held-out test. We mirror all 29 model, client, and reasoning-effort configurations as display-only evidence.

AI4AI-Bench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (29 configurations)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · medium reasoning · $166.31 exploration · 1.20M output tokens | Anthropic | 0.288 |
| 2 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · high reasoning · $181.09 exploration · 1.21M output tokens | Anthropic | 0.278 |
| 3 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · low reasoning · $181.27 exploration · 1.22M output tokens | Anthropic | 0.272 |
| 4 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · max reasoning · $626.19 exploration · 1.25M output tokens | OpenAI | 0.245 |
| 5 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · high reasoning · $419.93 exploration · 0.62M output tokens | OpenAI | 0.235 |
| 6 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · max reasoning · $345.99 exploration · 1.57M output tokens | OpenAI | 0.228 |
| 7 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · max reasoning · $195.47 exploration · 1.34M output tokens | Anthropic | 0.225 |
| 8 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · high reasoning · $95.53 exploration · 1.03M output tokens | Anthropic | 0.216 |
| 9 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · low reasoning · $336.86 exploration · 0.44M output tokens | OpenAI | 0.198 |
| 10 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · xhigh reasoning · $521.20 exploration · 0.83M output tokens | OpenAI | 0.195 |
| 11 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · xhigh reasoning · $184.67 exploration · 1.34M output tokens | Anthropic | 0.190 |
| 12 | [Kimi K3](/models/kimi-k3) | Claude Code · max reasoning · $29.93 exploration · 0.61M output tokens | Moonshot AI | 0.174 |
| 13 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · max reasoning · $108.62 exploration · 1.10M output tokens | Anthropic | 0.164 |
| 14 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · xhigh reasoning · $109.89 exploration · 0.71M output tokens | OpenAI | 0.147 |
| 15 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · medium reasoning · $448.69 exploration · 0.61M output tokens | OpenAI | 0.144 |
| 16 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · xhigh reasoning · $228.57 exploration · 0.91M output tokens | OpenAI | 0.144 |
| 17 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · max reasoning · $107.87 exploration · 0.77M output tokens | OpenAI | 0.139 |
| 18 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · xhigh reasoning · $100.78 exploration · 1.02M output tokens | Anthropic | 0.137 |
| 19 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · medium reasoning · $43.34 exploration · 0.24M output tokens | OpenAI | 0.135 |
| 20 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · high reasoning · $213.87 exploration · 0.73M output tokens | OpenAI | 0.129 |
| 21 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex · none reasoning · $339.34 exploration · 0.35M output tokens | OpenAI | 0.126 |
| 22 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · medium reasoning · $30.35 exploration · 0.26M output tokens | OpenAI | 0.124 |
| 23 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · high reasoning · $65.93 exploration · 0.50M output tokens | OpenAI | 0.119 |
| 24 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · medium reasoning · $97.75 exploration · 0.91M output tokens | Anthropic | 0.109 |
| 25 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · low reasoning · $34.56 exploration · 0.18M output tokens | OpenAI | 0.108 |
| 26 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · low reasoning · $93.07 exploration · 0.91M output tokens | Anthropic | 0.099 |
| 27 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · none reasoning · $16.91 exploration · 0.12M output tokens | OpenAI | 0.091 |
| 28 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Codex · low reasoning · $6.51 exploration · 0.08M output tokens | OpenAI | 0.079 |
| 29 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Codex · none reasoning · $3.53 exploration · 0.02M output tokens | OpenAI | 0.065 |

## FAQ

### What does AI4AI-Bench measure?

Tests whether coding agents can improve the training algorithm inside an existing AI research codebase, then survive a sealed training run and held-out evaluation.

### Which model leads the published AI4AI-Bench snapshot?

Claude Opus 5 currently leads the published AI4AI-Bench snapshot with a score of 0.288.

### How many models are evaluated on AI4AI-Bench?

The October 15, 2026 snapshot contains 29 configurations across 6 AI models.

### Does AI4AI-Bench affect BenchLM's overall score?

Not directly. AI4AI-Bench is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.