Skip to main content
BenchLM
Data

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement (AI4AI-Bench)

We show this table for reference; we do not rank on it.

Tests whether coding agents can improve the training algorithm inside an existing AI research codebase, then survive a sealed training run and held-out evaluation.

Mean normalized score on AI4AI-Bench — October 15, 2026 snapshot

We mirror the published mean normalized score view for AI4AI-Bench. Claude Opus 5 leads the public snapshot at 0.288, followed by Claude Opus 5 (0.278) and Claude Opus 5 (0.272). We do not use these results to rank models overall.

1Closed

Claude Opus 5

Anthropic

Claude Code · medium reasoning

$166.31 exploration · 1.20M output tokens

0.288
Overall 79.63Context 1M
2Closed

Claude Opus 5

Anthropic

Claude Code · high reasoning

$181.09 exploration · 1.21M output tokens

0.278
Overall 79.63Context 1M
3Closed

Claude Opus 5

Anthropic

Claude Code · low reasoning

$181.27 exploration · 1.22M output tokens

0.272
Overall 79.63Context 1M

29 configurationsAgenticCurrentDisplay onlyUpdated October 15, 2026 snapshot

Mean normalized score table (29 configurations)

Mean normalized score results for AI4AI-Bench
RankModel / configurationScoreParameters (B)Open / closed
1Claude Opus 5AnthropicClaude Code · medium reasoning$166.31 exploration · 1.20M output tokens
0.288
Not reportedClosed
2Claude Opus 5AnthropicClaude Code · high reasoning$181.09 exploration · 1.21M output tokens
0.278
Not reportedClosed
3Claude Opus 5AnthropicClaude Code · low reasoning$181.27 exploration · 1.22M output tokens
0.272
Not reportedClosed
4GPT-5.6 SolOpenAICodex · max reasoning$626.19 exploration · 1.25M output tokens
0.245
Not reportedClosed
5GPT-5.6 SolOpenAICodex · high reasoning$419.93 exploration · 0.62M output tokens
0.235
Not reportedClosed
6GPT-5.6 TerraOpenAICodex · max reasoning$345.99 exploration · 1.57M output tokens
0.228
Not reportedClosed
7Claude Opus 5AnthropicClaude Code · max reasoning$195.47 exploration · 1.34M output tokens
0.225
Not reportedClosed
8Claude Sonnet 5AnthropicClaude Code · high reasoning$95.53 exploration · 1.03M output tokens
0.216
Not reportedClosed
9GPT-5.6 SolOpenAICodex · low reasoning$336.86 exploration · 0.44M output tokens
0.198
Not reportedClosed
10GPT-5.6 SolOpenAICodex · xhigh reasoning$521.20 exploration · 0.83M output tokens
0.195
Not reportedClosed
11Claude Opus 5AnthropicClaude Code · xhigh reasoning$184.67 exploration · 1.34M output tokens
0.190
Not reportedClosed
12Kimi K3Moonshot AIClaude Code · max reasoning$29.93 exploration · 0.61M output tokens
0.174
Not reportedPending
13Claude Sonnet 5AnthropicClaude Code · max reasoning$108.62 exploration · 1.10M output tokens
0.164
Not reportedClosed
14GPT-5.6 LunaOpenAICodex · xhigh reasoning$109.89 exploration · 0.71M output tokens
0.147
Not reportedClosed
15GPT-5.6 SolOpenAICodex · medium reasoning$448.69 exploration · 0.61M output tokens
0.144
Not reportedClosed
16GPT-5.6 TerraOpenAICodex · xhigh reasoning$228.57 exploration · 0.91M output tokens
0.144
Not reportedClosed
17GPT-5.6 LunaOpenAICodex · max reasoning$107.87 exploration · 0.77M output tokens
0.139
Not reportedClosed
18Claude Sonnet 5AnthropicClaude Code · xhigh reasoning$100.78 exploration · 1.02M output tokens
0.137
Not reportedClosed
19GPT-5.6 TerraOpenAICodex · medium reasoning$43.34 exploration · 0.24M output tokens
0.135
Not reportedClosed
20GPT-5.6 TerraOpenAICodex · high reasoning$213.87 exploration · 0.73M output tokens
0.129
Not reportedClosed
21GPT-5.6 SolOpenAICodex · none reasoning$339.34 exploration · 0.35M output tokens
0.126
Not reportedClosed
22GPT-5.6 LunaOpenAICodex · medium reasoning$30.35 exploration · 0.26M output tokens
0.124
Not reportedClosed
23GPT-5.6 LunaOpenAICodex · high reasoning$65.93 exploration · 0.50M output tokens
0.119
Not reportedClosed
24Claude Sonnet 5AnthropicClaude Code · medium reasoning$97.75 exploration · 0.91M output tokens
0.109
Not reportedClosed
25GPT-5.6 TerraOpenAICodex · low reasoning$34.56 exploration · 0.18M output tokens
0.108
Not reportedClosed
26Claude Sonnet 5AnthropicClaude Code · low reasoning$93.07 exploration · 0.91M output tokens
0.099
Not reportedClosed
27GPT-5.6 LunaOpenAICodex · none reasoning$16.91 exploration · 0.12M output tokens
0.091
Not reportedClosed
28GPT-5.6 LunaOpenAICodex · low reasoning$6.51 exploration · 0.08M output tokens
0.079
Not reportedClosed
29GPT-5.6 TerraOpenAICodex · none reasoning$3.53 exploration · 0.02M output tokens
0.065
Not reportedClosed

How to read this leaderboard

Each task maps its native metric onto a common scale where 0 means no useful learning, 0.1 matches the repository's original method, and 1 represents a perfect result. Scores above 0.1 improve on the shipped algorithm; scores below it made the method worse.

Operator receipt: 29 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 0.288.

Honest limit: Each row combines model weights, reasoning effort, coding client, exploration budget, and ten different training environments. The normalized mean supports configuration comparison, not a controlled base-model rank.

Official benchmarkTask catalogPublished score dataGitHub repository

How we show AI4AI-Bench

We mirror the benchmark-owned AI4AI-Bench configuration table from October 15, 2026 snapshot. The source reports 29 configurations across 10 AI research codebases, for 290 four-hour agent runs.

Each task maps its native held-out metric onto one scale: 0 means no useful learning, 0.1 matches the repository's original method, and 1 is perfect. We keep reasoning-effort and client configurations separate because the source result belongs to the full research-agent setup.

Snapshot

6 models29 configurations10 research codebases290 agent runs124 below baselineDisplay only
AI4AI-Bench project and resultsTask catalogAgent trajectoriesPublished score dataResearch paperGitHub repository

The published AI4AI-Bench snapshot places Claude Opus 5 first at 0.288. The third row is 0.016 score units behind. The broader top-10 range is 0.093 score units, so many of the published results sit in a relatively narrow band.

29 configurations have been evaluated on AI4AI-Bench. The benchmark falls in the Agentic category. AI4AI-Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About AI4AI-Bench

Year

2026

Tasks

10 AI training-algorithm design tasks

Format

Four-hour code rewrite followed by sealed training and held-out evaluation

Difficulty

End-to-end AI research and algorithm design

AI4AI-Bench freezes 10 research codebases spanning supervised fine-tuning, agentic reinforcement learning, distillation, reward modeling, preference optimization, diffusion reinforcement learning, machine unlearning, graph diffusion, model merging, and pruning. Agents receive four hours and one GPU to rewrite the method. A sealed pipeline then trains the submission from scratch for up to 12 hours and scores it on a held-out test. We mirror all 29 model, client, and reasoning-effort configurations as display-only evidence.

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-ImprovementPublic benchmark source

Freshness and provenance

Version

AI4AI-Bench v1.5

Refresh cadence

Quarterly

Staleness state

Current

Question availability

10 public tasks with score data and trajectories

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does AI4AI-Bench measure?

Tests whether coding agents can improve the training algorithm inside an existing AI research codebase, then survive a sealed training run and held-out evaluation.

Which model leads the published AI4AI-Bench snapshot?

Claude Opus 5 currently leads the published AI4AI-Bench snapshot with 0.288 mean normalized score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on AI4AI-Bench?

The October 15, 2026 snapshot snapshot contains 29 configurations across 6 AI models.

Last updated: October 15, 2026 snapshot · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.