InferenceBench
We show this table for reference; we do not rank on it.
A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.
Aggregate speedup on InferenceBench — May 20, 2026
We mirror the published aggregate speedup view for InferenceBench. Claude Opus 5.5 leads the public snapshot at 12.08x, followed by Claude Fable 5.1 (9.83x) and Claude Fable 5.1 (9.31x). We do not use these results to rank models overall.
Claude Opus 5.5
Anthropic
Claude Code · v2.1.280 · prompt v1.0.1
Claude Fable 5.1
Anthropic
Claude Code · v2.1.258 · prompt v1.0.1
Claude Fable 5.1
Anthropic
Claude Code · v2.1.258 · prompt v1.0.1
42 modelsAgenticCurrentDisplay onlyUpdated May 20, 2026
Aggregate speedup table (42 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 5.5AnthropicClaude Code · v2.1.280 · prompt v1.0.1 | 12.08x | Not reported | Closed |
| 2 | Claude Fable 5.1AnthropicClaude Code · v2.1.258 · prompt v1.0.1 | 9.83x | Not reported | Closed |
| 3 | Claude Fable 5.1AnthropicClaude Code · v2.1.258 · prompt v1.0.1 | 9.31x | Not reported | Closed |
| 4 | Claude Opus 5AnthropicClaude Code · v2.1.119 · prompt v1.0.1 | 8.90x | Not reported | Closed |
| 5 | Claude Fable 5AnthropicClaude Code · v2.1.175 · prompt v1.0.1 | 8.74x | Not reported | Closed |
| 6 | Claude Opus 4.7AnthropicClaude Code · v2.1.175 | 8.53x | Not reported | Closed |
| 7 | GPT-6 AstraOpenAICodex CLI · prompt v1.0.1 | 7.90x | Not reported | Closed |
| 8 | GPT-6 SolOpenAICodex CLI · prompt v1.0.1 | 7.64x | Not reported | Closed |
| 9 | Claude Opus 4.8AnthropicClaude Code · v2.1.175 | 7.60x | Not reported | Closed |
| 10 | Claude Fable 5AnthropicClaude Code · v2.1.175 · prompt v1.0.1 | 7.52x | Not reported | Closed |
| 11 | GPT-5.6 SolOpenAICodex CLI · prompt v1.0.1 | 7.34x | Not reported | Closed |
| 12 | Claude Opus 4.8AnthropicClaude Code | 7.34x | Not reported | Closed |
| 13 | GLM-5.2Z.AIClaude Code · v2.1.119 · prompt v1.0.1 | 7.00x | Not reported | Open |
| 14 | Grok 4.6xAIGrok Build · prompt v1.0.1 | 6.65x | Not reported | Closed |
| 15 | Claude Sonnet 5AnthropicClaude Code · v2.1.119 · prompt v1.0.1 | 6.43x | Not reported | Closed |
| 16 | Kimi K2.7 CodeMoonshot AIOpenCode · prompt v1.0.1 | 6.27x | Not reported | Open |
| 17 | GPT-5.4OpenAICodex CLI | 6.16x | Not reported | Closed |
| 18 | Kimi K3Moonshot AIOpenCode · prompt v1.0.1 | 5.70x | Not reported | Pending |
| 19 | Claude Sonnet 4.6AnthropicClaude Code | 5.56x | Not reported | Closed |
| 20 | GPT-5.3 CodexOpenAICodex CLI | 5.53x | Not reported | Closed |
| 21 | GPT-5.5OpenAICodex CLI | 5.45x | Not reported | Closed |
| 22 | GLM-5.3Z.AIClaude Code · v2.1.119 · prompt v1.0.1 | 4.98x | Not reported | Open |
| 23 | Gemini 3.1 ProGoogleOpenCode | 4.92x | Not reported | Closed |
| 24 | Muse Spark 1.3MetaOpenCode · prompt v1.0.1 | 4.53x | Not reported | Closed |
| 25 | Kimi K2.6Moonshot AIOpenCode | 4.51x | Not reported | Open |
| 26 | GLM-5.3-FlashZ.AIOpenCode · prompt v1.0.1 | 4.49x | Not reported | Open |
| 27 | Claude Opus 4.6AnthropicClaude Code | 4.38x | Not reported | Closed |
| 28 | GPT-5.2OpenAICodex CLI | 4.28x | Not reported | Closed |
| 29 | GPT-5.5OpenAICodex CLI | 4.22x | Not reported | Closed |
| 30 | Gemini 3.5 FlashGoogleOpenCode | 4.16x | Not reported | Closed |
| 31 | Claude Opus 4.5AnthropicClaude Code | 3.76x | Not reported | Closed |
| 32 | Grok 4.5xAIGrok Build · prompt v1.0.1 | 3.70x | Not reported | Closed |
| 33 | GPT-5.1-Codex-MaxOpenAICodex CLI | 3.59x | Not reported | Closed |
| 34 | Grok 4.5xAIOpenCode · prompt v1.0.1 | 3.42x | Not reported | Closed |
| 35 | GLM-5Z.AIOpenCode | 3.22x | Not reported | Open |
| 36 | Claude Sonnet 4.5AnthropicClaude Code | 3.18x | Not reported | Closed |
| 37 | Claude Fable 5AnthropicClaude Code · v2.1.175 | 3.16x | Not reported | Closed |
| 38 | Claude Haiku 4.5AnthropicClaude Code | 2.78x | Not reported | Closed |
| 39 | GPT-5.3 CodexOpenAICodex CLI | 2.32x | Not reported | Closed |
| 40 | Claude Opus 4.7AnthropicClaude Code · v2.1.114 | 2.25x | Not reported | Closed |
| 41 | Claude Fable 5AnthropicClaude Code · v2.1.175 | 2.15x | Not reported | Closed |
| 42 | GPT-5.2-CodexOpenAICodex CLI | 1.98x | Not reported | Closed |
How InferenceBench is shown here
BenchLM mirrors the public InferenceBench agent leaderboard captured on May 20, 2026. The source evaluates 42 frontier agent configurations over 4 inference-serving scenarios with a 2 h budget on 1 NVIDIA H100 80 GB.
InferenceBench is display only on BenchLM. It is a useful agentic systems-engineering signal, but the rows combine model capability, agent scaffold, inference framework choices, hardware, and final-server validity, so BenchLM keeps it separate from weighted model-only rankings.
Snapshot
The published InferenceBench snapshot places Claude Opus 5.5 first at 12.08x. The third row is 2.77 points behind. The broader top-10 range is 4.56 points, so many of the published results sit in a relatively narrow band.
42 models have been evaluated on InferenceBench. The benchmark falls in the Agentic category. InferenceBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About InferenceBench
Year
2026
Tasks
4 inference-serving optimization scenarios
Format
Two-hour autonomous CLI agent run
Difficulty
Open-ended ML systems engineering
BenchLM mirrors the public InferenceBench agent leaderboard as a display-only agentic systems-engineering benchmark. The primary score is aggregate geometric-mean speedup over a PyTorch baseline across prefill latency, decode latency, throughput, and all-in-one serving scenarios.
Freshness and provenance
Version
InferenceBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark harness and aggregate leaderboard
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does InferenceBench measure?
A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.
Which model leads the published InferenceBench snapshot?
Claude Opus 5.5 currently leads the published InferenceBench snapshot with 12.08x aggregate speedup. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on InferenceBench?
The May 20, 2026 snapshot contains 42 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.