# InferenceBench

> A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.

Canonical page: https://benchlm.ai/benchmarks/inferencebench

- Category: [Agentic](/agentic)
- Last updated: May 20, 2026

## About InferenceBench

- Year: 2026
- Tasks: 4 inference-serving optimization scenarios
- Format: Two-hour autonomous CLI agent run
- Difficulty: Open-ended ML systems engineering
- Paper: [InferenceBench](https://inferencebench.ai/)

BenchLM mirrors the public InferenceBench agent leaderboard as a display-only agentic systems-engineering benchmark. The primary score is aggregate geometric-mean speedup over a PyTorch baseline across prefill latency, decode latency, throughput, and all-in-one serving scenarios.

InferenceBench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (36 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Claude Opus 5](/models/claude-opus-5) | Claude Code · v2.1.119 · strict prompt | Anthropic | 8.90x |
| 2 | [Claude Fable 5](/models/claude-fable) | Claude Code · v2.1.175 · strict prompt | Anthropic | 8.74x |
| 3 | [Claude Opus 4.7](/models/claude-opus-4-7) | Claude Code · v2.1.175 | Anthropic | 8.53x |
| 4 | [Claude Opus 4.8](/models/claude-opus-4-8) | Claude Code · v2.1.175 | Anthropic | 7.60x |
| 5 | [Claude Fable 5](/models/claude-fable) | Claude Code · v2.1.175 · strict prompt | Anthropic | 7.52x |
| 6 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Codex CLI · strict prompt | OpenAI | 7.34x |
| 7 | [Claude Opus 4.8](/models/claude-opus-4-8) | Claude Code | Anthropic | 7.34x |
| 8 | [GLM-5.2](/models/glm-5-2) | Claude Code · v2.1.119 · strict prompt | Z.AI | 7.00x |
| 9 | [Grok 4.6](/models/grok-4-6) | Grok Build · strict prompt | xAI | 6.65x |
| 10 | [Claude Sonnet 5](/models/claude-sonnet-5) | Claude Code · v2.1.119 · strict prompt | Anthropic | 6.43x |
| 11 | [Kimi K2.7 Code](/models/kimi-k2-7-code) | OpenCode · strict prompt | Moonshot AI | 6.27x |
| 12 | [GPT-5.4](/models/gpt-5-4) | Codex CLI | OpenAI | 6.16x |
| 13 | [Kimi K3](/models/kimi-k3) | OpenCode · strict prompt | Moonshot AI | 5.70x |
| 14 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Claude Code | Anthropic | 5.56x |
| 15 | [GPT-5.3 Codex](/models/gpt-5-3-codex) | Codex CLI | OpenAI | 5.49x |
| 16 | [GPT-5.5](/models/gpt-5-5) | Codex CLI | OpenAI | 5.45x |
| 17 | [GLM-5.3](/models/glm-5-3) | Claude Code · v2.1.119 · strict prompt | Z.AI | 4.98x |
| 18 | [Gemini 3.1 Pro](/models/gemini-3-1-pro) | OpenCode | Google | 4.92x |
| 19 | [Kimi K2.6](/models/kimi-2-6) | OpenCode | Moonshot AI | 4.51x |
| 20 | [Ox Alpha (stealth)](https://inferencebench.ai/#results) | OpenCode · strict prompt | Unknown | 4.49x |
| 21 | [Claude Opus 4.6](/models/claude-opus-4-6) | Claude Code | Anthropic | 4.38x |
| 22 | [GPT-5.2](/models/gpt-5-2) | Codex CLI | OpenAI | 4.28x |
| 23 | [GPT-5.5](/models/gpt-5-5) | Codex CLI | OpenAI | 4.22x |
| 24 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | OpenCode | Google | 4.16x |
| 25 | [Claude Opus 4.5](/models/claude-opus-4-5) | Claude Code | Anthropic | 3.76x |
| 26 | [Grok 4.5](/models/grok-4-5) | Grok Build · strict prompt | xAI | 3.70x |
| 27 | [GPT-5.1-Codex-Max](/models/gpt-5-1-codex-max) | Codex CLI | OpenAI | 3.59x |
| 28 | [Grok 4.5](/models/grok-4-5) | OpenCode · strict prompt | xAI | 3.42x |
| 29 | [GLM-5](/models/glm-5) | OpenCode | Z.AI | 3.22x |
| 30 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Claude Code | Anthropic | 3.18x |
| 31 | [Claude Fable 5](/models/claude-fable) | Claude Code · v2.1.175 | Anthropic | 3.16x |
| 32 | [Claude Haiku 4.5](/models/claude-haiku-4-5) | Claude Code | Anthropic | 2.78x |
| 33 | [GPT-5.3 Codex](/models/gpt-5-3-codex) | Codex CLI | OpenAI | 2.32x |
| 34 | [Claude Opus 4.7](/models/claude-opus-4-7) | Claude Code · v2.1.114 | Anthropic | 2.25x |
| 35 | [Claude Fable 5](/models/claude-fable) | Claude Code · v2.1.175 | Anthropic | 2.15x |
| 36 | [GPT-5.2-Codex](/models/gpt-5-2-codex) | Codex CLI | OpenAI | 1.98x |

## FAQ

### What does InferenceBench measure?

A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.

### Which model leads the published InferenceBench snapshot?

Claude Opus 5 currently leads the published InferenceBench snapshot with a score of 8.90x.

### How many models are evaluated on InferenceBench?

The May 20, 2026 contains 36 AI models.

### Does InferenceBench affect BenchLM's overall score?

Not directly. InferenceBench is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.