A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.
BenchLM mirrors the public InferenceBench agent leaderboard captured on May 20, 2026. The source evaluates 15 frontier agent configurations over 4 inference-serving scenarios with a 2 h budget on 1 NVIDIA H100 80 GB.
InferenceBench is display only on BenchLM. It is a useful agentic systems-engineering signal, but the rows combine model capability, agent scaffold, inference framework choices, hardware, and final-server validity, so BenchLM keeps it separate from weighted model-only rankings.
BenchLM mirrors the published aggregate speedup view for InferenceBench. Claude Sonnet 4.6 leads the public snapshot at 8.08x , followed by GLM-5 (6.20x) and Gemini 3.1 Pro (6.16x). BenchLM does not use these results to rank models overall.
Claude Sonnet 4.6
Anthropic
GLM-5
Z.AI
Gemini 3.1 Pro
The published InferenceBench snapshot is tightly clustered at the top: Claude Sonnet 4.6 sits at 8.08x, while the third row is only 1.92 points behind. The broader top-10 spread is 4.54 points, so many of the published scores sit in a relatively narrow band.
15 models have been evaluated on InferenceBench. The benchmark falls in the Agentic category. This category carries a 22% weight in BenchLM.ai's overall scoring system. InferenceBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
Year
2026
Tasks
4 inference-serving optimization scenarios
Format
Two-hour autonomous CLI agent run
Difficulty
Open-ended ML systems engineering
BenchLM mirrors the public InferenceBench agent leaderboard as a display-only agentic systems-engineering benchmark. The primary score is aggregate geometric-mean speedup over a PyTorch baseline across prefill latency, decode latency, throughput, and all-in-one serving scenarios.
Version
InferenceBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark harness and aggregate leaderboard
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
A benchmark for open-ended LLM inference optimization by AI agents. Agents receive a base model, one H100, and a fixed time budget to build a valid OpenAI-compatible inference server that improves serving speed.
Claude Sonnet 4.6 currently leads the published InferenceBench snapshot with 8.08x aggregate speedup. BenchLM shows this benchmark for display only and does not use it in overall rankings.
15 AI models are included in BenchLM's mirrored InferenceBench snapshot, based on the public leaderboard captured on May 20, 2026.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.