Skip to main content
Radar

Five or fewer confirmed AI changes, with original sources, on mornings when something changed.A free source-linked morning brief.

Start free brief
BenchLM

Vending-Bench 2

Tests long-horizon coherence by asking agents to operate a simulated vending-machine business for one year.

Data verified 28 confirmed releases in the last 30 daysStart free brief

How we show Vending-Bench 2

We mirror Andon Labs' arithmetic-mean Vending-Bench 2 table. The current source contains 59 model-configuration rows and 307 completed runs, with 4 to 6 runs behind each row. The table ranks final bank balance after a one-year simulation and keeps each row's standard error attached.

Agents start with $500, source products, negotiate with suppliers, stock a simulated vending machine, and handle sales and customer messages. The snapshot also retains the publisher's geometric mean, but this page follows the source table's default arithmetic-mean ordering. Reasoning settings, custom-tool runs, and provider endpoints remain separate configurations.

59 model-configuration rows307 completed runs365-day simulationArithmetic meanDisplay only
Vending-Bench 2 leaderboard and methodologyOriginal Vending-Bench paper

Arithmetic mean final bank balance on Vending-Bench 2 — August 14, 2026

BenchLM mirrors the published arithmetic mean final bank balance view for Vending-Bench 2. Claude Opus 5 leads the public snapshot at $11,181.87 , followed by Claude Opus 4.7 ($10,936.76) and GPT-5.6 Sol ($9,619.37). We do not use these results to rank models overall.

1Closed

Claude Opus 5

Anthropic

6 runs · SEM $2,093.58

claude-opus-5

$11,181.87
Overall 82.9
2Closed

Claude Opus 4.7

Anthropic

6 runs · SEM $1,181.31

claude-opus-4-7

$10,936.76
Overall 71.7Context 1M
3Closed

GPT-5.6 Sol

OpenAI

5 runs · SEM $1,337.80

gpt-5-6-sol

$9,619.37
Overall 81.94Context 1.05M
59 modelsExternal benchmark mirrorsCurrentDisplay onlyUpdated August 14, 2026

Arithmetic mean final bank balance table (59 models)

Score
1
Claude Opus 5Anthropic · Closed6 runs · SEM $2,093.58
$11,181.87
2
Claude Opus 4.7Anthropic · Closed6 runs · SEM $1,181.31
$10,936.76
3
GPT-5.6 SolOpenAI · Closed5 runs · SEM $1,337.80
$9,619.37
4
Grok 4.6xAI · Closed4 runs · SEM $1,603.96
$9,047.03
5
GLM-5.2Z.AI · Open weight6 runs · SEM $1,084.11
$8,313.78
6
Claude Opus 4.6Anthropic · Closed5 runs · SEM $1,366.99
$8,017.59
7
GPT-5.5OpenAI · Closed5 runs · SEM $1,345.98
$7,523.84
8
GPT-5.6 TerraOpenAI · Closed5 runs · SEM $373.27
$7,343.21
9
Claude Sonnet 4.6Anthropic · Closed5 runs · SEM $721.98
$7,204.14
10
Muse Spark 1.1Meta · Closed6 runs · SEM $1,016.33
$6,520.47
11
Claude Sonnet 5Anthropic · Closed6 runs · SEM $783.74
$6,377.70
12
Kimi K2.6Moonshot AI · Open weight5 runs · SEM $660.87
$6,204.57
13
GPT-5.4OpenAI · Closed5 runs · SEM $871.10
$6,144.18
14
GPT-5.3 CodexOpenAI · Closed5 runs · SEM $795.98
$5,940.12
15
Claude Opus 4.8Anthropic · ClosedHigh reasoning · 5 runs · SEM $830.67
$5,787.43
16
Claude Fable 5Anthropic · ClosedHigh reasoning · 5 runs · SEM $1,074.93
$5,680.26
17
GLM-5.1Z.AI · Open weight5 runs · SEM $333.62
$5,634.41
18
Gemini 3 ProGoogle · Closed5 runs · SEM $905.03
$5,478.16
19
Gemini 3.5 FlashGoogle · Closed6 runs · SEM $1,058.73
$5,396.42
20
Kimi K3Moonshot AI · ClosedMoonshot endpoint · 6 runs · SEM $691.16
$5,165.04
21
Qwen 3.6 PlusAlibaba5 runs · SEM $1,097.12
$5,114.87
22
Kimi K2.7 CodeMoonshot AI · Open weight5 runs · SEM $1,069.37
$5,082.94
23
Claude Fable 5Anthropic · ClosedLow reasoning · 5 runs · SEM $1,225.23
$5,018.52
24
Claude Opus 4.5Anthropic · Closed6 runs · SEM $979.25
$4,967.06
25
Claude Fable 5Anthropic · ClosedMax reasoning · 5 runs · SEM $953.63
$4,966.64
26
Kimi K3Moonshot AI · ClosedFireworks endpoint · 6 runs · SEM $1,652.42
$4,906.96
27
Grok 4.20xAI · Closed6 runs · SEM $1,276.33
$4,662.85
28
Claude Fable 5Anthropic · ClosedNone reasoning · 5 runs · SEM $662.39
$4,529.94
29
GLM-5Z.AI · Open weight5 runs · SEM $393.55
$4,432.12
30
Claude Fable 5Anthropic · ClosedMedium reasoning · 4 runs · SEM $1,426.83
$4,339.81
31
Qwen 3.6 Max (preview)Alibaba · Closed6 runs · SEM $1,469.56
$4,254.19
32
GPT-5.6 LunaOpenAI · Closed5 runs · SEM $1,344.16
$4,094.71
33
Grok 4.5xAI · Closed5 runs · SEM $435.28
$3,887.43
34
Claude Sonnet 4.5Anthropic · Closed5 runs · SEM $1,102.10
$3,838.74
35
Gemini 3.1 ProGoogle · ClosedCustom tools · 5 runs · SEM $1,241.57
$3,774.25
36
Gemini 3 FlashGoogle · Closed5 runs · SEM $639.34
$3,634.72
37
GPT-5.2OpenAI · Closed5 runs · SEM $392.08
$3,591.33
38
DeepSeek V4 ProDeepSeek6 runs · SEM $570.70
$3,284.52
39
Claude Opus 4.8Anthropic · ClosedMax reasoning · 5 runs · SEM $722.02
$2,992.34
40
GLM-4.7Z.AI · Open weight5 runs · SEM $620.60
$2,376.82
41
MiniMax M3MiniMax · Open weight6 runs · SEM $1,151.04
$2,157.77
42
GPT-5.1OpenAI · Closed5 runs · SEM $756.17
$1,473.43
43
Kimi K2.5Moonshot AI · Open weight5 runs · SEM $596.70
$1,198.46
44
Grok 4.1 FastxAI · Closed5 runs · SEM $457.65
$1,106.63
45
DeepSeek V3.2DeepSeek · Open weight5 runs · SEM $503.15
$1,034.00
46
Gemini 3.1 ProGoogle · Closed5 runs · SEM $652.55
$911.21
47
Gemini 2.5 ProGoogle · Closed5 runs · SEM $465.30
$573.64
48
Gemini 2.5 FlashGoogle · Closed5 runs · SEM $365.69
$548.84
49
Qwen 3.5 35B A3BAlibaba4 runs · SEM $486.47
$462.69
50
Claude Haiku 4.5Anthropic · Closed5 runs · SEM $274.94
$458.89
51
Qwen 3.5 27BAlibaba5 runs · SEM $134.55
$201.98
52
MiniMax-M2MiniMax5 runs · SEM $97.40
$160.60
53
Qwen3 MaxAlibaba · Closed6 runs · SEM $94.52
$71.57
54
Grok 4.3xAI · Closed6 runs · SEM $58.90
$35.26
55
Qwen 3.5 PlusAlibaba5 runs · SEM $20.31
$0.54
56
Qwen3 235B A22B ThinkingAlibaba5 runs · SEM $14.59
$-11.34
57
GPT-OSS 120BOpenAI · Open weight5 runs · SEM $2.87
$-21.53
58
MiniMax M2.5MiniMax · Closed5 runs · SEM $0.28
$-23.16
59
GPT-5 miniOpenAI · Closed5 runs · SEM $5.97
$-31.18

The published Vending-Bench 2 snapshot places Claude Opus 5 first at $11,181.87. The third row is 1562.50 dollars behind. The broader top-10 range is 4661.40 dollars, so the table still separates the published systems.

59 models have been evaluated on Vending-Bench 2. The benchmark falls in the External benchmark mirrors category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Vending-Bench 2 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Vending-Bench 2

Year

2026

Tasks

One year of simulated autonomous business operation per run

Format

Arithmetic mean final bank balance in USD

Difficulty

Long-horizon agent coherence and business operation

Agents begin with $500 and manage product sourcing, supplier negotiation, inventory, pricing, sales, and customer requests over a 365-day simulation. The leaderboard uses arithmetic mean final bank balance by default and publishes row-level standard errors. We keep model configurations separate and exclude the benchmark from weighted rankings.

Vending-Bench 2Public benchmark source

BenchLM freshness & provenance

Version

Vending-Bench 2 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does Vending-Bench 2 measure?

Tests long-horizon coherence by asking agents to operate a simulated vending-machine business for one year.

Which model leads the published Vending-Bench 2 snapshot?

Claude Opus 5 currently leads the published Vending-Bench 2 snapshot with $11,181.87 arithmetic mean final bank balance. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on Vending-Bench 2?

59 AI models are included in BenchLM's mirrored Vending-Bench 2 snapshot, based on the public leaderboard captured on August 14, 2026.

Last updated: August 14, 2026 · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.