Vending-Bench 2
We show this table for reference; we do not rank on it.
Tests long-horizon coherence by asking agents to operate a simulated vending-machine business for one year.
Arithmetic mean final bank balance on Vending-Bench 2 — November 12, 2026
We mirror the published arithmetic mean final bank balance view for Vending-Bench 2. Gemini 4 Carbon leads the public snapshot at $16,484.21, followed by GPT-6 Astra ($15,514.70) and GPT-6 Sol ($14,427.85). We do not use these results to rank models overall.
Gemini 4 Carbon
6 runs · SEM $1,096.46
GPT-6 Astra
OpenAI
6 runs · SEM $1,074.48
GPT-6 Sol
OpenAI
6 runs · SEM $1,050.68
74 modelsAgenticCurrentDisplay onlyUpdated November 12, 2026
Arithmetic mean final bank balance table (74 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Gemini 4 CarbonGoogle6 runs · SEM $1,096.46 | $16,484.21 | Not reported | Closed |
| 2 | GPT-6 AstraOpenAI6 runs · SEM $1,074.48 | $15,514.70 | Not reported | Closed |
| 3 | GPT-6 SolOpenAI6 runs · SEM $1,050.68 | $14,427.85 | Not reported | Closed |
| 4 | Gemini 4 ArgonGoogle6 runs · SEM $3,100.01 | $13,718.16 | Not reported | Closed |
| 5 | GPT-6.1 SolOpenAI6 runs · SEM $1,043.09 | $12,860.61 | Not reported | Closed |
| 6 | Muse Wonder FullMeta6 runs · SEM $884.77 | $12,291.56 | Not reported | Closed |
| 7 | Claude Opus 5Anthropic6 runs · SEM $2,093.58 | $11,181.87 | Not reported | Closed |
| 8 | Claude Sonnet 5.5Anthropic6 runs · SEM $1,585.34 | $11,165.28 | Not reported | Closed |
| 9 | Claude Opus 4.7Anthropic6 runs · SEM $1,181.31 | $10,936.76 | Not reported | Closed |
| 10 | Grok 4.7xAI6 runs · SEM $651.73 | $10,536.83 | Not reported | Closed |
| 11 | GPT-5.6 SolOpenAI5 runs · SEM $1,337.80 | $9,619.37 | Not reported | Closed |
| 12 | Muse WonderMeta6 runs · SEM $965.94 | $9,432.18 | Not reported | Closed |
| 13 | Claude Opus 5.5Anthropic6 runs · SEM $784.72 | $9,235.25 | Not reported | Closed |
| 14 | GPT-6.1 TerraOpenAI6 runs · SEM $767.65 | $9,107.46 | Not reported | Closed |
| 15 | Grok 4.6xAI4 runs · SEM $1,603.96 | $9,047.03 | Not reported | Closed |
| 16 | GLM-5.2Z.AI6 runs · SEM $1,084.11 | $8,313.78 | Not reported | Open |
| 17 | GLM-5.3Z.AI6 runs · SEM $786.92 | $8,163.61 | Not reported | Open |
| 18 | Claude Opus 4.6Anthropic5 runs · SEM $1,366.99 | $8,017.59 | Not reported | Closed |
| 19 | GPT-5.5OpenAI5 runs · SEM $1,345.98 | $7,523.84 | Not reported | Closed |
| 20 | GPT-5.6 TerraOpenAI5 runs · SEM $373.27 | $7,343.21 | Not reported | Closed |
| 21 | Claude Sonnet 4.6Anthropic5 runs · SEM $721.98 | $7,204.14 | Not reported | Closed |
| 22 | Muse Spark 1.1Meta6 runs · SEM $1,016.33 | $6,520.47 | Not reported | Closed |
| 23 | Claude Sonnet 5Anthropic6 runs · SEM $783.74 | $6,377.70 | Not reported | Closed |
| 24 | Kimi K2.6Moonshot AI5 runs · SEM $660.87 | $6,204.57 | Not reported | Open |
| 25 | GPT-5.4OpenAI5 runs · SEM $871.10 | $6,144.18 | Not reported | Closed |
| 26 | GPT-5.3 CodexOpenAI5 runs · SEM $795.98 | $5,940.12 | Not reported | Closed |
| 27 | Claude Opus 4.8AnthropicHigh reasoning · 5 runs · SEM $830.67 | $5,787.43 | Not reported | Closed |
| 28 | Claude Fable 5AnthropicHigh reasoning · 5 runs · SEM $1,074.93 | $5,680.26 | Not reported | Closed |
| 29 | GLM-5.1Z.AI5 runs · SEM $333.62 | $5,634.41 | Not reported | Open |
| 30 | Gemini 3 ProGoogle5 runs · SEM $905.03 | $5,478.16 | Not reported | Closed |
| 31 | Claude Fable 5.1Anthropic6 runs · SEM $1,244.44 | $5,421.56 | Not reported | Closed |
| 32 | Gemini 3.5 FlashGoogle6 runs · SEM $1,058.73 | $5,396.42 | Not reported | Closed |
| 33 | Kimi K3Moonshot AIMoonshot endpoint · 6 runs · SEM $691.16 | $5,165.04 | Not reported | Open |
| 34 | Qwen 3.6 PlusAlibaba5 runs · SEM $1,097.12 | $5,114.87 | Not reported | Unknown |
| 35 | Gemini 3.8 FlashGoogle6 runs · SEM $490.16 | $5,093.79 | Not reported | Closed |
| 36 | Kimi K2.7 CodeMoonshot AI5 runs · SEM $1,069.37 | $5,082.94 | Not reported | Open |
| 37 | Claude Fable 5AnthropicLow reasoning · 5 runs · SEM $1,225.23 | $5,018.52 | Not reported | Closed |
| 38 | Claude Opus 4.5Anthropic6 runs · SEM $979.25 | $4,967.06 | Not reported | Closed |
| 39 | Claude Fable 5AnthropicMax reasoning · 5 runs · SEM $953.63 | $4,966.64 | Not reported | Closed |
| 40 | Kimi K3Moonshot AIFireworks endpoint · 6 runs · SEM $1,652.42 | $4,906.96 | Not reported | Open |
| 41 | Grok 4.20xAI6 runs · SEM $1,276.33 | $4,662.85 | Not reported | Closed |
| 42 | Claude Fable 5AnthropicNone reasoning · 5 runs · SEM $662.39 | $4,529.94 | Not reported | Closed |
| 43 | GLM-5Z.AI5 runs · SEM $393.55 | $4,432.12 | Not reported | Open |
| 44 | Claude Fable 5AnthropicMedium reasoning · 4 runs · SEM $1,426.83 | $4,339.81 | Not reported | Closed |
| 45 | Qwen 3.6 Max (preview)Alibaba6 runs · SEM $1,469.56 | $4,254.19 | Not reported | Closed |
| 46 | GPT-5.6 LunaOpenAI5 runs · SEM $1,344.16 | $4,094.71 | Not reported | Closed |
| 47 | Grok 4.5xAI5 runs · SEM $435.28 | $3,887.43 | Not reported | Closed |
| 48 | Claude Sonnet 4.5Anthropic5 runs · SEM $1,102.10 | $3,838.74 | Not reported | Closed |
| 49 | Gemini 3.1 ProGoogleCustom tools · 5 runs · SEM $1,241.57 | $3,774.25 | Not reported | Closed |
| 50 | Gemini 3 FlashGoogle5 runs · SEM $639.34 | $3,634.72 | Not reported | Closed |
| 51 | Claude Haiku 5.5Anthropic6 runs · SEM $690.64 | $3,602.46 | Not reported | Closed |
| 52 | GPT-5.2OpenAI5 runs · SEM $392.08 | $3,591.33 | Not reported | Closed |
| 53 | DeepSeek V4 Pro 0813DeepSeek6 runs · SEM $570.70 | $3,284.52 | Not reported | Open |
| 54 | Claude Opus 4.8AnthropicMax reasoning · 5 runs · SEM $722.02 | $2,992.34 | Not reported | Closed |
| 55 | GLM-4.7Z.AI5 runs · SEM $620.60 | $2,376.82 | Not reported | Open |
| 56 | MiniMax M3MiniMax6 runs · SEM $1,151.04 | $2,157.77 | Not reported | Open |
| 57 | GPT-5.1OpenAI5 runs · SEM $756.17 | $1,473.43 | Not reported | Closed |
| 58 | Kimi K2.5Moonshot AI5 runs · SEM $596.70 | $1,198.46 | Not reported | Open |
| 59 | Grok 4.1 FastxAI5 runs · SEM $457.65 | $1,106.63 | Not reported | Closed |
| 60 | DeepSeek V3.2DeepSeek5 runs · SEM $503.15 | $1,034.00 | Not reported | Open |
| 61 | Gemini 3.1 ProGoogle5 runs · SEM $652.55 | $911.21 | Not reported | Closed |
| 62 | Gemini 2.5 ProGoogle5 runs · SEM $465.30 | $573.64 | Not reported | Closed |
| 63 | Gemini 2.5 FlashGoogle5 runs · SEM $365.69 | $548.84 | Not reported | Closed |
| 64 | Qwen 3.5 35B A3BAlibaba4 runs · SEM $486.47 | $462.69 | Not reported | Unknown |
| 65 | Claude Haiku 4.5Anthropic5 runs · SEM $274.94 | $458.89 | Not reported | Closed |
| 66 | Qwen 3.5 27BAlibaba5 runs · SEM $134.55 | $201.98 | Not reported | Unknown |
| 67 | MiniMax-M2MiniMax5 runs · SEM $97.40 | $160.60 | Not reported | Unknown |
| 68 | Qwen3 MaxAlibaba6 runs · SEM $94.52 | $71.57 | Not reported | Closed |
| 69 | Grok 4.3xAI6 runs · SEM $58.90 | $35.26 | Not reported | Closed |
| 70 | Qwen 3.5 PlusAlibaba5 runs · SEM $20.31 | $0.54 | Not reported | Unknown |
| 71 | Qwen3 235B-A22B (Reasoning)Alibaba5 runs · SEM $14.59 | $-11.34 | Not reported | Open |
| 72 | GPT-OSS 120BOpenAI5 runs · SEM $2.87 | $-21.53 | Not reported | Open |
| 73 | MiniMax M2.5MiniMax5 runs · SEM $0.28 | $-23.16 | Not reported | Closed |
| 74 | GPT-5 miniOpenAI5 runs · SEM $5.97 | $-31.18 | Not reported | Closed |
How we show Vending-Bench 2
We mirror Andon Labs' arithmetic-mean Vending-Bench 2 table. The current source contains 74 model-configuration rows and 397 completed runs, with 4 to 6 runs behind each row. The table ranks final bank balance after a one-year simulation and keeps each row's standard error attached.
Agents start with $500, source products, negotiate with suppliers, stock a simulated vending machine, and handle sales and customer messages. The snapshot also retains the publisher's geometric mean, but this page follows the source table's default arithmetic-mean ordering. Reasoning settings, custom-tool runs, and provider endpoints remain separate configurations.
Snapshot
The published Vending-Bench 2 snapshot places Gemini 4 Carbon first at $16,484.21. The third row is 2056.36 score units behind. The broader top-10 range is 5947.38 score units, so the table still separates the published systems.
74 models have been evaluated on Vending-Bench 2. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Vending-Bench 2 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Vending-Bench 2
Year
2026
Tasks
One year of simulated autonomous business operation per run
Format
Arithmetic mean final bank balance in USD
Difficulty
Long-horizon agent coherence and business operation
Agents begin with $500 and manage product sourcing, supplier negotiation, inventory, pricing, sales, and customer requests over a 365-day simulation. The leaderboard uses arithmetic mean final bank balance by default and publishes row-level standard errors. We keep model configurations separate and exclude the benchmark from weighted rankings.
Freshness and provenance
Version
Vending-Bench 2 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Vending-Bench 2 measure?
Tests long-horizon coherence by asking agents to operate a simulated vending-machine business for one year.
Which model leads the published Vending-Bench 2 snapshot?
Gemini 4 Carbon currently leads the published Vending-Bench 2 snapshot with $16,484.21 arithmetic mean final bank balance. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Vending-Bench 2?
The November 12, 2026 snapshot contains 74 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.