PinchBench
We show this table for reference; we do not rank on it.
The public PinchBench snapshot ranks anthropic/claude-opus-4.8-fast first at 93.5%, ahead of Qwen3.7 Max (92.5%) and Claude Opus 4.8 (90.5%) among 61 models. We mirror the table as display-only evidence; it does not affect overall rankings.
An OpenClaw agent benchmark from Kilo that measures successful task completion across standardized real-world agent workflows.
Average success rate on PinchBench — 10/10/2026, 3:05 AM
We mirror the published average success rate view for PinchBench. anthropic/claude-opus-4.8-fast leads the public snapshot at 93.5%, followed by Qwen3.7 Max (92.5%) and Claude Opus 4.8 (90.5%). We do not use these results to rank models overall.
anthropic/claude-opus-4.8-fast
Anthropic
Qwen3.7 Max
Alibaba
Claude Opus 4.8
Anthropic
61 modelsAgenticCurrentDisplay onlyUpdated 10/10/2026, 3:05 AM
Average success rate table (61 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | anthropic/claude-opus-4.8-fastAnthropic | 93.5% | Not reported | Unknown |
| 2 | Qwen3.7 MaxAlibaba | 92.5% | Not reported | Closed |
| 3 | Claude Opus 4.8Anthropic | 90.5% | Not reported | Closed |
| 4 | Nemotron 3.5 Bolt 74B-A9B BF16NVIDIA | 90.4% | Not reported | Open |
| 5 | nvidia/nemotron-3-ultra-550b-a55bnvidia | 89.9% | Not reported | Unknown |
| 6 | MiMo-V2.5Xiaomi | 89.7% | Not reported | Closed |
| 7 | Grok Build 0.1xAI | 88.9% | Not reported | Closed |
| 8 | GPT-5.6 LunaOpenAI | 88.7% | Not reported | Closed |
| 9 | qwen/qwen3.6-flashAlibaba | 88.1% | Not reported | Unknown |
| 10 | Nemotron 3.5 Bolt 74B-A9BNVIDIA | 87.8% | Not reported | Open |
| 11 | MiMo-V2.5-ProXiaomi | 87.5% | Not reported | Closed |
| 12 | GLM-5.2Z.AI | 87.0% | Not reported | Open |
| 13 | nvidia/nemotron-3.5-lightning-30b-a3bnvidia | 86.4% | Not reported | Unknown |
| 14 | GPT-5.6 SolOpenAI | 84.2% | Not reported | Closed |
| 15 | inclusionai/ling-2.6-1tinclusionai | 82.6% | Not reported | Unknown |
| 16 | DeepSeek V4 Flash 0731DeepSeek | 81.7% | Not reported | Open |
| 17 | Gemini 3.1 ProGoogle | 81.0% | Not reported | Closed |
| 18 | Gemini 3.1 Flash-LiteGoogle | 80.5% | Not reported | Closed |
| 19 | Grok 4.20xAI | 80.3% | Not reported | Closed |
| 20 | Step 3.5 FlashStepFun | 79.4% | Not reported | Open |
| 21 | GPT-5.4 miniOpenAI | 79.2% | Not reported | Closed |
| 22 | Kimi K2.7 CodeMoonshot AI | 76.1% | Not reported | Open |
| 23 | Claude Opus 4.7Anthropic | 76.0% | Not reported | Closed |
| 24 | GPT-5.6 TerraOpenAI | 75.9% | Not reported | Closed |
| 25 | GPT-5.4OpenAI | 75.7% | Not reported | Closed |
| 26 | GPT-5.5OpenAI | 75.5% | Not reported | Closed |
| 27 | Grok 4.5xAI | 75.2% | Not reported | Closed |
| 28 | Seed-2.0-LiteByteDance | 75.0% | Not reported | Closed |
| 29 | Gemini 3.5 FlashGoogle | 74.2% | Not reported | Closed |
| 30 | Grok 4.3xAI | 73.7% | Not reported | Closed |
| 31 | Qwen3.6 PlusAlibaba | 72.5% | Not reported | Closed |
| 32 | Gemini 3 FlashGoogle | 72.1% | Not reported | Closed |
| 33 | GLM-5-TurboZ.AI | 71.8% | Not reported | Closed |
| 34 | Claude Opus 4.6Anthropic | 69.9% | Not reported | Closed |
| 35 | sakana/fugu-ultrasakana | 69.5% | Not reported | Unknown |
| 36 | mistralai/devstral-2512Mistral | 69.4% | Not reported | Unknown |
| 37 | GPT-5.4 nanoOpenAI | 69.0% | Not reported | Closed |
| 38 | Claude Haiku 4.5Anthropic | 67.7% | Not reported | Closed |
| 39 | GLM-5V-TurboZ.AI | 67.6% | Not reported | Closed |
| 40 | MiniMax M2.7MiniMax | 66.8% | Not reported | Open |
| 41 | Trinity-Large-PreviewArcee AI | 65.7% | Not reported | Open |
| 42 | mistralai/mistral-small-2603Mistral | 64.6% | Not reported | Unknown |
| 43 | Claude Sonnet 4.6Anthropic | 62.7% | Not reported | Closed |
| 44 | aion-labs/aion-3.0aion-labs | 61.1% | Not reported | Unknown |
| 45 | DeepSeek V4 Pro 0813DeepSeek | 61.1% | Not reported | Open |
| 46 | GLM-5.1Z.AI | 59.9% | Not reported | Open |
| 47 | google/gemma-4-26b-a4b-itGoogle | 56.4% | Not reported | Unknown |
| 48 | Claude Fable 5Anthropic | 54.8% | Not reported | Closed |
| 49 | Kimi K2.5Moonshot AI | 54.6% | Not reported | Open |
| 50 | mistralai/mistral-large-2512Mistral | 54.5% | Not reported | Unknown |
| 51 | Trinity-Large-PreviewArcee AI | 53.0% | Not reported | Open |
| 52 | google/gemma-4-31b-itGoogle | 52.7% | Not reported | Unknown |
| 53 | anthropic/claude-sonnet-4Anthropic | 48.8% | Not reported | Unknown |
| 54 | GPT-OSS 120BOpenAI | 44.8% | Not reported | Open |
| 55 | Nemotron 3 Super 120B A12BNVIDIA | 42.2% | Not reported | Open |
| 56 | Mercury 2Inception | 39.6% | Not reported | Closed |
| 57 | amazon/nova-2-lite-v1amazon | 37.6% | Not reported | Unknown |
| 58 | GPT-OSS 20BOpenAI | 36.3% | Not reported | Open |
| 59 | GPT-5.5 ProOpenAI | 21.4% | Not reported | Closed |
| 60 | Llama 3.1 70B InstructMeta | 10.7% | Not reported | Unknown |
| 61 | Llama 4 ScoutMeta | 3.2% | Not reported | Open |
How PinchBench is shown here
BenchLM mirrors the public PinchBench average-success-rate view using the official snapshot updated on 10/10/2026, 3:05 AM: 61 models and 640 runs. PinchBench grades runs with automated checks plus an LLM judge.
This benchmark is display only on BenchLM. It is excluded from BenchLM overall rankings, category rankings, and weighted scoring. The table below uses average scores only, matching the public PinchBench average view rather than the best-run view.
Snapshot
The published PinchBench snapshot places anthropic/claude-opus-4.8-fast first at 93.5%. The third row is 3.0 points behind. The broader top-10 range is 5.7 points, so many of the published results sit in a relatively narrow band.
61 models have been evaluated on PinchBench. The benchmark falls in the Agentic category. PinchBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About PinchBench
Year
2026
Tasks
23 OpenClaw agent tasks
Format
Average success rate from official runs
Difficulty
Long-horizon agent workflows
PinchBench publishes official OpenClaw runs across 23 tasks and grades results with automated checks plus an LLM judge. BenchLM mirrors the public average-score view as a display-only benchmark.
Freshness and provenance
Version
PinchBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does PinchBench measure?
An OpenClaw agent benchmark from Kilo that measures successful task completion across standardized real-world agent workflows.
Which model leads the published PinchBench snapshot?
anthropic/claude-opus-4.8-fast currently leads the published PinchBench snapshot with 93.5% average success rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on PinchBench?
The 10/10/2026, 3:05 AM snapshot contains 61 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.