# Pencil Puzzle Bench

> A multi-step verifiable reasoning benchmark that evaluates whether models can solve pencil puzzles with unique solutions.

Canonical page: https://benchlm.ai/benchmarks/ppbench

- Category: [Reasoning](/reasoning)
- Last updated: September 23, 2026 snapshot

## About Pencil Puzzle Bench

- Year: 2026
- Tasks: 300 evaluation puzzles
- Format: Direct and agentic puzzle solve rate
- Difficulty: Multi-step verifiable reasoning
- Paper: [Pencil Puzzle Bench](https://arxiv.org/abs/2603.02119)

BenchLM mirrors the public Pencil Puzzle Bench leaderboard as a display-only reasoning benchmark. The public site reports direct-ask and agentic solve rates across a 300-puzzle evaluation selection from the 62,231-puzzle dataset.

Pencil Puzzle Bench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (73 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Claude Fable 5](/models/claude-fable) | Anthropic | 97.6% |
| 2 | [GPT-5.5](/models/gpt-5-5) | OpenAI | 83.3% |
| 3 | [GPT-5.4](/models/gpt-5-4) | OpenAI | 70.2% |
| 4 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 56.0% |
| 5 | [Claude Opus 4.7](/models/claude-opus-4-7) | Anthropic | 50.0% |
| 6 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Google | 41.9% |
| 7 | [Qwen3.7 Max](/models/qwen3-7-max) | Alibaba | 40.0% |
| 8 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 36.7% |
| 9 | [claude-opus-4-6-1m](https://ppbench.com/model/claude-opus-4-6-1m.html) | Anthropic | 36.7% |
| 10 | [Claude Opus 4.6 (Adaptive)](/models/claude-opus-4-6-thinking) | Anthropic | 33.3% |
| 11 | [Gemini 3.1 Pro](/models/gemini-3-1-pro) | Google | 33.3% |
| 12 | [Claude Opus 4.6](/models/claude-opus-4-6) | Anthropic | 30.0% |
| 13 | [GLM-5.2](/models/glm-5-2) | Z.AI | 26.7% |
| 14 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Anthropic | 26.7% |
| 15 | [GPT-5.2 Pro](/models/gpt-5-2-pro) | OpenAI | 26.7% |
| 16 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 23.3% |
| 17 | [Claude Opus 4.6](/models/claude-opus-4-6) | Anthropic | 23.3% |
| 18 | [claude-sonnet-4-6-1m](https://ppbench.com/model/claude-sonnet-4-6-1m.html) | Anthropic | 23.3% |
| 19 | [Kimi K2.6](/models/kimi-2-6) | Moonshot AI | 20.0% |
| 20 | [Kimi K2.7 Code](/models/kimi-k2-7-code) | Moonshot AI | 16.7% |
| 21 | [Qwen3.7 Plus](/models/qwen3-7-plus) | Alibaba | 16.7% |
| 22 | [Gemini 3 Pro](/models/gemini-3-pro) | Google | 16.7% |
| 23 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Anthropic | 16.7% |
| 24 | [Gemini 3 Pro](/models/gemini-3-pro) | Google | 13.3% |
| 25 | [Gemini 3 Pro](/models/gemini-3-pro) | Google | 10.0% |
| 26 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 10.0% |
| 27 | [Qwen3.6 Plus](/models/qwen3-6-plus) | Alibaba | 10.0% |
| 28 | [GPT-5.1](/models/gpt-5-1) | OpenAI | 7.7% |
| 29 | [MiniMax M3](/models/minimax-m3) | MiniMax | 7.1% |
| 30 | [Claude Opus 4.5 Thinking](/models/claude-opus-4-5-thinking) | Anthropic | 6.7% |
| 31 | [Gemini 3 Flash](/models/gemini-3-flash) | Google | 6.7% |
| 32 | [Gemini 3 Flash](/models/gemini-3-flash) | Google | 6.7% |
| 33 | [Grok 4.20](/models/grok-4-20-beta) | xAI | 6.7% |
| 34 | [GPT-5 (high)](/models/gpt-5-high) | OpenAI | 6.0% |
| 35 | [Kimi K2.5](/models/kimi-k2-5) | Moonshot AI | 6.0% |
| 36 | [Grok 4.1 Fast](/models/grok-4-1-fast) | xAI | 5.7% |
| 37 | [Grok 4.1 Fast (Reasoning)](/models/grok-4-1-fast-reasoning) | xAI | 5.3% |
| 38 | [DeepSeek V4 Pro 0813](/models/deepseek-v4-pro-0813) | DeepSeek | 4.0% |
| 39 | [Grok 4.3](/models/grok-4-3) | xAI | 3.3% |
| 40 | [o3](/models/o3) | OpenAI | 3.3% |
| 41 | [nemotron-3-ultra-550b-a55b](https://ppbench.com/model/nemotron-3-ultra-550b-a55b.html) | Other | 3.3% |
| 42 | [MiniMax M2.5](/models/minimax-m2-5) | MiniMax | 3.3% |
| 43 | [Claude Opus 4.5](/models/claude-opus-4-5) | Anthropic | 3.3% |
| 44 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Anthropic | 3.3% |
| 45 | [deepseek-v3.2-speciale](https://ppbench.com/model/deepseek-v3.2-speciale.html) | DeepSeek | 2.3% |
| 46 | [Claude Sonnet 4.5 Thinking](/models/claude-sonnet-4-5-thinking) | Anthropic | 2.3% |
| 47 | [DeepSeek V3.2](/models/deepseek-v3-2) | DeepSeek | 2.0% |
| 48 | [Grok 4.3](/models/grok-4-3) | xAI | 2.0% |
| 49 | [Kimi K2](/models/kimi-k2) | Moonshot AI | 1.3% |
| 50 | [MiMo-V2-Pro](/models/mimo-v2-pro) | Xiaomi | 1.0% |
| 51 | [o1](/models/o1) | OpenAI | 0.7% |
| 52 | [MiniMax M2.7](/models/minimax-m2-7) | MiniMax | 0.7% |
| 53 | [qwen3.5-397b-a17b](https://ppbench.com/model/qwen3.5-397b-a17b.html) | Alibaba | 0.7% |
| 54 | [GLM-5](/models/glm-5) | Z.AI | 0.7% |
| 55 | [Gemini 2.5 Pro](/models/gemini-2-5-pro) | Google | 0.3% |
| 56 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 0.3% |
| 57 | [gemma-4-31b-it](https://ppbench.com/model/gemma-4-31b-it.html) | Other | 0.3% |
| 58 | [minimax-m2.1](https://ppbench.com/model/minimax-m2.1.html) | MiniMax | 0.3% |
| 59 | [GPT-OSS 120B](/models/gpt-oss-120b) | OpenAI | 0.3% |
| 60 | [qwen3-235b-a22b-thinking-2507](https://ppbench.com/model/qwen3-235b-a22b-thinking-2507.html) | Alibaba | 0.3% |
| 61 | [qwen3-next-80b-a3b-thinking](https://ppbench.com/model/qwen3-next-80b-a3b-thinking.html) | Alibaba | 0.3% |
| 62 | [qwen3-vl-235b-a22b-thinking](https://ppbench.com/model/qwen3-vl-235b-a22b-thinking.html) | Alibaba | 0.3% |
| 63 | [MiMo-V2-Flash](/models/mimo-v2-flash) | Xiaomi | 0.3% |
| 64 | [GLM-4.7](/models/glm-4-7) | Z.AI | 0.3% |
| 65 | [Grok Code Fast 1](/models/grok-code-fast-1) | xAI | 0.3% |
| 66 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Google | 0.0% |
| 67 | [gpt-3.5-turbo](https://ppbench.com/model/gpt-3.5-turbo.html) | OpenAI | 0.0% |
| 68 | [GPT-4.1](/models/gpt-4-1) | OpenAI | 0.0% |
| 69 | [GPT-4o](/models/gpt-4o) | OpenAI | 0.0% |
| 70 | [devstral-2512](https://ppbench.com/model/devstral-2512.html) | Mistral | 0.0% |
| 71 | [mistral-large-2512](https://ppbench.com/model/mistral-large-2512.html) | Mistral | 0.0% |
| 72 | [mistral-small-2603](https://ppbench.com/model/mistral-small-2603.html) | Mistral | 0.0% |
| 73 | [qwen3-coder](https://ppbench.com/model/qwen3-coder.html) | Alibaba | 0.0% |

## FAQ

### What does Pencil Puzzle Bench measure?

A multi-step verifiable reasoning benchmark that evaluates whether models can solve pencil puzzles with unique solutions.

### Which model leads the published Pencil Puzzle Bench snapshot?

Claude Fable 5 currently leads the published Pencil Puzzle Bench snapshot with a score of 97.6%.

### How many models are evaluated on Pencil Puzzle Bench?

The September 23, 2026 snapshot contains 73 AI models.

### Does Pencil Puzzle Bench affect BenchLM's overall score?

Not directly. Pencil Puzzle Bench is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.