Pencil Puzzle Bench
We show this table for reference; we do not rank on it.
A multi-step verifiable reasoning benchmark that evaluates whether models can solve pencil puzzles with unique solutions.
Best solve rate on Pencil Puzzle Bench — November 12, 2026 snapshot
We mirror the published best solve rate view for Pencil Puzzle Bench. Claude Fable 5 leads the public snapshot at 97.6%, followed by GPT-5.5 (83.3%) and GPT-5.4 (70.2%). We do not use these results to rank models overall.
Claude Fable 5
Anthropic
GPT-5.5
OpenAI
GPT-5.4
OpenAI
73 modelsReasoningCurrentDisplay onlyUpdated November 12, 2026 snapshot
Best solve rate table (73 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Fable 5Anthropic | 97.6% | Not reported | Closed |
| 2 | GPT-5.5OpenAI | 83.3% | Not reported | Closed |
| 3 | GPT-5.4OpenAI | 70.2% | Not reported | Closed |
| 4 | GPT-5.2OpenAI | 56.0% | Not reported | Closed |
| 5 | Claude Opus 4.7Anthropic | 50.0% | Not reported | Closed |
| 6 | Gemini 3.5 FlashGoogle | 41.9% | Not reported | Closed |
| 7 | Qwen3.7 MaxAlibaba | 40.0% | Not reported | Closed |
| 8 | GPT-5.2OpenAI | 36.7% | Not reported | Closed |
| 9 | claude-opus-4-6-1mAnthropic | 36.7% | Not reported | Unknown |
| 10 | Claude Opus 4.6 (Adaptive)Anthropic | 33.3% | Not reported | Closed |
| 11 | Gemini 3.1 ProGoogle | 33.3% | Not reported | Closed |
| 12 | Claude Opus 4.6Anthropic | 30.0% | Not reported | Closed |
| 13 | GLM-5.2Z.AI | 26.7% | Not reported | Open |
| 14 | Claude Sonnet 4.6Anthropic | 26.7% | Not reported | Closed |
| 15 | GPT-5.2 ProOpenAI | 26.7% | Not reported | Closed |
| 16 | GPT-5.2OpenAI | 23.3% | Not reported | Closed |
| 17 | Claude Opus 4.6Anthropic | 23.3% | Not reported | Closed |
| 18 | claude-sonnet-4-6-1mAnthropic | 23.3% | Not reported | Unknown |
| 19 | Kimi K2.6Moonshot AI | 20.0% | Not reported | Open |
| 20 | Kimi K2.7 CodeMoonshot AI | 16.7% | Not reported | Open |
| 21 | Qwen3.7 PlusAlibaba | 16.7% | Not reported | Closed |
| 22 | Gemini 3 ProGoogle | 16.7% | Not reported | Closed |
| 23 | Claude Sonnet 4.6Anthropic | 16.7% | Not reported | Closed |
| 24 | Gemini 3 ProGoogle | 13.3% | Not reported | Closed |
| 25 | Gemini 3 ProGoogle | 10.0% | Not reported | Closed |
| 26 | GPT-5.2OpenAI | 10.0% | Not reported | Closed |
| 27 | Qwen3.6 PlusAlibaba | 10.0% | Not reported | Closed |
| 28 | GPT-5.1OpenAI | 7.7% | Not reported | Closed |
| 29 | MiniMax M3MiniMax | 7.1% | Not reported | Open |
| 30 | Claude Opus 4.5 ThinkingAnthropic | 6.7% | Not reported | Closed |
| 31 | Gemini 3 FlashGoogle | 6.7% | Not reported | Closed |
| 32 | Gemini 3 FlashGoogle | 6.7% | Not reported | Closed |
| 33 | Grok 4.20xAI | 6.7% | Not reported | Closed |
| 34 | GPT-5 (high)OpenAI | 6.0% | Not reported | Closed |
| 35 | Kimi K2.5Moonshot AI | 6.0% | Not reported | Open |
| 36 | Grok 4.1 FastxAI | 5.7% | Not reported | Closed |
| 37 | Grok 4.1 Fast (Reasoning)xAI | 5.3% | Not reported | Closed |
| 38 | DeepSeek V4 Pro 0813DeepSeek | 4.0% | Not reported | Open |
| 39 | Grok 4.3xAI | 3.3% | Not reported | Closed |
| 40 | o3OpenAI | 3.3% | Not reported | Closed |
| 41 | nemotron-3-ultra-550b-a55bOther | 3.3% | Not reported | Unknown |
| 42 | MiniMax M2.5MiniMax | 3.3% | Not reported | Closed |
| 43 | Claude Opus 4.5Anthropic | 3.3% | Not reported | Closed |
| 44 | Claude Sonnet 4.5Anthropic | 3.3% | Not reported | Closed |
| 45 | deepseek-v3.2-specialeDeepSeek | 2.3% | Not reported | Unknown |
| 46 | Claude Sonnet 4.5 ThinkingAnthropic | 2.3% | Not reported | Closed |
| 47 | DeepSeek V3.2DeepSeek | 2.0% | Not reported | Open |
| 48 | Grok 4.3xAI | 2.0% | Not reported | Closed |
| 49 | Kimi K2Moonshot AI | 1.3% | Not reported | Closed |
| 50 | MiMo-V2-ProXiaomi | 1.0% | Not reported | Closed |
| 51 | o1OpenAI | 0.7% | Not reported | Closed |
| 52 | MiniMax M2.7MiniMax | 0.7% | Not reported | Open |
| 53 | qwen3.5-397b-a17bAlibaba | 0.7% | Not reported | Unknown |
| 54 | GLM-5Z.AI | 0.7% | Not reported | Open |
| 55 | Gemini 2.5 ProGoogle | 0.3% | Not reported | Closed |
| 56 | GPT-5.2OpenAI | 0.3% | Not reported | Closed |
| 57 | gemma-4-31b-itOther | 0.3% | Not reported | Unknown |
| 58 | minimax-m2.1MiniMax | 0.3% | Not reported | Unknown |
| 59 | GPT-OSS 120BOpenAI | 0.3% | Not reported | Open |
| 60 | qwen3-235b-a22b-thinking-2507Alibaba | 0.3% | Not reported | Unknown |
| 61 | qwen3-next-80b-a3b-thinkingAlibaba | 0.3% | Not reported | Unknown |
| 62 | qwen3-vl-235b-a22b-thinkingAlibaba | 0.3% | Not reported | Unknown |
| 63 | MiMo-V2-FlashXiaomi | 0.3% | Not reported | Open |
| 64 | GLM-4.7Z.AI | 0.3% | Not reported | Open |
| 65 | Grok Code Fast 1xAI | 0.3% | Not reported | Closed |
| 66 | Gemini 3.5 FlashGoogle | 0.0% | Not reported | Closed |
| 67 | gpt-3.5-turboOpenAI | 0.0% | Not reported | Unknown |
| 68 | GPT-4.1OpenAI | 0.0% | Not reported | Closed |
| 69 | GPT-4oOpenAI | 0.0% | Not reported | Closed |
| 70 | devstral-2512Mistral | 0.0% | Not reported | Unknown |
| 71 | mistral-large-2512Mistral | 0.0% | Not reported | Unknown |
| 72 | mistral-small-2603Mistral | 0.0% | Not reported | Unknown |
| 73 | qwen3-coderAlibaba | 0.0% | Not reported | Unknown |
How Pencil Puzzle Bench is shown here
BenchLM mirrors the public Pencil Puzzle Bench leaderboard from November 12, 2026 snapshot. The source benchmark evaluates 51 frontier models on 300 curated puzzles spanning 20 puzzle types, with direct-ask and agentic solve rates reported separately.
Pencil Puzzle Bench is display only on BenchLM. It is a useful multi-step reasoning reference, but the public table mixes direct prompting and agentic runs and exposes variant-specific reasoning settings, so BenchLM keeps it out of weighted model rankings for now.
Snapshot
The published Pencil Puzzle Bench snapshot places Claude Fable 5 first at 97.6%. The third row is 27.4 points behind. The broader top-10 range is 64.3 points, so the table still separates the published systems.
73 models have been evaluated on Pencil Puzzle Bench. The benchmark falls in the Reasoning category. Pencil Puzzle Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Pencil Puzzle Bench
Year
2026
Tasks
300 evaluation puzzles
Format
Direct and agentic puzzle solve rate
Difficulty
Multi-step verifiable reasoning
BenchLM mirrors the public Pencil Puzzle Bench leaderboard as a display-only reasoning benchmark. The public site reports direct-ask and agentic solve rates across a 300-puzzle evaluation selection from the 62,231-puzzle dataset.
Freshness and provenance
Version
Pencil Puzzle Bench 2026
Refresh cadence
Static
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Pencil Puzzle Bench measure?
A multi-step verifiable reasoning benchmark that evaluates whether models can solve pencil puzzles with unique solutions.
Which model leads the published Pencil Puzzle Bench snapshot?
Claude Fable 5 currently leads the published Pencil Puzzle Bench snapshot with 97.6% best solve rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Pencil Puzzle Bench?
The November 12, 2026 snapshot snapshot contains 73 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.