BIG-Bench Hard (BBH)
We show this table for reference; we do not rank on it.
A suite of 23 challenging tasks from the BIG-Bench collaborative benchmark where prior language models failed to exceed average human performance, even with chain-of-thought prompting.
Benchmark score on BBH — October 29, 2026
We compile the BBH rows from provider self-reports. DeepSeek V4 Pro Base leads the table at 87.5%, followed by DeepSeek V4 Flash Base (86.9%) and Claude 3.5 Haiku (86.6%). We do not use these results to rank models overall.
DeepSeek V4 Pro Base
DeepSeek
DeepSeek V4 Flash Base
DeepSeek
Claude 3.5 Haiku
Anthropic
10 modelsReasoningStaleSaturatedDisplay onlyUpdated October 29, 2026
Benchmark score table (10 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | DeepSeek V4 Pro BaseDeepSeek | 87.5% | Not reported | Open |
| 2 | DeepSeek V4 Flash BaseDeepSeek | 86.9% | Not reported | Open |
| 3 | Claude 3.5 HaikuAnthropic | 86.6% | Not reported | Closed |
| 4 | Soofi S 30B-A3BSoofi Project | 78.8% | Not reported | Open |
| 5 | MiniCPM5-1BOpenBMB | 71.9% | Not reported | Open |
| 6 | Phi-3.5-miniMicrosoft | 69% | Not reported | Open |
| 7 | Granite-4.0-1BIBM | 60.4% | Not reported | Open |
| 8 | Granite-4.0-H-1BIBM | 59.7% | Not reported | Open |
| 9 | Granite-4.0-H-350MIBM | 33.3% | Not reported | Open |
| 10 | Granite-4.0-350MIBM | 33.1% | Not reported | Open |
Among the reported BBH rows, DeepSeek V4 Pro Base is first at 87.5%. The third row is 0.9 points behind. The broader top-10 range is 54.4 points, so the table still separates the published systems.
10 models have been evaluated on BBH. The benchmark falls in the Reasoning category. BBH is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About BBH
Year
2022
Tasks
23 tasks
Format
Mixed reasoning tasks
Difficulty
Advanced reasoning
BBH focuses on 23 tasks from BIG-Bench that remain challenging for language models. Tasks include logical deduction, tracking shuffled objects, causal judgement, and other complex reasoning scenarios.
Freshness and provenance
Version
BBH 2022
Refresh cadence
Static
Staleness state
Stale
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does BBH measure?
A suite of 23 challenging tasks from the BIG-Bench collaborative benchmark where prior language models failed to exceed average human performance, even with chain-of-thought prompting.
Which model scores highest on BBH?
DeepSeek V4 Pro Base by DeepSeek currently leads with a score of 87.5% on BBH.
How many models are evaluated on BBH?
10 AI models have published results on BBH in the BenchLM catalog.
Compare top models on BBH
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.