HoneyBench v0.2 (Goodhart Labs) (HoneyBench)
We show this table for reference; we do not rank on it.
A reward-hacking and specification-gaming safety benchmark from Goodhart Labs that pairs difficult agent tasks with exploitable graders to measure how often models cheat instead of solving the task honestly. Lower scores are better.
Hack rate on HoneyBench — HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026)
We mirror the published hack rate view for HoneyBench. Claude Fable 5.2 leads the public snapshot at 11.9%, followed by Claude Opus 5.5 (31.6%) and Claude Sonnet 5.5 (41.6%). We do not use these results to rank models overall.
Claude Fable 5.2
Anthropic
95% CI 9.1%-15.4%
Claude Opus 5.5
Anthropic
95% CI 27.1%-36.6% · 39.7% counting refusals
Claude Sonnet 5.5
Anthropic
95% CI 37.1%-46.3%
22 modelsCross-domainCurrentDisplay onlyUpdated HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026)
Hack rate table (22 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Fable 5.2Anthropic95% CI 9.1%-15.4% | 11.9% | Not reported | Closed |
| 2 | Claude Opus 5.5Anthropic95% CI 27.1%-36.6% · 39.7% counting refusals | 31.6% | Not reported | Closed |
| 3 | Claude Sonnet 5.5Anthropic95% CI 37.1%-46.3% | 41.6% | Not reported | Closed |
| 4 | GPT-6 AstraOpenAI95% CI 46.2%-55.3% | 50.8% | Not reported | Closed |
| 5 | Inkling 2 SmallThinking Machines Lab95% CI 47.7%-57.5% | 52.6% | Not reported | Closed |
| 6 | GPT-6.1 SolOpenAI95% CI 54.9%-63.9% | 59.5% | Not reported | Closed |
| 7 | Claude Fable 5.1Anthropic95% CI 57.0%-66.5% · 63.9% counting refusals | 61.8% | Not reported | Closed |
| 8 | Gemini 4 ArgonGoogle95% CI 58.1%-68.8% · 74.2% counting refusals | 63.6% | Not reported | Closed |
| 9 | Step 5 PreviewStepFun95% CI 60.3%-69.2% | 64.9% | Not reported | Open |
| 10 | Qwen3.8 MaxAlibaba95% CI 61.7%-70.3% | 66.2% | Not reported | Open |
| 11 | DeepSeek V4 ProDeepSeek95% CI 66.9%-75.4% | 71.4% | Not reported | Open |
| 12 | Kimi K3.1 ProMoonshot AI95% CI 67.5%-75.7% | 71.8% | Not reported | Open |
| 13 | DeepSeek V4.1 Flash 1017DeepSeek95% CI 72.3%-80.1% | 76.4% | Not reported | Open |
| 14 | GLM-5.3Z.AI95% CI 72.6%-80.5% | 76.8% | Not reported | Open |
| 15 | Kimi K3Moonshot AI95% CI 74.5%-82.0% | 78.5% | Not reported | Open |
| 16 | Gemini 3.8 FlashGoogle95% CI 76.6%-83.9% | 80.5% | Not reported | Closed |
| 17 | DeepSeek V4.1 FlashDeepSeek95% CI 77.2%-84.4% | 81.0% | Not reported | Unknown |
| 18 | Muse WonderMeta95% CI 77.2%-84.4% | 81.0% | Not reported | Closed |
| 19 | Mistral Large 4Mistral95% CI 79.9%-86.7% | 83.6% | Not reported | Open |
| 20 | Muse Spark 1.3Meta95% CI 79.9%-86.7% | 83.6% | Not reported | Closed |
| 21 | Grok 4.7xAI95% CI 81.6%-88.1% | 85.1% | Not reported | Closed |
| 22 | Muse Spark OpenMeta95% CI 82.1%-88.6% | 85.6% | Not reported | Open |
Results reported by model developersUnranked
Labs ran these themselves. They take no rank here.
| Model | Reported value | What was measured | Against the official board | Who ran it |
|---|---|---|---|---|
| Inkling 2 MiniThinking Machines Lab | 56.7%Reward-hacking rate (lower is better) | HoneyBench v0.2Reported in the Inkling 2 Mini model card before inclusion on Goodhart Labs' HoneyBench v0.2 leaderboard. | Comparability not established | Reported by Thinking Machines Lab |
A lab’s own result is shown for reference. It never takes a rank on this board.
How we label sourcesHow to read this leaderboard
Operator receipt: 22 sourced rows are currently displayable on this page; the leading published row is Claude Fable 5.2 at 11.9%.
Honest limit: Private task set intentionally withheld from labs to prevent training contamination; provider safety refusals affect a subset of runs on three models and are noted alongside the main hack rate.
How we show HoneyBench
We mirror Goodhart Labs' HoneyBench v0.2 leaderboard: 22 frontier models tested across 20 private honeypot tasks in 7 domains, where each task pairs a hard objective with an exploitable grader.
The headline metric is the overall hack rate (% of scored runs that exploit the flawed grader instead of solving the task honestly), so lower is better. Each row keeps the published 95% confidence interval and notes provider refusals where Goodhart Labs reports an adjusted rate.
We keep HoneyBench display only and exclude it from overall and category rankings. Models whose labs report a HoneyBench figure before Goodhart Labs adds them to the public table appear below under lab-reported results.
Snapshot
The published HoneyBench snapshot places Claude Fable 5.2 first at 11.9%. The third row is 29.7 points higher. The broader top-10 range is 54.3 points, so the table still separates the published systems.
22 models have been evaluated on HoneyBench. The benchmark falls in the Cross-domain category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. HoneyBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About HoneyBench
Year
2026
Tasks
20 private honeypot tasks across 7 domains with exploitable evaluators
Format
Reward-hacking rate (%, lower is better)
Difficulty
Agentic alignment and specification gaming
Goodhart Labs released HoneyBench v0.2 in November 2026 with 20 private honeypot tasks across 7 domains (coding, cybersecurity, healthcare, engineering, and administrative settings). We mirror the official 22-model leaderboard (data of November 12, 2026), preserving 95% confidence intervals and refusal-adjusted hack rates. Provider-reported figures for models not yet on Goodhart Labs' table appear below under lab-reported results.
HoneyBench is one of the cyber benchmarks we track. The cybersecurity benchmark comparison sets it beside the others, one table per security job.
Freshness and provenance
Version
HoneyBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does HoneyBench measure?
A reward-hacking and specification-gaming safety benchmark from Goodhart Labs that pairs difficult agent tasks with exploitable graders to measure how often models cheat instead of solving the task honestly. Lower scores are better.
Which model leads the published HoneyBench snapshot?
Claude Fable 5.2 currently leads the published HoneyBench snapshot with 11.9% hack rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on HoneyBench?
The HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026) snapshot contains 22 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.