Skip to main content
BenchLM
Data

HoneyBench v0.2 (Goodhart Labs) (HoneyBench)

We show this table for reference; we do not rank on it.

A reward-hacking and specification-gaming safety benchmark from Goodhart Labs that pairs difficult agent tasks with exploitable graders to measure how often models cheat instead of solving the task honestly. Lower scores are better.

Hack rate on HoneyBench — HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026)

We mirror the published hack rate view for HoneyBench. Claude Fable 5.2 leads the public snapshot at 11.9%, followed by Claude Opus 5.5 (31.6%) and Claude Sonnet 5.5 (41.6%). We do not use these results to rank models overall.

22 modelsCross-domainCurrentDisplay onlyUpdated HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026)

Hack rate table (22 models)

Hack rate results for HoneyBench
RankModel / configurationScoreParameters (B)Open / closed
1Claude Fable 5.2Anthropic95% CI 9.1%-15.4%
11.9%
Not reportedClosed
2Claude Opus 5.5Anthropic95% CI 27.1%-36.6% · 39.7% counting refusals
31.6%
Not reportedClosed
3Claude Sonnet 5.5Anthropic95% CI 37.1%-46.3%
41.6%
Not reportedClosed
4GPT-6 AstraOpenAI95% CI 46.2%-55.3%
50.8%
Not reportedClosed
5Inkling 2 SmallThinking Machines Lab95% CI 47.7%-57.5%
52.6%
Not reportedClosed
6GPT-6.1 SolOpenAI95% CI 54.9%-63.9%
59.5%
Not reportedClosed
7Claude Fable 5.1Anthropic95% CI 57.0%-66.5% · 63.9% counting refusals
61.8%
Not reportedClosed
8Gemini 4 ArgonGoogle95% CI 58.1%-68.8% · 74.2% counting refusals
63.6%
Not reportedClosed
9Step 5 PreviewStepFun95% CI 60.3%-69.2%
64.9%
Not reportedOpen
10Qwen3.8 MaxAlibaba95% CI 61.7%-70.3%
66.2%
Not reportedOpen
11DeepSeek V4 ProDeepSeek95% CI 66.9%-75.4%
71.4%
Not reportedOpen
12Kimi K3.1 ProMoonshot AI95% CI 67.5%-75.7%
71.8%
Not reportedOpen
13DeepSeek V4.1 Flash 1017DeepSeek95% CI 72.3%-80.1%
76.4%
Not reportedOpen
14GLM-5.3Z.AI95% CI 72.6%-80.5%
76.8%
Not reportedOpen
15Kimi K3Moonshot AI95% CI 74.5%-82.0%
78.5%
Not reportedOpen
16Gemini 3.8 FlashGoogle95% CI 76.6%-83.9%
80.5%
Not reportedClosed
17DeepSeek V4.1 FlashDeepSeek95% CI 77.2%-84.4%
81.0%
Not reportedUnknown
18Muse WonderMeta95% CI 77.2%-84.4%
81.0%
Not reportedClosed
19Mistral Large 4Mistral95% CI 79.9%-86.7%
83.6%
Not reportedOpen
20Muse Spark 1.3Meta95% CI 79.9%-86.7%
83.6%
Not reportedClosed
21Grok 4.7xAI95% CI 81.6%-88.1%
85.1%
Not reportedClosed
22Muse Spark OpenMeta95% CI 82.1%-88.6%
85.6%
Not reportedOpen

Results reported by model developersUnranked

Labs ran these themselves. They take no rank here.

HoneyBench results reported by model developers, unranked
Inkling 2 MiniThinking Machines Lab56.7%Reward-hacking rate (lower is better)HoneyBench v0.2Reported in the Inkling 2 Mini model card before inclusion on Goodhart Labs' HoneyBench v0.2 leaderboard.Comparability not establishedReported by Thinking Machines Lab

A lab’s own result is shown for reference. It never takes a rank on this board.

How we label sources

How to read this leaderboard

Operator receipt: 22 sourced rows are currently displayable on this page; the leading published row is Claude Fable 5.2 at 11.9%.

Honest limit: Private task set intentionally withheld from labs to prevent training contamination; provider safety refusals affect a subset of runs on three models and are noted alongside the main hack rate.

How we show HoneyBench

We mirror Goodhart Labs' HoneyBench v0.2 leaderboard: 22 frontier models tested across 20 private honeypot tasks in 7 domains, where each task pairs a hard objective with an exploitable grader.

The headline metric is the overall hack rate (% of scored runs that exploit the flawed grader instead of solving the task honestly), so lower is better. Each row keeps the published 95% confidence interval and notes provider refusals where Goodhart Labs reports an adjusted rate.

We keep HoneyBench display only and exclude it from overall and category rankings. Models whose labs report a HoneyBench figure before Goodhart Labs adds them to the public table appear below under lab-reported results.

Snapshot

22 models20 private tasks7 domainsLower hack rate is betterDisplay only

The published HoneyBench snapshot places Claude Fable 5.2 first at 11.9%. The third row is 29.7 points higher. The broader top-10 range is 54.3 points, so the table still separates the published systems.

22 models have been evaluated on HoneyBench. The benchmark falls in the Cross-domain category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. HoneyBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About HoneyBench

Year

2026

Tasks

20 private honeypot tasks across 7 domains with exploitable evaluators

Format

Reward-hacking rate (%, lower is better)

Difficulty

Agentic alignment and specification gaming

Goodhart Labs released HoneyBench v0.2 in November 2026 with 20 private honeypot tasks across 7 domains (coding, cybersecurity, healthcare, engineering, and administrative settings). We mirror the official 22-model leaderboard (data of November 12, 2026), preserving 95% confidence intervals and refusal-adjusted hack rates. Provider-reported figures for models not yet on Goodhart Labs' table appear below under lab-reported results.

HoneyBench is one of the cyber benchmarks we track. The cybersecurity benchmark comparison sets it beside the others, one table per security job.

Freshness and provenance

Version

HoneyBench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does HoneyBench measure?

A reward-hacking and specification-gaming safety benchmark from Goodhart Labs that pairs difficult agent tasks with exploitable graders to measure how often models cheat instead of solving the task honestly. Lower scores are better.

Which model leads the published HoneyBench snapshot?

Claude Fable 5.2 currently leads the published HoneyBench snapshot with 11.9% hack rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on HoneyBench?

The HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026) snapshot contains 22 AI models.

Last updated: HoneyBench v0.2 data of November 12, 2026 (captured November 20, 2026) · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 7,000+ subscribers.

One email each week. Unsubscribe anytime.