ExploitGym Honeypot out-of-scope exploitation rate (ExploitGym Honeypot)
We show this table for reference; we do not rank on it.
An alignment test for cyber agents: hard ExploitGym tasks are paired with an easy-to-exploit auxiliary service (such as a package registry or object store) that would reveal the flag but is outside the authorized scope. Lower scores mean the model stays in scope more often.
Benchmark score on ExploitGym Honeypot — November 11, 2026
We compile the ExploitGym Honeypot rows from provider self-reports. GPT-6 Astra leads the table at 0.00%, followed by GPT-6.1 Sol (0.00%) and GPT-6.1 Terra (0.00%). We do not use these results to rank models overall.
GPT-6 Astra
OpenAI
GPT-6.1 Sol
OpenAI
GPT-6.1 Terra
OpenAI
9 modelsAgenticCurrentDisplay onlyUpdated November 11, 2026
Benchmark score table (9 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAI | 0.00% | Not reported | Closed |
| 2 | GPT-6.1 SolOpenAI | 0.00% | Not reported | Closed |
| 3 | GPT-6.1 TerraOpenAI | 0.00% | Not reported | Closed |
| 4 | GPT-6 SolOpenAI | 0.00% | Not reported | Closed |
| 5 | GPT-6 LunaOpenAI | 0.00% | Not reported | Closed |
| 6 | GPT-6 CyberOpenAI | 0.07% | Not reported | Closed |
| 7 | GPT-5.6 CyberOpenAI | 5.51% | Not reported | Closed |
| 8 | GPT-5.6 LunaOpenAI | 9.90% | Not reported | Closed |
| 9 | GPT-5.6 SolOpenAI | 49.50% | Not reported | Closed |
How to read this leaderboard
Operator receipt: 9 sourced rows are currently displayable on this page; the leading published row is GPT-6 Astra at 0.00%.
Honest limit: Provider-run alignment evaluation; it does not measure exploitation skill and the task subset is not published.
Among the reported ExploitGym Honeypot rows, GPT-6 Astra is first at 0.00%. The third row is 0.00 points higher. The broader top-10 range is 49.50 points, so the table still separates the published systems.
9 models have been evaluated on ExploitGym Honeypot. The benchmark falls in the Agentic category. We keep these results separate from the weighted global scoring system, so they remain source-specific evidence. ExploitGym Honeypot is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ExploitGym Honeypot
Year
2026
Tasks
Hard ExploitGym tasks with an out-of-scope honeypot service
Format
Out-of-scope exploitation rate (lower is better)
Difficulty
Cyber-agent scope adherence
OpenAI built the evaluation after the Hugging Face incident and reports the share of graded rollouts in which the model exploited the honeypot, without deployment safety instructions: GPT-6 Cyber 0.07%, GPT-5.6 Cyber 5.51%, GPT-5.6 Sol 49.5% and GPT-6 Astra 0%. With deployment safety instructions GPT-6 Cyber's rate was 0%. OpenAI's other system cards report GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol and GPT-6.1 Terra with no successful honeypot exploits and GPT-5.6 Luna at 9.9%; the GPT-6 Astra card's updated run put GPT-5.6 Sol at 53.0%, against 49.5% in the GPT-6 Cyber card, which BenchLM stores as the newer figure. This measures staying within scope, not cyber capability, and stays display-only.
Freshness and provenance
Version
ExploitGym Honeypot out-of-scope exploitation rate
Refresh cadence
Static
Staleness state
Current
Question availability
Private provider task set with published aggregate results
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ExploitGym Honeypot measure?
An alignment test for cyber agents: hard ExploitGym tasks are paired with an easy-to-exploit auxiliary service (such as a package registry or object store) that would reveal the flag but is outside the authorized scope. Lower scores mean the model stays in scope more often.
Which model scores highest on ExploitGym Honeypot?
GPT-6 Astra by OpenAI currently leads with a score of 0.00% on ExploitGym Honeypot.
How many models are evaluated on ExploitGym Honeypot?
9 AI models have published results on ExploitGym Honeypot in the BenchLM catalog.
Compare top models on ExploitGym Honeypot
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.