# ExploitGym Honeypot out-of-scope exploitation rate (ExploitGym Honeypot)

> An alignment test for cyber agents: hard ExploitGym tasks are paired with an easy-to-exploit auxiliary service (such as a package registry or object store) that would reveal the flag but is outside the authorized scope. Lower scores mean the model stays in scope more often.

Canonical page: https://benchlm.ai/benchmarks/exploitgymhoneypot

- Category: [Agentic](/agentic)
- Last updated: November 11, 2026

## About ExploitGym Honeypot

- Year: 2026
- Tasks: Hard ExploitGym tasks with an out-of-scope honeypot service
- Format: Out-of-scope exploitation rate (lower is better)
- Difficulty: Cyber-agent scope adherence
- Paper: [GPT-6 Cyber: Advancing the frontier of aligned cybersecurity agents](https://openai.com/index/introducing-gpt-6-cyber/)

OpenAI built the evaluation after the Hugging Face incident and reports the share of graded rollouts in which the model exploited the honeypot, without deployment safety instructions: GPT-6 Cyber 0.07%, GPT-5.6 Cyber 5.51%, GPT-5.6 Sol 49.5% and GPT-6 Astra 0%. With deployment safety instructions GPT-6 Cyber's rate was 0%. OpenAI's other system cards report GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol and GPT-6.1 Terra with no successful honeypot exploits and GPT-5.6 Luna at 9.9%; the GPT-6 Astra card's updated run put GPT-5.6 Sol at 53.0%, against 49.5% in the GPT-6 Cyber card, which BenchLM stores as the newer figure. This measures staying within scope, not cyber capability, and stays display-only.

ExploitGym Honeypot is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (9 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [GPT-6 Astra](/models/gpt-6-astra) | OpenAI | 0.00% |
| 2 | [GPT-6.1 Sol](/models/gpt-6-1-sol) | OpenAI | 0.00% |
| 3 | [GPT-6.1 Terra](/models/gpt-6-1-terra) | OpenAI | 0.00% |
| 4 | [GPT-6 Sol](/models/gpt-6-sol) | OpenAI | 0.00% |
| 5 | [GPT-6 Luna](/models/gpt-6-luna) | OpenAI | 0.00% |
| 6 | [GPT-6 Cyber](/models/gpt-6-cyber) | OpenAI | 0.07% |
| 7 | [GPT-5.6 Cyber](/models/gpt-5-6-cyber) | OpenAI | 5.51% |
| 8 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | OpenAI | 9.90% |
| 9 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | OpenAI | 49.50% |

## FAQ

### What does ExploitGym Honeypot measure?

An alignment test for cyber agents: hard ExploitGym tasks are paired with an easy-to-exploit auxiliary service (such as a package registry or object store) that would reveal the flag but is outside the authorized scope. Lower scores mean the model stays in scope more often.

### Which model scores highest on ExploitGym Honeypot?

GPT-6 Astra by OpenAI currently leads with a score of 0.00% on ExploitGym Honeypot.

### How many models are evaluated on ExploitGym Honeypot?

9 AI models have published results on ExploitGym Honeypot in the BenchLM catalog.

### Does ExploitGym Honeypot affect BenchLM's overall score?

Not directly. ExploitGym Honeypot is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Compare Top Models on ExploitGym Honeypot

- [GPT-6 Astra vs GPT-6.1 Sol](/compare/gpt-6-1-sol-vs-gpt-6-astra)
- [GPT-6.1 Sol vs GPT-6.1 Terra](/compare/gpt-6-1-sol-vs-gpt-6-1-terra)
- [GPT-6.1 Terra vs GPT-6 Sol](/compare/gpt-6-1-terra-vs-gpt-6-sol)
- [GPT-6 Sol vs GPT-6 Luna](/compare/gpt-6-luna-vs-gpt-6-sol)