# xdev-bench-hard (Internal, pass@3) (xdev-bench-hard)

> Moonshot AI's internal benchmark of expert-curated, source-code-based vulnerability discovery and exploitation tasks, most reproducing recently disclosed vulnerabilities with mitigations enabled.

Canonical page: https://benchlm.ai/benchmarks/xdevbenchhard

- Category: [Agentic](/agentic)
- Last updated: October 29, 2026

## About xdev-bench-hard

- Year: 2026
- Tasks: Vulnerability exploitation tasks
- Format: Pass@3
- Difficulty: Offensive security research
- Paper: [Kimi K3.1 Pro Tech Blog: Advancing Open Intelligence](https://www.kimi.ai/blog/kimi-k3-1-pro)

Each task provides vulnerable source code, a short bug hint, and the binary; the agent must achieve arbitrary code execution in a separate sandbox and retrieve a secret string. Moonshot reports pass@3 over a random subset of runs with no hard timeouts or output limits. BenchLM stores the provider-published internal benchmark value as display-only launch evidence and does not treat it as an independently reproducible score.

xdev-bench-hard is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (1 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Kimi K3.1 Pro](/models/kimi-3-1-pro) | Moonshot AI | 81.0% |

## FAQ

### What does xdev-bench-hard measure?

Moonshot AI's internal benchmark of expert-curated, source-code-based vulnerability discovery and exploitation tasks, most reproducing recently disclosed vulnerabilities with mitigations enabled.

### Which model scores highest on xdev-bench-hard?

Kimi K3.1 Pro by Moonshot AI currently leads with a score of 81.0% on xdev-bench-hard.

### How many models are evaluated on xdev-bench-hard?

1 AI models have published results on xdev-bench-hard in the BenchLM catalog.

### Does xdev-bench-hard affect BenchLM's overall score?

Not directly. xdev-bench-hard is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.