SRE-Bench binary reverse engineering (SRE-Bench)
We show this table for reference; we do not rank on it.
A contamination-controlled benchmark that asks agents to reverse engineer compiled binaries without source access and satisfy six task-specific objectives per challenge.
Benchmark score on SRE-Bench — October 29, 2026
We compile the SRE-Bench rows from provider self-reports. GPT-6 Astra leads the table at 88.0%. We do not use these results to rank models overall.
1 modelAgenticCurrentDisplay onlyUpdated October 29, 2026
Benchmark score table (1 model)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAI | 88.0% | Not reported | Closed |
About SRE-Bench
Year
2026
Tasks
262 binary reverse-engineering instances
Format
Fully solved challenge rate (pass@1)
Difficulty
Binary reverse engineering
SRE-Bench contains 262 binary instances derived from 19 privately developed programs spanning network protocols, file formats, malware remediation, and firmware across C, C++, Go, and Rust. A challenge counts as solved only when all six objectives are satisfied. BenchLM stores the single-attempt launch-table value as a display-only external security row and records pass@4 results in provenance notes.
Freshness and provenance
Version
SRE-Bench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SRE-Bench measure?
A contamination-controlled benchmark that asks agents to reverse engineer compiled binaries without source access and satisfy six task-specific objectives per challenge.
Which model scores highest on SRE-Bench?
GPT-6 Astra by OpenAI currently leads with a score of 88.0% on SRE-Bench.
How many models are evaluated on SRE-Bench?
1 AI models have published results on SRE-Bench in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.