Skip to main content
BenchLM
Data

Vals SRE Bench (formerly ReverseEngBench) (SRE Bench)

We show this table for reference; we do not rank on it.

Data verified

A contamination-resistant agent benchmark for reverse engineering real-world binaries.

Evaluation results published by Vals AI. Published leaderboard aggregates and evaluation settings only. Dataset contents are not included.

SRE Bench score on SRE Bench — October 14, 2026

We mirror the published sre bench score view for SRE Bench. GPT-6 Astra leads the public snapshot at 56.87%, followed by GPT-6.1 Sol (50.76%) and Gemini 4 Argon (44.27%). We do not use these results to rank models overall.

1Closed

GPT-6 Astra

OpenAI

max reasoning

56.87%
Overall 83.88Context 1.05M
2Closed

GPT-6.1 Sol

OpenAI

max reasoning

50.76%
Overall 84.07Context 1.05M
3Closed

Gemini 4 Argon

Google

high reasoning

44.27%
Overall 80.97Context 1M

35 modelsAgenticCurrentDisplay onlyUpdated October 14, 2026

SRE Bench score table (35 models)

SRE Bench score results for SRE Bench
RankModel / configurationScoreParameters (B)Open / closed
1GPT-6 AstraOpenAImax reasoning
56.87%
Not reportedClosed
2GPT-6.1 SolOpenAImax reasoning
50.76%
Not reportedClosed
3Gemini 4 ArgonGooglehigh reasoning
44.27%
Not reportedClosed
4Claude Opus 5.5Anthropic
33.59%
Not reportedClosed
5GPT Daybreak BlueOpenAImax reasoning
31.68%
Not reportedUnknown
6GPT-5.6 SolOpenAImax reasoning
30.53%
Not reportedClosed
7GPT-6 SolOpenAImax reasoning
30.53%
Not reportedClosed
8Claude Sonnet 5.5Anthropic
30.15%
Not reportedClosed
9Claude Fable 5.1Anthropic
22.90%
Not reportedClosed
10Claude Opus 5.5 RangeAnthropic
21.76%
Not reportedUnknown
11Claude Opus 5Anthropic
12.21%
Not reportedClosed
12Muse Wonder FullMetamax reasoning
11.83%
Not reportedClosed
13Gemini 3.8 FlashGooglehigh reasoning
11.07%
Not reportedClosed
14GPT-5.6 TerraOpenAImax reasoning
9.54%
Not reportedClosed
15GLM-5.3Z.AImax reasoning
9.16%
Not reportedOpen
16Claude Haiku 5.5Anthropic
5.72%
Not reportedClosed
17Gemini 3.7 FlashGooglehigh reasoning
4.58%
Not reportedClosed
18MiMo-V2.6-FlashXiaomi
4.20%
Not reportedOpen
19Grok 4.7xAIxhigh reasoning
3.82%
Not reportedClosed
20GPT-5.5OpenAIxhigh reasoning
3.82%
Not reportedClosed
21Muse Spark 1.3 MaxMetamax reasoning
3.82%
Not reportedUnknown
22MiMo-V2.6-ProXiaomi
3.05%
Not reportedOpen
23GPT-6 LunaOpenAImax reasoning
2.67%
Not reportedClosed
24Hy4 previewTencent
2.29%
Not reportedOpen
25DeepSeek V4 Pro 0813DeepSeekmax reasoning
1.91%
Not reportedOpen
26GPT-5.6 LunaOpenAImax reasoning
1.53%
Not reportedClosed
27DeepSeek V4.1 Flash 1017DeepSeekmax reasoning
0.76%
Not reportedOpen
28GLM-5.3-FlashZ.AImax reasoning
0.76%
Not reportedOpen
29Grok 4.5xAIhigh reasoning
0.76%
Not reportedClosed
30Claude Sonnet 5Anthropic
0.38%
Not reportedClosed
31Step 5 PreviewStepFun
0.38%
Not reportedOpen
32GLM-5.2Z.AImax reasoning
0.00%
Not reportedOpen
33Muse Spark 1.2Metaxhigh reasoning
0.00%
Not reportedClosed
34Gemini 3.1 Pro PreviewGooglehigh reasoning
0.00%
Not reportedUnknown
35InklingThinking Machines Lab0.99 reasoning
0.00%
Not reportedOpen

How SRE Bench is shown here

BenchLM mirrors the public Vals AI SRE Bench leaderboard captured from https://www.vals.ai/benchmarks/srebench and updated by Vals on October 14, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.

SRE Bench is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.

Snapshot

35 Vals rows3 task viewsprivate datasetTasks: Overall, Fully Solved, Capability ScoreDisplay only
SRE Bench on Vals AIVals methodologyVals home

The published SRE Bench snapshot places GPT-6 Astra first at 56.87%. The third row is 12.59 points behind. The broader top-10 range is 35.11 points, so the table still separates the published systems.

35 models have been evaluated on SRE Bench. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. SRE Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About SRE Bench

Year

2026

Tasks

Real-world binary reverse-engineering tasks

Format

Fully solved rate and capability score

Difficulty

Agentic reverse engineering

Vals published this board as ReverseEngBench until September 2026 and now lists it as SRE Bench under the same sre_bench benchmark id. The public table reports fully solved and partial-capability scores through an external agent runner. We keep it display only because Vals marks the dataset private and the result measures the complete agent setup.

SRE BenchPublic benchmark source

Freshness and provenance

Version

SRE Bench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does SRE Bench measure?

A contamination-resistant agent benchmark for reverse engineering real-world binaries.

Which model leads the published SRE Bench snapshot?

GPT-6 Astra currently leads the published SRE Bench snapshot with 56.87% sre bench score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on SRE Bench?

The October 14, 2026 snapshot contains 35 AI models.

Last updated: October 14, 2026 · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.