Benchmark profile
ProgramBench hidden-test pass rate after episode 1 (ProgramBench (episode 1))
Program-reconstruction hidden-test pass rate after the first of five sequential long-context episodes.
Data verifiedBenchmark score on ProgramBench (episode 1) — July 24, 2026
BenchLM mirrors the published score view for ProgramBench (episode 1). Claude Opus 5 leads the public snapshot at 83.0%. BenchLM does not use these results to rank models overall.
Benchmark score table (1 model)
ScoreAbout ProgramBench (episode 1)
Year
2026
Tasks
166 golden program-reconstruction tasks
Format
Hidden-test pass rate after episode 1
Difficulty
Long-context clean-room software engineering
Anthropic evaluated 166 golden tasks with no internet or decompilation tools and a fresh context budget of up to one million tokens per episode. This is kept separate from the episode-5 result.
BenchLM freshness & provenance
Version
ProgramBench (episode 1) 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does ProgramBench (episode 1) measure?
Program-reconstruction hidden-test pass rate after the first of five sequential long-context episodes.
Which model scores highest on ProgramBench (episode 1)?
Claude Opus 5 by Anthropic currently leads with a score of 83.0% on ProgramBench (episode 1).
How many models are evaluated on ProgramBench (episode 1)?
1 AI models have been evaluated on ProgramBench (episode 1) on BenchLM.