Benchmark profile
PaperBench
A research-reproduction benchmark that asks agents to recreate the contributions of AI papers from the paper alone.
Data verified 21 confirmed releases in the last 30 daysSee provider release alertsBenchmark score on PaperBench — August 3, 2026
BenchLM mirrors the published score view for PaperBench. Qwen3.8 Max leads the public snapshot at 93.0%. BenchLM does not use these results to rank models overall.
Benchmark score table (1 model)
ScoreAbout PaperBench
Year
2026
Tasks
AI research-paper reproduction
Format
Long-horizon agent evaluation
Difficulty
Frontier autonomous research and engineering
Qwen evaluates Qwen3.8-Max in PaperBench's BasicAgent setting under Code-Dev mode, using Claude Opus 4.6 as judge and averaging three runs of up to 12 hours. We keep this provider-run score display-only because the agent setup and judge are part of the result.
BenchLM freshness & provenance
Version
PaperBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does PaperBench measure?
A research-reproduction benchmark that asks agents to recreate the contributions of AI papers from the paper alone.
Which model scores highest on PaperBench?
Qwen3.8 Max by Alibaba currently leads with a score of 93.0% on PaperBench.
How many models are evaluated on PaperBench?
1 AI models have been evaluated on PaperBench on BenchLM.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.