Skip to main content
BenchLM
Data

Vals ProgramBench (ProgramBench)

We show this table for reference; we do not rank on it.

Data verified

Can language models rebuild programs from scratch?

Evaluation results published by Vals AI. Published leaderboard aggregates and evaluation settings only. Dataset contents are not included.

Vals ProgramBench mirror score on ProgramBench — November 9, 2026

We mirror the published vals programbench mirror score view for ProgramBench. Claude Fable 5.2 leads the public snapshot at 19.00%, followed by Claude Opus 5.5 (18.50%) and Claude Fable 5.1 (7.00%). We do not use these results to rank models overall.

1Closed

Claude Fable 5.2

Anthropic

19.00%
Overall 93.65Context 1M
2Closed

Claude Opus 5.5

Anthropic

18.50%
Overall 86.62Context 1M
3Closed

Claude Fable 5.1

Anthropic

7.00%
Overall 79.21Context 1M

66 modelsCodingCurrentDisplay onlyUpdated November 9, 2026

Vals ProgramBench mirror score table (66 models)

Vals ProgramBench mirror score results for ProgramBench
RankModel / configurationScoreParameters (B)Open / closed
1Claude Fable 5.2Anthropic
19.00%
Not reportedClosed
2Claude Opus 5.5Anthropic
18.50%
Not reportedClosed
3Claude Fable 5.1Anthropic
7.00%
Not reportedClosed
4Claude Sonnet 5.5Anthropic
6.50%
Not reportedClosed
5GPT-6 AstraOpenAImax reasoning
5.50%
Not reportedClosed
6GPT-6.1 SolOpenAImax reasoning
3.00%
Not reportedClosed
7Claude Opus 5Anthropic
3.00%
Not reportedClosed
8Gemini 4 ArgonGooglehigh reasoning
2.50%
Not reportedClosed
9Claude Haiku 5.5Anthropic
2.50%
Not reportedClosed
10Muse Spark 1.3 MaxMetamax reasoning
2.50%
Not reportedUnknown
11Claude Fable 5Anthropic
2.00%
Not reportedClosed
12GPT-6 SolOpenAImax reasoning
2.00%
Not reportedClosed
13Kimi K3Moonshot AI
2.00%
Not reportedOpen
14GPT-5.6 SolOpenAImax reasoning
1.50%
Not reportedClosed
15GLM-5.3Z.AImax reasoning
1.50%
Not reportedOpen
16Grok 4.6xAIhigh reasoning
1.00%
Not reportedClosed
17Claude Opus 4.8Anthropic
1.00%
Not reportedClosed
18Gemini 3.8 FlashGooglehigh reasoning
1.00%
Not reportedClosed
19Grok 4.7xAIxhigh reasoning
0.50%
Not reportedClosed
20MiMo-V2.6-FlashXiaomi
0.50%
Not reportedOpen
21DeepSeek V4.1 Flash 1017DeepSeekmax reasoning
0.50%
Not reportedOpen
22GPT-5.6 TerraOpenAImax reasoning
0.50%
Not reportedClosed
23Muse Spark 1.3Metaxhigh reasoning
0.50%
Not reportedClosed
24MiMo-V2.6-ProXiaomi
0.50%
Not reportedOpen
25Inkling 2 SmallThinking Machines Lab0.99 reasoning
0.50%
Not reportedClosed
26GPT-5.5OpenAIxhigh reasoning
0.50%
Not reportedClosed
27GLM-5.2Z.AImax reasoning
0.50%
Not reportedOpen
28GPT-6 LunaOpenAImax reasoning
0.50%
Not reportedClosed
29Muse Wonder FullMetamax reasoning
0.50%
Not reportedClosed
30GPT-5.4 2026 03 05 HighOpenAIhigh reasoning
0.50%
Not reportedUnknown
31Muse Spark 1.2Metaxhigh reasoning
0.50%
Not reportedClosed
32Claude Sonnet 4.6Anthropic
0.50%
Not reportedClosed
33Inkling-SmallThinking Machines Lab0.99 reasoning
0.50%
Not reportedOpen
34Hy4 previewTencent
0.00%
Not reportedOpen
35DeepSeek V4 Pro 0813DeepSeekmax reasoning
0.00%
Not reportedOpen
36Claude Sonnet 5Anthropic
0.00%
Not reportedClosed
37Qwen3.8 MaxAlibaba
0.00%
Not reportedOpen
38Gemini 3.7 FlashGooglehigh reasoning
0.00%
Not reportedClosed
39Claude Opus 4.7Anthropic
0.00%
Not reportedClosed
40GPT-5.6 LunaOpenAImax reasoning
0.00%
Not reportedClosed
41DeepSeek V4 Flash 0731DeepSeekhigh reasoning
0.00%
Not reportedOpen
42GPT-5.4OpenAIxhigh reasoning
0.00%
Not reportedClosed
43GLM-5.3-FlashZ.AImax reasoning
0.00%
Not reportedOpen
44Grok 4.5xAIhigh reasoning
0.00%
Not reportedClosed
45GPT-5.4 miniOpenAIxhigh reasoning
0.00%
Not reportedClosed
46Gemini 3.6 FlashGooglehigh reasoning
0.00%
Not reportedClosed
47Muse Spark 1.1Metaxhigh reasoning
0.00%
Not reportedClosed
48Gemini 3.5 FlashGooglehigh reasoning
0.00%
Not reportedClosed
49GLM-5.1Z.AI
0.00%
Not reportedOpen
50Gemini 3.1 Pro PreviewGooglehigh reasoning
0.00%
Not reportedUnknown
51Qwen3.6 PlusAlibaba
0.00%
Not reportedClosed
52Qwen3.8-27BAlibabaxhigh reasoning
0.00%
Not reportedOpen
53Kimi K2.7 CodeMoonshot AI
0.00%
Not reportedOpen
54Kimi K2.6Moonshot AI
0.00%
Not reportedOpen
55Nemotron 3 Ultra 550b A55bNvidia
0.00%
Not reportedUnknown
56DeepSeek V4 Pro 0813DeepSeekmax reasoning
0.00%
Not reportedOpen
57Gemini 3 Flash PreviewGooglehigh reasoning
0.00%
Not reportedUnknown
58Gemini 3.5 Flash-LiteGooglehigh reasoning
0.00%
Not reportedClosed
59Claude Haiku 4.5 ThinkingAnthropic
0.00%
Not reportedClosed
60Grok 4.3xAIhigh reasoning
0.00%
Not reportedClosed
61Nemotron Lightning 3p5 30b A3bFireworks AI
0.00%
Not reportedUnknown
62Laguna XS.2Poolside
0.00%
Not reportedOpen
63InklingThinking Machines Lab0.99 reasoning
0.00%
Not reportedOpen
64Laguna M.1Poolside
0.00%
Not reportedClosed
65Gemini 3.1 Flash Lite PreviewGooglehigh reasoning
0.00%
Not reportedUnknown
66MiniMax M2.7MiniMax
0.00%
Not reportedOpen

How Vals ProgramBench mirror is shown here

BenchLM mirrors the public Vals AI Vals ProgramBench mirror leaderboard captured from https://www.vals.ai/benchmarks/programbench and updated by Vals on November 9, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.

Vals ProgramBench mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.

Snapshot

66 Vals rows4 task viewspublic datasetTasks: Overall, Fully Resolved, Almost Resolved, Raw Pass RateDisplay only
Vals ProgramBench mirror on Vals AIVals methodologyVals home

The published ProgramBench snapshot places Claude Fable 5.2 first at 19.00%. The third row is 12.00 points behind. The broader top-10 range is 16.50 points, so the table still separates the published systems.

66 models have been evaluated on ProgramBench. The benchmark falls in the Coding category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. ProgramBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About ProgramBench

Year

2026

Tasks

Program reconstruction tasks

Format

Accuracy score

Difficulty

Cleanroom software engineering

BenchLM mirrors the public Vals AI ProgramBench leaderboard as display-only external evidence. The captured snapshot preserves overall scores, task-level scores where Vals publishes them, uncertainty, latency, and cost-per-test metadata. It is excluded from BenchLM weighted rankings.

ProgramBenchPublic benchmark source

Freshness and provenance

Version

ProgramBench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does ProgramBench measure?

Can language models rebuild programs from scratch?

Which model leads the published ProgramBench snapshot?

Claude Fable 5.2 currently leads the published ProgramBench snapshot with 19.00% vals programbench mirror score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on ProgramBench?

The November 9, 2026 snapshot contains 66 AI models.

Last updated: November 9, 2026 · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 7,000+ subscribers.

One email each week. Unsubscribe anytime.