GDPval — Mercor evaluation (GDPval (Mercor))

We show this table for reference; we do not rank on it.

Producing workplace deliverables from reference files. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Mean Score on GDPval (Mercor) — October 15, 2026 capture

We mirror the published mean score view for GDPval (Mercor). Opus 5.5 leads the public snapshot at 88.30%, followed by DeepSeek-V4.1-Flash (86.70%) and GPT-6 Astra (86.70%). We do not use these results to rank models overall.

Opus 5.5

Anthropic

Stirrup · max reasoning

Published margin ±1.5 · 832 reported samples

88.30%

DeepSeek-V4.1-Flash

DeepSeek

Stirrup · max reasoning

Published margin ±1.5 · 852 reported samples

86.70%

GPT-6 Astra

OpenAI

Stirrup · max reasoning

Published margin ±1.7 · 847 reported samples

86.70%

45 configurationsAgenticDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Mean Score table (45 configurations)

Mean Score results for GDPval (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	Opus 5.5AnthropicStirrup · max reasoningPublished margin ±1.5 · 832 reported samples	88.30%	Not reported	Unknown
2	DeepSeek-V4.1-FlashDeepSeekStirrup · max reasoningPublished margin ±1.5 · 852 reported samples	86.70%	Not reported	Unknown
3	GPT-6 AstraOpenAIStirrup · max reasoningPublished margin ±1.7 · 847 reported samples	86.70%	Not reported	Unknown
4	Fable 5.1AnthropicStirrup · max reasoningPublished margin ±1.7 · 851 reported samples	86.60%	Not reported	Unknown
5	Sonnet 5.5AnthropicStirrup · max reasoningPublished margin ±1.7 · 819 reported samples	86.50%	Not reported	Unknown
6	Grok 4.6xAIStirrup · xhigh reasoningPublished margin ±1.6 · 850 reported samples	86.00%	Not reported	Unknown
7	Opus 5AnthropicStirrup · max reasoningPublished margin ±1.9 · 854 reported samples	85.60%	Not reported	Unknown
8	Haiku 5.5AnthropicStirrup · max reasoningPublished margin ±1.6 · 834 reported samples	85.50%	Not reported	Unknown
9	Gemini 3.7 FlashGoogleStirrup · high reasoningPublished margin ±1.7 · 849 reported samples	85.20%	Not reported	Unknown
10	GPT-6 AstraOpenAIStirrup · xhigh reasoningPublished margin ±2.1 · 868 reported samples	84.80%	Not reported	Unknown
11	Opus 4.8AnthropicStirrup · max reasoningPublished margin ±1.7 · 852 reported samples	84.50%	Not reported	Unknown
12	Gemini 3.6 FlashGoogleStirrup · high reasoningPublished margin ±1.7 · 852 reported samples	83.80%	Not reported	Unknown
13	Muse Spark 1.2MetaStirrup · xhigh reasoningPublished margin ±2 · 845 reported samples	83.80%	Not reported	Unknown
14	GPT-6 SolOpenAIStirrup · max reasoningPublished margin ±1.9 · 852 reported samples	83.50%	Not reported	Unknown
15	Gemini 3.8 FlashGoogleStirrup · high reasoningPublished margin ±1.6 · 848 reported samples	83.50%	Not reported	Unknown
16	Qwen3.8-MaxAlibabaStirrup · xhigh reasoningPublished margin ±2 · 852 reported samples	83.30%	Not reported	Unknown
17	Muse Spark 1.1MetaStirrup · xhigh reasoningPublished margin ±2 · 851 reported samples	82.40%	Not reported	Unknown
18	Fable 5.1AnthropicStirrup · high reasoningPublished margin ±2.7 · 852 reported samples	81.40%	Not reported	Unknown
19	GPT-6 LunaOpenAIStirrup · max reasoningPublished margin ±2.2 · 852 reported samples	80.10%	Not reported	Unknown
20	GPT-5.5OpenAIStirrup · xhigh reasoningPublished margin ±5.2 · 852 reported samples	79.90%	Not reported	Unknown
21	Opus 4.7AnthropicStirrup · max reasoningPublished margin ±5.1 · 852 reported samples	79.80%	Not reported	Unknown
22	Sonnet 4.6AnthropicStirrup · high reasoningPublished margin ±5.2 · 852 reported samples	79.00%	Not reported	Unknown
23	Grok 4.5xAIStirrup · high reasoningPublished margin ±5.2 · 852 reported samples	78.80%	Not reported	Unknown
24	Kimi K3KimiStirrup · max reasoningPublished margin ±5.1 · 852 reported samples	78.80%	Not reported	Unknown
25	Fable 5AnthropicStirrup · max reasoningPublished margin ±5.1 · 852 reported samples	78.60%	Not reported	Unknown
26	DeepSeek-V4-FlashDeepSeekStirrup · max reasoningPublished margin ±5 · 852 reported samples	78.50%	Not reported	Unknown
27	GLM-5.2ZhipuStirrup · max reasoningPublished margin ±5.2 · 852 reported samples	78.20%	Not reported	Unknown
28	Sonnet 5AnthropicStirrup · max reasoningPublished margin ±5.2 · 852 reported samples	77.60%	Not reported	Unknown
29	GPT-5.6 TerraOpenAIStirrup · max reasoningPublished margin ±5.2 · 852 reported samples	77.10%	Not reported	Unknown
30	GPT-5.6 Sol (Pro)OpenAIStirrup · max reasoningPublished margin ±5.3 · 852 reported samples	75.60%	Not reported	Unknown
31	MiniMax-M3MiniMaxStirrup · high reasoningPublished margin ±5.3 · 852 reported samples	74.80%	Not reported	Unknown
32	DeepSeek-V4-ProDeepSeekStirrup · max reasoningPublished margin ±5.1 · 852 reported samples	74.30%	Not reported	Unknown
33	GPT-5.6 LunaOpenAIStirrup · max reasoningPublished margin ±5.5 · 852 reported samples	72.50%	Not reported	Unknown
34	Gemini 3.5 FlashGoogleStirrup · high reasoningPublished margin ±5.2 · 852 reported samples	72.40%	Not reported	Unknown
35	DeepSeek-V3.2DeepSeekStirrupPublished margin ±2.1 · 852 reported samples	71.90%	Not reported	Unknown
36	Qwen3.5AlibabaStirrupPublished margin ±2.2 · 851 reported samples	71.00%	Not reported	Unknown
37	Kimi K2.7 CodeKimiStirrup · high reasoningPublished margin ±5.4 · 852 reported samples	70.10%	Not reported	Unknown
38	Gemini 3.1 ProGoogleStirrup · high reasoningPublished margin ±5.5 · 852 reported samples	68.50%	Not reported	Unknown
39	MiniMax-M2.7MiniMaxStirrup · high reasoningPublished margin ±5.3 · 852 reported samples	68.30%	Not reported	Unknown
40	InklingThinking MachinesStirrup · high reasoningPublished margin ±5.6 · 852 reported samples	65.40%	Not reported	Unknown
41	Nemotron 3 UltraNVIDIAStirrup · high reasoningPublished margin ±3.1 · 852 reported samples	63.50%	Not reported	Unknown
42	GPT-5.4OpenAIStirrup · xhigh reasoningPublished margin ±5.7 · 852 reported samples	60.00%	Not reported	Unknown
43	Kimi K2 (Thinking)KimiStirrup · high reasoningPublished margin ±5.6 · 852 reported samples	54.10%	Not reported	Unknown
44	Gemma 4 31BGoogleStirrupPublished margin ±5.8 · 852 reported samples	52.70%	Not reported	Unknown
45	GPT-OSS-120BOpenAIStirrup · high reasoningPublished margin ±5.5 · 852 reported samples	38.80%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's GDPval results on October 15, 2026. The table preserves 45 published configurations, their source identifiers, reasoning settings, and disclosed mean score values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

The headline is the published mean score, which can include partial credit. It must not be read as a strict task-completion rate.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists llm-as-a-judge (lmaaj) and a judge of Gemini 3 Flash Preview. Its run-count label is k = 4. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Mean Score45 configurationsMercor evaluationDisplay only

Mercor GDPval results Mercor APEX benchmark index Mercor methodology Source paper Upstream dataset

The published GDPval (Mercor) snapshot places Opus 5.5 first at 88.30%. The third row is 1.60 points behind. The broader top-10 range is 3.50 points, so many of the published results sit in a relatively narrow band.

45 configurations are shown for GDPval (Mercor). The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. GDPval (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About GDPval (Mercor)

Tasks

213 source-reported tasks

Format

Mean Score

Difficulty

Source-specific evaluation

The selected headline is Mean Score. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

GDPval evaluation results Public benchmark source

Freshness and provenance

Version

GDPval; Mercor Mean Score

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does GDPval (Mercor) measure?

Producing workplace deliverables from reference files. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published GDPval (Mercor) snapshot?

Opus 5.5 currently leads the published GDPval (Mercor) snapshot with 88.30% mean score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on GDPval (Mercor)?

The October 15, 2026 capture snapshot contains 45 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.