HLE Extended — Mercor evaluation (HLE Extended (Mercor))

We show this table for reference; we do not rank on it.

Answering difficult academic questions across many subjects. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Pass@1 on HLE Extended (Mercor) — October 15, 2026 capture

We mirror the published pass@1 view for HLE Extended (Mercor). GPT-5.6 Sol (Pro) leads the public snapshot at 77.30%, followed by GPT-5.6 Terra (70.70%) and Opus 5 (61.00%). We do not use these results to rank models overall.

GPT-5.6 Sol (Pro)

OpenAI

Mercor extended-set run · max reasoning

Published margin ±3.7 · Original set not reported · 1,325 reported samples

77.30%

GPT-5.6 Terra

OpenAI

Mercor extended-set run · max reasoning

Published margin ±3.9 · Original set 44.9% ±2 · 1,341 reported samples

70.70%

Opus 5

Anthropic

Mercor extended-set run · high reasoning

Published margin ±4 · Original set 52% ±2 · 1,341 reported samples

61.00%

31 configurationsKnowledgeDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Pass@1 table (31 configurations)

Pass@1 results for HLE Extended (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	GPT-5.6 Sol (Pro)OpenAIMercor extended-set run · max reasoningPublished margin ±3.7 · Original set not reported · 1,325 reported samples	77.30%	Not reported	Unknown
2	GPT-5.6 TerraOpenAIMercor extended-set run · max reasoningPublished margin ±3.9 · Original set 44.9% ±2 · 1,341 reported samples	70.70%	Not reported	Unknown
3	Opus 5AnthropicMercor extended-set run · high reasoningPublished margin ±4 · Original set 52% ±2 · 1,341 reported samples	61.00%	Not reported	Unknown
4	Fable 5AnthropicMercor extended-set run · high reasoningPublished margin ±4.1 · Original set 51.5% ±2 · 1,341 reported samples	58.70%	Not reported	Unknown
5	GPT-5.6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±4 · Original set 39.6% ±1.9 · 1,319 reported samples	57.50%	Not reported	Unknown
6	GPT-5.4OpenAIMercor extended-set run · xhigh reasoningPublished margin ±3.8 · Original set 44.1% ±2 · 1,341 reported samples	56.10%	Not reported	Unknown
7	GPT-6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±4 · Original set 38.7% ±1.9 · 1,341 reported samples	53.30%	Not reported	Unknown
8	Opus 4.8AnthropicMercor extended-set run · high reasoningPublished margin ±4 · Original set 44.8% ±1.9 · 1,341 reported samples	49.40%	Not reported	Unknown
9	Qwen3.8-MaxAlibabaMercor extended-set run · xhigh reasoningPublished margin ±3.8 · Original set 42.2% ±1.9 · 1,341 reported samples	47.10%	Not reported	Unknown
10	Grok 4.6xAIMercor extended-set run · xhigh reasoningPublished margin ±3.8 · Original set 41.3% ±1.9 · 1,341 reported samples	45.80%	Not reported	Unknown
11	DeepSeek-V4.1-FlashDeepSeekMercor extended-set run · max reasoningPublished margin ±3.7 · Original set 40.5% ±1.9 · 1,339 reported samples	45.30%	Not reported	Unknown
12	Gemini 3.8 FlashGoogleMercor extended-set run · high reasoningPublished margin ±3.9 · Original set 47% ±2 · 1,341 reported samples	36.80%	Not reported	Unknown
13	DeepSeek-V4-FlashDeepSeekMercor extended-set run · max reasoningPublished margin ±3.5 · Original set 39% ±1.8 · 1,340 reported samples	36.10%	Not reported	Unknown
14	Kimi K3KimiMercor extended-set run · max reasoningPublished margin ±3.3 · Original set 46.5% ±2 · 1,341 reported samples	35.00%	Not reported	Unknown
15	Gemini 3.7 FlashGoogleMercor extended-set run · high reasoningPublished margin ±3.5 · Original set 46% ±1.9 · 1,341 reported samples	31.60%	Not reported	Unknown
16	Opus 4.7AnthropicMercor extended-set run · high reasoningPublished margin ±3.5 · Original set 42.2% ±1.9 · 1,341 reported samples	31.20%	Not reported	Unknown
17	Sonnet 5AnthropicMercor extended-set run · high reasoningPublished margin ±3.7 · Original set 36.3% ±1.8 · 1,341 reported samples	29.10%	Not reported	Unknown
18	GLM-5.2ZhipuMercor extended-set run · max reasoningPublished margin ±3.1 · Original set 31.7% ±1.6 · 1,341 reported samples	25.80%	Not reported	Unknown
19	Grok 4.5xAIMercor extended-set run · high reasoningPublished margin ±3.1 · Original set 41.9% ±1.9 · 1,341 reported samples	25.40%	Not reported	Unknown
20	Gemini 3.1 ProGoogleMercor extended-set run · high reasoningPublished margin ±2.6 · Original set 44.1% ±1.9 · 1,341 reported samples	19.60%	Not reported	Unknown
21	Gemini 3.5 FlashGoogleMercor extended-set run · high reasoningPublished margin ±2.5 · Original set 40.6% ±1.9 · 1,341 reported samples	14.80%	Not reported	Unknown
22	Kimi K2.7 CodeKimiMercor extended-set run · high reasoningPublished margin ±2.4 · Original set 34% ±1.8 · 1,341 reported samples	13.70%	Not reported	Unknown
23	Gemini 3.6 FlashGoogleMercor extended-set run · high reasoningPublished margin ±2.3 · Original set 41.2% ±1.9 · 1,341 reported samples	13.30%	Not reported	Unknown
24	Sonnet 4.6AnthropicMercor extended-set run · high reasoningPublished margin ±2.4 · Original set 24.4% ±1.6 · 1,341 reported samples	10.90%	Not reported	Unknown
25	DeepSeek-V3.2DeepSeekMercor extended-set runPublished margin ±1.9 · Original set 22.3% ±1.5 · 1,341 reported samples	8.10%	Not reported	Unknown
26	GPT-OSS-120BOpenAIMercor extended-set run · high reasoningPublished margin ±1.9 · Original set 18.5% ±1.4 · 1,341 reported samples	8.10%	Not reported	Unknown
27	MiniMax-M2.7MiniMaxMercor extended-set run · high reasoningPublished margin ±1.8 · Original set 27.1% ±1.6 · 1,341 reported samples	6.60%	Not reported	Unknown
28	MiniMax-M3MiniMaxMercor extended-set run · high reasoningPublished margin ±1.6 · Original set 37.8% ±1.8 · 1,341 reported samples	5.40%	Not reported	Unknown
29	Qwen3.5AlibabaMercor extended-set runPublished margin ±1.4 · Original set 29.2% ±1.7 · 1,341 reported samples	4.20%	Not reported	Unknown
30	Gemma 4 31BGoogleMercor extended-set runPublished margin ±1.4 · Original set 23.2% ±1.6 · 1,341 reported samples	3.80%	Not reported	Unknown
31	GLM-5 (Thinking)ZhipuMercor extended-set runPublished margin ±0.9 · Original set not reported · 1,341 reported samples	2.00%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's HLE Extended results on October 15, 2026. The table preserves 31 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

The headline uses the 447 Mercor-built tasks. The paired original-set values refer to the separately listed 2158-task public benchmark. Missing baselines remain unreported. We keep the extension and original set separate from their upstream benchmark-owner and provider-run tables.

The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists llm-as-a-judge (lmaaj) and a judge of Gemini 3.5 Flash. Its run-count label is k = 3. Those notes describe the original public benchmark; extension-specific changes are not established by that reference.

Snapshot

Pass@131 configurationsMercor task extensionDisplay only

Mercor HLE Extended results Mercor APEX benchmark index Mercor methodology Paired Mercor original-set results Source paper Upstream evaluation code Upstream dataset

The published HLE Extended (Mercor) snapshot places GPT-5.6 Sol (Pro) first at 77.30%. The third row is 16.30 points behind. The broader top-10 range is 31.50 points, so the table still separates the published systems.

31 configurations are shown for HLE Extended (Mercor). The benchmark falls in the Knowledge category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. HLE Extended (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About HLE Extended (Mercor)

Tasks

447 Mercor tasks; 2158 tasks in the separately reported original set

Format

Pass@1

Difficulty

Source-specific evaluation

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

HLE Extended evaluation results Public benchmark source

Freshness and provenance

Version

HLE Extended; Mercor Pass@1

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does HLE Extended (Mercor) measure?

Answering difficult academic questions across many subjects. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published HLE Extended (Mercor) snapshot?

GPT-5.6 Sol (Pro) currently leads the published HLE Extended (Mercor) snapshot with 77.30% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on HLE Extended (Mercor)?

The October 15, 2026 capture snapshot contains 31 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.