Skip to main content
BenchLM
Data

MedXpertQA MM Extended — Mercor evaluation (MedXpertQA MM Extended (Mercor))

We show this table for reference; we do not rank on it.

Medical multiple-choice reasoning over images and clinical details. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Pass@1 on MedXpertQA MM Extended (Mercor) — October 15, 2026 capture

We mirror the published pass@1 view for MedXpertQA MM Extended (Mercor). Opus 5.5 leads the public snapshot at 71.20%, followed by GPT-6 Sol (68.00%) and Fable 5.1 (66.00%). We do not use these results to rank models overall.

1

Opus 5.5

Anthropic

Mercor extended-set run · max reasoning

Published margin ±11.8 · Original set 85.7% ±4.4 · 153 reported samples

71.20%
2

GPT-6 Sol

OpenAI

Mercor extended-set run · max reasoning

Published margin ±12.1 · Original set 81.3% ±5.2 · 153 reported samples

68.00%
3

Fable 5.1

Anthropic

Mercor extended-set run · high reasoning

Published margin ±11.1 · Original set 79.7% ±5.2 · 153 reported samples

66.00%

40 configurationsMultimodal & GroundedDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Pass@1 table (40 configurations)

Pass@1 results for MedXpertQA MM Extended (Mercor)
RankModel / configurationScoreParameters (B)Open / closed
1Opus 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±11.8 · Original set 85.7% ±4.4 · 153 reported samples
71.20%
Not reportedUnknown
2GPT-6 SolOpenAIMercor extended-set run · max reasoningPublished margin ±12.1 · Original set 81.3% ±5.2 · 153 reported samples
68.00%
Not reportedUnknown
3Fable 5.1AnthropicMercor extended-set run · high reasoningPublished margin ±11.1 · Original set 79.7% ±5.2 · 153 reported samples
66.00%
Not reportedUnknown
4Fable 5.1AnthropicMercor extended-set run · max reasoningPublished margin ±11.4 · Original set 83.7% ±4.7 · 153 reported samples
65.40%
Not reportedUnknown
5GPT-5.6 Sol (Pro)OpenAIMercor extended-set run · max reasoningPublished margin ±12.1 · Original set 79.5% ±5.3 · 153 reported samples
64.70%
Not reportedUnknown
6GPT-6.1 SolOpenAIMercor extended-set run · max reasoningPublished margin ±12.1 · Original set 85.7% ±4.7 · 153 reported samples
64.70%
Not reportedUnknown
7Opus 5AnthropicMercor extended-set run · max reasoningPublished margin ±11.8 · Original set 79.8% ±5.2 · 153 reported samples
64.10%
Not reportedUnknown
8Gemini 3.7 FlashGoogleMercor extended-set run · high reasoningPublished margin ±13.4 · Original set 83.5% ±5 · 153 reported samples
62.70%
Not reportedUnknown
9GPT-5.4OpenAIMercor extended-set run · xhigh reasoningPublished margin ±11.8 · Original set 74.2% ±5.7 · 153 reported samples
62.10%
Not reportedUnknown
10GPT-5.5OpenAIMercor extended-set run · xhigh reasoningPublished margin ±11.4 · Original set 76.7% ±5.4 · 153 reported samples
60.80%
Not reportedUnknown
11Gemini 3.1 ProGoogleMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 80.2% ±5.2 · 153 reported samples
60.80%
Not reportedUnknown
12Gemini 3.8 FlashGoogleMercor extended-set run · high reasoningPublished margin ±13.1 · Original set 84.7% ±4.8 · 153 reported samples
60.80%
Not reportedUnknown
13GPT-6 AstraOpenAIMercor extended-set run · max reasoningPublished margin ±13.1 · Original set 86.8% ±4.6 · 153 reported samples
60.80%
Not reportedUnknown
14Muse Spark 1.3MetaMercor extended-set run · max reasoningPublished margin ±10.8 · Original set 76.3% ±5.2 · 153 reported samples
60.10%
Not reportedUnknown
15Muse Spark 1.1MetaMercor extended-set run · xhigh reasoningPublished margin ±11.8 · Original set 77% ±5.4 · 153 reported samples
58.20%
Not reportedUnknown
16Muse Spark 1.2MetaMercor extended-set run · xhigh reasoningPublished margin ±12.1 · Original set 75.7% ±5.3 · 153 reported samples
58.20%
Not reportedUnknown
17Gemini 3.5 FlashGoogleMercor extended-set run · high reasoningPublished margin ±12.7 · Original set 83.5% ±4.9 · 153 reported samples
57.50%
Not reportedUnknown
18Fable 5AnthropicMercor extended-set run · max reasoningPublished margin ±13.1 · Original set 76.2% ±5.7 · 153 reported samples
56.90%
Not reportedUnknown
19GPT-5.6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±13.1 · Original set 70.2% ±5.8 · 153 reported samples
56.20%
Not reportedUnknown
20Sonnet 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±12.7 · Original set 80.7% ±5 · 152 reported samples
55.60%
Not reportedUnknown
21Gemini 3.6 FlashGoogleMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 82.8% ±4.8 · 153 reported samples
54.20%
Not reportedUnknown
22GLM-5.3-FlashZhipuMercor extended-set run · max reasoningPublished margin ±11.8 · Original set 73.7% ±5.5 · 153 reported samples
52.90%
Not reportedUnknown
23Grok 4.7xAIMercor extended-set run · xhigh reasoningPublished margin ±11.4 · Original set 69% ±6.1 · 150 reported samples
52.00%
Not reportedUnknown
24Kimi K2.7 CodeKimiMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 65.7% ±5.7 · 153 reported samples
51.60%
Not reportedUnknown
25Qwen3.8-MaxAlibabaMercor extended-set run · xhigh reasoningPublished margin ±12.4 · Original set 78.3% ±5.2 · 153 reported samples
51.60%
Not reportedUnknown
26Opus 4.8AnthropicMercor extended-set run · max reasoningPublished margin ±12.4 · Original set 72.5% ±5.9 · 153 reported samples
51.60%
Not reportedUnknown
27GPT-5.6 TerraOpenAIMercor extended-set run · max reasoningPublished margin ±12.4 · Original set 75.7% ±5.5 · 153 reported samples
51.00%
Not reportedUnknown
28InklingThinking MachinesMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 60% ±6 · 153 reported samples
51.00%
Not reportedUnknown
29Opus 4.7AnthropicMercor extended-set run · max reasoningPublished margin ±12.4 · Original set 72.8% ±5.7 · 153 reported samples
51.00%
Not reportedUnknown
30Grok 4.6xAIMercor extended-set run · xhigh reasoningPublished margin ±12.4 · Original set 75.2% ±5.7 · 153 reported samples
48.40%
Not reportedUnknown
31DeepSeek-V4.1-FlashDeepSeekMercor extended-set run · max reasoningPublished margin ±12.4 · Original set 63.5% ±5.8 · 153 reported samples
48.40%
Not reportedUnknown
32Haiku 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±11.8 · Original set 67.8% ±5.8 · 153 reported samples
48.40%
Not reportedUnknown
33Kimi K3KimiMercor extended-set run · max reasoningPublished margin ±11.4 · Original set 73.2% ±5.3 · 153 reported samples
47.70%
Not reportedUnknown
34Gemini 3.5 Flash LiteGoogleMercor extended-set run · high reasoningPublished margin ±12.1 · Original set 66.7% ±6 · 153 reported samples
47.70%
Not reportedUnknown
35GPT-6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±13.1 · Original set 64.8% ±6.1 · 153 reported samples
46.40%
Not reportedUnknown
36Qwen3.5AlibabaMercor extended-set runPublished margin ±12.4 · Original set 65.8% ±6 · 153 reported samples
45.10%
Not reportedUnknown
37Sonnet 5AnthropicMercor extended-set run · max reasoningPublished margin ±12.1 · Original set 63% ±6 · 153 reported samples
43.80%
Not reportedUnknown
38Grok 4.5xAIMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 69.5% ±6.1 · 153 reported samples
41.20%
Not reportedUnknown
39Sonnet 4.6AnthropicMercor extended-set run · high reasoningPublished margin ±12.4 · Original set 61.7% ±6.2 · 153 reported samples
41.20%
Not reportedUnknown
40MiniMax-M3MiniMaxMercor extended-set run · high reasoningPublished margin ±10.5 · Original set 59.7% ±6 · 153 reported samples
34.60%
Not reportedUnknown

How to read this Mercor evaluation

We captured Mercor's MedXpertQA MM Extended results on October 15, 2026. The table preserves 40 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

The headline uses the 51 Mercor-built tasks. The paired original-set values refer to the separately listed 200-task public benchmark. Missing baselines remain unreported. We keep the extension and original set separate from their upstream benchmark-owner and provider-run tables.

The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists programmatic multiple-choice answer extraction and a judge of None. Its run-count label is k = 3. Those notes describe the original public benchmark; extension-specific changes are not established by that reference.

Snapshot

Pass@140 configurationsMercor task extensionDisplay only
Mercor MedXpertQA MM Extended resultsMercor APEX benchmark indexMercor methodologyPaired Mercor original-set resultsSource paperUpstream evaluation codeUpstream dataset

The published MedXpertQA MM Extended (Mercor) snapshot places Opus 5.5 first at 71.20%. The third row is 5.20 points behind. The broader top-10 range is 10.40 points, so the table still separates the published systems.

40 configurations are shown for MedXpertQA MM Extended (Mercor). The benchmark falls in the Multimodal & Grounded category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. MedXpertQA MM Extended (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About MedXpertQA MM Extended (Mercor)

Tasks

51 Mercor tasks; 200 tasks in the separately reported original set

Format

Pass@1

Difficulty

Source-specific evaluation

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

MedXpertQA MM Extended evaluation resultsPublic benchmark source

Freshness and provenance

Version

MedXpertQA MM Extended; Mercor Pass@1

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does MedXpertQA MM Extended (Mercor) measure?

Medical multiple-choice reasoning over images and clinical details. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published MedXpertQA MM Extended (Mercor) snapshot?

Opus 5.5 currently leads the published MedXpertQA MM Extended (Mercor) snapshot with 71.20% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on MedXpertQA MM Extended (Mercor)?

The October 15, 2026 capture snapshot contains 40 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.