MedXpertQA MM — Mercor evaluation (MedXpertQA MM (Mercor))

We show this table for reference; we do not rank on it.

Medical multiple-choice reasoning over images and clinical details. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Pass@1 on MedXpertQA MM (Mercor) — October 15, 2026 capture

We mirror the published pass@1 view for MedXpertQA MM (Mercor). GPT-6 Astra leads the public snapshot at 86.80%, followed by GPT-6 Astra (86.50%) and Opus 5.5 (85.70%). We do not use these results to rank models overall.

GPT-6 Astra

OpenAI

Single-shot multimodal · max reasoning

Published margin ±4.6 · 600 reported samples

86.80%

GPT-6 Astra

OpenAI

Single-shot multimodal · xhigh reasoning

Published margin ±4.4 · 600 reported samples

86.50%

Opus 5.5

Anthropic

Single-shot multimodal · max reasoning

Published margin ±4.4 · 600 reported samples

85.70%

42 configurationsMultimodal & GroundedDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Pass@1 table (42 configurations)

Pass@1 results for MedXpertQA MM (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	GPT-6 AstraOpenAISingle-shot multimodal · max reasoningPublished margin ±4.6 · 600 reported samples	86.80%	Not reported	Unknown
2	GPT-6 AstraOpenAISingle-shot multimodal · xhigh reasoningPublished margin ±4.4 · 600 reported samples	86.50%	Not reported	Unknown
3	Opus 5.5AnthropicSingle-shot multimodal · max reasoningPublished margin ±4.4 · 600 reported samples	85.70%	Not reported	Unknown
4	GPT-6.1 SolOpenAISingle-shot multimodal · max reasoningPublished margin ±4.7 · 600 reported samples	85.70%	Not reported	Unknown
5	Gemini 3.8 FlashGoogleSingle-shot multimodal · high reasoningPublished margin ±4.8 · 600 reported samples	84.70%	Not reported	Unknown
6	Fable 5.1AnthropicSingle-shot multimodal · max reasoningPublished margin ±4.7 · 600 reported samples	83.70%	Not reported	Unknown
7	Gemini 3.5 FlashGoogleSingle-shot multimodal · high reasoningPublished margin ±4.9 · 600 reported samples	83.50%	Not reported	Unknown
8	Gemini 3.7 FlashGoogleSingle-shot multimodal · high reasoningPublished margin ±5 · 600 reported samples	83.50%	Not reported	Unknown
9	Gemini 3.6 FlashGoogleSingle-shot multimodal · high reasoningPublished margin ±4.8 · 600 reported samples	82.80%	Not reported	Unknown
10	GPT-6 SolOpenAISingle-shot multimodal · max reasoningPublished margin ±5.2 · 600 reported samples	81.30%	Not reported	Unknown
11	Sonnet 5.5AnthropicSingle-shot multimodal · max reasoningPublished margin ±5 · 598 reported samples	80.70%	Not reported	Unknown
12	Gemini 3.1 ProGoogleSingle-shot multimodal · high reasoningPublished margin ±5.2 · 600 reported samples	80.20%	Not reported	Unknown
13	Opus 5AnthropicSingle-shot multimodal · max reasoningPublished margin ±5.2 · 600 reported samples	79.80%	Not reported	Unknown
14	Fable 5.1AnthropicSingle-shot multimodal · high reasoningPublished margin ±5.2 · 600 reported samples	79.70%	Not reported	Unknown
15	GPT-5.6 Sol (Pro)OpenAISingle-shot multimodal · max reasoningPublished margin ±5.3 · 600 reported samples	79.50%	Not reported	Unknown
16	Qwen3.8-MaxAlibabaSingle-shot multimodal · xhigh reasoningPublished margin ±5.2 · 600 reported samples	78.30%	Not reported	Unknown
17	Muse Spark 1.1MetaSingle-shot multimodal · xhigh reasoningPublished margin ±5.4 · 600 reported samples	77.00%	Not reported	Unknown
18	GPT-5.5OpenAISingle-shot multimodal · xhigh reasoningPublished margin ±5.4 · 600 reported samples	76.70%	Not reported	Unknown
19	Muse Spark 1.3MetaSingle-shot multimodal · max reasoningPublished margin ±5.2 · 600 reported samples	76.30%	Not reported	Unknown
20	Fable 5AnthropicSingle-shot multimodal · max reasoningPublished margin ±5.7 · 600 reported samples	76.20%	Not reported	Unknown
21	GPT-5.6 TerraOpenAISingle-shot multimodal · max reasoningPublished margin ±5.5 · 600 reported samples	75.70%	Not reported	Unknown
22	Muse Spark 1.2MetaSingle-shot multimodal · xhigh reasoningPublished margin ±5.3 · 600 reported samples	75.70%	Not reported	Unknown
23	Grok 4.6xAISingle-shot multimodal · xhigh reasoningPublished margin ±5.7 · 600 reported samples	75.20%	Not reported	Unknown
24	GPT-5.4OpenAISingle-shot multimodal · xhigh reasoningPublished margin ±5.7 · 600 reported samples	74.20%	Not reported	Unknown
25	GLM-5.3-FlashZhipuSingle-shot multimodal · max reasoningPublished margin ±5.5 · 600 reported samples	73.70%	Not reported	Unknown
26	Kimi K3KimiSingle-shot multimodal · max reasoningPublished margin ±5.3 · 600 reported samples	73.20%	Not reported	Unknown
27	Opus 4.7AnthropicSingle-shot multimodal · max reasoningPublished margin ±5.7 · 600 reported samples	72.80%	Not reported	Unknown
28	Opus 4.8AnthropicSingle-shot multimodal · max reasoningPublished margin ±5.9 · 600 reported samples	72.50%	Not reported	Unknown
29	GPT-5.6 LunaOpenAISingle-shot multimodal · max reasoningPublished margin ±5.8 · 600 reported samples	70.20%	Not reported	Unknown
30	Grok 4.5xAISingle-shot multimodal · high reasoningPublished margin ±6.1 · 600 reported samples	69.50%	Not reported	Unknown
31	Grok 4.7xAISingle-shot multimodal · xhigh reasoningPublished margin ±6.1 · 578 reported samples	69.00%	Not reported	Unknown
32	Haiku 5.5AnthropicSingle-shot multimodal · max reasoningPublished margin ±5.8 · 600 reported samples	67.80%	Not reported	Unknown
33	Gemini 3.5 Flash LiteGoogleSingle-shot multimodal · high reasoningPublished margin ±6 · 600 reported samples	66.70%	Not reported	Unknown
34	Qwen3.5AlibabaSingle-shot multimodalPublished margin ±6 · 600 reported samples	65.80%	Not reported	Unknown
35	Kimi K2.7 CodeKimiSingle-shot multimodal · high reasoningPublished margin ±5.7 · 600 reported samples	65.70%	Not reported	Unknown
36	GPT-6 LunaOpenAISingle-shot multimodal · max reasoningPublished margin ±6.1 · 600 reported samples	64.80%	Not reported	Unknown
37	DeepSeek-V4.1-FlashDeepSeekSingle-shot multimodal · max reasoningPublished margin ±5.8 · 600 reported samples	63.50%	Not reported	Unknown
38	Sonnet 5AnthropicSingle-shot multimodal · max reasoningPublished margin ±6 · 600 reported samples	63.00%	Not reported	Unknown
39	Sonnet 4.6AnthropicSingle-shot multimodal · high reasoningPublished margin ±6.2 · 600 reported samples	61.70%	Not reported	Unknown
40	InklingThinking MachinesSingle-shot multimodal · high reasoningPublished margin ±6 · 600 reported samples	60.00%	Not reported	Unknown
41	MiniMax-M3MiniMaxSingle-shot multimodal · high reasoningPublished margin ±6 · 600 reported samples	59.70%	Not reported	Unknown
42	GLM-5.3ZhipuSingle-shot multimodal · max reasoningPublished margin ±6.1 · 600 reported samples	49.20%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's MedXpertQA MM results on October 15, 2026. The table preserves 42 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists programmatic multiple-choice answer extraction and a judge of None. Its run-count label is k = 3. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Pass@142 configurationsMercor evaluationDisplay only

Mercor MedXpertQA MM results Mercor APEX benchmark index Mercor methodology Source paper Upstream evaluation code Upstream dataset

The published MedXpertQA MM (Mercor) snapshot places GPT-6 Astra first at 86.80%. The third row is 1.10 points behind. The broader top-10 range is 5.50 points, so many of the published results sit in a relatively narrow band.

42 configurations are shown for MedXpertQA MM (Mercor). The benchmark falls in the Multimodal & Grounded category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. MedXpertQA MM (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About MedXpertQA MM (Mercor)

Tasks

200 source-reported tasks

Format

Pass@1

Difficulty

Source-specific evaluation

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

MedXpertQA MM evaluation results Public benchmark source

Freshness and provenance

Version

MedXpertQA MM; Mercor Pass@1

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does MedXpertQA MM (Mercor) measure?

Medical multiple-choice reasoning over images and clinical details. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published MedXpertQA MM (Mercor) snapshot?

GPT-6 Astra currently leads the published MedXpertQA MM (Mercor) snapshot with 86.80% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on MedXpertQA MM (Mercor)?

The October 15, 2026 capture snapshot contains 42 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.