SciCode — Mercor evaluation (SciCode (Mercor))

We show this table for reference; we do not rank on it.

Solving scientific programming subproblems with executable checks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Mean Score on SciCode (Mercor) — October 15, 2026 capture

We mirror the published mean score view for SciCode (Mercor). Opus 5.5 leads the public snapshot at 57.40%, followed by Fable 5.1 (54.60%) and Sonnet 5.5 (51.20%). We do not use these results to rank models overall.

Opus 5.5

Anthropic

SciCode chain agent · max reasoning

Published margin ±8.3 · 256 reported samples

57.40%

Fable 5.1

Anthropic

SciCode chain agent · max reasoning

Published margin ±8.4 · 260 reported samples

54.60%

Sonnet 5.5

Anthropic

SciCode chain agent · max reasoning

Published margin ±8.8 · 206 reported samples

51.20%

39 configurationsCodingDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Mean Score table (39 configurations)

Mean Score results for SciCode (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	Opus 5.5AnthropicSciCode chain agent · max reasoningPublished margin ±8.3 · 256 reported samples	57.40%	Not reported	Unknown
2	Fable 5.1AnthropicSciCode chain agent · max reasoningPublished margin ±8.4 · 260 reported samples	54.60%	Not reported	Unknown
3	Sonnet 5.5AnthropicSciCode chain agent · max reasoningPublished margin ±8.8 · 206 reported samples	51.20%	Not reported	Unknown
4	GPT-6 AstraOpenAISciCode chain agent · max reasoningPublished margin ±8.1 · 260 reported samples	49.30%	Not reported	Unknown
5	Fable 5AnthropicSciCode chain agent · max reasoningPublished margin ±8.2 · 260 reported samples	49.00%	Not reported	Unknown
6	GPT-6 AstraOpenAISciCode chain agent · xhigh reasoningPublished margin ±8.5 · 260 reported samples	47.50%	Not reported	Unknown
7	Gemini 3.8 FlashGoogleSciCode chain agent · high reasoningPublished margin ±8.3 · 260 reported samples	46.60%	Not reported	Unknown
8	GPT-6.1 SolOpenAISciCode chain agent · max reasoningPublished margin ±7.9 · 256 reported samples	46.50%	Not reported	Unknown
9	Fable 5.1AnthropicSciCode chain agent · high reasoningPublished margin ±7.9 · 260 reported samples	46.30%	Not reported	Unknown
10	GPT-6 LunaOpenAISciCode chain agent · max reasoningPublished margin ±7.9 · 260 reported samples	46.00%	Not reported	Unknown
11	GPT-5.6 SolOpenAISciCode chain agent · xhigh reasoningPublished margin ±8.2 · 260 reported samples	45.80%	Not reported	Unknown
12	GPT-6 SolOpenAISciCode chain agent · max reasoningPublished margin ±7.7 · 260 reported samples	45.80%	Not reported	Unknown
13	Grok 4.6xAISciCode chain agent · xhigh reasoningPublished margin ±7.7 · 257 reported samples	45.60%	Not reported	Unknown
14	Opus 5AnthropicSciCode chain agent · max reasoningPublished margin ±8.3 · 259 reported samples	45.30%	Not reported	Unknown
15	Opus 4.8AnthropicSciCode chain agent · max reasoningPublished margin ±8.3 · 260 reported samples	44.70%	Not reported	Unknown
16	GPT-5.4OpenAISciCode chain agent · xhigh reasoningPublished margin ±7.8 · 260 reported samples	44.00%	Not reported	Unknown
17	Kimi K3KimiSciCode chain agent · max reasoningPublished margin ±8 · 260 reported samples	43.80%	Not reported	Unknown
18	Opus 4.7AnthropicSciCode chain agent · max reasoningPublished margin ±8.3 · 260 reported samples	43.30%	Not reported	Unknown
19	GPT-5.5OpenAISciCode chain agent · xhigh reasoningPublished margin ±7.6 · 260 reported samples	43.10%	Not reported	Unknown
20	DeepSeek-V4.1-FlashDeepSeekSciCode chain agent · max reasoningPublished margin ±7.4 · 260 reported samples	43.10%	Not reported	Unknown
21	Gemini 3.1 ProGoogleSciCode chain agent · high reasoningPublished margin ±8 · 260 reported samples	42.90%	Not reported	Unknown
22	Grok 4.5xAISciCode chain agent · high reasoningPublished margin ±7.7 · 260 reported samples	42.60%	Not reported	Unknown
23	GPT-5.6 TerraOpenAISciCode chain agent · max reasoningPublished margin ±8 · 260 reported samples	42.40%	Not reported	Unknown
24	Haiku 5.5AnthropicSciCode chain agent · max reasoningPublished margin ±8.5 · 246 reported samples	41.80%	Not reported	Unknown
25	Gemini 3.6 FlashGoogleSciCode chain agent · high reasoningPublished margin ±7.7 · 260 reported samples	41.60%	Not reported	Unknown
26	DeepSeek-V4-ProDeepSeekSciCode chain agent · max reasoningPublished margin ±7.2 · 260 reported samples	40.80%	Not reported	Unknown
27	Gemini 3.5 FlashGoogleSciCode chain agent · high reasoningPublished margin ±7.7 · 260 reported samples	40.60%	Not reported	Unknown
28	GPT-5.6 LunaOpenAISciCode chain agent · max reasoningPublished margin ±7.8 · 260 reported samples	40.50%	Not reported	Unknown
29	DeepSeek-V4-FlashDeepSeekSciCode chain agent · max reasoningPublished margin ±7.5 · 260 reported samples	40.00%	Not reported	Unknown
30	GLM-5.2ZhipuSciCode chain agent · max reasoningPublished margin ±7.4 · 260 reported samples	38.80%	Not reported	Unknown
31	InklingThinking MachinesSciCode chain agent · high reasoningPublished margin ±7.3 · 260 reported samples	37.00%	Not reported	Unknown
32	MiniMax-M3MiniMaxSciCode chain agent · high reasoningPublished margin ±7.1 · 260 reported samples	36.80%	Not reported	Unknown
33	Gemma 4 31BGoogleSciCode chain agentPublished margin ±8.1 · 260 reported samples	36.40%	Not reported	Unknown
34	Kimi K2.7 CodeKimiSciCode chain agent · high reasoningPublished margin ±7 · 260 reported samples	35.70%	Not reported	Unknown
35	Sonnet 4.6AnthropicSciCode chain agent · high reasoningPublished margin ±7.4 · 259 reported samples	34.20%	Not reported	Unknown
36	MiniMax-M2.7MiniMaxSciCode chain agent · high reasoningPublished margin ±6.1 · 260 reported samples	32.90%	Not reported	Unknown
37	Kimi K2 (Thinking)KimiSciCode chain agent · high reasoningPublished margin ±6.1 · 260 reported samples	32.40%	Not reported	Unknown
38	DeepSeek-V3.2DeepSeekSciCode chain agentPublished margin ±6.3 · 260 reported samples	31.30%	Not reported	Unknown
39	GPT-OSS-120BOpenAISciCode chain agent · high reasoningPublished margin ±6.3 · 260 reported samples	29.60%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's SciCode results on October 15, 2026. The table preserves 39 published configurations, their source identifiers, reasoning settings, and disclosed mean score values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

The selected mean score measures scientific subproblem performance. It is distinct from strict whole-problem Pass@1, which is not exposed as a selectable headline on this result page.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists programmatic unit tests for each subproblem (all must pass) and a judge of None. Its run-count label is k = 3. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Mean Score39 configurationsMercor evaluationDisplay only

Mercor SciCode results Mercor APEX benchmark index Mercor methodology Source paper Upstream evaluation code Upstream dataset

The published SciCode (Mercor) snapshot places Opus 5.5 first at 57.40%. The third row is 6.20 points behind. The broader top-10 range is 11.40 points, so the table still separates the published systems.

39 configurations are shown for SciCode (Mercor). The benchmark falls in the Coding category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. SciCode (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About SciCode (Mercor)

Tasks

65 source-reported tasks

Format

Mean Score

Difficulty

Source-specific evaluation

The selected headline is Mean Score. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

SciCode evaluation results Public benchmark source

Freshness and provenance

Version

SciCode; Mercor Mean Score

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does SciCode (Mercor) measure?

Solving scientific programming subproblems with executable checks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published SciCode (Mercor) snapshot?

Opus 5.5 currently leads the published SciCode (Mercor) snapshot with 57.40% mean score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on SciCode (Mercor)?

The October 15, 2026 capture snapshot contains 39 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.