# SciCode — Mercor evaluation (SciCode (Mercor))

> Solving scientific programming subproblems with executable checks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Canonical page: https://benchlm.ai/benchmarks/mercorscicode

- Category: [Coding](/coding)
- Last updated: October 15, 2026 capture

Evaluation results published by [Mercor](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/). Published evaluation aggregates only; task contents and dataset license grants are not included.

We captured Mercor's SciCode results on October 15, 2026. The table preserves 39 published configurations, their source identifiers, reasoning settings, and disclosed mean score values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

The selected mean score measures scientific subproblem performance. It is distinct from strict whole-problem Pass@1, which is not exposed as a selectable headline on this result page.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists programmatic unit tests for each subproblem (all must pass) and a judge of None. Its run-count label is k = 3. The linked result page takes precedence where its setup differs, as noted above.

- [Mercor SciCode results](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/)
- [Mercor APEX benchmark index](https://www.mercor.com/apex/)
- [Mercor methodology](https://www.mercor.com/apex/methodology/)
- [Source paper](https://arxiv.org/abs/2407.13168)
- [Upstream evaluation code](https://github.com/scicode-bench/SciCode)
- [Upstream dataset](https://huggingface.co/datasets/SciCode1/SciCode)

## About SciCode (Mercor)

- Tasks: 65 source-reported tasks
- Format: Mean Score
- Difficulty: Source-specific evaluation
- Paper: [SciCode evaluation results](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/)

The selected headline is Mean Score. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

SciCode (Mercor) is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (39 configurations)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Opus 5.5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.3 · 256 reported samples | Anthropic | 57.40% |
| 2 | [Fable 5.1](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.4 · 260 reported samples | Anthropic | 54.60% |
| 3 | [Sonnet 5.5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.8 · 206 reported samples | Anthropic | 51.20% |
| 4 | [GPT-6 Astra](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.1 · 260 reported samples | OpenAI | 49.30% |
| 5 | [Fable 5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.2 · 260 reported samples | Anthropic | 49.00% |
| 6 | [GPT-6 Astra](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · xhigh reasoning · Published margin ±8.5 · 260 reported samples | OpenAI | 47.50% |
| 7 | [Gemini 3.8 Flash](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±8.3 · 260 reported samples | Google | 46.60% |
| 8 | [GPT-6.1 Sol](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.9 · 256 reported samples | OpenAI | 46.50% |
| 9 | [Fable 5.1](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.9 · 260 reported samples | Anthropic | 46.30% |
| 10 | [GPT-6 Luna](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.9 · 260 reported samples | OpenAI | 46.00% |
| 11 | [GPT-5.6 Sol](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · xhigh reasoning · Published margin ±8.2 · 260 reported samples | OpenAI | 45.80% |
| 12 | [GPT-6 Sol](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.7 · 260 reported samples | OpenAI | 45.80% |
| 13 | [Grok 4.6](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · xhigh reasoning · Published margin ±7.7 · 257 reported samples | xAI | 45.60% |
| 14 | [Opus 5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.3 · 259 reported samples | Anthropic | 45.30% |
| 15 | [Opus 4.8](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.3 · 260 reported samples | Anthropic | 44.70% |
| 16 | [GPT-5.4](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · xhigh reasoning · Published margin ±7.8 · 260 reported samples | OpenAI | 44.00% |
| 17 | [Kimi K3](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8 · 260 reported samples | Kimi | 43.80% |
| 18 | [Opus 4.7](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.3 · 260 reported samples | Anthropic | 43.30% |
| 19 | [GPT-5.5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · xhigh reasoning · Published margin ±7.6 · 260 reported samples | OpenAI | 43.10% |
| 20 | [DeepSeek-V4.1-Flash](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.4 · 260 reported samples | DeepSeek | 43.10% |
| 21 | [Gemini 3.1 Pro](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±8 · 260 reported samples | Google | 42.90% |
| 22 | [Grok 4.5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.7 · 260 reported samples | xAI | 42.60% |
| 23 | [GPT-5.6 Terra](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8 · 260 reported samples | OpenAI | 42.40% |
| 24 | [Haiku 5.5](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±8.5 · 246 reported samples | Anthropic | 41.80% |
| 25 | [Gemini 3.6 Flash](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.7 · 260 reported samples | Google | 41.60% |
| 26 | [DeepSeek-V4-Pro](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.2 · 260 reported samples | DeepSeek | 40.80% |
| 27 | [Gemini 3.5 Flash](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.7 · 260 reported samples | Google | 40.60% |
| 28 | [GPT-5.6 Luna](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.8 · 260 reported samples | OpenAI | 40.50% |
| 29 | [DeepSeek-V4-Flash](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.5 · 260 reported samples | DeepSeek | 40.00% |
| 30 | [GLM-5.2](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · max reasoning · Published margin ±7.4 · 260 reported samples | Zhipu | 38.80% |
| 31 | [Inkling](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.3 · 260 reported samples | Thinking Machines | 37.00% |
| 32 | [MiniMax-M3](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.1 · 260 reported samples | MiniMax | 36.80% |
| 33 | [Gemma 4 31B](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · Published margin ±8.1 · 260 reported samples | Google | 36.40% |
| 34 | [Kimi K2.7 Code](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7 · 260 reported samples | Kimi | 35.70% |
| 35 | [Sonnet 4.6](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±7.4 · 259 reported samples | Anthropic | 34.20% |
| 36 | [MiniMax-M2.7](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±6.1 · 260 reported samples | MiniMax | 32.90% |
| 37 | [Kimi K2 (Thinking)](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±6.1 · 260 reported samples | Kimi | 32.40% |
| 38 | [DeepSeek-V3.2](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · Published margin ±6.3 · 260 reported samples | DeepSeek | 31.30% |
| 39 | [GPT-OSS-120B](https://www.mercor.com/apex/oss-benchmarks/oss-scicode-leaderboard/) | SciCode chain agent · high reasoning · Published margin ±6.3 · 260 reported samples | OpenAI | 29.60% |

## FAQ

### What does SciCode (Mercor) measure?

Solving scientific programming subproblems with executable checks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

### Which model leads the published SciCode (Mercor) snapshot?

Opus 5.5 currently leads the published SciCode (Mercor) snapshot with a score of 57.40%.

### How many models are evaluated on SciCode (Mercor)?

The October 15, 2026 capture contains 39 source configurations.

### Does SciCode (Mercor) affect BenchLM's overall score?

Not directly. SciCode (Mercor) is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.