# Long Context Reasoning (AA-LCR) Extended — Mercor evaluation (Long Context Reasoning (AA-LCR) Extended (Mercor))

> Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Canonical page: https://benchlm.ai/benchmarks/mercoraalcrextended

- Category: [Reasoning](/reasoning)
- Last updated: October 15, 2026 capture

Evaluation results published by [Mercor](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/). Published evaluation aggregates only; task contents and dataset license grants are not included.

We captured Mercor's Long Context Reasoning (AA-LCR) Extended results on October 15, 2026. The table preserves 48 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

The headline uses the 89 Mercor-built tasks. The paired original-set values refer to the separately listed 100-task public benchmark. Missing baselines remain unreported. We keep the extension and original set separate from their upstream benchmark-owner and provider-run tables.

The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists llm-as-a-judge (lmaaj) and a judge of Gemini 3.5 Flash. Its run-count label is k = 4. Those notes describe the original public benchmark; extension-specific changes are not established by that reference.

- [Mercor Long Context Reasoning (AA-LCR) Extended results](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/)
- [Mercor APEX benchmark index](https://www.mercor.com/apex/)
- [Mercor methodology](https://www.mercor.com/apex/methodology/)
- [Paired Mercor original-set results](https://www.mercor.com/apex/oss-benchmarks/oss-aa-lcr-leaderboard/)
- [Upstream dataset](https://huggingface.co/datasets/ArtificialAnalysis/AA-LCR)

## About Long Context Reasoning (AA-LCR) Extended (Mercor)

- Tasks: 89 Mercor tasks; 100 tasks in the separately reported original set
- Format: Pass@1
- Difficulty: Source-specific evaluation
- Paper: [Long Context Reasoning (AA-LCR) Extended evaluation results](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/)

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

Long Context Reasoning (AA-LCR) Extended (Mercor) is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (48 configurations)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [GPT-5.6 Sol (Pro)](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.4 · Original set 85.3% ±6.4 · 351 reported samples | OpenAI | 79.50% |
| 2 | [GLM-5.3](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.4 · Original set 78% ±6.2 · 351 reported samples | Zhipu | 76.80% |
| 3 | [GPT-5.6 Terra](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.4 · Original set 83.3% ±6.8 · 356 reported samples | OpenAI | 74.40% |
| 4 | [Opus 5.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.9 · Original set 85% ±6.4 · 356 reported samples | Anthropic | 74.20% |
| 5 | [Opus 5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.9 · Original set 82% ±7 · 356 reported samples | Anthropic | 73.90% |
| 6 | [Sonnet 5.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8 · Original set 83.5% ±7 · 341 reported samples | Anthropic | 73.80% |
| 7 | [Qwen3.8-Max](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±7.9 · Original set 80.7% ±6.9 · 355 reported samples | Alibaba | 73.30% |
| 8 | [Fable 5.1](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±7.9 · Original set 82.8% ±6.9 · 356 reported samples | Anthropic | 73.00% |
| 9 | [GPT-5.6 Luna](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.7 · Original set 81.2% ±6.9 · 356 reported samples | OpenAI | 72.20% |
| 10 | [Fable 5.1](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.6 · Original set 87.5% ±5.8 · 356 reported samples | Anthropic | 71.90% |
| 11 | [Fable 5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.9 · Original set 85.5% ±6.5 · 356 reported samples | Anthropic | 71.10% |
| 12 | [DeepSeek-V4.1-Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.7 · Original set 82.8% ±6.8 · 356 reported samples | DeepSeek | 70.80% |
| 13 | [GPT-5.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±8.3 · Original set 83.8% ±6.5 · 356 reported samples | OpenAI | 70.50% |
| 14 | [GPT-6 Sol](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8 · Original set 83.5% ±6.9 · 356 reported samples | OpenAI | 70.20% |
| 15 | [Kimi K3](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.3 · Original set 84.3% ±6.4 · 356 reported samples | Kimi | 69.10% |
| 16 | [GPT-5.4](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±8.1 · Original set 81.5% ±6.4 · 355 reported samples | OpenAI | 68.30% |
| 17 | [GPT-6 Luna](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8 · Original set 82.3% ±6.8 · 356 reported samples | OpenAI | 68.30% |
| 18 | [Grok 4.7](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±8.1 · Original set 77.4% ±7.4 · 350 reported samples | xAI | 68.20% |
| 19 | [Muse Spark 1.3](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.1 · Original set 82.3% ±7 · 356 reported samples | Meta | 66.90% |
| 20 | [GLM-5.3-Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.7 · Original set 79.7% ±6.6 · 344 reported samples | Zhipu | 66.90% |
| 21 | [Grok 4.6](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±8.1 · Original set 78.2% ±7 · 356 reported samples | xAI | 65.20% |
| 22 | [Haiku 5.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.3 · Original set 82.3% ±7 · 356 reported samples | Anthropic | 63.80% |
| 23 | [Opus 4.8](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.4 · Original set 79.5% ±7.3 · 356 reported samples | Anthropic | 63.20% |
| 24 | [DeepSeek-V4-Pro-0813](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8 · Original set 78.5% ±7.1 · 356 reported samples | DeepSeek | 62.90% |
| 25 | [Sonnet 5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.8 · Original set 81.8% ±6.9 · 356 reported samples | Anthropic | 61.80% |
| 26 | [DeepSeek-V4-Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8 · Original set 73% ±6.6 · 356 reported samples | DeepSeek | 61.80% |
| 27 | [Muse Spark 1.2](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · xhigh reasoning · Published margin ±8.6 · Original set 79% ±6.5 · 356 reported samples | Meta | 61.50% |
| 28 | [Grok 4.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.3 · Original set not reported · 356 reported samples | xAI | 60.40% |
| 29 | [Opus 4.7](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.6 · Original set 83% ±6.8 · 356 reported samples | Anthropic | 59.80% |
| 30 | [Sonnet 4.6](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.6 · Original set 80.5% ±6.9 · 356 reported samples | Anthropic | 58.10% |
| 31 | [Gemini 3.8 Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.3 · Original set 82.3% ±6.8 · 356 reported samples | Google | 55.60% |
| 32 | [Inkling](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±7.9 · Original set 74.5% ±7.6 · 356 reported samples | Thinking Machines | 50.60% |
| 33 | [Kimi K2.7 Code](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.8 · Original set 79.2% ±6.2 · 356 reported samples | Kimi | 49.20% |
| 34 | [Gemini 3.7 Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±9.1 · Original set 80.2% ±7 · 356 reported samples | Google | 48.90% |
| 35 | [Gemini 3.6 Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.8 · Original set 78.7% ±7.5 · 356 reported samples | Google | 46.30% |
| 36 | [MiniMax-M3](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.3 · Original set 83.3% ±5.6 · 356 reported samples | MiniMax | 45.50% |
| 37 | [GLM-5.2](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±7.6 · Original set 78.7% ±6.5 · 356 reported samples | Zhipu | 42.70% |
| 38 | [Nemotron 3 Ultra](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8 · Original set 73.5% ±7.3 · 356 reported samples | NVIDIA | 41.60% |
| 39 | [Gemini 3.5 Flash](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±9 · Original set 74.8% ±7.5 · 356 reported samples | Google | 39.90% |
| 40 | [GLM-5.1](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · Published margin ±7.7 · Original set not reported · 356 reported samples | Zhipu | 37.90% |
| 41 | [DeepSeek-V4-Pro](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · max reasoning · Published margin ±8.1 · Original set not reported · 356 reported samples | DeepSeek | 37.60% |
| 42 | [Gemini 3.1 Pro](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±8.3 · Original set 78.7% ±7.3 · 356 reported samples | Google | 35.40% |
| 43 | [Qwen3.5](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · Published margin ±7.9 · Original set 76% ±7 · 356 reported samples | Alibaba | 33.40% |
| 44 | [Kimi K2 (Thinking)](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±7.4 · Original set 73.3% ±6.5 · 356 reported samples | Kimi | 32.90% |
| 45 | [MiniMax-M2.7](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±7.4 · Original set not reported · 356 reported samples | MiniMax | 28.40% |
| 46 | [GPT-OSS-120B](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · high reasoning · Published margin ±7.6 · Original set not reported · 356 reported samples | OpenAI | 25.80% |
| 47 | [DeepSeek-V3.2](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · Published margin ±6.6 · Original set 69.5% ±7.2 · 356 reported samples | DeepSeek | 24.70% |
| 48 | [Gemma 4 31B](https://www.mercor.com/apex/mercor-extended/ots-held-out-aa-lcr-leaderboard/) | Mercor extended-set run · Published margin ±6.9 · Original set not reported · 353 reported samples | Google | 22.70% |

## FAQ

### What does Long Context Reasoning (AA-LCR) Extended (Mercor) measure?

Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.

### Which model leads the published Long Context Reasoning (AA-LCR) Extended (Mercor) snapshot?

GPT-5.6 Sol (Pro) currently leads the published Long Context Reasoning (AA-LCR) Extended (Mercor) snapshot with a score of 79.50%.

### How many models are evaluated on Long Context Reasoning (AA-LCR) Extended (Mercor)?

The October 15, 2026 capture contains 48 source configurations.

### Does Long Context Reasoning (AA-LCR) Extended (Mercor) affect BenchLM's overall score?

Not directly. Long Context Reasoning (AA-LCR) Extended (Mercor) is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.