Long Context Reasoning (AA-LCR) Extended — Mercor evaluation (Long Context Reasoning (AA-LCR) Extended (Mercor))
We show this table for reference; we do not rank on it.
Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.
Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.
Pass@1 on Long Context Reasoning (AA-LCR) Extended (Mercor) — October 15, 2026 capture
We mirror the published pass@1 view for Long Context Reasoning (AA-LCR) Extended (Mercor). GPT-5.6 Sol (Pro) leads the public snapshot at 79.50%, followed by GLM-5.3 (76.80%) and GPT-5.6 Terra (74.40%). We do not use these results to rank models overall.
GPT-5.6 Sol (Pro)
OpenAI
Mercor extended-set run · max reasoning
Published margin ±7.4 · Original set 85.3% ±6.4 · 351 reported samples
GLM-5.3
Zhipu
Mercor extended-set run · max reasoning
Published margin ±7.4 · Original set 78% ±6.2 · 351 reported samples
GPT-5.6 Terra
OpenAI
Mercor extended-set run · max reasoning
Published margin ±7.4 · Original set 83.3% ±6.8 · 356 reported samples
48 configurationsReasoningDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture
Pass@1 table (48 configurations)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-5.6 Sol (Pro)OpenAIMercor extended-set run · max reasoningPublished margin ±7.4 · Original set 85.3% ±6.4 · 351 reported samples | 79.50% | Not reported | Unknown |
| 2 | GLM-5.3ZhipuMercor extended-set run · max reasoningPublished margin ±7.4 · Original set 78% ±6.2 · 351 reported samples | 76.80% | Not reported | Unknown |
| 3 | GPT-5.6 TerraOpenAIMercor extended-set run · max reasoningPublished margin ±7.4 · Original set 83.3% ±6.8 · 356 reported samples | 74.40% | Not reported | Unknown |
| 4 | Opus 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±7.9 · Original set 85% ±6.4 · 356 reported samples | 74.20% | Not reported | Unknown |
| 5 | Opus 5AnthropicMercor extended-set run · max reasoningPublished margin ±7.9 · Original set 82% ±7 · 356 reported samples | 73.90% | Not reported | Unknown |
| 6 | Sonnet 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±8 · Original set 83.5% ±7 · 341 reported samples | 73.80% | Not reported | Unknown |
| 7 | Qwen3.8-MaxAlibabaMercor extended-set run · xhigh reasoningPublished margin ±7.9 · Original set 80.7% ±6.9 · 355 reported samples | 73.30% | Not reported | Unknown |
| 8 | Fable 5.1AnthropicMercor extended-set run · high reasoningPublished margin ±7.9 · Original set 82.8% ±6.9 · 356 reported samples | 73.00% | Not reported | Unknown |
| 9 | GPT-5.6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±7.7 · Original set 81.2% ±6.9 · 356 reported samples | 72.20% | Not reported | Unknown |
| 10 | Fable 5.1AnthropicMercor extended-set run · max reasoningPublished margin ±8.6 · Original set 87.5% ±5.8 · 356 reported samples | 71.90% | Not reported | Unknown |
| 11 | Fable 5AnthropicMercor extended-set run · max reasoningPublished margin ±7.9 · Original set 85.5% ±6.5 · 356 reported samples | 71.10% | Not reported | Unknown |
| 12 | DeepSeek-V4.1-FlashDeepSeekMercor extended-set run · max reasoningPublished margin ±7.7 · Original set 82.8% ±6.8 · 356 reported samples | 70.80% | Not reported | Unknown |
| 13 | GPT-5.5OpenAIMercor extended-set run · xhigh reasoningPublished margin ±8.3 · Original set 83.8% ±6.5 · 356 reported samples | 70.50% | Not reported | Unknown |
| 14 | GPT-6 SolOpenAIMercor extended-set run · max reasoningPublished margin ±8 · Original set 83.5% ±6.9 · 356 reported samples | 70.20% | Not reported | Unknown |
| 15 | Kimi K3KimiMercor extended-set run · max reasoningPublished margin ±7.3 · Original set 84.3% ±6.4 · 356 reported samples | 69.10% | Not reported | Unknown |
| 16 | GPT-5.4OpenAIMercor extended-set run · xhigh reasoningPublished margin ±8.1 · Original set 81.5% ±6.4 · 355 reported samples | 68.30% | Not reported | Unknown |
| 17 | GPT-6 LunaOpenAIMercor extended-set run · max reasoningPublished margin ±8 · Original set 82.3% ±6.8 · 356 reported samples | 68.30% | Not reported | Unknown |
| 18 | Grok 4.7xAIMercor extended-set run · xhigh reasoningPublished margin ±8.1 · Original set 77.4% ±7.4 · 350 reported samples | 68.20% | Not reported | Unknown |
| 19 | Muse Spark 1.3MetaMercor extended-set run · max reasoningPublished margin ±8.1 · Original set 82.3% ±7 · 356 reported samples | 66.90% | Not reported | Unknown |
| 20 | GLM-5.3-FlashZhipuMercor extended-set run · max reasoningPublished margin ±7.7 · Original set 79.7% ±6.6 · 344 reported samples | 66.90% | Not reported | Unknown |
| 21 | Grok 4.6xAIMercor extended-set run · xhigh reasoningPublished margin ±8.1 · Original set 78.2% ±7 · 356 reported samples | 65.20% | Not reported | Unknown |
| 22 | Haiku 5.5AnthropicMercor extended-set run · max reasoningPublished margin ±8.3 · Original set 82.3% ±7 · 356 reported samples | 63.80% | Not reported | Unknown |
| 23 | Opus 4.8AnthropicMercor extended-set run · max reasoningPublished margin ±8.4 · Original set 79.5% ±7.3 · 356 reported samples | 63.20% | Not reported | Unknown |
| 24 | DeepSeek-V4-Pro-0813DeepSeekMercor extended-set run · max reasoningPublished margin ±8 · Original set 78.5% ±7.1 · 356 reported samples | 62.90% | Not reported | Unknown |
| 25 | Sonnet 5AnthropicMercor extended-set run · max reasoningPublished margin ±8.8 · Original set 81.8% ±6.9 · 356 reported samples | 61.80% | Not reported | Unknown |
| 26 | DeepSeek-V4-FlashDeepSeekMercor extended-set run · max reasoningPublished margin ±8 · Original set 73% ±6.6 · 356 reported samples | 61.80% | Not reported | Unknown |
| 27 | Muse Spark 1.2MetaMercor extended-set run · xhigh reasoningPublished margin ±8.6 · Original set 79% ±6.5 · 356 reported samples | 61.50% | Not reported | Unknown |
| 28 | Grok 4.5xAIMercor extended-set run · high reasoningPublished margin ±8.3 · Original set not reported · 356 reported samples | 60.40% | Not reported | Unknown |
| 29 | Opus 4.7AnthropicMercor extended-set run · max reasoningPublished margin ±8.6 · Original set 83% ±6.8 · 356 reported samples | 59.80% | Not reported | Unknown |
| 30 | Sonnet 4.6AnthropicMercor extended-set run · high reasoningPublished margin ±8.6 · Original set 80.5% ±6.9 · 356 reported samples | 58.10% | Not reported | Unknown |
| 31 | Gemini 3.8 FlashGoogleMercor extended-set run · high reasoningPublished margin ±8.3 · Original set 82.3% ±6.8 · 356 reported samples | 55.60% | Not reported | Unknown |
| 32 | InklingThinking MachinesMercor extended-set run · high reasoningPublished margin ±7.9 · Original set 74.5% ±7.6 · 356 reported samples | 50.60% | Not reported | Unknown |
| 33 | Kimi K2.7 CodeKimiMercor extended-set run · high reasoningPublished margin ±8.8 · Original set 79.2% ±6.2 · 356 reported samples | 49.20% | Not reported | Unknown |
| 34 | Gemini 3.7 FlashGoogleMercor extended-set run · high reasoningPublished margin ±9.1 · Original set 80.2% ±7 · 356 reported samples | 48.90% | Not reported | Unknown |
| 35 | Gemini 3.6 FlashGoogleMercor extended-set run · high reasoningPublished margin ±8.8 · Original set 78.7% ±7.5 · 356 reported samples | 46.30% | Not reported | Unknown |
| 36 | MiniMax-M3MiniMaxMercor extended-set run · high reasoningPublished margin ±8.3 · Original set 83.3% ±5.6 · 356 reported samples | 45.50% | Not reported | Unknown |
| 37 | GLM-5.2ZhipuMercor extended-set run · max reasoningPublished margin ±7.6 · Original set 78.7% ±6.5 · 356 reported samples | 42.70% | Not reported | Unknown |
| 38 | Nemotron 3 UltraNVIDIAMercor extended-set run · high reasoningPublished margin ±8 · Original set 73.5% ±7.3 · 356 reported samples | 41.60% | Not reported | Unknown |
| 39 | Gemini 3.5 FlashGoogleMercor extended-set run · high reasoningPublished margin ±9 · Original set 74.8% ±7.5 · 356 reported samples | 39.90% | Not reported | Unknown |
| 40 | GLM-5.1ZhipuMercor extended-set runPublished margin ±7.7 · Original set not reported · 356 reported samples | 37.90% | Not reported | Unknown |
| 41 | DeepSeek-V4-ProDeepSeekMercor extended-set run · max reasoningPublished margin ±8.1 · Original set not reported · 356 reported samples | 37.60% | Not reported | Unknown |
| 42 | Gemini 3.1 ProGoogleMercor extended-set run · high reasoningPublished margin ±8.3 · Original set 78.7% ±7.3 · 356 reported samples | 35.40% | Not reported | Unknown |
| 43 | Qwen3.5AlibabaMercor extended-set runPublished margin ±7.9 · Original set 76% ±7 · 356 reported samples | 33.40% | Not reported | Unknown |
| 44 | Kimi K2 (Thinking)KimiMercor extended-set run · high reasoningPublished margin ±7.4 · Original set 73.3% ±6.5 · 356 reported samples | 32.90% | Not reported | Unknown |
| 45 | MiniMax-M2.7MiniMaxMercor extended-set run · high reasoningPublished margin ±7.4 · Original set not reported · 356 reported samples | 28.40% | Not reported | Unknown |
| 46 | GPT-OSS-120BOpenAIMercor extended-set run · high reasoningPublished margin ±7.6 · Original set not reported · 356 reported samples | 25.80% | Not reported | Unknown |
| 47 | DeepSeek-V3.2DeepSeekMercor extended-set runPublished margin ±6.6 · Original set 69.5% ±7.2 · 356 reported samples | 24.70% | Not reported | Unknown |
| 48 | Gemma 4 31BGoogleMercor extended-set runPublished margin ±6.9 · Original set not reported · 353 reported samples | 22.70% | Not reported | Unknown |
How to read this Mercor evaluation
We captured Mercor's Long Context Reasoning (AA-LCR) Extended results on October 15, 2026. The table preserves 48 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.
The headline uses the 89 Mercor-built tasks. The paired original-set values refer to the separately listed 100-task public benchmark. Missing baselines remain unreported. We keep the extension and original set separate from their upstream benchmark-owner and provider-run tables.
The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.
The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.
The general methodology lists llm-as-a-judge (lmaaj) and a judge of Gemini 3.5 Flash. Its run-count label is k = 4. Those notes describe the original public benchmark; extension-specific changes are not established by that reference.
Snapshot
The published Long Context Reasoning (AA-LCR) Extended (Mercor) snapshot places GPT-5.6 Sol (Pro) first at 79.50%. The third row is 5.10 points behind. The broader top-10 range is 7.60 points, so many of the published results sit in a relatively narrow band.
48 configurations are shown for Long Context Reasoning (AA-LCR) Extended (Mercor). The benchmark falls in the Reasoning category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Long Context Reasoning (AA-LCR) Extended (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Long Context Reasoning (AA-LCR) Extended (Mercor)
Tasks
89 Mercor tasks; 100 tasks in the separately reported original set
Format
Pass@1
Difficulty
Source-specific evaluation
The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.
Freshness and provenance
Version
Long Context Reasoning (AA-LCR) Extended; Mercor Pass@1
Refresh cadence
Reviewed dated source capture
Staleness state
Dated Mercor configurations
Question availability
Upstream task access and terms; only aggregate results mirrored
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Long Context Reasoning (AA-LCR) Extended (Mercor) measure?
Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.
Which model leads the published Long Context Reasoning (AA-LCR) Extended (Mercor) snapshot?
GPT-5.6 Sol (Pro) currently leads the published Long Context Reasoning (AA-LCR) Extended (Mercor) snapshot with 79.50% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Long Context Reasoning (AA-LCR) Extended (Mercor)?
The October 15, 2026 capture snapshot contains 48 source configurations.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.