Long Context Reasoning (AA-LCR v1.1) — Mercor evaluation (Long Context Reasoning (AA-LCR v1.1) (Mercor))

We show this table for reference; we do not rank on it.

Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Should you trust a Long Context Reasoning (AA-LCR v1.1) (Mercor) score?

Read with care: scores cluster at the top; the benchmark is not marked current.

Saturation Active: Top three within 2.20 points; frontier rows are clustering, so read gaps at the top with care.
Who produced the rows Mirrored table: Every row is mirrored from Mercor Long Context Reasoning (AA-LCR v1.1) as it publishes it; BenchLM did not re-run these results. Source.
Freshness and weight Display only: Dated Mercor configurations, refreshed reviewed dated source capture. Display only; it does not move rankings. Upstream task access and terms; only aggregate results mirrored.

Row statuses come from the verification registry; saturation and freshness from the benchmark lifecycle. A provider self-report is displayable evidence, not an independent run.

Pass@1 on Long Context Reasoning (AA-LCR v1.1) (Mercor) — October 15, 2026 capture

We mirror the published pass@1 view for Long Context Reasoning (AA-LCR v1.1) (Mercor). Fable 5.1 leads the public snapshot at 87.50%, followed by Fable 5 (85.50%) and GPT-5.6 Sol (Pro) (85.30%). We do not use these results to rank models overall.

Fable 5.1

Anthropic

Single-shot agent · max reasoning

Published margin ±5.8 · 400 reported samples

87.50%

Fable 5

Anthropic

Single-shot agent · max reasoning

Published margin ±6.5 · 400 reported samples

85.50%

GPT-5.6 Sol (Pro)

OpenAI

Single-shot agent · max reasoning

Published margin ±6.4 · 400 reported samples

85.30%

49 configurationsReasoningDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Pass@1 table (49 configurations)

Pass@1 results for Long Context Reasoning (AA-LCR v1.1) (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	Fable 5.1AnthropicSingle-shot agent · max reasoningPublished margin ±5.8 · 400 reported samples	87.50%	Not reported	Unknown
2	Fable 5AnthropicSingle-shot agent · max reasoningPublished margin ±6.5 · 400 reported samples	85.50%	Not reported	Unknown
3	GPT-5.6 Sol (Pro)OpenAISingle-shot agent · max reasoningPublished margin ±6.4 · 400 reported samples	85.30%	Not reported	Unknown
4	Opus 5.5AnthropicSingle-shot agent · max reasoningPublished margin ±6.4 · 400 reported samples	85.00%	Not reported	Unknown
5	Kimi K3KimiSingle-shot agent · max reasoningPublished margin ±6.4 · 400 reported samples	84.30%	Not reported	Unknown
6	GPT-5.5OpenAISingle-shot agent · xhigh reasoningPublished margin ±6.5 · 400 reported samples	83.80%	Not reported	Unknown
7	GPT-6 SolOpenAISingle-shot agent · max reasoningPublished margin ±6.9 · 400 reported samples	83.50%	Not reported	Unknown
8	Sonnet 5.5AnthropicSingle-shot agent · max reasoningPublished margin ±7 · 400 reported samples	83.50%	Not reported	Unknown
9	MiniMax-M3MiniMaxSingle-shot agent · high reasoningPublished margin ±5.6 · 400 reported samples	83.30%	Not reported	Unknown
10	GPT-5.6 TerraOpenAISingle-shot agent · max reasoningPublished margin ±6.8 · 400 reported samples	83.30%	Not reported	Unknown
11	Opus 4.7AnthropicSingle-shot agent · max reasoningPublished margin ±6.8 · 400 reported samples	83.00%	Not reported	Unknown
12	Fable 5.1AnthropicSingle-shot agent · high reasoningPublished margin ±6.9 · 400 reported samples	82.80%	Not reported	Unknown
13	DeepSeek-V4.1-FlashDeepSeekSingle-shot agent · max reasoningPublished margin ±6.8 · 400 reported samples	82.80%	Not reported	Unknown
14	Gemini 3.8 FlashGoogleSingle-shot agent · high reasoningPublished margin ±6.8 · 400 reported samples	82.30%	Not reported	Unknown
15	Muse Spark 1.3MetaSingle-shot agent · max reasoningPublished margin ±7 · 400 reported samples	82.30%	Not reported	Unknown
16	Qwen 3.8 27BAlibabaSingle-shot agent · xhigh reasoningPublished margin ±6.6 · 400 reported samples	82.30%	Not reported	Unknown
17	GPT-6 LunaOpenAISingle-shot agent · max reasoningPublished margin ±6.8 · 400 reported samples	82.30%	Not reported	Unknown
18	Haiku 5.5AnthropicSingle-shot agent · max reasoningPublished margin ±7 · 400 reported samples	82.30%	Not reported	Unknown
19	Opus 5AnthropicSingle-shot agent · max reasoningPublished margin ±7 · 400 reported samples	82.00%	Not reported	Unknown
20	Sonnet 5AnthropicSingle-shot agent · max reasoningPublished margin ±6.9 · 400 reported samples	81.80%	Not reported	Unknown
21	GPT-5.4OpenAISingle-shot agent · xhigh reasoningPublished margin ±6.4 · 400 reported samples	81.50%	Not reported	Unknown
22	GPT-5.6 LunaOpenAISingle-shot agent · max reasoningPublished margin ±6.9 · 400 reported samples	81.20%	Not reported	Unknown
23	GPT-6.1 SolOpenAISingle-shot agent · max reasoningPublished margin ±7 · 400 reported samples	81.20%	Not reported	Unknown
24	Qwen3.8-MaxAlibabaSingle-shot agent · xhigh reasoningPublished margin ±6.9 · 400 reported samples	80.70%	Not reported	Unknown
25	GPT-6 AstraOpenAISingle-shot agent · max reasoningPublished margin ±7.4 · 400 reported samples	80.50%	Not reported	Unknown
26	Sonnet 4.6AnthropicSingle-shot agent · high reasoningPublished margin ±6.9 · 400 reported samples	80.50%	Not reported	Unknown
27	Gemini 3.7 FlashGoogleSingle-shot agent · high reasoningPublished margin ±7 · 400 reported samples	80.20%	Not reported	Unknown
28	GLM-5.3-FlashZhipuSingle-shot agent · max reasoningPublished margin ±6.6 · 400 reported samples	79.70%	Not reported	Unknown
29	Opus 5.5AnthropicSingle-shot agent · medium reasoningPublished margin ±7.1 · 400 reported samples	79.70%	Not reported	Unknown
30	Opus 4.8AnthropicSingle-shot agent · max reasoningPublished margin ±7.3 · 400 reported samples	79.50%	Not reported	Unknown
31	Kimi K2.7 CodeKimiSingle-shot agent · high reasoningPublished margin ±6.2 · 400 reported samples	79.20%	Not reported	Unknown
32	Muse Spark 1.2MetaSingle-shot agent · xhigh reasoningPublished margin ±6.5 · 400 reported samples	79.00%	Not reported	Unknown
33	Gemini 3.6 FlashGoogleSingle-shot agent · high reasoningPublished margin ±7.5 · 400 reported samples	78.70%	Not reported	Unknown
34	Gemini 3.1 ProGoogleSingle-shot agent · high reasoningPublished margin ±7.3 · 400 reported samples	78.70%	Not reported	Unknown
35	GLM-5.2ZhipuSingle-shot agent · max reasoningPublished margin ±6.5 · 400 reported samples	78.70%	Not reported	Unknown
36	DeepSeek-V4-Pro-0813DeepSeekSingle-shot agent · max reasoningPublished margin ±7.1 · 400 reported samples	78.50%	Not reported	Unknown
37	Muse Spark 1.1MetaSingle-shot agent · xhigh reasoningPublished margin ±6.1 · 400 reported samples	78.20%	Not reported	Unknown
38	Grok 4.6xAISingle-shot agent · xhigh reasoningPublished margin ±7 · 400 reported samples	78.20%	Not reported	Unknown
39	GLM-5.3ZhipuSingle-shot agent · max reasoningPublished margin ±6.2 · 400 reported samples	78.00%	Not reported	Unknown
40	Grok 4.7xAISingle-shot agent · xhigh reasoningPublished margin ±7.4 · 384 reported samples	77.40%	Not reported	Unknown
41	Opus 4.6AnthropicSingle-shot agent · max reasoningPublished margin ±7.4 · 400 reported samples	77.20%	Not reported	Unknown
42	Gemini 3.5 Flash LiteGoogleSingle-shot agent · high reasoningPublished margin ±7.4 · 400 reported samples	76.70%	Not reported	Unknown
43	Qwen3.5AlibabaSingle-shot agentPublished margin ±7 · 400 reported samples	76.00%	Not reported	Unknown
44	Gemini 3.5 FlashGoogleSingle-shot agent · high reasoningPublished margin ±7.5 · 400 reported samples	74.80%	Not reported	Unknown
45	InklingThinking MachinesSingle-shot agent · high reasoningPublished margin ±7.6 · 400 reported samples	74.50%	Not reported	Unknown
46	Nemotron 3 UltraNVIDIASingle-shot agent · high reasoningPublished margin ±7.3 · 400 reported samples	73.50%	Not reported	Unknown
47	Kimi K2 (Thinking)KimiSingle-shot agent · high reasoningPublished margin ±6.5 · 400 reported samples	73.30%	Not reported	Unknown
48	DeepSeek-V4-FlashDeepSeekSingle-shot agent · max reasoningPublished margin ±6.6 · 400 reported samples	73.00%	Not reported	Unknown
49	DeepSeek-V3.2DeepSeekSingle-shot agentPublished margin ±7.2 · 400 reported samples	69.50%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's Long Context Reasoning (AA-LCR v1.1) results on October 15, 2026. The table preserves 49 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

The headline is the source page's selected Pass@1 metric. The task set, harness, tools, and grader belong to this Mercor run and may differ from another evaluator's results.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists llm-as-a-judge (lmaaj) and a judge of Gemini 3.5 Flash. Its run-count label is k = 4. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Pass@149 configurationsMercor evaluationDisplay only

Mercor Long Context Reasoning (AA-LCR v1.1) results Mercor APEX benchmark index Mercor methodology Upstream dataset

The published Long Context Reasoning (AA-LCR v1.1) (Mercor) snapshot places Fable 5.1 first at 87.50%. The third row is 2.20 points behind. The broader top-10 range is 4.20 points, so many of the published results sit in a relatively narrow band.

49 configurations are shown for Long Context Reasoning (AA-LCR v1.1) (Mercor). The benchmark falls in the Reasoning category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Long Context Reasoning (AA-LCR v1.1) (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Long Context Reasoning (AA-LCR v1.1) (Mercor)

Tasks

100 source-reported tasks

Format

Pass@1

Difficulty

Source-specific evaluation

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

Long Context Reasoning (AA-LCR v1.1) evaluation results Public benchmark source

Freshness and provenance

Version

Long Context Reasoning (AA-LCR v1.1); Mercor Pass@1

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does Long Context Reasoning (AA-LCR v1.1) (Mercor) measure?

Synthesizing information across long collections of documents. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published Long Context Reasoning (AA-LCR v1.1) (Mercor) snapshot?

Fable 5.1 currently leads the published Long Context Reasoning (AA-LCR v1.1) (Mercor) snapshot with 87.50% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on Long Context Reasoning (AA-LCR v1.1) (Mercor)?

The October 15, 2026 capture snapshot contains 49 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.