Harvey LAB — Mercor evaluation (Harvey LAB (Mercor))

We show this table for reference; we do not rank on it.

Producing legal work across practice areas and contract tasks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Pass@1 on Harvey LAB (Mercor) — October 15, 2026 capture

We mirror the published pass@1 view for Harvey LAB (Mercor). Grok 4.7 leads the public snapshot at 19.50%, followed by Muse Spark 1.2 (19.40%) and Opus 5 (17.80%). We do not use these results to rank models overall.

Grok 4.7

xAI

Harvey LAB Agent · xhigh reasoning

Published margin ±4.5 · 630 reported samples

19.50%

Muse Spark 1.2

Pass@1 table (40 configurations)

Pass@1 results for Harvey LAB (Mercor)
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	Grok 4.7xAIHarvey LAB Agent · xhigh reasoningPublished margin ±4.5 · 630 reported samples	19.50%	Not reported	Unknown
2	Muse Spark 1.2MetaHarvey LAB Agent · xhigh reasoningPublished margin ±4.6 · 639 reported samples	19.40%	Not reported	Unknown
3	Opus 5AnthropicHarvey LAB Agent · max reasoningPublished margin ±1.5 · 5,236 reported samples	17.80%	Not reported	Unknown
4	Fable 5.1AnthropicHarvey LAB Agent · max reasoningPublished margin ±4.2 · 639 reported samples	17.40%	Not reported	Unknown
5	Muse Spark 1.3MetaHarvey LAB Agent · max reasoningPublished margin ±4.1 · 639 reported samples	15.00%	Not reported	Unknown
6	Kimi K3KimiHarvey LAB Agent · max reasoningPublished margin ±1.4 · 5,216 reported samples	14.90%	Not reported	Unknown
7	GLM-5.3ZhipuHarvey LAB Agent · max reasoningPublished margin ±3.8 · 639 reported samples	14.60%	Not reported	Unknown
8	Fable 5AnthropicHarvey LAB Agent · max reasoningPublished margin ±1.4 · 5,212 reported samples	14.40%	Not reported	Unknown
9	Qwen3.8-MaxAlibabaHarvey LAB Agent · xhigh reasoningPublished margin ±1.3 · 5,204 reported samples	11.90%	Not reported	Unknown
10	DeepSeek-V4-Pro-0813DeepSeekHarvey LAB Agent · max reasoningPublished margin ±1.2 · 5,209 reported samples	11.40%	Not reported	Unknown
11	Gemini 3.8 FlashGoogleHarvey LAB Agent · high reasoningPublished margin ±3.5 · 639 reported samples	11.40%	Not reported	Unknown
12	GLM-5.3-FlashZhipuHarvey LAB Agent · max reasoningPublished margin ±3.4 · 638 reported samples	11.10%	Not reported	Unknown
13	Opus 5.5AnthropicHarvey LAB Agent · max reasoningPublished margin ±3.4 · 622 reported samples	10.80%	Not reported	Unknown
14	Grok 4.5xAIHarvey LAB Agent · high reasoningPublished margin ±1.2 · 5,209 reported samples	10.70%	Not reported	Unknown
15	Sonnet 5.5AnthropicHarvey LAB Agent · max reasoningPublished margin ±3.6 · 545 reported samples	10.00%	Not reported	Unknown
16	Opus 4.8AnthropicHarvey LAB Agent · max reasoningPublished margin ±1.1 · 5,227 reported samples	9.40%	Not reported	Unknown
17	Gemini 3.7 FlashGoogleHarvey LAB Agent · high reasoningPublished margin ±1.1 · 5,247 reported samples	9.30%	Not reported	Unknown
18	GPT-6 AstraOpenAIHarvey LAB Agent · max reasoningPublished margin ±3.3 · 638 reported samples	9.20%	Not reported	Unknown
19	GPT-6.1 SolOpenAIHarvey LAB Agent · max reasoningPublished margin ±3.4 · 616 reported samples	8.80%	Not reported	Unknown
20	Opus 4.7AnthropicHarvey LAB Agent · max reasoningPublished margin ±1.1 · 5,236 reported samples	8.70%	Not reported	Unknown
21	Sonnet 5AnthropicHarvey LAB Agent · max reasoningPublished margin ±1.1 · 5,188 reported samples	8.60%	Not reported	Unknown
22	GPT-6 AstraOpenAIHarvey LAB Agent · xhigh reasoningPublished margin ±2.8 · 635 reported samples	6.10%	Not reported	Unknown
23	MiniMax-M3MiniMaxHarvey LAB Agent · high reasoningPublished margin ±0.8 · 5,209 reported samples	5.30%	Not reported	Unknown
24	Haiku 5.5AnthropicHarvey LAB Agent · max reasoningPublished margin ±2.1 · 639 reported samples	4.90%	Not reported	Unknown
25	DeepSeek-V4-ProDeepSeekHarvey LAB Agent · max reasoningPublished margin ±0.7 · 5,247 reported samples	4.50%	Not reported	Unknown
26	Nemotron 3 UltraNVIDIAHarvey LAB Agent · high reasoningPublished margin ±2.3 · 639 reported samples	4.50%	Not reported	Unknown
27	GPT-5.5OpenAIHarvey LAB Agent · xhigh reasoningPublished margin ±0.7 · 5,235 reported samples	4.10%	Not reported	Unknown
28	Gemini 3.6 FlashGoogleHarvey LAB Agent · high reasoningPublished margin ±0.8 · 5,242 reported samples	4.10%	Not reported	Unknown
29	GPT-6 SolOpenAIHarvey LAB Agent · max reasoningPublished margin ±2.2 · 639 reported samples	4.10%	Not reported	Unknown
30	GPT-5.6 LunaOpenAIHarvey LAB Agent · max reasoningPublished margin ±1.8 · 639 reported samples	3.90%	Not reported	Unknown
31	GPT-6 LunaOpenAIHarvey LAB Agent · max reasoningPublished margin ±2 · 611 reported samples	3.10%	Not reported	Unknown
32	Kimi K2.7 CodeKimiHarvey LAB Agent · high reasoningPublished margin ±0.6 · 5,226 reported samples	2.70%	Not reported	Unknown
33	GPT-5.6 Sol (Pro)OpenAIHarvey LAB Agent · max reasoningPublished margin ±0.6 · 5,134 reported samples	2.70%	Not reported	Unknown
34	GLM-5.1ZhipuHarvey LAB AgentPublished margin ±1.6 · 639 reported samples	2.70%	Not reported	Unknown
35	GPT-5.6 TerraOpenAIHarvey LAB Agent · max reasoningPublished margin ±1.4 · 639 reported samples	2.30%	Not reported	Unknown
36	MiniMax-M2.7MiniMaxHarvey LAB Agent · high reasoningPublished margin ±0.4 · 5,244 reported samples	1.90%	Not reported	Unknown
37	Gemini 3.5 Flash LiteGoogleHarvey LAB Agent · high reasoningPublished margin ±1 · 639 reported samples	1.30%	Not reported	Unknown
38	GPT-5.4OpenAIHarvey LAB Agent · xhigh reasoningPublished margin ±0.4 · 5,242 reported samples	1.20%	Not reported	Unknown
39	Gemini 3.5 FlashGoogleHarvey LAB Agent · high reasoningPublished margin ±0.3 · 5,345 reported samples	0.70%	Not reported	Unknown
40	Gemini 3.1 ProGoogleHarvey LAB Agent · high reasoningPublished margin ±0.1 · 5,247 reported samples	0.10%	Not reported	Unknown

How to read this Mercor evaluation

We captured Mercor's Harvey LAB results on October 15, 2026. The table preserves 40 published configurations, their source identifiers, reasoning settings, and disclosed pass@1 values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

Mercor lists Harvey LAB twice on the index. We use the single linked result page once and retain its current scores, rather than combining the duplicate index cards.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists expert-authored rubrics, scored criterion by criterion and a judge of Gemini 3 Flash Preview. Its run-count label is k = 3. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Pass@140 configurationsMercor evaluationDisplay only

Mercor Harvey LAB results Mercor APEX benchmark index Mercor methodology Upstream evaluation code

The published Harvey LAB (Mercor) snapshot places Grok 4.7 first at 19.50%. The third row is 1.70 points behind. The broader top-10 range is 8.10 points, so many of the published results sit in a relatively narrow band.

40 configurations are shown for Harvey LAB (Mercor). The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Harvey LAB (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Harvey LAB (Mercor)

Tasks

1,749 source-reported tasks

Format

Pass@1

Difficulty

Source-specific evaluation

The selected headline is Pass@1. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

Harvey LAB evaluation results Public benchmark source

Freshness and provenance

Version

Harvey LAB; Mercor Pass@1

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does Harvey LAB (Mercor) measure?

Producing legal work across practice areas and contract tasks. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published Harvey LAB (Mercor) snapshot?

Grok 4.7 currently leads the published Harvey LAB (Mercor) snapshot with 19.50% pass@1. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on Harvey LAB (Mercor)?

The October 15, 2026 capture snapshot contains 40 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.