Harvey LAB-AA v1.1 Criterion Pass
We show this table for reference; we do not rank on it.
The share of legal-deliverable rubric criteria passed, pooled across three judges before the hallucination gate.
Evaluation results published by Artificial Analysis and Harvey. Published v1.1 aggregate results; legal task documents and submissions remain private.
Previous release
Harvey LAB-AA v1.0 remains a separate historical table. Version 1.1 changes the task set and grading, so the scores are not comparable across versions.
View Harvey LAB-AA v1.0Criterion Pass Rate on Harvey LAB-AA v1.1 Criterion Pass — 2026-10-15 capture
We mirror the published criterion pass rate view for Harvey LAB-AA v1.1 Criterion Pass. Muse Spark 1.3 (Max) leads the public snapshot at 96.0%, followed by Muse Wonder Full (Max) (95.5%) and Muse Wonder (Max) (95.1%). We do not use these results to rank models overall.
Muse Spark 1.3 (Max)
Meta
Max · Stirrup
Muse Wonder Full (Max)
Meta
Max · Stirrup
Muse Wonder (Max)
Meta
Max · Stirrup
24 modelsAgenticCurrentDisplay onlyUpdated 2026-10-15 capture
Criterion Pass Rate table (24 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Muse Spark 1.3 (Max)MetaMax · Stirrup | 96.0% | Not reported | Closed |
| 2 | Muse Wonder Full (Max)MetaMax · Stirrup | 95.5% | Not reported | Closed |
| 3 | Muse Wonder (Max)MetaMax · Stirrup | 95.1% | Not reported | Closed |
| 4 | Kimi K3 (Max)Moonshot AIMax · Stirrup | 93.0% | Not reported | Pending |
| 5 | Mistral Large 4 PreviewMistralStirrup | 93.0% | Not reported | Pending |
| 6 | GLM-5.3 (Max)Z.AIMax · Stirrup | 93.0% | Not reported | Open |
| 7 | Grok 4.7 (Xhigh)xAIXhigh · Stirrup | 92.8% | Not reported | Closed |
| 8 | DeepSeek V4.1 Flash (Max)DeepSeekMax · Stirrup | 92.5% | Not reported | Open |
| 9 | Claude Fable 5.1 (Max, Default Fallback)AnthropicMax, Default Fallback · Stirrup | 92.4% | Not reported | Closed |
| 10 | Claude Sonnet 5.5 (Max, Default Fallback)AnthropicMax, Default Fallback · Stirrup | 92.1% | Not reported | Closed |
| 11 | GLM-5.3-FlashZ.AIStirrup | 91.1% | Not reported | Open |
| 12 | GPT-6 Astra (Max)OpenAIMax · Stirrup | 90.6% | Not reported | Closed |
| 13 | GPT-6.1 Sol (Max)OpenAIMax · Stirrup | 90.6% | Not reported | Closed |
| 14 | Gemini 3.8 Flash (High)GoogleHigh · Stirrup | 90.5% | Not reported | Closed |
| 15 | Claude Opus 5.5 (Max, Default Fallback)AnthropicMax, Default Fallback · Stirrup | 90.0% | Not reported | Closed |
| 16 | Claude Haiku 5.5 (Xhigh)AnthropicXhigh · Stirrup | 88.0% | Not reported | Closed |
| 17 | Claude Haiku 5.5 (High)AnthropicHigh · Stirrup | 87.9% | Not reported | Closed |
| 18 | Claude Haiku 5.5 (Max)AnthropicMax · Stirrup | 87.7% | Not reported | Closed |
| 19 | Claude Haiku 5.5 (Medium)AnthropicMedium · Stirrup | 87.5% | Not reported | Closed |
| 20 | Claude Haiku 5.5 (Low)AnthropicLow · Stirrup | 87.5% | Not reported | Closed |
| 21 | GPT-6 Sol (Max)OpenAIMax · Stirrup | 86.9% | Not reported | Closed |
| 22 | GPT-6 Luna (Max)OpenAIMax · Stirrup | 86.0% | Not reported | Closed |
| 23 | MiniMax-M3MiniMaxStirrup | 85.6% | Not reported | Open |
| 24 | Inkling (Xhigh)Thinking Machines LabXhigh · Stirrup | 81.3% | Not reported | Open |
How to read this leaderboard
Operator receipt: 24 sourced rows are currently displayable on this page; the leading published row is Muse Spark 1.3 (Max) at 96.0%.
Honest limit: Private tasks and model judges limit independent reproduction. Criterion pass and hallucination-gated all-pass rates measure different outcomes.
How to read Harvey LAB-AA v1.1
Version 1.1 tests legal deliverables on 120 private tasks across 24 practice areas in the Stirrup agent harness. A three-judge panel grades each deliverable, and a separate check compares its claims with the source documents.
The headline hallucination-gated all-pass rate credits a task only when every rubric criterion passes and the deliverable contains no material hallucination, averaged across three judges. Criterion pass rate pools passed criteria across all three judges before the hallucination gate. The two metrics have separate tables.
Artificial Analysis developed v1.1 with Harvey and says its scores are not comparable with v1.0. We keep every published effort configuration and preserve the earlier table separately. These new results stay outside overall and category rankings.
Snapshot
The published Harvey LAB-AA v1.1 Criterion Pass snapshot places Muse Spark 1.3 (Max) first at 96.0%. The third row is 0.9 points behind. The broader top-10 range is 3.9 points, so many of the published results sit in a relatively narrow band.
24 models are shown for Harvey LAB-AA v1.1 Criterion Pass. The benchmark falls in the Agentic category. Harvey LAB-AA v1.1 Criterion Pass is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Harvey LAB-AA v1.1 Criterion Pass
Year
2026
Tasks
120 private legal tasks across 24 practice areas
Format
Criterion Pass Rate
Difficulty
Professional legal work
Stirrup agent harness on 120 private legal tasks across 24 practice areas. Version 1.1 changes the task set and grading; scores are not comparable with v1.0. Each effort and fallback setting remains a separate configuration. Results are display only.
Freshness and provenance
Version
Harvey LAB-AA v1.1
Refresh cadence
Published leaderboard snapshots
Staleness state
Current
Question availability
Private legal tasks; only published aggregates mirrored
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Harvey LAB-AA v1.1 Criterion Pass measure?
The share of legal-deliverable rubric criteria passed, pooled across three judges before the hallucination gate.
Which model leads the published Harvey LAB-AA v1.1 Criterion Pass snapshot?
Muse Spark 1.3 (Max) currently leads the published Harvey LAB-AA v1.1 Criterion Pass snapshot with 96.0% criterion pass rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Harvey LAB-AA v1.1 Criterion Pass?
The 2026-10-15 capture snapshot contains 24 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.