# Harvey LAB-AA v1.1 Criterion Pass

> The share of legal-deliverable rubric criteria passed, pooled across three judges before the hallucination gate.

Canonical page: https://benchlm.ai/benchmarks/harvey-lab-aa-v1-1-criterion-pass

- Category: [Agentic](/agentic)
- Last updated: 2026-10-15 capture

Evaluation results published by [Artificial Analysis and Harvey](https://artificialanalysis.ai/evaluations/harvey-lab-aa). Published v1.1 aggregate results; legal task documents and submissions remain private.

Version 1.1 tests legal deliverables on 120 private tasks across 24 practice areas in the Stirrup agent harness. A three-judge panel grades each deliverable, and a separate check compares its claims with the source documents.

The headline hallucination-gated all-pass rate credits a task only when every rubric criterion passes and the deliverable contains no material hallucination, averaged across three judges. Criterion pass rate pools passed criteria across all three judges before the hallucination gate. The two metrics have separate tables.

Artificial Analysis developed v1.1 with Harvey and says its scores are not comparable with v1.0. We keep every published effort configuration and preserve the earlier table separately. These new results stay outside overall and category rankings.

- [Official Harvey LAB-AA v1.1 results](https://artificialanalysis.ai/evaluations/harvey-lab-aa)
- [Evaluation methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking#harvey-lab-aa)
- [Hallucination-gated all-pass rates](/benchmarks/harvey-lab-aa-v1-1)
- [Archived Harvey LAB-AA v1.0 results](/benchmarks/aaharveylab)

## About Harvey LAB-AA v1.1 Criterion Pass

- Year: 2026
- Tasks: 120 private legal tasks across 24 practice areas
- Format: Criterion Pass Rate
- Difficulty: Professional legal work
- Paper: [Harvey LAB-AA v1.1 Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/harvey-lab-aa)

Stirrup agent harness on 120 private legal tasks across 24 practice areas. Version 1.1 changes the task set and grading; scores are not comparable with v1.0. Each effort and fallback setting remains a separate configuration. Results are display only.

Harvey LAB-AA v1.1 Criterion Pass is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (24 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Muse Spark 1.3 (Max)](/models/muse-spark-1-3) | Max · Stirrup | Meta | 96.0% |
| 2 | [Muse Wonder Full (Max)](/models/muse-wonder-full) | Max · Stirrup | Meta | 95.5% |
| 3 | [Muse Wonder (Max)](/models/muse-wonder) | Max · Stirrup | Meta | 95.1% |
| 4 | [Kimi K3 (Max)](/models/kimi-k3) | Max · Stirrup | Moonshot AI | 93.0% |
| 5 | [Mistral Large 4 Preview](/models/mistral-large-4) | Stirrup | Mistral | 93.0% |
| 6 | [GLM-5.3 (Max)](/models/glm-5-3) | Max · Stirrup | Z.AI | 93.0% |
| 7 | [Grok 4.7 (Xhigh)](/models/grok-4-7) | Xhigh · Stirrup | xAI | 92.8% |
| 8 | [DeepSeek V4.1 Flash (Max)](/models/deepseek-v4-1-flash) | Max · Stirrup | DeepSeek | 92.5% |
| 9 | [Claude Fable 5.1 (Max, Default Fallback)](/models/claude-fable-5-1) | Max, Default Fallback · Stirrup | Anthropic | 92.4% |
| 10 | [Claude Sonnet 5.5 (Max, Default Fallback)](/models/claude-sonnet-5-5) | Max, Default Fallback · Stirrup | Anthropic | 92.1% |
| 11 | [GLM-5.3-Flash](/models/glm-5-3-flash) | Stirrup | Z.AI | 91.1% |
| 12 | [GPT-6 Astra (Max)](/models/gpt-6-astra) | Max · Stirrup | OpenAI | 90.6% |
| 13 | [GPT-6.1 Sol (Max)](/models/gpt-6-1-sol) | Max · Stirrup | OpenAI | 90.6% |
| 14 | [Gemini 3.8 Flash (High)](/models/gemini-3-8-flash) | High · Stirrup | Google | 90.5% |
| 15 | [Claude Opus 5.5 (Max, Default Fallback)](/models/claude-opus-5-5) | Max, Default Fallback · Stirrup | Anthropic | 90.0% |
| 16 | [Claude Haiku 5.5 (Xhigh)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Xhigh · Stirrup | Anthropic | 88.0% |
| 17 | [Claude Haiku 5.5 (High)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | High · Stirrup | Anthropic | 87.9% |
| 18 | [Claude Haiku 5.5 (Max)](/models/claude-haiku-5-5) | Max · Stirrup | Anthropic | 87.7% |
| 19 | [Claude Haiku 5.5 (Medium)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Medium · Stirrup | Anthropic | 87.5% |
| 20 | [Claude Haiku 5.5 (Low)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Low · Stirrup | Anthropic | 87.5% |
| 21 | [GPT-6 Sol (Max)](/models/gpt-6-sol) | Max · Stirrup | OpenAI | 86.9% |
| 22 | [GPT-6 Luna (Max)](/models/gpt-6-luna) | Max · Stirrup | OpenAI | 86.0% |
| 23 | [MiniMax-M3](/models/minimax-m3) | Stirrup | MiniMax | 85.6% |
| 24 | [Inkling (Xhigh)](/models/inkling) | Xhigh · Stirrup | Thinking Machines Lab | 81.3% |

## FAQ

### What does Harvey LAB-AA v1.1 Criterion Pass measure?

The share of legal-deliverable rubric criteria passed, pooled across three judges before the hallucination gate.

### Which model leads the published Harvey LAB-AA v1.1 Criterion Pass snapshot?

Muse Spark 1.3 (Max) currently leads the published Harvey LAB-AA v1.1 Criterion Pass snapshot with a score of 96.0%.

### How many models are evaluated on Harvey LAB-AA v1.1 Criterion Pass?

The 2026-10-15 capture contains 24 AI models.

### Does Harvey LAB-AA v1.1 Criterion Pass affect BenchLM's overall score?

Not directly. Harvey LAB-AA v1.1 Criterion Pass is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.