# Harvey LAB-AA v1.1

> Legal-agent task success requiring every rubric criterion to pass and no material hallucination, averaged across three judges.

Canonical page: https://benchlm.ai/benchmarks/harvey-lab-aa-v1-1

- Category: [Agentic](/agentic)
- Last updated: 2026-10-15 capture

Evaluation results published by [Artificial Analysis and Harvey](https://artificialanalysis.ai/evaluations/harvey-lab-aa). Published v1.1 aggregate results; legal task documents and submissions remain private.

Version 1.1 tests legal deliverables on 120 private tasks across 24 practice areas in the Stirrup agent harness. A three-judge panel grades each deliverable, and a separate check compares its claims with the source documents.

The headline hallucination-gated all-pass rate credits a task only when every rubric criterion passes and the deliverable contains no material hallucination, averaged across three judges. Criterion pass rate pools passed criteria across all three judges before the hallucination gate. The two metrics have separate tables.

Artificial Analysis developed v1.1 with Harvey and says its scores are not comparable with v1.0. We keep every published effort configuration and preserve the earlier table separately. These new results stay outside overall and category rankings.

- [Official Harvey LAB-AA v1.1 results](https://artificialanalysis.ai/evaluations/harvey-lab-aa)
- [Evaluation methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking#harvey-lab-aa)
- [Criterion pass rates](/benchmarks/harvey-lab-aa-v1-1-criterion-pass)
- [Archived Harvey LAB-AA v1.0 results](/benchmarks/aaharveylab)

## About Harvey LAB-AA v1.1

- Year: 2026
- Tasks: 120 private legal tasks across 24 practice areas
- Format: Hallucination-Gated All-Pass Rate
- Difficulty: Professional legal work
- Paper: [Harvey LAB-AA v1.1 Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/harvey-lab-aa)

Stirrup agent harness on 120 private legal tasks across 24 practice areas. Version 1.1 changes the task set and grading; scores are not comparable with v1.0. Each effort and fallback setting remains a separate configuration. Results are display only.

Harvey LAB-AA v1.1 is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (24 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Muse Wonder Full (Max)](/models/muse-wonder-full) | Max · Stirrup | Meta | 13.1% |
| 2 | [Muse Wonder (Max)](/models/muse-wonder) | Max · Stirrup | Meta | 10.0% |
| 3 | [Grok 4.7 (Xhigh)](/models/grok-4-7) | Xhigh · Stirrup | xAI | 9.4% |
| 4 | [Muse Spark 1.3 (Max)](/models/muse-spark-1-3) | Max · Stirrup | Meta | 8.9% |
| 5 | [GPT-6 Astra (Max)](/models/gpt-6-astra) | Max · Stirrup | OpenAI | 8.6% |
| 6 | [GPT-6.1 Sol (Max)](/models/gpt-6-1-sol) | Max · Stirrup | OpenAI | 6.9% |
| 7 | [Claude Fable 5.1 (Max, Default Fallback)](/models/claude-fable-5-1) | Max, Default Fallback · Stirrup | Anthropic | 6.4% |
| 8 | [Kimi K3 (Max)](/models/kimi-k3) | Max · Stirrup | Moonshot AI | 5.3% |
| 9 | [Claude Opus 5.5 (Max, Default Fallback)](/models/claude-opus-5-5) | Max, Default Fallback · Stirrup | Anthropic | 4.2% |
| 10 | [GPT-6 Sol (Max)](/models/gpt-6-sol) | Max · Stirrup | OpenAI | 3.6% |
| 11 | [GPT-6 Luna (Max)](/models/gpt-6-luna) | Max · Stirrup | OpenAI | 3.3% |
| 12 | [Mistral Large 4 Preview](/models/mistral-large-4) | Stirrup | Mistral | 3.1% |
| 13 | [Claude Sonnet 5.5 (Max, Default Fallback)](/models/claude-sonnet-5-5) | Max, Default Fallback · Stirrup | Anthropic | 2.8% |
| 14 | [Claude Haiku 5.5 (Xhigh)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Xhigh · Stirrup | Anthropic | 2.8% |
| 15 | [DeepSeek V4.1 Flash (Max)](/models/deepseek-v4-1-flash) | Max · Stirrup | DeepSeek | 1.7% |
| 16 | [Claude Haiku 5.5 (Low)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Low · Stirrup | Anthropic | 1.7% |
| 17 | [Claude Haiku 5.5 (High)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | High · Stirrup | Anthropic | 1.1% |
| 18 | [GLM-5.3-Flash](/models/glm-5-3-flash) | Stirrup | Z.AI | 0.8% |
| 19 | [Claude Haiku 5.5 (Max)](/models/claude-haiku-5-5) | Max · Stirrup | Anthropic | 0.6% |
| 20 | [Claude Haiku 5.5 (Medium)](https://artificialanalysis.ai/evaluations/harvey-lab-aa#score) | Medium · Stirrup | Anthropic | 0.6% |
| 21 | [GLM-5.3 (Max)](/models/glm-5-3) | Max · Stirrup | Z.AI | 0.3% |
| 22 | [Gemini 3.8 Flash (High)](/models/gemini-3-8-flash) | High · Stirrup | Google | 0.0% |
| 23 | [MiniMax-M3](/models/minimax-m3) | Stirrup | MiniMax | 0.0% |
| 24 | [Inkling (Xhigh)](/models/inkling) | Xhigh · Stirrup | Thinking Machines Lab | 0.0% |

## FAQ

### What does Harvey LAB-AA v1.1 measure?

Legal-agent task success requiring every rubric criterion to pass and no material hallucination, averaged across three judges.

### Which model leads the published Harvey LAB-AA v1.1 snapshot?

Muse Wonder Full (Max) currently leads the published Harvey LAB-AA v1.1 snapshot with a score of 13.1%.

### How many models are evaluated on Harvey LAB-AA v1.1?

The 2026-10-15 capture contains 24 AI models.

### Does Harvey LAB-AA v1.1 affect BenchLM's overall score?

Not directly. Harvey LAB-AA v1.1 is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.