What Legal AI Benchmarks Actually Measure

Legal AI benchmark scores answer different questions. Vals AI's October 1, 2026 tables put Claude Fable 5 first on LegalBench at 88.6%, while three models tie on Legal Research Bench at 55.3%.

Muse Spark 1.2 has the highest Harvey LAB task pass rate, 25.4%. Archived CaseLaw v2 still lists Grok 4.3 at 79.3%, with a May 4 source update.

We keep these results as separate, display-only evidence. Grading rules, tasks, and source dates determine what a row can tell you about the work you want to test.

Four tests do not share a scale

Table 1 Download CSV

Benchmark	Task	First source row	Score	Source updated
LegalBench	Legal reasoning questions	Claude Fable 5	88.6%	October 1, 2026
CaseLaw v2	Questions about Canadian court cases	Grok 4.3	79.3%	May 4, 2026 (archived)
Legal Research Bench	Agentic US-law research	Muse Spark 1.3 Max, Claude Opus 5, and Claude Fable 5.1 tied	55.3%	October 1, 2026
Harvey's Legal Agent Benchmark	Completed legal work with files and tools	Muse Spark 1.2	25.4%	October 1, 2026

Those dates are Vals' source update stamps. We checked the rows on October 5. Capturing a table does not give it a new evaluation date.

It is tempting to read those scores as a decline from an easy quiz to a hard assignment. Tasks differ, and their grading changes too. Subtracting 25.4 on one test from 88.6 on another would not measure how much ability a model lost.

For selection, the useful question is narrower: which test contains the kind of task you need, and what counts as passing it? A source row at the top of a legal reasoning table is a reason to inspect it. It is not a substitute for a document-production or research result.

Each test asks for a different kind of answer

LegalBench checks legal reasoning on public tasks

Vals' LegalBench implementation covers issue spotting, rule identification, conclusions, interpretation, and rhetoric. Its current table has 149 source rows. Claude Fable 5 scores 88.561%, Claude Fable 5.1 scores 88.511%, and Gemini 4 Argon scores 88.299%.

We calculate a 1.587-point spread between the first and eighth rows, about 1.6 percentage points. That small range is worth keeping beside the leader's name: the order is precise, but its practical meaning depends on the task and uncertainty behind the measurements. It does not show that every leading model handles the same legal question equally well.

LegalBench is useful when its question format resembles what you are checking. It gives much less direct evidence about an agent that must locate a source, inspect a file, and produce a finished document. Pair the reasoning result with an evaluation of those steps rather than treating the high score as a general legal qualification.

CaseLaw v2 needs its archive label

CaseLaw v2 is a question-answer benchmark over Canadian cases. Its 54-row table is archived and carries a May 4, 2026 update. Grok 4.3 has 79.314%, followed by GPT-5.1 at 73.419%.

Archiving changes how we read that first row. It identifies the highest result in that recorded cohort, not a current comparison with every newer model. We retain the table because the Canadian case-law task is specific and useful historical context. Buyers comparing present-day options need to see the missing newer evaluations rather than silently treating them as losses.

Our missing-benchmark explanation covers the broader scoring mistake: an absent test result and a low test result are different things. CaseLaw v2 makes that distinction easy to see because its coverage stopped while the model catalog kept changing.

Legal Research Bench has a costly three-way tie

Legal Research Bench reports strict all-pass results on 208 private test questions across eight US-law practice areas. Vals also publishes a weighted-pass view, which gives credit for individual rubric checks. Its source table now has 73 rows.

Table 2 Download CSV

Model	Strict all-pass	Weighted pass	Reported cost per test
Muse Spark 1.3 Max	55.288%	86.8%	$0.59
Claude Opus 5	55.288%	90.6%	$6.76
Claude Fable 5.1	55.288%	90.2%	$23.06

With the overall scores tied, cost becomes more interesting than a winner's headline. Using the unrounded source costs, the Fable row costs about 39 times the Muse row per test. That is arithmetic on this benchmark's published attempts, not a quotation for your firm's monthly research bill.

Equal overall pass rates do not make the three configurations interchangeable. Their weighted scores differ, and a shared aggregate can hide different results in a practice area. We would inspect that breakdown and run the relevant questions before paying a premium for a model whose extra value the overall column cannot establish.

Harvey LAB grades a completed assignment

Harvey's benchmark extends the task to finished legal work across 24 practice areas. Its open-source harness includes task instructions, documents, and grading rubrics. Vals' published scores use the held-out set with internet access disabled.

Vals' Harvey LAB table reports both task pass and criteria pass. Passing a task requires all its criteria to pass, and the final task score averages the pass rates from two LLM judges, GPT-5.5 and Claude Sonnet 4.6.

Table 3 Download CSV

Model	Task pass	Criteria pass	Reported cost per test
Muse Spark 1.2	25.4%	94.5%	$2.09
Muse Spark 1.3 Max	23.8%	94.7%	$2.26
Muse Spark 1.3	22.9%	92.2%	$2.76

Muse Spark 1.2 leads by task pass, while Muse Spark 1.3 Max leads by criteria pass. That changes what a leader's name means before any price comparison begins. Read the metric beside the rank, and keep this Vals protocol separate from results labeled with an Anthropic internal harness or another partner's task set.

Partial credit is a different answer from all-pass

Harvey's 94.5% criteria result and 25.4% task result describe the same configuration under different aggregation rules. Individual checks can pass across many assignments while few assignments satisfy every required check. Calling both numbers accuracy without naming the rule hides the difference.

We see a related gap in Finance Agent v2. On Vals' October 1 table, Gemini 4 Argon has 65.4% partial credit and 55.8% all-pass. Finance Agent's primary score is a weighted check score gated by required dealbreakers. Its secondary score requires every check to pass. That is a separate financial task set, not a fifth legal benchmark.

Our reading is to choose the acceptance rule before comparing models. If one missing required field makes your deliverable unusable, a score that rewards many correct fields can overstate how often the complete output meets your needs. If you are evaluating a first draft that will be reviewed and corrected, individual-check results can help identify what still needs work.

Neither interpretation turns a benchmark pass rate into a legal outcome or a production-safety assessment. Each test evaluates defined tasks under its reported harness. They do not measure the consequences of relying on a wrong answer in a particular matter.

Source rows and linked profiles are different counts

We checked the current source tables against the compact mirror and counted model links using the site's resolver on October 5, 2026. Our linked count measures how many source rows connect to public model profiles. It is not the number of results the benchmark owner published.

Table 4 Download CSV

Table	Source rows	Rows linked to public model profiles	Source updated
LegalBench	149	79	October 1, 2026
Legal Research Bench	73	61	October 1, 2026
Harvey LAB	74	62	October 1, 2026
CaseLaw v2	54	35	May 4, 2026

Those counts are our coverage check, separate from the benchmark scores. A source row without a model link is still part of the owner's table. We do not invent a mapping to make the counts agree.

We also keep the rows out of the weighted model ranking. Task sets, grading rules, settings, and access differ. LegalBench is public. Legal Research Bench and CaseLaw use private data. Vals' Harvey results use a held-out set even though Harvey publishes an open harness and public tasks. Open code alone does not make every displayed score independently reproducible.

When the same benchmark name appears on two pages, check the harness and split before transferring a score. A private held-out result, an internal-harness result, and a public sample run are separate pieces of evidence until their protocols are shown to match.

Match the test to the work you need checked

Table 5 Download CSV

Work you are evaluating	Starting evidence	What to inspect next
Short legal reasoning answers	LegalBench	Relevant question types and the narrow gap among leading rows
US-law research	Legal Research Bench	Practice-area results, strict versus weighted grading, and cost per test
Documents and multi-step legal assignments	Harvey LAB	Task rubrics, the all-pass rule, harness, and held-out split
Canadian case-law questions	CaseLaw v2	Its archived cohort and missing newer model evaluations

Our legal benchmark collection groups the disclosed legal protocols without pooling them into a legal score. Use it to locate the right test, then build a small evaluation of the work you actually need done.

For research, include source checks in that evaluation: whether an authority exists, whether it supports the claim, and whether the answer addresses the requested jurisdiction and question. For document work, check the finished artifact rather than only the prose that accompanies it. Have a person review the result before relying on client-facing work.

We would investigate a result that improves the relevant all-pass rate on a disclosed protocol while retaining the source checks your task requires. A larger overall number on a different task set would leave that decision unresolved.

→ Agentic evidence · How missing tests affect a model score

---

Questions

What is the best LLM for legal work in 2026?

On Vals' October 1, 2026 tables, Claude Fable 5 has 88.6% on LegalBench, Muse Spark 1.2 has 25.4% on Harvey LAB, and Muse Spark 1.3 Max, Claude Opus 5, and Claude Fable 5.1 tie at 55.3% on Legal Research Bench. Grok 4.3's 79.3% CaseLaw result is archived. These are separate task rankings.

Why are legal AI benchmark scores so low?

Harvey's all-pass grading rejects a task if any criterion fails. Its October 1 leader satisfies 94.5% of criteria but passes 25.4% of tasks. Finance Agent v2 shows a related strictness gap: Gemini 4 Argon scores 65.4% partial credit and 55.8% all-pass. Those percentages use different rules, so read the grading definition.

What does Harvey's Legal Agent Benchmark measure?

Harvey LAB evaluates completed legal assignments across 24 practice areas using files, documents, spreadsheets, and presentations. Vals runs the held-out set with six tools, no internet, and two LLM judges. Its October 1, 2026 table lists Muse Spark 1.2 at 25.4% task pass rate. Those results apply to that harness.

Is LegalBench still a useful benchmark?

LegalBench can check legal reasoning on its public tasks, but its leading rows are close together. Vals' October 1, 2026 table contains 149 source rows, with the top eight within 1.6 percentage points. Use it alongside tests resembling the intended work. That narrow gap alone does not settle a deployment choice.

Can AI do legal research reliably?

Legal Research Bench's October 1, 2026 leaders tie at 55.3% strict all-pass accuracy on a private US-law test set. That result does not establish reliability on other questions, jurisdictions, or workflows. Inspect the relevant practice-area results and test actual research tasks, including whether the cited sources support the answer.

Where do these legal benchmark scores come from?

Vals AI publishes the four source tables. We checked their current rows and linked-profile counts on October 5, 2026. LegalBench, Legal Research Bench, and Harvey LAB carry October 1 source updates. Archived CaseLaw v2 carries May 4. We keep mirrored rows display-only because their grading, data access, and coverage differ.

---

Scores and methods: Vals AI source tables and Harvey's published benchmark materials, checked October 5, 2026. Vals update dates are October 1 for LegalBench, Legal Research Bench, Harvey LAB, and Finance Agent v2. CaseLaw v2 remains archived with a May 4 update. Coverage counts and the cost comparison are our calculations.