What Legal AI Benchmarks Actually Measure

Four legal AI benchmarks matter in July 2026, and each crowns a different model. Claude Fable 5 leads LegalBench at 88.6. Grok 4.3 leads CaseLaw v2 at 79.3. GPT-5.6 Sol leads Legal Research Bench at 48.1. Muse Spark 1.1 leads Harvey's Legal Agent Benchmark at 20.0. Those four numbers, all from Vals AI's July 9 runs, line up into the only conclusion that matters: the closer a legal benchmark gets to actual legal work, the lower the best score gets.

We mirror Vals AI's benchmark tables in our dataset as display-only evidence, the same way we handle ResearchClawBench: visible, source-backed, and excluded from weighted rankings. This post walks the four legal tests in that snapshot, what each one measures, and where the numbers stop meaning what they appear to mean.

The ladder: four tests, four leaders

Table 1 Download CSV

Benchmark	What it asks	Leader (July 9, 2026)	Top score	Models run
LegalBench	Answer IRAC-style legal reasoning tasks	Claude Fable 5	88.6	123
CaseLaw v2	Answer questions over Canadian court cases	Grok 4.3	79.3	54
Legal Research Bench	Research questions across US law, agentically	GPT-5.6 Sol	48.1	18
Harvey's Legal Agent Benchmark	Produce finished legal work product with tools	Muse Spark 1.1	20.0	19

Read the ladder top to bottom: 88.6, then 79.3, then 48.1, then 20.0.

That slope is the finding. Legal reasoning as a quiz is close to solved. Legal research as an agent task earns half marks. Legal work product, the thing a client actually pays for, fails four times out of five for the best model anyone has measured.

Rung by rung

LegalBench: legal reasoning as a quiz

LegalBench is the open, Stanford CodeX-partnered suite that grades models on issue spotting, rule identification, conclusions, interpretation, and rhetoric. It has the broadest coverage of the four by far: 123 models, from GPT-3.5 Turbo to Claude Fable 5. And it is saturating the way MMLU did. Fable 5's 88.6 leads, but Gemini 3.1 Pro Preview sits at 87.4, GPT-5.6 Sol at 87.0, and the top eight land within roughly 2.5 points of each other.

A gap that small between models, on public tasks that have been circulating since 2023, tells you a model is not broken at legal reasoning. It no longer tells you which model to hire.

CaseLaw v2: check the date before the leader

CaseLaw v2 is a private question-answer set over Canadian court cases, and its table says Grok 4.3 leads at 79.3, ahead of GPT-5.1 at 73.4. Before quoting that anywhere, look at the metadata: the benchmark is archived, last updated May 4, 2026, and its 54 rows include no Claude Fable 5, no Claude Opus 4.8, and no GPT-5.6 anything.

Grok 4.3 is not the best model at case law. It is the best model that was still being tested when the benchmark stopped. We keep flagging this pattern because it is the single most common way benchmark tables mislead: a leaderboard with a coverage cutoff quietly becomes a leaderboard of older models.

Legal Research Bench: half marks with real stakes

This is the interesting middle rung. Legal Research Bench poses research tasks across US practice areas, written with law firms including Fisher Phillips, McDermott, and Reed Smith, and it is fresh: updated July 9 with the newest models included.

Table 2 Download CSV

Model	Score	Cost per task
GPT-5.6 Sol	48.1	$21.61
Claude Opus 4.8	43.8	$2.82
Claude Sonnet 5	41.8	$4.08
GPT-5.6 Terra	40.9	$7.60
GPT-5.5	40.4	$7.40
Claude Sonnet 4.6	38.5	$2.30

Notice the cost column before celebrating the leader. Sol's 4.3-point edge over Claude Opus 4.8 comes at 7.7 times the price per task; a firm running a thousand research queries a month would pay roughly $18,800 more for it. Whether four points of research quality is worth that depends entirely on what the queries are, which is a decision the overall column cannot make for you.

The domain split matters as much as the headline. The best score on health-law research is 80; the best on family law is 27.3, and Sol, the overall leader, scores 13.6 there. "AI legal research" is not one capability, and a model that handles a regulatory question competently can faceplant on a custody question the same afternoon.

Harvey's Legal Agent Benchmark: the work-product test

The bottom rung hands the agent documents, spreadsheets, presentations, and file-system tools, then asks for completed legal assignments across 24 practice areas, graded by expert rubrics.

Table 3 Download CSV

Model	Tasks completed	Criteria passed	Cost per task
Muse Spark 1.1	20.0%	92.9%	$0.80
Grok 4.5	12.9%	90.5%	$2.02
Claude Fable 5	11.25%	90.5%	$19.23
Claude Opus 4.8	9.6%	87.9%	$10.22
GLM 5.2	7.1%	85.6%	$2.06
Claude Sonnet 4.6	5.0%	86.7%	$3.04

Claude Fable 5 sits at the top of our overall leaderboard and completes one Harvey task in nine here, at the highest cost per attempt on the board.

Twenty out of one hundred is the current public ceiling for AI doing a lawyer's actual job.

Almost right is the expensive kind of wrong

The Harvey table hides a second number that explains the first. Alongside the overall completion score, it tracks a criteria pass rate: the share of individual rubric checks a model satisfies. Muse Spark 1.1 passes 92.9% of criteria while fully completing 20% of tasks. Fable 5 passes 90.5% of criteria and fully completes 11.25%.

So the failure mode is not gibberish. The models produce work that is roughly 90% correct on a checklist and still not a deliverable, because legal work is conjunctive: one misread authority, one missed filing requirement, one wrong number in the spreadsheet, and the assignment fails as a whole. A brief that is 90% right is not 90% useful.

Two details from the same table sharpen the point. Tax scored zero for all three top models. And the cost spread means "almost right" is priced like "right": Fable 5's attempts run twenty-four times Muse Spark's, and the meter runs whether or not the deliverable lands.

The same gap shows up one rung higher, in a different disguise. Legal Research Bench tracks a weighted-pass rate alongside its overall score, and GPT-5.6 Sol posts 87.8 on the first against 48.1 on the second. Two independent benchmarks, two grading schemes, one consistent shape: models satisfy most of the checklist and still miss the answer that would let a lawyer stop checking.

Outside the lab, the pattern replicates. An independent audit that graded 3,000 AI legal answers found that roughly a quarter cited or applied law that does not say what the model claimed. The polished tone survives; the authority underneath it does not.

How we read these rows, and why we don't rank on them

Every number above comes from Vals AI's published runs, captured in our July 9, 2026 snapshot with source URLs attached. They stay display-only. We do not blend them into our weighted rankings, for three reasons we can defend row by row.

The run protocols differ across models. In the Harvey table alone, Muse Spark 1.1 ran at xhigh reasoning effort, Grok 4.5 at high, and Fable 5 at max compute effort; those are different machines wearing the same column header. Second, two of the four datasets are private, so nobody outside the vendor can re-run a disputed row. Third, the coverage is uneven in exactly the way the CaseLaw v2 rung shows, and averaging a fresh 19-model table with an archived 54-model one is how composite scores go quietly wrong.

We would rather show you the evidence with its seams visible than launder it into one tidy legal score.

Which test to check before you pick a model

Match the rung to your job.

Table 4 Download CSV

Your job	Table to read	What it says today
Screening for basic legal literacy	LegalBench	Pass/fail gate; the high-80s cluster all qualify, gaps inside it are noise
Research memos, first-pass analysis	Legal Research Bench	Read the practice-area columns, not the overall; weigh Sol's quality edge against its cost
Autonomous drafting or transaction agents	Harvey's Legal Agent Benchmark	No model clears it; budget for human completion of every task
Case-law question answering	CaseLaw v2	Archived; treat as historical context only

Also know what none of the four measures. There is no public benchmark for contract redlining against a playbook, none for negotiation support, none for privilege and confidentiality handling, and none for whether a model formats citations to the jurisdiction you file in. Those are the tasks legal teams actually route to AI first, and the evidence for them today is vendor demos.

For anything client-facing, pair whichever rung fits with a citation-checking workflow and a human signature. The benchmarks above measure capability; they do not measure the malpractice exposure of the 10% a model gets wrong.

The row we are watching is Harvey's overall column. The first model to hold 30 or better there, at a cost per task a firm can stomach, changes legal AI from a research assistant into a junior associate. As of July 9, nobody is close.

→ Overall leaderboard · Agentic rankings · How leaderboards reward hidden weak scores

---