Skip to main content
BenchLM
Data

APEX-Accounting — Mercor evaluation (APEX-Accounting (Mercor))

We show this table for reference; we do not rank on it.

Month-end accounting tasks that require work across files and applications. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Evaluation results published by Mercor. Published evaluation aggregates only; task contents and dataset license grants are not included.

Mean Score on APEX-Accounting (Mercor) — October 15, 2026 capture

We mirror the published mean score view for APEX-Accounting (Mercor). Opus 5.5 leads the public snapshot at 61.80%, followed by Fable 5.1 (61.00%) and Opus 5.5 (59.90%). We do not use these results to rank models overall.

1

Opus 5.5

Anthropic

Loop · max reasoning

Published margin ±4 · Pass@1 14.8% ±4.9 · 637 reported samples

61.80%
2

Fable 5.1

Anthropic

Loop · max reasoning

Published margin ±3.7 · Pass@1 11.7% ±4.3 · 639 reported samples

61.00%
3

Opus 5.5

Anthropic

Loop · medium reasoning

Published margin ±3.7 · Pass@1 12.7% ±4.6 · 640 reported samples

59.90%

34 configurationsAgenticDated Mercor configurationsDisplay onlyUpdated October 15, 2026 capture

Mean Score table (34 configurations)

Mean Score results for APEX-Accounting (Mercor)
RankModel / configurationScoreParameters (B)Open / closed
1Opus 5.5AnthropicLoop · max reasoningPublished margin ±4 · Pass@1 14.8% ±4.9 · 637 reported samples
61.80%
Not reportedUnknown
2Fable 5.1AnthropicLoop · max reasoningPublished margin ±3.7 · Pass@1 11.7% ±4.3 · 639 reported samples
61.00%
Not reportedUnknown
3Opus 5.5AnthropicLoop · medium reasoningPublished margin ±3.7 · Pass@1 12.7% ±4.6 · 640 reported samples
59.90%
Not reportedUnknown
4GPT-6 AstraOpenAILoop · max reasoningPublished margin ±3.8 · Pass@1 11.9% ±4.5 · 640 reported samples
57.90%
Not reportedUnknown
5Fable 5AnthropicLoop · max reasoningPublished margin ±3.6 · Pass@1 9.8% ±4 · 480 reported samples
56.40%
Not reportedUnknown
6Opus 5AnthropicLoop · max reasoningPublished margin ±3.9 · Pass@1 10.5% ±3.9 · 640 reported samples
54.50%
Not reportedUnknown
7Muse Spark 1.3MetaLoop · max reasoningPublished margin ±3.7 · Pass@1 10.8% ±3.9 · 630 reported samples
53.50%
Not reportedUnknown
8GLM-5.3ZhipuLoop · max reasoningPublished margin ±3.7 · Pass@1 9.2% ±3.3 · 639 reported samples
53.10%
Not reportedUnknown
9Muse Spark 1.1MetaLoop · xhigh reasoningPublished margin ±3.6 · Pass@1 7.7% ±3.1 · 480 reported samples
52.60%
Not reportedUnknown
10Gemini 3.8 FlashGoogleLoop · high reasoningPublished margin ±3.9 · Pass@1 8.7% ±3.7 · 640 reported samples
51.70%
Not reportedUnknown
11GPT-5.6 SolOpenAIHarness not reported for this record · max reasoningPublished margin ±3.9 · Pass@1 8.4% ±3.5 · Sample count not reported
51.50%
Not reportedUnknown
12GPT-5.6 Sol (Pro)OpenAILoop · max reasoningPublished margin ±3.9 · Pass@1 9.6% ±4.1 · 480 reported samples
51.50%
Not reportedUnknown
13Muse Spark 1.2MetaLoop · xhigh reasoningPublished margin ±4 · Pass@1 8.9% ±3.6 · 599 reported samples
51.40%
Not reportedUnknown
14Kimi K3KimiLoop · max reasoningPublished margin ±3.6 · Pass@1 6.2% ±2.7 · 635 reported samples
49.90%
Not reportedUnknown
15GPT-5.5OpenAILoop · xhigh reasoningPublished margin ±3.6 · Pass@1 6.2% ±3 · 633 reported samples
48.20%
Not reportedUnknown
16DeepSeek-V4.1-FlashDeepSeekLoop · max reasoningPublished margin ±3.6 · Pass@1 7.8% ±3.4 · 640 reported samples
48.10%
Not reportedUnknown
17Opus 4.8AnthropicLoop · max reasoningPublished margin ±3.6 · Pass@1 4.2% ±2.5 · 477 reported samples
48.00%
Not reportedUnknown
18GPT-5.4OpenAILoop · xhigh reasoningPublished margin ±3.8 · Pass@1 5.3% ±2.7 · 637 reported samples
46.80%
Not reportedUnknown
19Grok 4.7xAILoop · xhigh reasoningPublished margin ±3.1 · Pass@1 4.2% ±1.8 · 638 reported samples
46.70%
Not reportedUnknown
20Grok 4.5xAILoop · high reasoningPublished margin ±3.5 · Pass@1 4.5% ±2.3 · 631 reported samples
43.70%
Not reportedUnknown
21Opus 4.6AnthropicLoop · max reasoningPublished margin ±3.5 · Pass@1 4.4% ±2.7 · 616 reported samples
43.30%
Not reportedUnknown
22GLM-5.2ZhipuHarness not reported for this recordPublished margin ±3.7 · Pass@1 4% ±1.8 · Sample count not reported
42.70%
Not reportedUnknown
23GPT-5.6 TerraOpenAILoop · max reasoningPublished margin ±3.9 · Pass@1 6% ±2.6 · 634 reported samples
40.30%
Not reportedUnknown
24MiniMax-M3MiniMaxLoop · high reasoningPublished margin ±3.3 · Pass@1 2.2% ±1.2 · 623 reported samples
39.10%
Not reportedUnknown
25GPT-5.6 LunaOpenAILoop · max reasoningPublished margin ±3.9 · Pass@1 4.4% ±2.6 · 629 reported samples
38.00%
Not reportedUnknown
26Kimi K2.7 CodeKimiHarness not reported for this record · auto reasoningPublished margin ±3.3 · Pass@1 2.9% ±1.7 · Sample count not reported
37.00%
Not reportedUnknown
27Gemini 3.1 ProGoogleLoop · high reasoningPublished margin ±3 · Pass@1 3.1% ±2.1 · 622 reported samples
34.00%
Not reportedUnknown
28Qwen3.5AlibabaLoopPublished margin ±2.9 · Pass@1 0% ±0 · 471 reported samples
24.40%
Not reportedUnknown
29InklingThinking MachinesLoop · high reasoningPublished margin ±2.8 · Pass@1 0.8% ±0.9 · 638 reported samples
24.40%
Not reportedUnknown
30Nemotron 3 UltraNVIDIALoop · high reasoningPublished margin ±3 · Pass@1 0.3% ±0.4 · 602 reported samples
22.90%
Not reportedUnknown
31DeepSeek-V3.2DeepSeekLoopPublished margin ±2.8 · Pass@1 1.1% ±0.8 · 636 reported samples
20.70%
Not reportedUnknown
32GLM-5.1ZhipuLoopPublished margin ±4.3 · Pass@1 1.2% ±1.3 · 266 reported samples
18.60%
Not reportedUnknown
33GLM-5.3-FlashZhipuLoop · max reasoningPublished margin ±2.8 · Pass@1 1.7% ±1.4 · 640 reported samples
10.80%
Not reportedUnknown
34GPT-OSS-120BOpenAILoop · high reasoningPublished margin ±1 · Pass@1 0% ±0 · 636 reported samples
2.10%
Not reportedUnknown

How to read this Mercor evaluation

We captured Mercor's APEX-Accounting results on October 15, 2026. The table preserves 34 published configurations, their source identifiers, reasoning settings, and disclosed mean score values. These are results from Mercor's evaluation setup; we did not rerun them.

This dated source table is display only and does not enter overall or category rankings. Source configurations retain their external identity, including Pro descriptors and different effort settings.

Mean score is the percentage of rubric criteria passed; Pass@1 requires all criteria for a task to pass. The dedicated accounting page describes 160 tasks, the Loop harness, a three-attempt mean-criteria headline, and 95% task-level bootstrap intervals from 10,000 resamples. Its eight-attempt reliability description and the general methodology's four-run single-shot description differ. Three legacy scalar records omit a harness and sample count; their labels retain that missing information. This table remains separate from Ramp Accounting Bench.

The row labels retain published error margins and sample counts when supplied. Missing margins and counts remain unreported; a sample count is not used to reconstruct a task denominator. Except where the dedicated source defines an interval, the confidence level is unreported. Equal displayed scores and overlapping margins do not establish statistical ties. Capture time does not establish individual evaluation dates.

The general methodology lists expert-authored rubrics, scored criterion by criterion and a judge of LLM judge over expert rubrics. Its run-count label is k = 4. The linked result page takes precedence where its setup differs, as noted above.

Snapshot

Mean Score34 configurationsMercor evaluationDisplay only
Mercor APEX-Accounting resultsMercor APEX benchmark indexMercor methodologySource paperMercor evaluation notes

The published APEX-Accounting (Mercor) snapshot places Opus 5.5 first at 61.80%. The third row is 1.90 points behind. The broader top-10 range is 10.10 points, so the table still separates the published systems.

34 configurations are shown for APEX-Accounting (Mercor). The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. APEX-Accounting (Mercor) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About APEX-Accounting (Mercor)

Tasks

160 held-out accounting tasks

Format

Mean Score

Difficulty

Source-specific evaluation

The selected headline is Mean Score. Source model identifiers, effort settings, published margins, and reported sample counts remain attached to the source configuration. Public-set results and Mercor task extensions occupy separate tables.

APEX-Accounting evaluation resultsPublic benchmark source

Freshness and provenance

Version

APEX-Accounting; Mercor Mean Score; loop_agent

Refresh cadence

Reviewed dated source capture

Staleness state

Dated Mercor configurations

Question availability

Upstream task access and terms; only aggregate results mirrored

Dated Mercor configurationsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does APEX-Accounting (Mercor) measure?

Month-end accounting tasks that require work across files and applications. This table shows Mercor-run configurations for reference and is excluded from model rankings.

Which model leads the published APEX-Accounting (Mercor) snapshot?

Opus 5.5 currently leads the published APEX-Accounting (Mercor) snapshot with 61.80% mean score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on APEX-Accounting (Mercor)?

The October 15, 2026 capture snapshot contains 34 source configurations.

Last updated: October 15, 2026 capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.