Hermes Bench: 150 agent tasks graded on the files and state the agent leaves behind (Hermes Bench)

We show this table for reference; we do not rank on it.

Tests agents on 150 tasks across 25 categories, including Hermes skills, research, diagrams and art, memory, tool use and safety. Each agent starts in a workspace with real files, some tasks add follow-up turns, and the grader checks the files and state left behind.

How to read Hermes Bench scores

Read with care: Nous Research makes both the harness and one of the suites, and every score belongs to a model run in that harness.

Saturation Not assessed: Scores come from one harness at one effort setting. Gaps between neighboring rows do not establish saturation or a significant difference, and the Opus 5.5 and Sonnet 5.5 cost figures are provisional.
Who produced the rows Mirrored table: Nous Research publishes every row. We mirror the published table and did not rerun the models. Source.
Freshness and weight Display only: Current, refreshed published leaderboard snapshots. Display only; it does not move rankings. Aggregate results published; tasks and workspaces are not mirrored.

Row statuses come from the verification registry; saturation and freshness from the benchmark lifecycle. A provider self-report is displayable evidence, not an independent run.

Hermes Bench on Hermes Bench — October 15, 2026 snapshot

We mirror the published hermes bench view for Hermes Bench. Claude Opus 5.5 leads the public snapshot at 76.7%, followed by Grok 4.7 (74.4%) and GPT-6 Astra (73.7%). We do not use these results to rank models overall.

1Closed

Claude Opus 5.5

Anthropic

Hermes Agent · high where offered reasoning

Hermes Agent · $0.516 per task

76.7%

Overall 86.56Context 1M

2Closed

Grok 4.7

xAI

Hermes Agent · high where offered reasoning

Hermes Agent · $0.69 per task

74.4%

Context 500K

3Closed

GPT-6 Astra

OpenAI

Hermes Agent · high where offered reasoning

Hermes Agent · $1.52 per task

73.7%

Overall 83.88Context 1.05M

15 modelsAgenticCurrentDisplay onlyUpdated October 15, 2026 snapshot

Models run in Hermes Agent (15)
Rank	Model	Hermes Index	Hermes Bench	TerminalBench 4	TerminalBench Science	SkillsBench
1	Claude Opus 5.5Anthropic	63.31$4.99*	76.7$0.52	54.0$6.27*	52.9$12.65	69.6$0.51
2	Grok 4.7xAI	39.32$10.77	74.4$0.69	12.7$23.00	1.4$18.41	68.8$0.99 · 71.9 raw
3	GPT-6 AstraOpenAI	56.25$11.61	73.7$1.52	41.3$12.66	57.1$29.84	52.9$2.44
4	GPT-6 SolOpenAI	44.10$2.23	73.5$0.26	23.8$2.53	18.6$5.70	60.5$0.42
5	Claude Sonnet 5.5Anthropic	53.14$2.82*	73.5$0.25	36.5$2.77*	35.7$8.02	66.9$0.23
6	GLM-5.3Z.AI	39.23$3.78	71.8$0.18	38.1$7.68	1.4$6.82	45.6$0.44
7	Kimi K3Moonshot AI	39.16$6.50	71.5$0.18	19.0$8.57	8.6$16.74	57.5$0.53
8	DeepSeek V4.1 Flash 1017DeepSeek	36.91$0.26	70.7$0.019	17.5$0.62	2.9$0.38	56.5$0.016 · 61.1 raw
9	Qwen3.8 MaxAlibaba	36.17$6.69	70.5$0.32	17.5$6.84	5.7$19.09	51.0$0.52
10	GPT-6 LunaOpenAI	33.89$0.14	70.5$0.015	6.3$0.24	4.8$0.28	54.0$0.030
11	Step 5 PreviewStepFun	32.83$1.75	69.2$0.17	12.7$2.63	7.1$3.84	42.3$0.35 · 46.6 raw
12	GLM-5.3-FlashZ.AI	34.95$0.50	68.9$0.019	25.4$0.60	2.9$1.31	42.6$0.058
13	Gemini 3.8 FlashGoogle	38.12$3.17	62.2$0.64	23.8$5.94	8.6$5.15	57.9$0.94 · 67.7 raw
14	Hy4 previewTencent	26.13$0.48	58.6$0.090	12.7$0.70	1.4$0.98	31.8$0.16
15	Ling 3.0 FlashInclusionAI	21.56$0.054	53.8$0.004	0.0$0.080	0.0$0.12	32.4$0.013

Scores are pass@1 percentages. The line beneath each score is mean cost per task in USD. Every run uses the Hermes Agent harness with high reasoning effort where the model offers it, and TerminalBench 4 runs without its 4 GPU tasks. An asterisk marks a provisional result, a partial run or cost estimate pending a full rerun. SkillsBench shows the clean score, which drops tasks where the run found the suite's public repository, with the raw score beneath it.

How to read this leaderboard

The score is the share of Hermes Bench tasks the agent passed on its first attempt inside Hermes Agent. Most tasks are graded on the workspace the agent leaves behind rather than on its final message.

Operator receipt: 15 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5.5 at 76.7%.

Honest limit: Nous Research authors the suite and makes the harness it runs in, and some tasks depend on an LLM or vision judge. The score belongs to the model run in Hermes Agent with high reasoning effort where offered. We mirror the published score and did not rerun the models.

Hermes Index leaderboard

How we show Hermes Bench

Hermes Bench is the 150-task suite Nous Research built for agents run inside Hermes Agent. It is one of the four suites in the Hermes Index. We mirror its published score for 15 models, ordered by that suite, with the suite's own cost per task.

Tasks start from seeded workspaces with real files and the grader checks the files and state the agent leaves behind. Most use deterministic checks, many add an LLM judge, and eight use a vision judge on rendered diagrams.

Nous makes both the suite and the harness, so we show it as supporting evidence and keep it out of weighted rankings.

Snapshot

15 modelsHermes Agent harnesspass@1Cost per taskDisplay only

Hermes Index leaderboard

The published Hermes Bench snapshot places Claude Opus 5.5 first at 76.7%. The third row is 3.1 points behind. The broader top-10 range is 6.3 points, so many of the published results sit in a relatively narrow band.

15 models are shown for Hermes Bench. The benchmark falls in the Agentic category. Hermes Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Hermes Bench

Year

2026

Tasks

150 tasks across 25 categories

Format

pass@1 score

Difficulty

Frontier agentic

Of the 150 tasks, 64 are graded by deterministic checks on files, state and tool evidence, 77 by deterministic checks plus an LLM judge on a rubric, 1 by an LLM judge alone and 8 by a vision judge that scores rendered diagrams three times and keeps the median. Tasks include seeded workspaces, multi-turn follow-ups and a safety task on refusing destructive Git operations.

Hermes Bench (Hermes Index)Public benchmark source

Freshness and provenance

Version

Hermes Bench (150 tasks, 25 categories)

Refresh cadence

Published leaderboard snapshots

Staleness state

Current

Question availability

Aggregate results published; tasks and workspaces are not mirrored

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What is Hermes Bench?

Hermes Bench is Nous Research's 150-task suite for agents run inside Hermes Agent. It covers Hermes skills, research, diagrams and art, memory, tool use and safety, and grades the files and state an agent leaves behind.

How is Hermes Bench graded?

Of its 150 tasks, 64 use deterministic checks, 77 combine deterministic checks with an LLM judge, 1 is judged by an LLM alone and 8 use a vision judge on rendered diagrams, scored three times with the median kept.

Why is Hermes Bench display only?

The suite and the harness it runs in both come from Nous Research, and each score belongs to a model run in that harness with a fixed effort setting. We show it as supporting evidence and keep it out of the weighted ranking.

Last updated: October 15, 2026 snapshot · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.