# Hermes Bench: 150 agent tasks graded on the files and state the agent leaves behind (Hermes Bench)

> Tests agents on 150 tasks across 25 categories, including Hermes skills, research, diagrams and art, memory, tool use and safety. Each agent starts in a workspace with real files, some tasks add follow-up turns, and the grader checks the files and state left behind.

Canonical page: https://benchlm.ai/benchmarks/hermesbench

- Category: [Agentic](/agentic)
- Last updated: October 15, 2026 snapshot

Hermes Bench is the 150-task suite Nous Research built for agents run inside Hermes Agent. It is one of the four suites in the Hermes Index. We mirror its published score for 15 models, ordered by that suite, with the suite's own cost per task.

Tasks start from seeded workspaces with real files and the grader checks the files and state the agent leaves behind. Most use deterministic checks, many add an LLM judge, and eight use a vision judge on rendered diagrams.

Nous makes both the suite and the harness, so we show it as supporting evidence and keep it out of weighted rankings.

- [Hermes Index leaderboard](https://portal.nousresearch.com/bench)

## About Hermes Bench

- Year: 2026
- Tasks: 150 tasks across 25 categories
- Format: pass@1 score
- Difficulty: Frontier agentic
- Paper: [Hermes Bench (Hermes Index)](https://portal.nousresearch.com/bench)

Of the 150 tasks, 64 are graded by deterministic checks on files, state and tool evidence, 77 by deterministic checks plus an LLM judge on a rubric, 1 by an LLM judge alone and 8 by a vision judge that scores rendered diagrams three times and keeps the median. Tasks include seeded workspaces, multi-turn follow-ups and a safety task on refusing destructive Git operations.

Hermes Bench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (15 models)

| Rank | Model | Hermes Index | Hermes Bench | TerminalBench 4 | TerminalBench Science | SkillsBench (clean / raw) |
|---|---|---|---|---|---|---|
| 1 | [Claude Opus 5.5](/models/claude-opus-5-5) | 63.31 ($4.99*) | 76.7 ($0.516) | 54.0 ($6.27*) | 52.9 ($12.65) | 69.6 ($0.51) |
| 2 | [Grok 4.7](/models/grok-4-7) | 39.32 ($10.77) | 74.4 ($0.69) | 12.7 ($23) | 1.4 ($18.41) | 68.8 ($0.99) / 71.9 |
| 3 | [GPT-6 Astra](/models/gpt-6-astra) | 56.25 ($11.61) | 73.7 ($1.52) | 41.3 ($12.66) | 57.1 ($29.84) | 52.9 ($2.44) |
| 4 | [GPT-6 Sol](/models/gpt-6-sol) | 44.10 ($2.23) | 73.5 ($0.263) | 23.8 ($2.53) | 18.6 ($5.7) | 60.5 ($0.42) |
| 5 | [Claude Sonnet 5.5](/models/claude-sonnet-5-5) | 53.14 ($2.82*) | 73.5 ($0.25) | 36.5 ($2.77*) | 35.7 ($8.02) | 66.9 ($0.23) |
| 6 | [GLM-5.3](/models/glm-5-3) | 39.23 ($3.78) | 71.8 ($0.18) | 38.1 ($7.68) | 1.4 ($6.82) | 45.6 ($0.44) |
| 7 | [Kimi K3](/models/kimi-k3) | 39.16 ($6.5) | 71.5 ($0.176) | 19.0 ($8.57) | 8.6 ($16.74) | 57.5 ($0.53) |
| 8 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | 36.91 ($0.259) | 70.7 ($0.019) | 17.5 ($0.62) | 2.9 ($0.38) | 56.5 ($0.016) / 61.1 |
| 9 | [Qwen3.8 Max](/models/qwen3-8-max) | 36.17 ($6.69) | 70.5 ($0.322) | 17.5 ($6.84) | 5.7 ($19.09) | 51.0 ($0.52) |
| 10 | [GPT-6 Luna](/models/gpt-6-luna) | 33.89 ($0.141) | 70.5 ($0.015) | 6.3 ($0.24) | 4.8 ($0.28) | 54.0 ($0.03) |
| 11 | [Step 5 Preview](/models/step-5-preview) | 32.83 ($1.75) | 69.2 ($0.17) | 12.7 ($2.63) | 7.1 ($3.84) | 42.3 ($0.35) / 46.6 |
| 12 | [GLM-5.3-Flash](/models/glm-5-3-flash) | 34.95 ($0.497) | 68.9 ($0.019) | 25.4 ($0.6) | 2.9 ($1.31) | 42.6 ($0.058) |
| 13 | [Gemini 3.8 Flash](/models/gemini-3-8-flash) | 38.12 ($3.17) | 62.2 ($0.64) | 23.8 ($5.94) | 8.6 ($5.15) | 57.9 ($0.94) / 67.7 |
| 14 | [Hy4 preview](/models/hy4-preview) | 26.13 ($0.482) | 58.6 ($0.09) | 12.7 ($0.7) | 1.4 ($0.98) | 31.8 ($0.16) |
| 15 | [Ling 3.0 Flash](/models/ling-3-0-flash) | 21.56 ($0.054) | 53.8 ($0.0036) | 0.0 ($0.08) | 0.0 ($0.12) | 32.4 ($0.013) |

Scores are pass@1 percentages with mean cost per task in USD in parentheses. Every run uses Nous Research's Hermes Agent harness with high reasoning effort where offered. TerminalBench 4 runs without its 4 GPU tasks. An asterisk marks a provisional result pending a full rerun. SkillsBench shows the clean score first and the raw score after the slash.

## FAQ

### What is Hermes Bench?

Hermes Bench is Nous Research's 150-task suite for agents run inside Hermes Agent. It covers Hermes skills, research, diagrams and art, memory, tool use and safety, and grades the files and state an agent leaves behind.

### How is Hermes Bench graded?

Of its 150 tasks, 64 use deterministic checks, 77 combine deterministic checks with an LLM judge, 1 is judged by an LLM alone and 8 use a vision judge on rendered diagrams, scored three times with the median kept.

### Why is Hermes Bench display only?

The suite and the harness it runs in both come from Nous Research, and each score belongs to a model run in that harness with a fixed effort setting. We show it as supporting evidence and keep it out of the weighted ranking.