# Company Agent Bench

> Concept's benchmark of how well an AI agent understands the company it works in. Agents answer 205 questions drawn from one month of Concept's own Slack history: who wrote a message, who was tagged and who sent each reply. The score is overall accuracy.

Canonical page: https://benchlm.ai/benchmarks/company-agent-bench

- Category: [Agentic](/agentic)
- Last updated: Launch post, Nov 12, 2026

## About Company Agent Bench

- Year: 2026
- Tasks: 205 questions from one month of Concept's Slack: 100 Authorship, 50 Mentions and 55 Reply matching panels
- Format: Replay of the Slack history in sandbox workspaces; agent products at default settings, API models in a shared Slack harness
- Difficulty: Knowing who owns what, who works with whom and how each teammate thinks, from Slack alone
- Paper: [Introducing Company Agent Bench](https://concept.dev/blog/company-agent-bench)

Concept recreated its channels, people and conversations in sandbox Slack workspaces and replayed one month of history in date order. Each agent answers with its own search and memory and sees only what was posted before the message that gives the answer. Authorship (100 questions) asks who wrote an anonymized message, Mentions (50) asks who was tagged, and Reply matching (55 panels) asks which teammate sent each reply in a thread, scored as the share of replies matched and averaged across panels. For Authorship and Reply matching, Concept standardized spelling and grammar to remove typing habits as a clue. The overall score weights the three parts equally. Four agent products ran at default settings: Dots, Grok Bot, Claude Tag and Devin. Six API models (GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, Claude Fable 5.2, Opus 5.5 and Sonnet 5.5) ran in a shared Slack harness with native Slack search and read tools at low, medium and high reasoning effort; this page shows their high-effort runs, as Concept's headline chart does. Claude Tag ran with three models; Opus 5.5 is its default view in the post.

Company Agent Bench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (12 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Claude Fable 5.2](/models/claude-fable-5-2) | high reasoning · API model in a Slack harness · low 78.9% · medium 80.0% · $1.345 per question at high | Anthropic | 82.2% |
| 2 | [GPT-6 Astra](/models/gpt-6-astra) | high reasoning · API model in a Slack harness · low 76.7% · medium 81.1% · $0.557 per question at high | OpenAI | 80.8% |
| 3 | [Claude Opus 5.5](/models/claude-opus-5-5) | high reasoning · API model in a Slack harness · low 54.5% · medium 72.5% · $0.441 per question at high | Anthropic | 79.2% |
| 4 | [GPT-6.1 Sol](/models/gpt-6-1-sol) | high reasoning · API model in a Slack harness · low 73.9% · medium 78.8% · $0.108 per question at high | OpenAI | 79.1% |
| 5 | [Dots](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · Authorship 72.0% · Mentions 74.0% · Reply matching 81.8% | OpenAI | 75.9% |
| 6 | [GPT-6.1 Terra](/models/gpt-6-1-terra) | high reasoning · API model in a Slack harness · low 70.3% · medium 76.8% · $0.036 per question at high | OpenAI | 74.3% |
| 7 | [Claude Tag (Fable 5.1)](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · runs Claude Fable 5.1 · Authorship 69.0% · Mentions 74.0% · Reply matching 76.4% | Anthropic | 73.1% |
| 8 | [Grok Bot](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · Authorship 67.0% · Mentions 68.0% · Reply matching 83.0% | xAI | 72.7% |
| 9 | [Claude Sonnet 5.5](/models/claude-sonnet-5-5) | high reasoning · API model in a Slack harness · low 35.6% · medium 52.7% · $0.147 per question at high | Anthropic | 68.7% |
| 10 | [Claude Tag (Opus 5.5)](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · runs Claude Opus 5.5 · Authorship 64.0% · Mentions 66.0% · Reply matching 73.3% | Anthropic | 67.8% |
| 11 | [Devin](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · Authorship 60.0% · Mentions 68.0% · Reply matching 70.3% | Cognition | 66.1% |
| 12 | [Claude Tag (Sonnet 5.5)](https://concept.dev/blog/company-agent-bench) | Agent product · default settings · runs Claude Sonnet 5.5 · Authorship 58.0% · Mentions 60.0% · Reply matching 70.9% | Anthropic | 63.0% |

## FAQ

### What does Company Agent Bench measure?

Whether an AI agent connected to a company's Slack understands the people in it: who wrote a message, whom a teammate would tag and who sent each reply in a thread. Concept built the 205 questions from one month of its own Slack history.

### Which agent product scores highest on Company Agent Bench?

Dots, at 75.9% overall. Grok Bot scores 72.7%, Claude Tag with Opus 5.5 67.8% and Devin 66.1%. All four ran at default settings.

### Do API models beat the agent products?

In Concept's own Slack harness, the strongest did. Fable 5.2 at high effort scored 82.2% and GPT-6 Astra 80.8%, above Dots at 75.9%. At low effort, Opus 5.5 (54.5%) and Sonnet 5.5 (35.6%) scored below all four products.