Benign Asks, Risky Consequences (BARC-Bench)
We show this table for reference; we do not rank on it.
Scale AI's benchmark of whether AI agents stay safe while carrying out ordinary workplace requests. Agents handle 110 benign tasks in nine simulated workspaces, where the risk lies in how the work is done, such as leaking private details in a calendar invite. The score is safety pass^5: the share of tasks on which all five attempts were safe.
Safety pass^5 on BARC-Bench — Paper v2, Nov 3, 2026
We mirror the published safety pass^5 view for BARC-Bench. GPT-6 Astra leads the public snapshot at 72.7%, followed by Claude Fable 5.1 (37.3%) and Gemini 3.8 Flash (23.6%). We do not use these results to rank models overall.
GPT-6 Astra
OpenAI
OpenCode on the AgentEnv Framework · default reasoning
pass@5 90.0% · 75.8% on the 91 requests with no conflict · 57.9% on the 19 that conflict with a workspace rule
Claude Fable 5.1
Anthropic
OpenCode on the AgentEnv Framework · default reasoning
pass@5 71.8% · 37.4% on the 91 requests with no conflict · 36.8% on the 19 that conflict with a workspace rule
Gemini 3.8 Flash
OpenCode on the AgentEnv Framework · default reasoning
pass@5 46.4% · 26.4% on the 91 requests with no conflict · 10.5% on the 19 that conflict with a workspace rule
4 modelsAgenticCurrentDisplay onlyUpdated Paper v2, Nov 3, 2026
Safety pass^5 table (4 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAIOpenCode on the AgentEnv Framework · default reasoningpass@5 90.0% · 75.8% on the 91 requests with no conflict · 57.9% on the 19 that conflict with a workspace rule | 72.7% | Not reported | Closed |
| 2 | Claude Fable 5.1AnthropicOpenCode on the AgentEnv Framework · default reasoningpass@5 71.8% · 37.4% on the 91 requests with no conflict · 36.8% on the 19 that conflict with a workspace rule | 37.3% | Not reported | Closed |
| 3 | Gemini 3.8 FlashGoogleOpenCode on the AgentEnv Framework · default reasoningpass@5 46.4% · 26.4% on the 91 requests with no conflict · 10.5% on the 19 that conflict with a workspace rule | 23.6% | Not reported | Closed |
| 4 | DeepSeek V4 Pro 0813DeepSeekOpenCode on the AgentEnv Framework · default reasoningpass@5 51.8% · 9.9% on the 91 requests with no conflict · 5.3% on the 19 that conflict with a workspace rule | 9.1% | Not reported | Open |
How to read this leaderboard
Higher is better, and the bar is strict: a score of 72.7% means at least one of five attempts was unsafe on 27.3% of tasks. GPT-6 Astra leads at 72.7%, ahead of Claude Fable 5.1 (37.3%), Gemini 3.8 Flash (23.6%) and DeepSeek V4-Pro (9.1%). When one safe attempt out of five is enough (pass@5), scores rise to 90.0%, 71.8%, 46.4% and 51.8%, so the models can find a safe path but do not take it every time. Communication tasks were harder than file operations for all four models. The most common failure, for every model, was an agent that found the rule in the workspace and broke it anyway.
Operator receipt: 4 sourced rows are currently displayable on this page; the leading published row is GPT-6 Astra at 72.7%.
Honest limit: Scale tested four models, each at its default thinking budget in one harness, and the verdicts come from a model judge rather than people. The 110 tasks are synthetic workspaces, so results may not carry over to a real company. The scores measure safety, not whether the task was completed. Scale publishes no live leaderboard; BenchLM copied the scores from the paper. BenchLM shows BARC-Bench as display only; it does not enter overall or category rankings.
How BARC-Bench is shown here
BARC stands for Benign Asks, Risky Consequences. Scale AI wrote 110 ordinary workplace requests across 9 simulated workspaces, each with people, email and chat history, calendars, documents and records. Nothing in a request is harmful; the risk is in how the agent carries it out, such as putting a confidential performance plan in a calendar invite that coworkers can see.
Each model made 5 attempts per task in the OpenCode harness at its default thinking budget, and a GPT-5.6 Sol judge checked every attempt. An attempt fails if it takes any unsafe action, even if it later finishes the task or asks the user; it passes if it completes the task safely or defers to the user. The score is safety pass^5, the share of tasks on which all five attempts were safe. Each row also gives pass@5, the share on which at least one attempt was safe.
Scale tested four models and publishes no live leaderboard, so BenchLM copied the tables from the paper. The scores measure safety, not task success. BARC-Bench is display only on BenchLM.
Snapshot
The published BARC-Bench snapshot places GPT-6 Astra first at 72.7%. The third row is 49.1 points behind. The broader top-10 range is 63.6 points, so the table still separates the published systems.
4 models have been evaluated on BARC-Bench. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. BARC-Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About BARC-Bench
Year
2026
Tasks
110 benign workplace requests across nine simulated workspaces: 70 communication tasks, 34 file operations and 6 workspace application changes
Format
Agent runs in simulated workspaces with MCP tools; five attempts per task, judged by GPT-5.6 Sol against each task's rubric
Difficulty
Respecting boundaries the requester never stated, from workspace records and social norms
Each task is an everyday request from an employee to an agent working inside a simulated organization with people, email and chat history, calendars, documents and operational records. The requests are benign but leave safety boundaries unstated. Scale labels each boundary as explicit in the environment (stated in an artifact such as meeting notes, 61 tasks), implicit in the environment (inferred by connecting records, 23 tasks) or a social norm (26 tasks), and each request as communication or a workspace change. An attempt fails if it takes any unsafe action, even if it later completes the task or defers to the user; it passes if it completes the task safely or defers. Each model ran five times per task in the OpenCode harness on Scale's AgentEnv Framework at its default thinking budget. A GPT-5.6 Sol judge scored each attempt from the prompt, the full trajectory, the final artifacts and the environment state. On 300 trajectories checked by three human reviewers, the judge matched the majority verdict 90.0% of the time.
Freshness and provenance
Version
Paper v2 (November 3, 2026; same scores as v1 of October 23, 2026)
Refresh cadence
On source changes
Staleness state
Current
Question availability
Public: 110 tasks and nine environments on Hugging Face (CC BY-NC 4.0)
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does BARC-Bench measure?
Whether an AI agent avoids harmful actions while doing an ordinary, benign workplace task, such as scheduling a meeting or updating records, when the safety boundary is not stated in the request. The agent has to find the rule in the workspace or infer it from context and social norms.
Which model is safest on BARC-Bench?
GPT-6 Astra, safe on all five attempts for 72.7% of the 110 tasks. Claude Fable 5.1 scores 37.3%, Gemini 3.8 Flash 23.6% and DeepSeek V4-Pro 9.1%.
What is the difference between pass^5 and pass@5 on BARC-Bench?
Pass^5 counts a task only if all five attempts were safe; pass@5 counts it if at least one was. DeepSeek V4-Pro rises from 9.1% to 51.8% under pass@5, which shows that models can act safely but do not do so consistently.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.