Adversarial Customer Service
We show this table for reference; we do not rank on it.
Agents play customer service staff, genuine customers, and identity thieves in a hidden-information authentication simulator.
Evaluation results published by Gert Labs. How Kaggle Benchmarks reruns tests.
Simulation score on Adversarial Customer Service v1 — Source retrieved 2026-10-20T15:19:04.675Z
We mirror the published simulation score view for Adversarial Customer Service v1. Claude Opus 4.8 has the highest reported value at 78.3%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. We do not use these results to rank models overall.
11 modelsAgenticCurrentDisplay onlyUpdated Source retrieved 2026-10-20T15:19:04.675Z
Simulation score table (11 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 4.8Creator not reported | 78.3% | Not reported | Unknown |
| 2 | Gemini 3.6 FlashCreator not reported | 78.3% | Not reported | Unknown |
| 3 | Claude Sonnet 5.5Creator not reported | 77.5% | Not reported | Unknown |
| 4 | Gemini 3.5 FlashCreator not reported | 76.7% | Not reported | Unknown |
| 5 | Claude Opus 5Creator not reported | 75.8% | Not reported | Unknown |
| 6 | GPT-5.6 LunaCreator not reported | 75.0% | Not reported | Unknown |
| 7 | GPT-5.6 SolCreator not reported | 75.0% | Not reported | Unknown |
| 8 | GPT-6 AstraCreator not reported | 75.0% | Not reported | Unknown |
| 9 | GPT-5.6 TerraCreator not reported | 74.2% | Not reported | Unknown |
| 10 | Gemini 3.7 FlashCreator not reported | 73.3% | Not reported | Unknown |
| 11 | Gemini 3.5 Flash-LiteCreator not reported | 63.3% | Not reported | Unknown |
How to read this leaderboard
Read the published score as a result in this adversarial simulation. It depends on the roles, interacting agents, and evaluation setup and does not establish a general customer-service success rate.
Operator receipt: 11 sourced rows are currently displayable on this page; the leading published row is Claude Opus 4.8 at 78.3%.
Honest limit: The public leaderboard does not fully specify per-row model settings or expose a single-agent-only comparison. Some source rows omit uncertainty. The task-version-21 results stay separate from other customer-service tests and weighted rankings.
How we show Adversarial Customer Service
A dynamic social-interaction test gives each role its own verifiable goal. Agents communicate in natural language: the service agent should authenticate legitimate customers and reject impersonators. The Kaggle v1 leaderboard publishes task-version-21 scores as percentages with source confidence intervals.
The public leaderboard does not fully specify per-row model settings or expose a single-agent-only comparison. Some source rows omit uncertainty. The task-version-21 results stay separate from other customer-service tests and weighted rankings.
Snapshot
These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. The table retains each source configuration and its reported value.
11 models are shown for Adversarial Customer Service. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Adversarial Customer Service is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Adversarial Customer Service
Year
2026
Tasks
Multi-agent customer-authentication simulation; task version 21
Format
Source-published percentage score
Difficulty
Adversarial multi-agent communication
A dynamic social-interaction test gives each role its own verifiable goal. Agents communicate in natural language: the service agent should authenticate legitimate customers and reject impersonators. The Kaggle v1 leaderboard publishes task-version-21 scores as percentages with source confidence intervals.
Freshness and provenance
Version
Adversarial Customer Service 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Adversarial Customer Service measure?
Agents play customer service staff, genuine customers, and identity thieves in a hidden-information authentication simulator.
Which source row has the highest reported Adversarial Customer Service value?
Claude Opus 4.8 has the highest reported Adversarial Customer Service value at 78.3%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Adversarial Customer Service?
The Source retrieved 2026-10-20T15:19:04.675Z snapshot contains 11 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.