Skip to main content
Radar

Every change to the models you run, with its source and its date. Releases, price changes, retirements, API changes, and incidents.Every change to the models you run, with its source.

Follow model changes
BenchLM

AutomationBench

Data verified 36 confirmed releases in the last 30 daysFollow model changes

An agent benchmark for completing automation workflows in reproducible task environments.

Top models on AutomationBench — September 18, 2026

As of September 18, 2026, DeepSeek V4.1 Flash leads the AutomationBench leaderboard with 54.8% , followed by Atria Dawn Preview (53.8%) and Muse Spark 1.3 (49.4%).

1Open weights

DeepSeek V4.1 Flash

DeepSeek

54.8%
Context 1M
2Open weights

Atria Dawn Preview

Shanghai Artificial Intelligence Laboratory

53.8%
Context 256K
3Proprietary

Muse Spark 1.3

Meta

49.4%
Context 1M

14 modelsAgentic10% of category scoreCurrentUpdated September 18, 2026

Leaderboard (14 models)

Score
1
DeepSeek V4.1 FlashDeepSeek · Open weight
54.8%
2
Atria Dawn PreviewShanghai Artificial Intelligence Laboratory · Open weight
53.8%
3
Muse Spark 1.3Meta · Closed
49.4%
4
GLM-5.3-FlashZ.AI · Open weight
48.8%
5
GLM-5.3Z.AI · Open weight
48.2%
6
GPT-6 AstraOpenAI · Closed
41.4%
7
Hy4 previewTencent · Open weight
32.1%
8
DeepSeek V4 Pro 0813DeepSeek · Closed
31.8%
9
Claude Fable 5.1Anthropic · Closed
31.4%
10
Kimi K3Moonshot AI · Closed
30.8%
11
Gemini 3.7 FlashGoogle · Closed
30.4%
12
Qwen3.8 MaxAlibaba · Open weight
27.3%
13
Claude Opus 5Anthropic · Closed
26.0%
14
DeepSeek V4 Flash 0731DeepSeek · Closed
25.1%

According to BenchLM.ai, DeepSeek V4.1 Flash leads the AutomationBench benchmark with a score of 54.8%, followed by Atria Dawn Preview (53.8%) and Muse Spark 1.3 (49.4%). There is significant spread across the leaderboard, making this benchmark effective at differentiating model capabilities.

14 models have been evaluated on AutomationBench. The benchmark falls in the Agentic category. This category carries a 22% weight in BenchLM.ai's overall scoring system. Within that category, AutomationBench contributes 10% of the category score, so strong performance here directly affects a model's overall ranking.

About AutomationBench

Year

2026

Tasks

600 public automation tasks

Format

Agent task-completion score

Difficulty

Long-horizon automation

Moonshot evaluates the 600-task public subset while following the official GitHub setup. BenchLM stores this provider-run result separately from Artificial Analysis AutomationBench-AA.

Kimi K3: Open Frontier Intelligence

Freshness and provenance

Version

AutomationBench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

Current

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does AutomationBench measure?

An agent benchmark for completing automation workflows in reproducible task environments.

Which model scores highest on AutomationBench?

DeepSeek V4.1 Flash by DeepSeek currently leads with a score of 54.8% on AutomationBench.

How many models are evaluated on AutomationBench?

14 AI models have been evaluated on AutomationBench on BenchLM.

Compare top models on AutomationBench

DeepSeek V4.1 Flash vs Atria Dawn PreviewAtria Dawn Preview vs Muse Spark 1.3Muse Spark 1.3 vs GLM-5.3-FlashGLM-5.3-Flash vs GLM-5.3
Last updated: September 18, 2026 · BenchLM version AutomationBench 2026

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.