OSWorld-Verified
OSWorld-Verified is the July 2025 repaired release of OSWorld's real-computer evaluation. It measures whether a model-agent system can finish desktop and web tasks from configured starting states, with success checked by execution-based evaluators.
Two OSWorld protocols
OSWorld-Verified repairs the original desktop task suite. OSWorld 2.0 uses 108 long-horizon workflows and a separate completion metric. A score change between these pages is not a like-for-like model comparison.
Read the OSWorld 2.0 resultsTop models on OSWorld-Verified — October 15, 2026
As of October 15, 2026, Qwen3.8 Max leads the OSWorld-Verified leaderboard with 86.1% , followed by Claude Fable 5 (85%) and Claude Mythos 5 (85%).
Qwen3.8 Max
Alibaba
Claude Fable 5
Anthropic
Claude Mythos 5
Anthropic
37 modelsAgentic7% of Agentic reference weightCurrentUpdated October 15, 2026
Leaderboard (37 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Qwen3.8 MaxAlibaba | 86.1% | Not reported | Open |
| 2 | Claude Fable 5Anthropic | 85% | Not reported | Closed |
| 3 | Claude Mythos 5Anthropic | 85% | Not reported | Closed |
| 4 | Qwen3.8-27BAlibaba | 84.3% | Not reported | Open |
| 5 | Claude Opus 4.8Anthropic | 83.4% | Not reported | Closed |
| 6 | Gemini 3.6 FlashGoogle | 83% | Not reported | Closed |
| 7 | Holo3-35B-A3BH Company | 82.6% | Not reported | Open |
| 8 | MiMo-V2.6-ProXiaomi | 82% | Not reported | Open |
| 9 | Claude Sonnet 5Anthropic | 81.2% | Not reported | Closed |
| 10 | Muse Spark 1.1Meta | 80.8% | Not reported | Closed |
| 11 | MiMo-V2.6-FlashXiaomi | 80.8% | Not reported | Open |
| 12 | Holo3-122B-A10BH Company | 78.8% | Not reported | Closed |
| 13 | GPT-5.5OpenAI | 78.7% | Not reported | Closed |
| 14 | Gemini 3.5 FlashGoogle | 78.4% | Not reported | Closed |
| 15 | Claude Opus 4.7 (Adaptive)Anthropic | 78% | Not reported | Closed |
| 16 | UI-Mate-27BTencent | 77% | Not reported | Open |
| 17 | GPT-5.4OpenAI | 75% | Not reported | Closed |
| 18 | Gemini 3.5 Flash-LiteGoogle | 74% | Not reported | Closed |
| 19 | Qwen3.7 PlusAlibaba | 73.3% | Not reported | Closed |
| 20 | Kimi K2.6Moonshot AI | 73.1% | Not reported | Open |
| 21 | Claude Opus 4.6Anthropic | 72.7% | Not reported | Closed |
| 22 | Claude Sonnet 4.6Anthropic | 72.1% | Not reported | Closed |
| 23 | GPT-5.4 miniOpenAI | 72.1% | Not reported | Closed |
| 24 | MiniMax M3MiniMax | 70.1% | Not reported | Open |
| 25 | Claude Opus 4.5Anthropic | 66.3% | Not reported | Closed |
| 26 | UI-Mate-9BTencent | 66.2% | Not reported | Open |
| 27 | Muse Glimmer 30BMeta | 65.9% | Not reported | Open |
| 28 | GPT-5.3 CodexOpenAI | 64.7% | Not reported | Closed |
| 29 | Kimi K2.5 (Reasoning)Moonshot AI | 63.3% | Not reported | Closed |
| 30 | Kimi K2.5Moonshot AI | 63.3% | Not reported | Open |
| 31 | Qwen3.5 397BAlibaba | 62.2% | Not reported | Open |
| 32 | Claude Sonnet 4.5Anthropic | 61.4% | Not reported | Closed |
| 33 | Qwen3.5-122B-A10BAlibaba | 58% | Not reported | Open |
| 34 | Qwen3.5-27BAlibaba | 56.2% | Not reported | Open |
| 35 | Qwen3.5-35B-A3BAlibaba | 54.5% | Not reported | Open |
| 36 | GPT-5.2OpenAI | 47.3% | Not reported | Closed |
| 37 | GPT-5.4 nanoOpenAI | 39% | Not reported | Closed |
How to read this leaderboard
Editorial review by Glevd · 2026-07-15
Read this table as a ledger of sourced model-agent results. Match the task count, environment revision, model and agent variant, general model versus specialized model versus agentic framework, screenshot or accessibility-tree observation, action interface, prompt and scaffold, action budget, and attempt policy before comparing scores. The benchmark-hosted table uses unified settings; provider-reported rows are not automatically part of that controlled comparison.
Operator receipt: 37 sourced rows are currently displayable on this page; the leading published row is Qwen3.8 Max at 86.1%.
Honest limit: A success rate reflects the complete model-agent system, not the base model alone. Fixed applications and configured starting states do not reproduce every production login, permission boundary, network failure, application update, latency, cost, or safety control. Eight Google Drive tasks may be excluded under the official policy, and rows without displayable exact-source records remain withheld. OSWorld 2.0 results are not interchangeable with OSWorld-Verified results.
According to BenchLM.ai, Qwen3.8 Max leads the OSWorld-Verified benchmark with a score of 86.1%, followed by Claude Fable 5 (85%) and Claude Mythos 5 (85%). The top models are clustered within 1.1 points, suggesting this benchmark is nearing saturation for frontier models.
37 models have been evaluated on OSWorld-Verified. The benchmark falls in the Agentic category. BenchAlign v5.8 gives OSWorld-Verified 7% of the Agentic reference weight, so it moves the Agentic leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.
About OSWorld-Verified
Year
2025
Tasks
369 real-world computer tasks (361 when eight Google Drive tasks are excluded)
Format
Execution-based interactive task success
Difficulty
Multi-step desktop and cross-application workflows
The release covers 369 real-world tasks across desktop and web applications. The maintainers allow eight Google Drive tasks to be manually configured or excluded, making a 361-task run officially acceptable. Public evaluation requires the maintainers to run the agent or review monitoring data and trajectories. OSWorld 2.0 is a newer, separate protocol.
Freshness and provenance
Version
OSWorld Verified
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does OSWorld-Verified measure?
OSWorld-Verified measures whether a computer-use agent can complete 369 tasks across real desktop and web applications. Each task starts from a configured state and is checked with an execution-based evaluator. Eight Google Drive tasks may be manually configured or excluded, producing an officially accepted 361-task run.
Are OSWorld-Verified scores directly comparable?
Only when the setup matches. Compare the same task set, environment revision, model-agent variant, observation and action interface, prompt or scaffold, action budget, and attempt policy. The official leaderboard separates general models, specialized models, and agentic frameworks; provider-published scores may use different conditions.
Can OSWorld-Verified choose the best computer-use model?
Not alone. It is strong evidence for desktop task completion, but fixed applications cannot reproduce every production login, permission, network, app-version, latency, cost, or safety condition. Use matched OSWorld-Verified results with workflow trials, and treat OSWorld 2.0 as a separate newer protocol rather than interchangeable evidence.
Compare top models on OSWorld-Verified
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,500+ readers.
One email each week. Unsubscribe anytime.