RuneBench / runescape-bench (RuneScape-Bench)
We show this table for reference; we do not rank on it.
An agentic coding benchmark where models use a TypeScript SDK to play a RuneScape-like environment and optimize skill-training performance.
Avg ln(1 + XP/min) on RuneScape-Bench — 30-minute skill runs; complete 16-skill configurations
We mirror the published avg ln(1 + xp/min) view for RuneScape-Bench. GPT-6 Astra high leads the public snapshot at 7.3%, followed by GPT-6 Astra (6.9%) and Gemini 4 Argon (6.9%). We do not use these results to rank models overall.
GPT-6 Astra high
OpenAI
GPT-6 Astra
OpenAI
Gemini 4 Argon
103 modelsAgenticCurrentDisplay onlyUpdated 30-minute skill runs; complete 16-skill configurations
Avg ln(1 + XP/min) table (103 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 Astra highOpenAI | 7.3% | Not reported | Unknown |
| 2 | GPT-6 AstraOpenAI | 6.9% | Not reported | Closed |
| 3 | Gemini 4 ArgonGoogle | 6.9% | Not reported | Closed |
| 4 | GPT-6.1 Sol highOpenAI | 6.9% | Not reported | Unknown |
| 5 | GPT-6.1 SolOpenAI | 6.8% | Not reported | Closed |
| 6 | GPT-6.1 Sol lowOpenAI | 6.7% | Not reported | Unknown |
| 7 | Claude Opus 5.5 xhighAnthropic | 6.7% | Not reported | Unknown |
| 8 | Claude Opus 5.5Anthropic | 6.6% | Not reported | Closed |
| 9 | GPT-6.1 TerraOpenAI | 6.6% | Not reported | Closed |
| 10 | GPT-6 Sol highOpenAI | 6.4% | Not reported | Unknown |
| 11 | GPT-6 SolOpenAI | 6.4% | Not reported | Closed |
| 12 | Grok 4.6xAI | 6.3% | Not reported | Closed |
| 13 | Claude Fable 5.1 xhighAnthropic | 6.1% | Not reported | Unknown |
| 14 | Gemini 3.8 FlashGoogle | 6.0% | Not reported | Closed |
| 15 | Gemini 3.7 FlashGoogle | 6.0% | Not reported | Closed |
| 16 | Claude Fable 5.1Anthropic | 6.0% | Not reported | Closed |
| 17 | Claude Fable 5Anthropic | 6.0% | Not reported | Closed |
| 18 | GPT-5.6 Terra xhighOpenAI | 5.9% | Not reported | Unknown |
| 19 | GPT-5.6 Sol xhighOpenAI | 5.9% | Not reported | Unknown |
| 20 | Grok 4.6 mediumxAI | 5.8% | Not reported | Unknown |
| 21 | DeepSeek V4.1 Flash 1017DeepSeek | 5.8% | Not reported | Open |
| 22 | Claude Opus 5 xhighAnthropic | 5.7% | Not reported | Unknown |
| 23 | Claude Opus 5Anthropic | 5.7% | Not reported | Closed |
| 24 | GPT-5.6 SolOpenAI | 5.7% | Not reported | Closed |
| 25 | GPT-5.5 xhighOpenAI | 5.7% | Not reported | Unknown |
| 26 | Grok 4.5xAI | 5.7% | Not reported | Closed |
| 27 | DeepSeek V4.1 FlashDeepSeek | 5.7% | Not reported | Unknown |
| 28 | Claude Opus 5 mediumAnthropic | 5.7% | Not reported | Unknown |
| 29 | Kimi K3.1 ProMoonshot AI | 5.7% | Not reported | Pending |
| 30 | Claude Fable 5 xhighAnthropic | 5.7% | Not reported | Unknown |
| 31 | Grok 4.7xAI | 5.7% | Not reported | Closed |
| 32 | Grok 4.6 xhighxAI | 5.6% | Not reported | Unknown |
| 33 | Gemini 3.6 FlashGoogle | 5.6% | Not reported | Closed |
| 34 | GPT-5.6 TerraOpenAI | 5.5% | Not reported | Closed |
| 35 | Claude Opus 5 lowAnthropic | 5.5% | Not reported | Unknown |
| 36 | Claude Sonnet 5.5 xhighAnthropic | 5.5% | Not reported | Unknown |
| 37 | Muse Spark 1.1Meta | 5.5% | Not reported | Closed |
| 38 | Gemini 3.5 FlashGoogle | 5.4% | Not reported | Closed |
| 39 | Hy4 previewTencent | 5.4% | Not reported | Open |
| 40 | Grok 4.7 xhighxAI | 5.4% | Not reported | Unknown |
| 41 | GLM 5.3 (Prime)Z.AI | 5.4% | Not reported | Unknown |
| 42 | GPT-5.5OpenAI | 5.3% | Not reported | Closed |
| 43 | Qwen3.8 MaxAlibaba | 5.3% | Not reported | Open |
| 44 | Beam 1.0 501BUnknown | 5.3% | Not reported | Unknown |
| 45 | GPT-5.6 Luna xhighOpenAI | 5.3% | Not reported | Unknown |
| 46 | Grok 4.5 mediumxAI | 5.3% | Not reported | Unknown |
| 47 | Muse Spark 1.2Meta | 5.2% | Not reported | Closed |
| 48 | GLM 5.2 (WandB fp4)Z.AI | 5.2% | Not reported | Unknown |
| 49 | Claude Opus 4.8 maxAnthropic | 5.1% | Not reported | Unknown |
| 50 | GPT-6 Luna highOpenAI | 5.1% | Not reported | Unknown |
| 51 | GLM 5.3Z.AI | 5.0% | Not reported | Unknown |
| 52 | Muse WonderMeta | 5.0% | Not reported | Closed |
| 53 | Claude Opus 4.8Anthropic | 5.0% | Not reported | Closed |
| 54 | Gemini 3.5 Flash highGoogle | 4.9% | Not reported | Unknown |
| 55 | GPT-6 LunaOpenAI | 4.9% | Not reported | Closed |
| 56 | GPT-5.6 LunaOpenAI | 4.8% | Not reported | Closed |
| 57 | Claude Sonnet 5.5Anthropic | 4.8% | Not reported | Closed |
| 58 | Claude Sonnet 5 xhighAnthropic | 4.8% | Not reported | Unknown |
| 59 | GLM 5.2Z.AI | 4.8% | Not reported | Unknown |
| 60 | DeepSeek V4 FlashDeepSeek | 4.8% | Not reported | Unknown |
| 61 | Claude Opus 4.7 xhighAnthropic | 4.7% | Not reported | Unknown |
| 62 | GPT-5.4OpenAI | 4.7% | Not reported | Closed |
| 63 | Gemini 3 FlashGoogle | 4.7% | Not reported | Closed |
| 64 | Claude Opus 4.7Anthropic | 4.6% | Not reported | Closed |
| 65 | Muse Spark 1.3Meta | 4.6% | Not reported | Closed |
| 66 | Muse Spark OpenMeta | 4.6% | Not reported | Open |
| 67 | Gemini 3.1 ProGoogle | 4.5% | Not reported | Closed |
| 68 | Claude Sonnet 5Anthropic | 4.5% | Not reported | Closed |
| 69 | Claude Haiku 5.5 highAnthropic | 4.5% | Not reported | Unknown |
| 70 | Claude Opus 4.6Anthropic | 4.4% | Not reported | Closed |
| 71 | Nemotron 3.5 SuperNVIDIA | 4.4% | Not reported | Unknown |
| 72 | Codex CLI 5.3OpenAI | 4.3% | Not reported | Unknown |
| 73 | DeepSeek V4 ProDeepSeek | 4.3% | Not reported | Unknown |
| 74 | Claude Haiku 5.5Anthropic | 4.3% | Not reported | Closed |
| 75 | Gemini 3.5 Flash-LiteGoogle | 4.2% | Not reported | Closed |
| 76 | DeepSeek V4 Flash 0731DeepSeek | 4.2% | Not reported | Open |
| 77 | Kimi K2.7 CodeMoonshot AI | 4.2% | Not reported | Open |
| 78 | Claude Opus 4.5Anthropic | 4.1% | Not reported | Closed |
| 79 | GPT-5.4 MiniOpenAI | 4.1% | Not reported | Unknown |
| 80 | Qwen3.7 MaxAlibaba | 4.1% | Not reported | Closed |
| 81 | Mistral Large 4Mistral | 3.9% | Not reported | Pending |
| 82 | Kimi K3Moonshot AI | 3.8% | Not reported | Pending |
| 83 | InklingThinking Machines Lab | 3.8% | Not reported | Open |
| 84 | Gemini 3 ProGoogle | 3.8% | Not reported | Closed |
| 85 | GLM 5.3 FlashZ.AI | 3.8% | Not reported | Unknown |
| 86 | Nemotron 3 UltraNVIDIA | 3.8% | Not reported | Open |
| 87 | Claude Sonnet 4Anthropic | 3.7% | Not reported | Unknown |
| 88 | Grok 4.3xAI | 3.7% | Not reported | Closed |
| 89 | Kimi K2.6Moonshot AI | 3.6% | Not reported | Open |
| 90 | Mistral Large 4 highMistral | 3.3% | Not reported | Unknown |
| 91 | Claude Sonnet 4.6Anthropic | 3.2% | Not reported | Closed |
| 92 | Claude Sonnet 4.5Anthropic | 3.2% | Not reported | Closed |
| 93 | kimi3-lowMoonshot AI | 3.2% | Not reported | Unknown |
| 94 | Laguna S 2.1Poolside | 2.8% | Not reported | Open |
| 95 | Qwen3.8 27BAlibaba | 2.8% | Not reported | Unknown |
| 96 | GPT-5.4 NanoOpenAI | 2.3% | Not reported | Unknown |
| 97 | Kimi K2.5Moonshot AI | 2.1% | Not reported | Open |
| 98 | GLM 5Z.AI | 1.9% | Not reported | Unknown |
| 99 | Claude Haiku 4.5Anthropic | 1.6% | Not reported | Closed |
| 100 | Gemma 4 31BGoogle | 1.6% | Not reported | Open |
| 101 | GPT-OSS-120BOpenAI | 1.5% | Not reported | Unknown |
| 102 | Qwen3 MaxAlibaba | 1.4% | Not reported | Closed |
| 103 | Qwen3.5 35BAlibaba | 0.7% | Not reported | Unknown |
How RuneScape-Bench is shown here
BenchLM mirrors RuneBench, the public results site for runescape-bench. The snapshot averages ln(1 + XP/min) across 16 skill-training tasks where coding agents use a TypeScript SDK to control a RuneScape-like game environment.
RuneScape-Bench is display only on BenchLM because it measures agent scaffolds and gameplay automation strategies as much as base model quality. BenchLM keeps it as external agentic-coding context.
Snapshot
The published RuneScape-Bench snapshot places GPT-6 Astra high first at 7.3%. The third row is 0.4 points behind. The broader top-10 range is 0.9 points, so many of the published results sit in a relatively narrow band.
103 models have been evaluated on RuneScape-Bench. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. RuneScape-Bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About RuneScape-Bench
Year
2026
Tasks
16 RuneScape skill-training tasks
Format
Average log XP-rate score
Difficulty
Agentic gameplay automation
RuneBench evaluates gameplay automation and coding-agent strategy. BenchLM mirrors the public aggregate computed as average ln(1 + XP/min) across 16 skill-training tasks, while keeping the benchmark display-only because rows reflect agent harness and gameplay strategy.
Freshness and provenance
Version
RuneScape-Bench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does RuneScape-Bench measure?
An agentic coding benchmark where models use a TypeScript SDK to play a RuneScape-like environment and optimize skill-training performance.
Which model leads the published RuneScape-Bench snapshot?
GPT-6 Astra high currently leads the published RuneScape-Bench snapshot with 7.3% avg ln(1 + xp/min). BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on RuneScape-Bench?
The 30-minute skill runs; complete 16-skill configurations snapshot contains 103 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.