ExplorationBench AlienLogic Best@3 (ExplorationBench AlienLogic)
We show this table for reference; we do not rank on it.
Tests whether a system can discover altered logic rules and use them to prove or correctly decline unseen theorems.
Best@3 held-out accuracy on ExplorationBench AlienLogic — October 15, 2026 snapshot
We mirror the published best@3 held-out accuracy view for ExplorationBench AlienLogic. Grok 4.6 leads the public snapshot at 81.0%, followed by Claude Opus 5 (79.5%) and Qwen3.8-Max (77.6%). We do not use these results to rank models overall.
Grok 4.6
xAI
xhigh reasoning
Claude Opus 5
Anthropic
max reasoning
Qwen3.8-Max
Alibaba
max reasoning
10 systemsAgenticCurrentDisplay onlyUpdated October 15, 2026 snapshot
| Rank | System / reasoning | Before exploration | Best@3 | Mean@3 | Worst | Parameters (B) | Open / closed |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.6xAI · xhigh | 42.9% | 81.0% | 72.1% | 65.7% | Not reported | Closed |
| 2 | Claude Opus 5Anthropic · max | 43.3% | 79.5% | 75.9% | 69.5% | Not reported | Closed |
| 3 | Qwen3.8-MaxAlibaba · max | 45.2% | 77.6% | 74.3% | 71.9% | Not reported | Open |
| 4 | GPT-5.6 SolOpenAI · max | 51.0% | 76.7% | 75.1% | 72.4% | Not reported | Closed |
| 5 | Hy4 previewTencent · high | 35.2% | 76.2% | 69.2% | 61.4% | Not reported | Open |
| 6 | DeepSeek-V4-ProDeepSeek · max | 33.8% | 75.2% | 66.3% | 61.4% | Not reported | Unknown |
| 7 | Kimi K3Moonshot AI · max | 45.2% | 74.3% | 69.4% | 61.4% | Not reported | Unknown |
| 8 | DeepSeek-V4.1-FlashDeepSeek · max | 51.4% | 73.8% | 60.6% | 52.9% | Not reported | Open |
| 9 | Seed2.1 ProByteDance · high | 37.6% | 69.0% | 59.8% | 53.3% | Not reported | Closed |
| 10 | Gemini 3.8 FlashGoogle · high | 45.7% | 59.5% | 59.0% | 58.6% | Not reported | Closed |
Before exploration and Best@3 come from the same selected run. Mean@3 and Worst cover all three runs. Each checkpoint averages three answers per task.
How to read ExplorationBench AlienLogic
AlienLogic tests whether a system can discover hidden rules through experiments, then use them on 70 held-out tasks. Each system gets four exploration rounds and three independent runs. An interpreter or proof checker grades the answers.
Best@3 is the best complete run. Mean@3 and Worst show how repeatable that result was; Best@3 does not combine successful answers across runs. The code and tasks are private, so we keep these tables display only and outside overall and category rankings.
Scores follow the current web table. The September 24 paper describes the protocol but reports an earlier AlienCode maximum of 87.6% versus 89.0% on the website.
Snapshot
The published ExplorationBench AlienLogic snapshot places Grok 4.6 first at 81.0%. The third row is 3.4 points behind. The broader top-10 range is 21.5 points, so the table still separates the published systems.
10 systems have been evaluated on ExplorationBench AlienLogic. The benchmark falls in the Agentic category. ExplorationBench AlienLogic is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ExplorationBench AlienLogic
Year
2026
Tasks
70 held-out theorems; 24 altered rules
Format
Held-out accuracy (Best@3)
Difficulty
Learning and applying unfamiliar inference rules
A proof checker grades 70 held-out goals after four exploration rounds; 25 goals are unprovable and require refusal. Best@3 selects one complete run, with mean and worst results retained separately. This synthetic logic world is distinct from AlienCode. Private code and tasks keep the table outside overall and category rankings.
Freshness and provenance
Version
September 2026 web protocol: AlienLogic
Refresh cadence
On source changes
Staleness state
Current
Question availability
Code and held-out tasks private
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ExplorationBench AlienLogic measure?
Tests whether a system can discover altered logic rules and use them to prove or correctly decline unseen theorems.
Which model leads the published ExplorationBench AlienLogic snapshot?
Grok 4.6 currently leads the published ExplorationBench AlienLogic snapshot with 81.0% best@3 held-out accuracy. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on ExplorationBench AlienLogic?
The October 15, 2026 snapshot snapshot contains 10 systems at their published reasoning settings.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.