# STEMBench120: six-domain scientific research progress (STEMBench)

> Scores how far a model advances 120 open research tasks across biology, chemistry, physics, astronomy, Earth science and engineering on a 0 to 4 progress scale, crediting supported partial work and honest statements of what remains unresolved.

Canonical page: https://benchlm.ai/benchmarks/stembench

- Category: [Reasoning](/reasoning)
- Last updated: October 16, 2026 snapshot

## About STEMBench

- Year: 2026
- Tasks: 120 research tasks, 20 in each of six STEM domains
- Format: Audited progress level 0 to 4 per task, refined to a fractional score; complete-run mean across all tasks
- Difficulty: Open research targets; no run has reached a full-target success
- Paper: [STEMBench: six-domain model evaluation](https://www.ulam.ai/stembench)

STEMBench120 gives each model 120 research tasks, 20 per domain, and audits every response for progress toward the task's full target. An integer level records the outcome: 0 no usable progress, 1 framing or protocol, 2 meaningful partial progress with obligations still open, 3 substantial target-level progress, 4 full target met. An experimental quality index refines levels 1 to 3 into a fractional score (level plus 0.95 times the index) from supported achievement strength, obligation coverage, verification and reproducibility, scientific support, validity surviving decisive gaps, and scope honesty. The complete-run mean averages every task, zero-score attempts included; a usable-answer mean excludes assignments without usable content. Ulam publishes aggregates only for STEMBench120; the separate STEMBench-small set of 60 tasks is public under Apache 2.0 without baseline scores.

STEMBench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (10 audited runs)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [GPT-6 Astra](/models/gpt-6-astra) | With tools (Codex) · 120 of 120 usable · 9 level-3 | OpenAI | 2.81 |
| 2 | [GPT-6.1 Sol](/models/gpt-6-1-sol) | With tools (Codex) · 120 of 120 usable · 8 level-3 | OpenAI | 2.76 |
| 3 | [Muse Wonder](/models/muse-wonder) | Closed-book · 120 of 120 usable · 4 level-3 | Meta | 2.68 |
| 4 | [Claude Opus 5.5](/models/claude-opus-5-5) | Closed-book · 113 of 120 usable · 5 level-3 | Anthropic | 2.62 |
| 5 | [Hy4 preview](/models/hy4-preview) | Closed-book · 120 of 120 usable · 1 level-3 | Tencent | 2.59 |
| 6 | [Mistral Large 4](/models/mistral-large-4) | Closed-book · 118 of 120 usable · 0 level-3 | Mistral | 2.50 |
| 7 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | Closed-book · 117 of 120 usable · 0 level-3 | DeepSeek | 2.49 |
| 8 | [Gemini 4 Argon](/models/gemini-4-argon) | Closed-book · 105 of 120 usable · 3 level-3 | Google | 2.44 |
| 9 | [Qwen3.8-27B NVFP4 · Closed-book](https://www.ulam.ai/stembench#full-run-results) | Closed-book · 109 of 120 usable · 0 level-3 | Alibaba | 2.31 |
| 10 | [MiMo-V2.6-Pro](/models/mimo-v2-6-pro) | Closed-book · 100 of 120 usable · 0 level-3 | Xiaomi | 2.17 |

## FAQ

### What does STEMBench measure?

STEMBench measures research progress, not answer correctness. Each of its 120 tasks sets a research target in one of six STEM domains, and an audit scores how far the response advances it on a 0 to 4 scale, crediting supported partial work and honest statements of what remains unresolved. BenchLM shows the complete-run mean of the fractional score.

### Why do some rows say With tools and others Closed-book?

The two Codex runs used web or source retrieval, code execution, file operations and multi-day agent workflows. The closed-book runs called the model's API with no model-side tools. The settings measure different things, so every row names its setting and a model's page shows its best run with the setting in its note.

### Why is STEMBench display only?

Each score belongs to one complete agent configuration under a specific resource setting, the fractional index is experimental, and the private STEMBench120 tasks and judgments cannot be re-run. The results compare those exact runs, so they stay out of BenchLM's rankings.