Scale Labs SEAL Math (SEAL Math)
We show this table for reference; we do not rank on it.
Scale’s SEAL math board on GSM1k, a new set of grade-school word problems written to mirror GSM8K without its contamination. BenchLM shows it as display-only reference data, excluded from rankings. Scale marked this board deprecated as of January 2025 and no longer adds models.
Accuracy (%) on SEAL Math — November 12, 2026 snapshot
We mirror the published accuracy (%) view for SEAL Math. Claude 3.5 Sonnet (June 2024) leads the public snapshot at 96.6%, followed by GPT-4o (August 2024) (95.7%) and Llama 3.1 405B Instruct (95.6%). We do not use these results to rank models overall.
Claude 3.5 Sonnet (June 2024)
Anthropic
GPT-4o (August 2024)
OpenAI
Llama 3.1 405B Instruct
Meta
16 modelsMathematicsCurrentDisplay onlyUpdated November 12, 2026 snapshot
Accuracy (%) table (16 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude 3.5 Sonnet (June 2024)Anthropic | 96.6% | Not reported | Unknown |
| 2 | GPT-4o (August 2024)OpenAI | 95.7% | Not reported | Unknown |
| 3 | Llama 3.1 405B InstructMeta | 95.6% | Not reported | Unknown |
| 4 | Claude 3 OpusAnthropic | 95.2% | Not reported | Closed |
| 5 | GPT-4 Turbo PreviewOpenAI | 95.1% | Not reported | Unknown |
| 6 | GPT-4o (May 2024)OpenAI | 94.8% | Not reported | Unknown |
| 7 | Gemini 1.5 Pro (August 2024)Google | 94.7% | Not reported | Unknown |
| 8 | Mistral Large 2Mistral | 93.9% | Not reported | Closed |
| 9 | Claude 3 SonnetAnthropic | 93.3% | Not reported | Unknown |
| 10 | Gemini 1.5 Pro (May 2024)Google | 92.3% | Not reported | Unknown |
| 11 | Gemini 1.5 Pro (April 2024)Google | 90.5% | Not reported | Unknown |
| 12 | Gemini 1.5 FlashGoogle | 90.1% | Not reported | Closed |
| 13 | Llama 3 70B InstructMeta | 90.1% | Not reported | Unknown |
| 14 | Mistral LargeMistral | 87.5% | Not reported | Unknown |
| 15 | Gemini 1.0 ProGoogle | 79.8% | Not reported | Closed |
| 16 | CodeLlama 34B InstructMeta | 37.5% | Not reported | Unknown |
How to read this leaderboard
Compare the published configurations as complete evaluation systems. The source can combine a base model, agent scaffold, tools, budget, and inference setting in each result.
Operator receipt: 16 sourced rows are currently displayable on this page; the leading published row is Claude 3.5 Sonnet (June 2024) at 96.6%.
Honest limit: This Scale table is display-only context, not benchmark provenance or a weighted model-only comparison. Scale marked this board deprecated as of January 2025 and no longer adds models.
How BenchLM shows SEAL Math
BenchLM mirrors 16 published rows from Scale Labs’ public SEAL Math leaderboard, captured on November 12, 2026 snapshot.
The table is display only. It is useful context for a published agent or model configuration, but it does not enter BenchLM’s overall or category rankings. Scale marked this board deprecated as of January 2025 and no longer adds models.
Snapshot
The published SEAL Math snapshot places Claude 3.5 Sonnet (June 2024) first at 96.6%. The third row is 1.0 points behind. The broader top-10 range is 4.3 points, so many of the published results sit in a relatively narrow band.
16 models have been evaluated on SEAL Math. The benchmark falls in the Mathematics category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. SEAL Math is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About SEAL Math
Year
2026
Tasks
About 1,000 problems (GSM1k)
Format
Accuracy (%)
Difficulty
External agent and model evaluation
GSM1k contains about 1,000 problems at roughly a fifth-grade level. The score is accuracy in percent. BenchLM mirrors 16 published rows captured on November 12, 2026 snapshot.
Freshness and provenance
Version
SEAL Math 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SEAL Math measure?
Scale’s SEAL math board on GSM1k, a new set of grade-school word problems written to mirror GSM8K without its contamination. BenchLM shows it as display-only reference data, excluded from rankings. Scale marked this board deprecated as of January 2025 and no longer adds models.
Which model leads the published SEAL Math snapshot?
Claude 3.5 Sonnet (June 2024) currently leads the published SEAL Math snapshot with 96.6% accuracy (%). BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on SEAL Math?
The November 12, 2026 snapshot snapshot contains 16 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.