Skip to main content
BenchLM
Data

SciCode Subproblem Standard (Kaggle task v3) (SciCode Subproblem Standard (Kaggle))

We show this table for reference; we do not rank on it.

Data verified

A separate Kaggle SciCode track for decomposed scientific coding problems without background notes.

Evaluation results published by Kaggle Benchmarks. How Kaggle Benchmarks reruns tests.

Problem success rate on SciCode Subproblem Standard (Kaggle) — 2026-10-20 source capture

We mirror the published problem success rate view for SciCode Subproblem Standard (Kaggle). Claude Opus 5.5 has the highest reported value at 52.2%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. We do not use these results to rank models overall.

1

Claude Opus 5.5

Task version 3; Evaluated 2026-09-24

52.2%
2

Gemini 4 Argon

Task version 3; Evaluated 2026-10-12

45.0%
3

Gemini 3.1 Pro Preview

Task version 3

44.9%

62 configurationsCodingRefreshingDisplay onlyUpdated 2026-10-20 source capture

Problem success rate table (62 configurations)

Problem success rate results for SciCode Subproblem Standard (Kaggle)
RankModel / configurationScoreParameters (B)Open / closed
1Claude Opus 5.5Creator not reportedTask version 3; Evaluated 2026-09-24
52.2%
Not reportedUnknown
2Gemini 4 ArgonCreator not reportedTask version 3; Evaluated 2026-10-12
45.0%
Not reportedUnknown
3Gemini 3.1 Pro PreviewCreator not reportedTask version 3
44.9%
Not reportedUnknown
4Claude Sonnet 5.5Creator not reportedTask version 3; Evaluated 2026-09-30
43.3%
Not reportedUnknown
5GPT-6 AstraCreator not reportedTask version 3; Evaluated 2026-09-14
43.3%
Not reportedUnknown
6Gemini 3.7 FlashCreator not reportedTask version 3; Evaluated 2026-08-28
43.0%
Not reportedUnknown
7Claude Opus 5Creator not reportedTask version 3; Evaluated 2026-07-27
42.6%
Not reportedUnknown
8GPT-5.6 SolCreator not reportedTask version 3; Evaluated 2026-07-22
42.6%
Not reportedUnknown
9Grok 4.5Creator not reportedTask version 3; Evaluated 2026-07-21
41.9%
Not reportedUnknown
10Gemini 3.8 FlashCreator not reportedTask version 3; Evaluated 2026-09-06
40.2%
Not reportedUnknown
11Claude Sonnet 5Creator not reportedTask version 3; Evaluated 2026-07-23
40.1%
Not reportedUnknown
12Gemini 3.5 FlashCreator not reportedTask version 3
39.5%
Not reportedUnknown
13GPT-5.6 TerraCreator not reportedTask version 3; Evaluated 2026-07-22
39.2%
Not reportedUnknown
14Claude Opus 4.8Creator not reportedTask version 3
38.1%
Not reportedUnknown
15Gemini 3.6 FlashCreator not reportedTask version 3; Evaluated 2026-07-21
37.8%
Not reportedUnknown
16GPT-6.1 SolCreator not reportedTask version 3; Evaluated 2026-10-05
37.8%
Not reportedUnknown
17o3Creator not reportedTask version 3; Evaluated 2025-05-22
36.4%
Not reportedUnknown
18o4 miniCreator not reportedTask version 3; Evaluated 2025-05-22
36.1%
Not reportedUnknown
19o3 miniCreator not reportedTask version 3; Evaluated 2025-05-22
34.7%
Not reportedUnknown
20Claude Haiku 5.5Creator not reportedTask version 3; Evaluated 2026-10-09
34.4%
Not reportedUnknown
21GPT-5.6 LunaCreator not reportedTask version 3; Evaluated 2026-07-21
34.4%
Not reportedUnknown
22GPT-5Creator not reportedTask version 3
33.3%
Not reportedUnknown
23Gemma 4 31BCreator not reportedTask version 3
33.2%
Not reportedUnknown
24Claude Sonnet 4Creator not reportedTask version 3; Evaluated 2025-05-23
31.3%
Not reportedUnknown
25Gemini 3 Flash PreviewCreator not reportedTask version 3
30.9%
Not reportedUnknown
26Gemini 2.5 Pro PreviewCreator not reportedTask version 3; Evaluated 2025-06-06
30.9%
Not reportedUnknown
27GPT-4.1Creator not reportedTask version 3; Evaluated 2025-05-22
30.9%
Not reportedUnknown
28Gemini 2.5 Flash PreviewCreator not reportedTask version 3; Evaluated 2025-05-23
30.9%
Not reportedUnknown
29Claude Opus 4Creator not reportedTask version 3; Evaluated 2025-05-23
30.6%
Not reportedUnknown
30Claude 3.7 SonnetCreator not reportedTask version 3; Evaluated 2025-05-22
30.6%
Not reportedUnknown
31Gemini 3.5 Flash-LiteCreator not reportedTask version 3; Evaluated 2026-07-21
29.6%
Not reportedUnknown
32DeepSeek-R1Creator not reportedTask version 3; Evaluated 2025-06-04
29.6%
Not reportedUnknown
33o1Creator not reportedTask version 3; Evaluated 2025-05-22
29.2%
Not reportedUnknown
34Grok 3Creator not reportedTask version 3; Evaluated 2025-05-26
29.2%
Not reportedUnknown
35Gemini 1.5 ProCreator not reportedTask version 3; Evaluated 2025-05-22
28.5%
Not reportedUnknown
36DeepSeek-V3Creator not reportedTask version 3; Evaluated 2025-05-22
27.8%
Not reportedUnknown
37gpt-oss-120bCreator not reportedTask version 3; Evaluated 2025-08-15
27.5%
Not reportedUnknown
38Gemini 3.1 Flash-Lite PreviewCreator not reportedTask version 3
27.5%
Not reportedUnknown
39Claude Opus 4.1Creator not reportedTask version 3; Evaluated 2025-08-26
27.1%
Not reportedUnknown
40Grok 3 MiniCreator not reportedTask version 3; Evaluated 2025-05-26
26.8%
Not reportedUnknown
41deepseek-v3.1Creator not reportedTask version 3; Evaluated 2025-08-27
26.5%
Not reportedUnknown
42Qwen 3 235B A22B Instruct 2506Creator not reportedTask version 3; Evaluated 2025-09-03
25.1%
Not reportedUnknown
43o1 miniCreator not reportedTask version 3; Evaluated 2025-05-22
25.1%
Not reportedUnknown
44Claude 3.5 SonnetCreator not reportedTask version 3; Evaluated 2025-05-22
24.1%
Not reportedUnknown
45GPT-4oCreator not reportedTask version 3; Evaluated 2025-05-22
24.1%
Not reportedUnknown
46Gemini 2.0 FlashCreator not reportedTask version 3; Evaluated 2025-05-22
22.7%
Not reportedUnknown
47Mistral Large 2Creator not reportedTask version 3; Evaluated 2025-05-22
22.7%
Not reportedUnknown
48gpt-oss-20bCreator not reportedTask version 3; Evaluated 2025-08-15
22.7%
Not reportedUnknown
49Claude 3.5 HaikuCreator not reportedTask version 3; Evaluated 2025-05-22
22.0%
Not reportedUnknown
50Grok 2Creator not reportedTask version 3; Evaluated 2025-05-22
21.0%
Not reportedUnknown
51Gemini 1.5 FlashCreator not reportedTask version 3; Evaluated 2025-05-22
20.3%
Not reportedUnknown
52CodestralCreator not reportedTask version 3; Evaluated 2025-05-22
17.2%
Not reportedUnknown
53GPT-4o miniCreator not reportedTask version 3; Evaluated 2025-05-22
16.5%
Not reportedUnknown
54Granite 4.0 SmallCreator not reportedTask version 3; Evaluated 2025-09-30
16.5%
Not reportedUnknown
55Mixtral 8x22BCreator not reportedTask version 3; Evaluated 2025-05-22
15.5%
Not reportedUnknown
56Gemma 3 27BCreator not reportedTask version 3; Evaluated 2025-05-22
14.8%
Not reportedUnknown
57Gemini 1.5 Flash 8BCreator not reportedTask version 3; Evaluated 2025-05-22
14.4%
Not reportedUnknown
58GPT-3.5 TurboCreator not reportedTask version 3; Evaluated 2025-05-22
13.1%
Not reportedUnknown
59Gemma 3 12BCreator not reportedTask version 3; Evaluated 2025-05-22
12.0%
Not reportedUnknown
60Ministral 8BCreator not reportedTask version 3; Evaluated 2025-05-22
8.2%
Not reportedUnknown
61Ministral 3BCreator not reportedTask version 3; Evaluated 2025-05-22
6.9%
Not reportedUnknown
62Gemma 3 4BCreator not reportedTask version 3; Evaluated 2025-05-22
4.8%
Not reportedUnknown

How to read this leaderboard

Operator receipt: 62 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5.5 at 52.2%.

Honest limit: Background notes change the knowledge available in the prompt. Main-problem and subproblem results are different tasks. Source intervals have no disclosed confidence level in this API response, and missing settings are not inferred.

How benchmark confidence works
Kaggle leaderboard and methodologyPublished leaderboard dataSciCode project and original methodologySciCode paperApache 2.0 license shown by KaggleApache 2.0 license copy

How to read these results

This Kaggle SciCode track tests decomposed scientific coding problems without background notes. The four tracks retain separate scores because supplying background information and evaluating subproblems change the task.

We preserve Kaggle task version 3, source configuration names, evaluation dates, and published intervals. The source does not state an interval confidence level here. These rows remain separate from the core SciCode, Artificial Analysis, and Mercor evaluation protocols.

Snapshot

62 configurationsTask version 3Display only
Kaggle leaderboard and methodologyPublished leaderboard dataSciCode project and original methodologySciCode paperApache 2.0 license shown by Kaggle

These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. The table retains each source configuration and its reported value.

62 configurations are shown for SciCode Subproblem Standard (Kaggle). The benchmark falls in the Coding category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. SciCode Subproblem Standard (Kaggle) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About SciCode Subproblem Standard (Kaggle)

Year

2024

Tasks

Decomposed scientific coding problems

Format

Published Kaggle leaderboard aggregates

This track reports problem success for decomposed scientific coding problems without background notes. We show the published percentage and keep it separate from the other three Kaggle tracks and other SciCode evaluation providers.

SciCode: A Research Coding Benchmark Curated by ScientistsPublic benchmark source

Freshness and provenance

Version

SciCode Subproblem Standard (Kaggle) 2024

Refresh cadence

Annual

Staleness state

Refreshing

Question availability

Public benchmark set

RefreshingDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does SciCode Subproblem Standard (Kaggle) measure?

A separate Kaggle SciCode track for decomposed scientific coding problems without background notes.

Which source row has the highest reported SciCode Subproblem Standard (Kaggle) value?

Claude Opus 5.5 has the highest reported SciCode Subproblem Standard (Kaggle) value at 52.2%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on SciCode Subproblem Standard (Kaggle)?

The 2026-10-20 source capture snapshot contains 62 AI models.

Last updated: 2026-10-20 source capture · mirrored from the public benchmark leaderboard

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 6,100+ subscribers.

One email each week. Unsubscribe anytime.