Skip to main content
BenchLM
Data

Data as of October 8, 2026 · How the score is built

CurrentOpen WeightReasoning

Released Oct 8, 2026131072 tokens context

Mellum2.1-12B-A2.5B-Thinking

Decision readingMellum2.1-12B-A2.5B-Thinking is tracked, but not publicly ranked yet. The profile exposes 7 sourced benchmark rows and leaves unsupported fields blank until a published record exists.

Released Oct 8, 2026 — see all recent releases

Decision snapshot

Each value carries a field reference instead of floating alone. Markers compare this model with the current ranked and priced catalog; they are not absolute quality thresholds.

Capability

Unranked

field median 52Not eligible for a public rank

Price

Self-hosted; infrastructure cost varies

input median $0.95No comparable first-party hosted token rate

Speed

Not measured

field median 97 tok/sTime to first token not measured

Context

131072 tokenstokens

field median 256,000Maximum output length is tracked separately

Strongest published evidence

Published rows are visible, but no category has enough eligible evidence for a comparative rank.

Validate before choosing

7 published rows leave some tracked benchmark slots empty. No comparable first-party API token rate is published.

Source-linked · 7 displayable benchmark rows

Follow model changes

Category score record

Scores and ranks appear only where published evidence can be displayed. The table keeps the score, weight, cohort, and evidence state together.

Category scores, ranks, weighting, benchmark coverage, and evidence status
CategoryScore
AgenticWeight 22%1 benchmarkVerified
Score pending
CodingRank Not rankedWeight 20%3 benchmarksVerified
19.8
ReasoningWeight 17%0 benchmarksNot measured
Not measured
MultimodalWeight 12%0 benchmarksNot measured
Not measured
KnowledgeWeight 12%2 benchmarksVerified
Score pending
MultilingualWeight 7%0 benchmarksNot measured
Not measured
Inst. FollowingWeight 5%1 benchmarkVerified
Score pending
MathWeight 5%0 benchmarksNot measured
Not measured

7 of 665 tracked benchmark slots have displayable evidence · bars run 0–100

Coverage details

How much of this is verified

Coverage is split by category so a strong number never hides a thin evidence base. Verified means the row is tied to a published source; provisional rows remain visible but separate.

  1. Agentic1/1 verified
  2. Coding3/3 verified
  3. ReasoningNot measured
  4. MultimodalNot measured
  5. Knowledge2/2 verified
  6. MultilingualNot measured
  7. Inst. Following1/1 verified
  8. MathNot measured
Verified sourceProvisionalNot measured

Benchmark ledger

Coding opens by default. The marker compares each value with the best source-verified result in the catalog; provisional leaders do not set the reference. Expand the remaining categories for every published row.

Coding3 rows
Coding benchmark values, best verified comparison, weight, and source status
SWE-bench ProScore28%Versus best verified row

Best verified: Claude Opus 5.5 · 89.9%

Gap61.9 behindWeight26% ref. weight
LiveCodeBench v6Score82.0%Versus best verified row

Best verified: Sakana Fugu-Ultra · 93.2%

Gap11.2 behindWeightDisplay only
SWE-bench VerifiedSoftware Engineering Benchmark VerifiedScore47%Versus best verified row

Best verified: Claude Opus 5 · 96%

Gap49 behindWeightDisplay only
Agentic1 row
Agentic benchmark values, best verified comparison, weight, and source status
Terminal-Bench 2.1Score17.4%Versus best verified row

Best verified: SWE-2 · 92.8%

Gap75.4 behindWeight8% ref. weight
Knowledge2 rows
Knowledge benchmark values, best verified comparison, weight, and source status
MMLU-ReduxScore87.8%Versus best verified row

Best verified: Qwen3.7 Max · 95%

Gap7.2 behindWeightDisplay only
GPQA-DGPQA DiamondScore64.6%Versus best verified row

Best verified: GPT-6 Astra · 96.0%

Gap31.4 behindWeightDisplay only
Inst. Following1 row
Inst. Following benchmark values, best verified comparison, weight, and source status
IFEvalInstruction-Following EvalScore90.6%Versus best verified row

Best verified: Qwen3.5-27B · 95%

Gap4.4 behindWeightDisplay only

Bars run 0–100; the dark tick marks the best source-verified value

All 7 rows

Lineage

The sequence follows explicit supersedes links. Each score is estimated for that model; a relative can inform a sparse estimate but never sets a floor, so a newer release can score below an earlier one. Scores and prices remain blank when the corresponding public row or first-party rate is unavailable.

  1. May 28, 2026

    Mellum2-12B-A2.5B-Thinking

    Not publicly ranked · Price not listed

  2. Oct 8, 2026 · you are here

    Mellum2.1-12B-A2.5B-Thinking

    Not publicly ranked · Price not listed

Thinking

Radar

Mellum2.1-12B-A2.5B-Thinking release history

Full release history

Radar confirmed these at the source. Use Mellum2.1-12B-A2.5B-Thinking in your work? Explore Radar to follow supported changes and choose your alerts.

Radar

Spec sheet

Each documented value carries its source. Missing fields stay visible as not sourced or not published, rather than disappearing from the page.

Weights model ID
JetBrains/Mellum2.1-12B-A2.5B-ThinkingJetBrains Mellum2.1 Thinking model card
Context window
131072 tokensJetBrains Mellum2.1 Thinking config
Maximum output
Not sourced yet
Knowledge cutoff
Not sourced yet
Parameters
12B total · 2.5B active · MOE
Availability
Hugging Face · Self-hosted vLLM · GGUF: llama.cpp, Ollama, LM StudioJetBrains Mellum2.1 Thinking GGUF model card
Cloud regions
Not tracked yet
API capabilities
Tool calling, structured outputs, and batch support are not tracked yet
Prompt caching
Not documented in the pricing recordJetBrains Mellum2.1 Thinking model card
Self-host
Open weights available; hardware estimate not sourced
Rate limits
Not tracked yet

How to read this profile

The visual layer above carries the decisions. These notes preserve the model, ranking, coverage, and family context behind the numbers.

We track Mellum2.1-12B-A2.5B-Thinking, but the public leaderboard excludes this profile until enough non-generated benchmark coverage is available. Only published rows appear above.

Mellum2.1-12B-A2.5B-Thinking is a open weight model with a 131072 tokens context window. It uses an explicit reasoning mode, which can improve complex problem solving while adding latency and token use.

Apache 2.0 weights on Hugging Face, with vLLM serving instructions. Official GGUF builds support llama.cpp, Ollama, and LM Studio. No first-party hosted API token rate is published.

JetBrains released Mellum2.1 Thinking on October 8, 2026 under Apache 2.0. The MoE has 12B total parameters, 2.5B active, and a 131,072-token context confirmed by config.json. All stored scores are self-reported by JetBrains in thinking mode. Non-agentic tests use greedy decoding. SWE-bench Verified, SWE-bench Pro, and Terminal-Bench 2.1 use Pi v0.73.1 with shell and file tools, temperature 1.0, a 114K-token context, and up to 16K tokens per turn. Unmapped BFCL v4 is 62.3%, a macro-average of v1, v2, v3, web search, and memory subtasks. LiveCodeBench v6 and Terminal-Bench 2.1 remain separate from other versions. Unmapped results: HumanEval+ 91.5%, MBPP+ 79.4%, AIME 2025/2026 mean 83.3%, GSM-Plus 88.3%, WorkBench 44.6%, ToolHop 49.1%, MixEval-Hard 46.4%, XSTest 88.8%, and HarmBench harmful rate 8.5% (lower is better). The combined AIME result is not an individual-year score. GGUF builds are quantizations of this model, without separate task-benchmark results.

Its explicit predecessor is Mellum2-12B-A2.5B-Thinking. 7 of 665 tracked benchmark slots currently have displayable evidence. Missing categories stay blank.

Last updated October 8, 2026. Runtime fields remain blank until a sourced snapshot exists.

Deployment options

Self-host and provider-specific paths stay separate from benchmark evidence so operating constraints are visible before a score becomes the whole decision.

Published weights are available, but BenchLM does not yet have a sourced parameter and VRAM profile for this exact model. Hardware cost estimates stay unavailable until that sizing record is complete.

Estimate VRAM from known parameters

Questions

How does Mellum2.1-12B-A2.5B-Thinking perform overall in AI benchmarks?

Mellum2.1-12B-A2.5B-Thinking has 7 source-displayable benchmark rows, but it does not qualify for a public overall rank. The available rows remain visible by category without being converted into a site-wide score. Missing evidence stays blank instead of being estimated from an earlier model.

Is Mellum2.1-12B-A2.5B-Thinking good for knowledge and understanding?

Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for knowledge and understanding, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.

Is Mellum2.1-12B-A2.5B-Thinking good for coding and programming?

Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for coding and programming, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.

Is Mellum2.1-12B-A2.5B-Thinking good for agentic tool use and computer tasks?

Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for agentic tool use and computer tasks, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.

Is Mellum2.1-12B-A2.5B-Thinking good for instruction following?

Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for instruction following, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.

Is Mellum2.1-12B-A2.5B-Thinking open source?

Mellum2.1-12B-A2.5B-Thinking is an open-weight model from JetBrains. Its weights can be downloaded for local or hosted deployment, subject to the published license. Open weight does not automatically mean open source: training data and training code may remain private, and commercial restrictions can still apply.

Does Mellum2.1-12B-A2.5B-Thinking have full benchmark coverage on BenchLM?

No. Mellum2.1-12B-A2.5B-Thinking currently has 7 source-displayable rows across 665 tracked benchmark slots. The profile exposes published, non-generated evidence and leaves missing categories blank until an exact evaluation is available. Coverage describes how much was measured; it is not a penalty added to an individual benchmark result.

What is the context window size of Mellum2.1-12B-A2.5B-Thinking?

Mellum2.1-12B-A2.5B-Thinking has a documented context window of 131072 tokens. That figure is the maximum combined prompt and retained-conversation space reported for this exact model; it is not the maximum output length. The profile keeps output limits separate because providers often publish those limits independently.

Watch Mellum2.1-12B-A2.5B-Thinking in the weekly brief

Get one weekly email when material rank, price, availability, or benchmark evidence changes are worth revisiting.

Read a sample issue

Join 5,500+ readers.

Compare Mellum2.1-12B-A2.5B-Thinking with every tracked model887 comparisons