Data as of November 21, 2026 · How the score is built
Mellum2.1-12B-A2.5B-Thinking
Decision readingMellum2.1-12B-A2.5B-Thinking is tracked, but not publicly ranked yet. The profile exposes 7 sourced benchmark rows and leaves unsupported fields blank until a published record exists.
Email me when its price, rank or availability changes.
Capability
Unranked
field median 47.5Not eligible for a public rank
Price
Self-hosted; infrastructure cost varies
input median $0.60No comparable first-party hosted token rate
Speed
Not measured
field median 93.5 tok/sTime to first token not measured
Context
131072 tokenstokens
field median 256,000Maximum output length is tracked separately
Decision snapshot
How much published evidence this model has, where that evidence is strongest, and what to validate before choosing it. It has no public rank, so no field position is shown.
Strongest published evidence
Published rows are visible, but no category has enough eligible evidence for a comparative rank.
Validate before choosing
7 published rows leave some tracked benchmark slots empty. No comparable first-party API token rate is published.
Source-linked · 7 displayable benchmark rows
Category score record
Scores and ranks appear only where published evidence can be displayed. The table keeps the score, weight, cohort, and evidence state together.
| Category | Score | Rank | Percentile | Weight | Benchmarks | Evidence |
|---|---|---|---|---|---|---|
| AgenticWeight 22%1 benchmarkVerified | Score pending | 1 benchmark | Verified | |||
| CodingRank Not rankedWeight 20%3 benchmarksVerified | 21.1 | 3 benchmarks | Verified | |||
| ReasoningWeight 17%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| MultimodalWeight 12%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| KnowledgeWeight 12%2 benchmarksVerified | Score pending | 2 benchmarks | Verified | |||
| MultilingualWeight 7%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| Inst. FollowingWeight 5%1 benchmarkVerified | Score pending | 1 benchmark | Verified | |||
| MathWeight 5%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured |
7 of 952 tracked benchmark slots have displayable evidence · bars run 0–100
Coverage detailsHow much of this is verified
Coverage is split by category so a strong number never hides a thin evidence base. Verified means the row is tied to a published source; provisional rows remain visible but separate.
- Agentic1/1 verified
- Coding3/3 verified
- ReasoningNot measured
- MultimodalNot measured
- Knowledge2/2 verified
- MultilingualNot measured
- Inst. Following1/1 verified
- MathNot measured
Benchmark ledger
Every result says who ran the test. For this model, JetBrains ran all 17. Coding opens by default. The tick marks the best source-verified result in the catalog; provisional leaders do not set the reference.
Coding3 rows
| Benchmark | Score | Versus best verified row | Gap | Who ran it | Use in ranking |
|---|---|---|---|---|---|
| LiveCodeBench v6 | 82.0% | Best verified: Sakana Fugu-Ultra · 93.2% | Gap11.2 behind | Reported by JetBrains | Weighted benchmark5% ref. weight |
| SWE-bench Pro | 28% | Best verified: Claude Fable 5.2 · 93% | Gap65 behind | Reported by JetBrains | Weighted benchmark4% ref. weight |
| SWE-bench VerifiedSoftware Engineering Benchmark Verified | 47% | Best verified: Claude Opus 5 · 96% | Gap49 behind | Reported by JetBrains | Shown only |
Agentic1 row
| Benchmark | Score | Versus best verified row | Gap | Who ran it | Use in ranking |
|---|---|---|---|---|---|
| Terminal-Bench 2.1 | 17.4% | Best verified: SWE-2 · 92.8% | Gap75.4 behind | Reported by JetBrains | Weighted benchmark2% ref. weight |
Knowledge2 rows
| Benchmark | Score | Versus best verified row | Gap | Who ran it | Use in ranking |
|---|---|---|---|---|---|
| GPQA-DGPQA Diamond | 64.6% | Best verified: GPT-6.2 Astra · 97.0% | Gap32.4 behind | Reported by JetBrains | Weighted benchmark3% ref. weight |
| MMLU-Redux | 87.8% | Best verified: Qwen3.7 Max · 95% | Gap7.2 behind | Reported by JetBrains | Weighted benchmark1% ref. weight |
Inst. Following1 row
| Benchmark | Score | Versus best verified row | Gap | Who ran it | Use in ranking |
|---|---|---|---|---|---|
| IFEvalInstruction-Following Eval | 90.6% | Best verified: LFM2.5-24B-A2B · 96.8% | Gap6.2 behind | Reported by JetBrains | Shown only |
Other lab-reported results10 rowsAnother metric, split or setting, or a test without a page
| Benchmark | Score | On a 0–100 scale | Who ran it | Use in ranking |
|---|---|---|---|---|
| HumanEval+Score · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline; HumanEval+ (EvalPlus) adds tests to the HumanEval problems, the board stores base HumanEval | 91.5% | Reported by JetBrains | Shown only | |
| XSTestScore · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 88.8% | Reported by JetBrains | Shown only | |
| GSM-PlusScore · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 88.3% | Reported by JetBrains | Shown only | |
| AIME 2025/2026 (mean)Mean of AIME 2025 and AIME 2026 (30 questions each) · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline; a combined result, not a single-year score | 83.3% | Reported by JetBrains | Shown only | |
| MBPP+Score · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 79.4% | Reported by JetBrains | Shown only | |
| BFCL v4Macro-average of five subtasks (v1, v2, v3, web search, memory) · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline; not the official BFCL v4 overall weighting | 62.3% | Reported by JetBrains | Shown only | |
| ToolHopScore · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 49.1% | Reported by JetBrains | Shown only | |
| MixEval-HardScore · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 46.4% | Reported by JetBrains | Shown only | |
| WorkBenchScore · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 44.6% | Reported by JetBrains | Shown only | |
| HarmBenchHarmful rate (lower is better) · Thinking mode, greedy decoding; JetBrains ran every model in the table with the same pipeline | 8.5% | Reported by JetBrains | Shown only |
Outlined bars are lab-reported. Filled bars are benchmark-run or independent. Bars run 0–100; the dark tick marks the best source-verified value.
Lineage
The sequence follows explicit supersedes links. Each score is estimated for that model; a relative can inform a sparse estimate but never sets a floor, so a newer release can score below an earlier one. Scores and prices remain blank when the corresponding public row or first-party rate is unavailable.
Thinking
Spec sheet
Each documented value carries its source. Missing fields stay visible as not sourced or not published, rather than disappearing from the page.
- Weights model ID
- JetBrains/Mellum2.1-12B-A2.5B-ThinkingJetBrains Mellum2.1 Thinking model card
- Context window
- 131072 tokensJetBrains Mellum2.1 Thinking config
- Maximum output
- Not sourced yet
- Knowledge cutoff
- Not sourced yet
- Input modalities
- textJetBrains Mellum2.1 Thinking model card
- Output modalities
- textJetBrains Mellum2.1 Thinking model card
- Parameters
- 12B total · 2.5B active · MOE
- Availability
- Hugging Face · Self-hosted vLLM · GGUF: llama.cpp, Ollama, LM StudioJetBrains Mellum2.1 Thinking GGUF model card
- Cloud regions
- Not tracked yet
- Lifecycle
- activeJetBrains Mellum2.1 Thinking model card
- API capabilities
- Tool calling, structured outputs, and batch support are not tracked yet
- Prompt caching
- Not documented in the pricing recordJetBrains Mellum2.1 Thinking model card
- Self-host
- Mellum2.1-12B-A2.5B-Thinking needs ~24GB VRAM (1× NVIDIA RTX 4090 (24GB)).
- Rate limits
- Not tracked yet
How to read this profile
The visual layer above carries the decisions. These notes preserve the model, ranking, coverage, and family context behind the numbers.
We track Mellum2.1-12B-A2.5B-Thinking, but the public leaderboard excludes this profile until enough non-generated benchmark coverage is available. Only published rows appear above.
Mellum2.1-12B-A2.5B-Thinking is a open weight model with a 131072 tokens context window. It uses an explicit reasoning mode, which can improve complex problem solving while adding latency and token use.
Apache 2.0 weights on Hugging Face, with vLLM serving instructions. Official GGUF builds support llama.cpp, Ollama, and LM Studio. No first-party hosted API token rate is published.
JetBrains released Mellum2.1 Thinking on October 8, 2026 under Apache 2.0. The MoE has 12B total parameters, 2.5B active, and a 131,072-token context confirmed by config.json. All stored scores are self-reported by JetBrains in thinking mode. Non-agentic tests use greedy decoding. SWE-bench Verified, SWE-bench Pro, and Terminal-Bench 2.1 use Pi v0.73.1 with shell and file tools, temperature 1.0, a 114K-token context, and up to 16K tokens per turn. LiveCodeBench v6 and Terminal-Bench 2.1 remain separate from other versions. JetBrains' unmapped results on BFCL v4 (a macro-average of v1, v2, v3, web search, and memory subtasks), HumanEval+, MBPP+, AIME 2025/2026 (mean), GSM-Plus, WorkBench, ToolHop, MixEval-Hard, XSTest, and HarmBench (harmful rate, lower is better) are listed in the benchmark ledger. The combined AIME result is not an individual-year score. GGUF builds are quantizations of this model, without separate task-benchmark results.
Its explicit predecessor is Mellum2-12B-A2.5B-Thinking. 7 of 952 tracked benchmark slots currently have displayable evidence. Missing categories stay blank.
Last updated November 21, 2026. Runtime fields remain blank until a sourced snapshot exists.
Deployment options
Self-host and provider-specific paths stay separate from benchmark evidence so operating constraints are visible before a score becomes the whole decision.
Self-host vs API cost
Estimates at 50,000 req/day · 1000 tokens/req average.
Questions
How does Mellum2.1-12B-A2.5B-Thinking perform overall in AI benchmarks?
Mellum2.1-12B-A2.5B-Thinking has 7 source-displayable benchmark rows, but it does not qualify for a public overall rank. The available rows remain visible by category without being converted into a site-wide score. Missing evidence stays blank instead of being estimated from an earlier model.
Is Mellum2.1-12B-A2.5B-Thinking good for knowledge and understanding?
Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for knowledge and understanding, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Mellum2.1-12B-A2.5B-Thinking good for coding and programming?
Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for coding and programming, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Mellum2.1-12B-A2.5B-Thinking good for agentic tool use and computer tasks?
Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for agentic tool use and computer tasks, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Mellum2.1-12B-A2.5B-Thinking good for instruction following?
Mellum2.1-12B-A2.5B-Thinking has source-displayable benchmark coverage for instruction following, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Mellum2.1-12B-A2.5B-Thinking open source?
Mellum2.1-12B-A2.5B-Thinking is an open-weight model from JetBrains. Its weights can be downloaded for local or hosted deployment, subject to the published license. Open weight does not automatically mean open source: training data and training code may remain private, and commercial restrictions can still apply.
Does Mellum2.1-12B-A2.5B-Thinking have full benchmark coverage on BenchLM?
No. Mellum2.1-12B-A2.5B-Thinking currently has 7 source-displayable rows. The profile exposes published, non-generated evidence and leaves missing categories blank until an exact evaluation is available. Coverage describes how much was measured; it is not a penalty added to an individual benchmark result.
What is the context window size of Mellum2.1-12B-A2.5B-Thinking?
Mellum2.1-12B-A2.5B-Thinking has a documented context window of 131072 tokens. That figure is the maximum combined prompt and retained-conversation space reported for this exact model; it is not the maximum output length. The profile keeps output limits separate because providers often publish those limits independently.