Model profile
Nemotron 3 Ultra
Evidence coverage
39 of 313 tracked benchmarks are published. 22 are verified and 17 provisional. 7 of 8 categories are measured.
- Published / tracked
- 39 / 313
- Verified
- 22
- Provisional
- 17
- Categories measured
- 7 / 8
Evidence by category
- Agentic12 benchmarksMixed evidence
- Coding8 benchmarksMixed evidence
- Reasoning3 benchmarksVerified
- Knowledge12 benchmarksMixed evidence
- Math0 benchmarksNot measured
- Multilingual1 benchmarkVerified
- Multimodal1 benchmarkReported
- Inst. Following2 benchmarksMixed evidence
According to BenchLM.ai, Nemotron 3 Ultra ranks #41 out of 78 models on the provisional leaderboard with an overall score of 60/100. It also ranks #24 out of 32 on the verified leaderboard. While not a frontier model, it offers specific advantages depending on the use case.
Nemotron 3 Ultra is a open weight model with a 1M token context window. It uses explicit chain-of-thought reasoning, which typically improves performance on math and complex reasoning tasks at the cost of higher latency and token usage.
This profile currently has 39 of 313 tracked benchmarks. BenchLM only exposes non-generated benchmark rows publicly, so missing categories stay blank until a sourced evaluation is available.
Its strongest category is Instruction Following (#6), while its weakest is Agentic (#66). This performance profile makes it a well-rounded choice across a range of tasks.
Peer position
Exact provisional scores and ranks for the closest listed peers.
Range 60.0–61.0
- Grok 4xAICompare#3961.0Grok 4 is #39 with a score of 61.0.
- Kimi K2.5Moonshot AICompare#4061.0Kimi K2.5 is #40 with a score of 61.0.
- Nemotron 3 UltraCurrent modelNVIDIA#4160.0Nemotron 3 Ultra is #41 with a score of 60.0.
- DeepSeek V3.2 (Thinking)DeepSeekCompare#4260.0DeepSeek V3.2 (Thinking) is #42 with a score of 60.0.
- Composer 2CursorCompareUnranked61.0Composer 2 is Unranked with a score of 61.0.
- Mistral Medium 3.5 128BMistralCompareUnranked61.0Mistral Medium 3.5 128B is Unranked with a score of 61.0.
- GPT-5.4 nanoOpenAICompareUnranked60.0GPT-5.4 nano is Unranked with a score of 60.0.
Category percentile
More
Relative position among models eligible for each sourced category. A higher percentile means a stronger position within that category's ranked cohort; 100 is highest.
- Inst. Following96%Eligible cohort rank #6 of 135Category score 90.0
- Multilingual72%Eligible cohort rank #28 of 99Category score 72.8
- Knowledge65%Eligible cohort rank #38 of 107Category score 64.9
- Agentic46%Eligible cohort rank #66 of 121Category score 45.4
Category evidence
Scores and ranks appear only where this model has published benchmark evidence. Categories without displayable source records remain not measured.
| Category | Score | Rank | Percentile | Weight | Benchmarks | Evidence |
|---|---|---|---|---|---|---|
| AgenticRank #66 of 121Percentile 46thWeight 22%12 benchmarksMixed sources | 45.4 | #66 of 121 | 46th | 22% | 12 benchmarks | Mixed sources |
| CodingRank Not rankedWeight 20%8 benchmarksMixed sources | 61.3 | Not ranked | Not available | 20% | 8 benchmarks | Mixed sources |
| ReasoningRank Not rankedWeight 17%3 benchmarksVerified | 43.5 | Not ranked | Not available | 17% | 3 benchmarks | Verified |
| KnowledgeRank #38 of 107Percentile 65thWeight 12%12 benchmarksMixed sources | 64.9 | #38 of 107 | 65th | 12% | 12 benchmarks | Mixed sources |
| MathWeight 5%0 benchmarksNot measured | Not measured | Not ranked | Not available | 5% | 0 benchmarks | Not measured |
| MultilingualRank #28 of 99Percentile 72ndWeight 7%1 benchmarkVerified | 72.8 | #28 of 99 | 72nd | 7% | 1 benchmark | Verified |
| MultimodalRank Not rankedWeight 12%1 benchmarkReported | 0.0 | Not ranked | Not available | 12% | 1 benchmark | Reported |
| Inst. FollowingRank #6 of 135Percentile 96thWeight 5%2 benchmarksMixed sources | 90.0 | #6 of 135 | 96th | 5% | 2 benchmarks | Mixed sources |
Benchmark Details
Rows below have a displayable published verification record. Each source link and provenance note remains in the page HTML while its category is closed. Source-unverified manual rows and generated rows stay hidden.
Agentic12 benchmarks
τ³-Bench Tool-Agent-User Evaluation
GDPval-AA normalized
Humanity's Last Exam with tools
Artificial Analysis Agentic Index
τ²-Bench Tool-Agent-User Evaluation
Artificial Analysis Briefcase
Artificial Analysis EnterpriseOps-Gym
Artificial Analysis Harvey LAB-AA
Coding8 benchmarks
Software Engineering Benchmark Verified
Scientific Code Benchmark
Artificial Analysis Coding Index
Artificial Analysis SciCode
Reasoning3 benchmarks
Artificial Analysis Long Context Reasoning
Critical Physics Tasks
Knowledge12 benchmarks
Humanity's Last Exam
Massive Multitask Language Understanding Professional
Graduate-Level Google-Proof Q&A
GPQA Diamond
Humanity's Last Exam without tools
Artificial Analysis Omniscience Accuracy
Artificial Analysis GPQA Diamond
Artificial Analysis Humanity's Last Exam
Artificial Analysis Omniscience Index
Artificial Analysis Omniscience Hallucination Rate
Artificial Analysis Openness Index
Multilingual1 benchmark
Multimodal1 benchmark
Design Arena Website Elo
Inst. Following2 benchmarks
Instruction Following Benchmark
Artificial Analysis IFBench
Frequently Asked Questions
How does Nemotron 3 Ultra perform overall in AI benchmarks?
Nemotron 3 Ultra currently ranks #41 out of 78 models on BenchLM's provisional leaderboard with an overall score of 60. It also ranks #24 out of 32 on the verified leaderboard. It is created by NVIDIA and features a 1M context window.
Is Nemotron 3 Ultra good for knowledge and understanding?
Nemotron 3 Ultra ranks #38 out of 78 models in knowledge and understanding benchmarks with an average score of 64.9. There are stronger options in this category.
Is Nemotron 3 Ultra good for coding and programming?
Nemotron 3 Ultra has visible benchmark coverage in coding and programming, but BenchLM does not currently assign it a global category rank there.
Is Nemotron 3 Ultra good for reasoning and logic?
Nemotron 3 Ultra has visible benchmark coverage in reasoning and logic, but BenchLM does not currently assign it a global category rank there.
Is Nemotron 3 Ultra good for agentic tool use and computer tasks?
Nemotron 3 Ultra ranks #66 out of 78 models in agentic tool use and computer tasks benchmarks with an average score of 45.4. There are stronger options in this category.
Is Nemotron 3 Ultra good for multimodal and grounded tasks?
Nemotron 3 Ultra has visible benchmark coverage in multimodal and grounded tasks, but BenchLM does not currently assign it a global category rank there.
Is Nemotron 3 Ultra good for instruction following?
Nemotron 3 Ultra ranks #6 out of 78 models in instruction following benchmarks with an average score of 90. It is among the top performers in this category.
Is Nemotron 3 Ultra good for multilingual tasks?
Nemotron 3 Ultra ranks #28 out of 78 models in multilingual tasks benchmarks with an average score of 72.8. There are stronger options in this category.
Is Nemotron 3 Ultra open source?
Yes, Nemotron 3 Ultra is an open weight model created by NVIDIA, meaning it can be downloaded and run locally or fine-tuned for specific use cases.
Does Nemotron 3 Ultra have full benchmark coverage on BenchLM?
Not yet. Nemotron 3 Ultra currently has 39 published benchmark scores out of the 313 benchmarks BenchLM tracks. BenchLM only exposes non-generated public benchmark rows, so missing categories stay blank until a sourced evaluation is available.
What is the context window size of Nemotron 3 Ultra?
Nemotron 3 Ultra has a context window of 1M, which determines how much text it can process in a single interaction.
Related Resources
Don't miss the next GPT moment
Which models moved up, what is new, and what it costs. One email each week.
Free. One email per week.