Data as of October 1, 2026 · How the score is built
Skywork-Critic-Llama-3.1-70B (JudgeBench)
Decision readingSkywork-Critic-Llama-3.1-70B (JudgeBench) is tracked, but not publicly ranked yet. The original benchmark tables below include the published metrics for this evaluated configuration. Their distinct protocols remain outside general model rankings.
Skywork-Critic-Llama-3.1-70B (JudgeBench) will be repriced, updated or retired. Get each notice with its source and date. Follow model changes
Share or export
Original benchmark results
Every published metric for this configuration, with source precision and evaluation limits. Download results (JSON) · Numeric metrics (CSV)
Maintainer results on GPT-4o response pairs
JudgeBench’s paper and maintained leaderboard report judges under different prompts, reasoning efforts, and response-model splits. Accuracy is a percentage. The paper’s 700-pair GPT-4o split is distinct from the 270-pair Claude split. The app appends the same Nemotron CSV to both tabs without specifying a response split; we show that supplement once with its split unresolved. Live one-decimal values and paper two-decimal values are preserved separately.
GPT-4o-2024-05-13 response split. The public app rounds to one decimal; paper tables retain two decimals.
Published source · Full JudgeBench results
JudgeBench: A Benchmark for Evaluating LLM-Based Judges — Sijun Tan and colleagues. Numeric results transcribed and reformatted; evaluation notes and model links added by BenchLM. Published numeric precision is retained.
| Judge | Type | Knowledge (%) | Reasoning (%) | Math (%) | Coding (%) | Overall (%) |
|---|---|---|---|---|---|---|
| Skywork-Critic-Llama-3.1-70B | Fine-Tuned Judge | 55.8 | 55.1 | 73.2 | 47.6 | 57.4 |
Paper: judge methods
JudgeBench’s paper and maintained leaderboard report judges under different prompts, reasoning efforts, and response-model splits. Accuracy is a percentage. The paper’s 700-pair GPT-4o split is distinct from the 270-pair Claude split. The app appends the same Nemotron CSV to both tabs without specifying a response split; we show that supplement once with its split unresolved. Live one-decimal values and paper two-decimal values are preserved separately.
Paper v2; source group labels and published precision are retained.
Published source · Full JudgeBench results
JudgeBench: A Benchmark for Evaluating LLM-Based Judges — Sijun Tan and colleagues. Numeric results transcribed and reformatted; evaluation notes and model links added by BenchLM. Published numeric precision is retained.
| Judge | Knowledge (%) | Reasoning (%) | Math (%) | Coding (%) | Overall (%) |
|---|---|---|---|---|---|
| Skywork-LLaMA-3.1B-70B | 55.84 | 55.10 | 73.21 | 47.62 | 57.43 |
Paper: judgment outcomes
JudgeBench’s paper and maintained leaderboard report judges under different prompts, reasoning efforts, and response-model splits. Accuracy is a percentage. The paper’s 700-pair GPT-4o split is distinct from the 270-pair Claude split. The app appends the same Nemotron CSV to both tabs without specifying a response split; we show that supplement once with its split unresolved. Live one-decimal values and paper two-decimal values are preserved separately.
Paper v2; source group labels and published precision are retained.
Published source · Full JudgeBench results
JudgeBench: A Benchmark for Evaluating LLM-Based Judges — Sijun Tan and colleagues. Numeric results transcribed and reformatted; evaluation notes and model links added by BenchLM. Published numeric precision is retained.
| Judge | A>B count | A<B count | Tie count | Invalid count |
|---|---|---|---|---|
| Skywork-LLaMA-3.1B-70B | 390 | 310 | 0 | 0 |
Paper: order inconsistency
JudgeBench’s paper and maintained leaderboard report judges under different prompts, reasoning efforts, and response-model splits. Accuracy is a percentage. The paper’s 700-pair GPT-4o split is distinct from the 270-pair Claude split. The app appends the same Nemotron CSV to both tabs without specifying a response split; we show that supplement once with its split unresolved. Live one-decimal values and paper two-decimal values are preserved separately.
Paper v2; source group labels and published precision are retained.
Published source · Full JudgeBench results
JudgeBench: A Benchmark for Evaluating LLM-Based Judges — Sijun Tan and colleagues. Numeric results transcribed and reformatted; evaluation notes and model links added by BenchLM. Published numeric precision is retained.
| Judge | Inconsistent (%) |
|---|---|
| Skywork-Llama-3.1B-70B | 18.29% |
Lineage
The sequence follows explicit supersedes links. Each score is estimated for that model; a relative can inform a sparse estimate but never sets a floor, so a newer release can score below an earlier one. Scores and prices remain blank when the corresponding public row or first-party rate is unavailable.
Release date not sourced · you are here
Skywork-Critic-Llama-3.1-70B (JudgeBench)Not publicly ranked · Price not listed
Benchmark-system
Spec sheet
Each documented value carries its source. Missing fields stay visible as not sourced or not published, rather than disappearing from the page.
- API model ID
- Not publishedProvider pricing
- Context window
- Not published
- Maximum output
- Not sourced yet
- Knowledge cutoff
- Not sourced yet
- Input modalities
- Not sourced yet
- Output modalities
- Not sourced yet
- Parameters
- Not sourced yet
- Availability
- Not sourced yet
- Cloud regions
- Not tracked yet
- Lifecycle
- Tracked
- API capabilities
- Tool calling, structured outputs, and batch support are not tracked yet
- Prompt caching
- Not documented in the pricing recordProvider pricing
- Self-host
- Open weights available; hardware estimate not sourced
- Rate limits
- Not tracked yet
How to read this profile
The visual layer above carries the decisions. These notes preserve the model, ranking, coverage, and family context behind the numbers.
The original benchmark tables show this evaluated configuration’s published results. Training choices, prompts, response splits, and metrics remain attached to each table. The profile stays outside general model rankings.
Skywork-Critic-Llama-3.1-70B (JudgeBench) is a open weight model. No explicit reasoning mode is documented in this profile.
Evaluated system: Skywork-Critic-Llama-3.1-70B (JudgeBench). The linked benchmark-owner report supplies the full numeric results and protocol. This profile represents that evaluated configuration; it does not transfer results to a base model or another prompt. Context limit, current API tariff, and release date are not established by this evaluation.
The original benchmark results appear in the source tables above; they remain separate from weighted benchmark slots.
Last updated October 1, 2026. Runtime fields remain blank until a sourced snapshot exists.
Deployment options
Self-host and provider-specific paths stay separate from benchmark evidence so operating constraints are visible before a score becomes the whole decision.
Published weights are available, but BenchLM does not yet have a sourced parameter and VRAM profile for this exact model. Hardware cost estimates stay unavailable until that sizing record is complete.
Estimate VRAM from known parametersQuestions
How does Skywork-Critic-Llama-3.1-70B (JudgeBench) perform overall in AI benchmarks?
This configuration has original benchmark results in the source tables on this profile. Every published metric retains its evaluated configuration, source precision, and protocol. These results stay outside general model rankings; they do not produce a public overall score or transfer to a base model with different settings.
Is Skywork-Critic-Llama-3.1-70B (JudgeBench) open source?
Skywork-Critic-Llama-3.1-70B (JudgeBench) is an open-weight model from Skywork. Its weights can be downloaded for local or hosted deployment, subject to the published license. Open weight does not automatically mean open source: training data and training code may remain private, and commercial restrictions can still apply.
Does Skywork-Critic-Llama-3.1-70B (JudgeBench) have full benchmark coverage on BenchLM?
This configuration has results in the original benchmark tables on this profile. Those tables preserve every imported source metric, but they do not fill the site’s general scoring slots. Other benchmarks remain unmeasured, and compatible configurations are required before comparing results across different reports.
What is the context window size of Skywork-Critic-Llama-3.1-70B (JudgeBench)?
Skywork-Critic-Llama-3.1-70B (JudgeBench)'s context window is not documented in a source tied to this exact model yet. The profile leaves the value unavailable instead of borrowing a limit from an earlier family member or an unverified route. Maximum output length remains a separate field.