ToolSandbox
We show this table for reference; we do not rank on it.
Apple benchmark of stateful, conversational tool use, where tool calls change state and a simulated user takes part.
Benchmark score on ToolSandbox — October 29, 2026
We compile the ToolSandbox rows from provider self-reports. The table contains 3 models. LFM3-1.7B has the highest published score at 80.36%, but the available coverage and evaluation setups do not establish a market leader. We do not use these results to rank models overall.
LFM3-1.7B
LiquidAI
Nemotron 3.5 Bolt E4B
NVIDIA
Nemotron 3.5 Bolt E4B BF16
NVIDIA
3 modelsAgenticRefreshingDisplay onlyUpdated October 29, 2026
Benchmark score table (3 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | LFM3-1.7BLiquidAI | 80.36% | Not reported | Open |
| 2 | Nemotron 3.5 Bolt E4BNVIDIA | 74.13% | Not reported | Open |
| 3 | Nemotron 3.5 Bolt E4B BF16NVIDIA | 68.77% | Not reported | Open |
How to read this leaderboard
Operator receipt: 3 sourced rows are currently displayable on this page; the highest published score among these 3 models is LFM3-1.7B at 80.36%, which does not establish a market leader.
Honest limit: Only NVIDIA-run results are stored so far, from one harness. Display-only and excluded from weighted ranking.
Among the reported ToolSandbox rows, LFM3-1.7B is first at 80.36%. The third row is 11.59 points behind. The broader top-10 range is 11.59 points, so the table still separates the published systems.
3 models have been evaluated on ToolSandbox. The benchmark falls in the Agentic category. ToolSandbox is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ToolSandbox
Year
2024
Tasks
Stateful tool-use conversations
Format
Average accuracy
Difficulty
Tool use with changing state
NVIDIA reports it for Nemotron 3.5 Bolt E4B (October 23, 2026), run in its NeMo Gym harness, as average accuracy in reasoning mode.
Freshness and provenance
Version
ToolSandbox 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ToolSandbox measure?
Apple benchmark of stateful, conversational tool use, where tool calls change state and a simulated user takes part.
Which model scores highest on ToolSandbox?
LFM3-1.7B by LiquidAI currently leads with a score of 80.36% on ToolSandbox.
How many models are evaluated on ToolSandbox?
3 AI models have published results on ToolSandbox in the BenchLM catalog.
Compare top models on ToolSandbox
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.