# ToolSandbox

> Apple benchmark of stateful, conversational tool use, where tool calls change state and a simulated user takes part.

Canonical page: https://benchlm.ai/benchmarks/toolsandbox

- Category: [Agentic](/agentic)
- Last updated: October 29, 2026

## About ToolSandbox

- Year: 2024
- Tasks: Stateful tool-use conversations
- Format: Average accuracy
- Difficulty: Tool use with changing state
- Paper: [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://github.com/apple-aiml-research/ToolSandbox)

NVIDIA reports it for Nemotron 3.5 Bolt E4B (October 23, 2026), run in its NeMo Gym harness, as average accuracy in reasoning mode.

ToolSandbox is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (3 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [LFM3-1.7B](/models/lfm3-1-7b) | LiquidAI | 80.36% |
| 2 | [Nemotron 3.5 Bolt E4B](/models/nemotron-3-5-bolt-e4b) | NVIDIA | 74.13% |
| 3 | [Nemotron 3.5 Bolt E4B BF16](/models/nemotron-3-5-bolt-e4b-bf16) | NVIDIA | 68.77% |

## FAQ

### What does ToolSandbox measure?

Apple benchmark of stateful, conversational tool use, where tool calls change state and a simulated user takes part.

### Which model scores highest on ToolSandbox?

LFM3-1.7B by LiquidAI currently leads with a score of 80.36% on ToolSandbox.

### How many models are evaluated on ToolSandbox?

3 AI models have published results on ToolSandbox in the BenchLM catalog.

### Does ToolSandbox affect BenchLM's overall score?

Not directly. ToolSandbox is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Compare Top Models on ToolSandbox

- [LFM3-1.7B vs Nemotron 3.5 Bolt E4B](/compare/lfm3-1-7b-vs-nemotron-3-5-bolt-e4b)
- [Nemotron 3.5 Bolt E4B vs Nemotron 3.5 Bolt E4B BF16](/compare/nemotron-3-5-bolt-e4b-vs-nemotron-3-5-bolt-e4b-bf16)