# Vals SWE-bench Multimodal evaluation (SWE-bench Multimodal (Vals))

> JavaScript issue resolution where bug reports depend on screenshots, evaluated by Vals AI with no agent internet access.

Canonical page: https://benchlm.ai/benchmarks/valsswebenchmultimodal

- Category: [Coding](/coding)
- Last updated: October 13, 2026

Evaluation results published by [Vals AI](https://www.vals.ai/benchmarks/swebench-multimodal). Published leaderboard aggregates and evaluation settings only. Dataset contents are not included.

## About SWE-bench Multimodal (Vals)

- Year: 2026
- Tasks: 480 JavaScript issues with visual bug reports
- Format: Resolved issue percentage
- Difficulty: Visual software issue resolution
- Paper: [SWE-bench Multimodal on Vals AI](https://www.vals.ai/benchmarks/swebench-multimodal)

Vals AI evaluates 480 tasks across eleven repositories with mini-swe-agent, one trial per model, screenshots supplied before the run, and no agent internet access. These source-specific results stay display only and remain separate from text-only SWE-bench scores.

SWE-bench Multimodal (Vals) is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (18 models)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Claude Opus 5.5](/models/claude-opus-5-5) | — | Anthropic | 70.21% |
| 2 | [Claude Fable 5.1](/models/claude-fable-5-1) | — | Anthropic | 61.25% |
| 3 | [Claude Sonnet 5.5](/models/claude-sonnet-5-5) | — | Anthropic | 58.33% |
| 4 | [Gemini 4 Argon](/models/gemini-4-argon) | high reasoning | Google | 56.04% |
| 5 | [Muse Wonder Full](/models/muse-wonder-full) | max reasoning | Meta | 47.29% |
| 6 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | max reasoning | DeepSeek | 46.46% |
| 7 | [Muse Wonder](/models/muse-wonder) | max reasoning | Meta | 45.42% |
| 8 | [Claude Haiku 5.5](/models/claude-haiku-5-5) | — | Anthropic | 43.96% |
| 9 | [GPT-6.1 Sol](/models/gpt-6-1-sol) | max reasoning | OpenAI | 43.54% |
| 10 | [GPT-6 Astra](/models/gpt-6-astra) | max reasoning | OpenAI | 43.33% |
| 11 | [Muse Spark 1.3 Max](https://www.vals.ai/models/meta_muse_spark_1_3_max) | max reasoning | Meta | 43.13% |
| 12 | [Grok 4.7](/models/grok-4-7) | xhigh reasoning | xAI | 42.92% |
| 13 | [MiMo-V2.6-Pro](/models/mimo-v2-6-pro) | — | Xiaomi | 41.25% |
| 14 | [Gemini 3.8 Flash](/models/gemini-3-8-flash) | high reasoning | Google | 40.83% |
| 15 | [Step 5 Preview](/models/step-5-preview) | — | StepFun | 40.83% |
| 16 | [Qwen3.8 Max](/models/qwen3-8-max) | — | Alibaba | 39.17% |
| 17 | [GPT-6 Luna](/models/gpt-6-luna) | max reasoning | OpenAI | 37.50% |
| 18 | [Kimi K3](/models/kimi-k3) | max reasoning | Moonshot AI | 35.83% |

## FAQ

### What does SWE-bench Multimodal (Vals) measure?

JavaScript issue resolution where bug reports depend on screenshots, evaluated by Vals AI with no agent internet access.

### Which model leads the published SWE-bench Multimodal (Vals) snapshot?

Claude Opus 5.5 currently leads the published SWE-bench Multimodal (Vals) snapshot with a score of 70.21%.

### How many models are evaluated on SWE-bench Multimodal (Vals)?

The October 13, 2026 contains 18 AI models.

### Does SWE-bench Multimodal (Vals) affect BenchLM's overall score?

Not directly. SWE-bench Multimodal (Vals) is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.