# MathVista

> MathVista tests mathematical reasoning about images. This table shows published accuracy on the testmini split, with each source configuration kept separate. The results are display only and do not enter overall or category rankings.

Canonical page: https://benchlm.ai/benchmarks/mathvista

- Category: [Multimodal & Grounded](/multimodal-grounded)
- Last updated: June 9, 2025

MathVista by [MathVista authors and website maintainers](https://mathvista.github.io/) and contributors. [Original benchmark](https://mathvista.github.io/) · [CC-BY-SA-4.0 license](https://creativecommons.org/licenses/by-sa/4.0/).

Selected aggregate-result compilation and its adaptation. Task content, source images, model outputs, and model weights are excluded. Selected aggregate scores and adjacent metadata; converted numeric strings to numbers and missing-score markers to null; separated testmini from test; added configuration notes and three named profile references.

This table contains 78 model or configuration records from the MathVista website compilation for testmini. The split has 1,000 examples; the aggregate files do not state a separate completed-example count for each record. The pinned source commit is dated June 9, 2025, and the source files were observed on October 15, 2026.

Each score belongs to the published configuration shown below it. Prompting, tools, image processing, and test-time compute can change the result. We keep the two splits separate and exclude these scores from overall and category rankings.

Reference baselines are outside the model table: Human Performance*: 60.3%; Random Chance: 17.86%.

MathVista website compilation, CC BY-SA 4.0. Original result sources remain credited in each record. This adaptation retains CC BY-SA 4.0. Changes: Selected aggregate scores and adjacent metadata; converted numeric strings to numbers and missing-score markers to null; separated testmini from test; added configuration notes and three named profile references.

The compilation contains different evaluation setups; the highest reported value does not establish a comparable model ranking.

- [MathVista project and leaderboard](https://mathvista.github.io/)
- [Pinned MathVista result compilation](https://github.com/mathvista/mathvista.github.io/tree/a9daca33c6b3ae348b776e6586ec6b8a62656d29)
- [MathVista paper](https://arxiv.org/abs/2310.02255)
- [Website compilation license declaration](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/README.md)
- [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)

## About MathVista

- Year: 2023
- Tasks: 1,000 examples in the testmini split
- Format: Published accuracy (%)
- Difficulty: Mathematical reasoning in visual contexts
- Paper: [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://arxiv.org/abs/2310.02255)

The testmini split contains 1,000 examples. The pinned website compilation was committed on June 9, 2025 and observed on October 1, 2026. Source record counts describe published configurations, not distinct models or confirmed complete evaluation runs.

MathVista is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (78 model or configuration records)

| Source configuration | Accuracy | Evaluation setup | Sources |
|---|---|---|---|
| [DreamPRM (o4-mini)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/DreamPRM%20(o4-mini)/scores_dreamprm_testmini.json) | 85.2% | DreamPRM selection pipeline with o4-mini; separate from a bare o4-mini evaluation; Source date 2025-06-04 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/DreamPRM%20(o4-mini)/scores_dreamprm_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/DreamPRM%20(o4-mini)/meta.json); [original result](https://arxiv.org/abs/2505.20241) |
| [VL-Rethinker](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/VL-Rethinker/scores_mathvista_VL-Rethinker.json) | 80.3% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-04-10 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/VL-Rethinker/scores_mathvista_VL-Rethinker.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/VL-Rethinker/meta.json); [original result](https://arxiv.org/abs/2504.08837) |
| [Step R1-V-Mini](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Step%20R1-V-Mini/scores_mathvista_Step%20R1-V-Mini.json) | 80.1% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-04-07 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Step%20R1-V-Mini/scores_mathvista_Step%20R1-V-Mini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Step%20R1-V-Mini/meta.json); [original result](https://platform.stepfun.com/) |
| [Kimi-k1.6-preview-20250308](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.6-preview-20250308/scores_mathvista_kimi-1.6.json) | 80% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-03-10 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.6-preview-20250308/scores_mathvista_kimi-1.6.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.6-preview-20250308/meta.json); [original result](https://github.com/MoonshotAI/Kimi-k1.5) |
| [Doubao-pro-1.5](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Doubao-pro-1.5/scores_mathvista_doubao-pro-1.5.json) | 79.5% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-01-22 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Doubao-pro-1.5/scores_mathvista_doubao-pro-1.5.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Doubao-pro-1.5/meta.json); [original result](https://team.doubao.com/en/special/doubao_1_5_pro) |
| [Ovis2_34B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis2_34B/scores_mathvista_Ovis2_34B.json) | 77.1% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2025-02-10 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis2_34B/scores_mathvista_Ovis2_34B.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis2_34B/meta.json); [original result](https://huggingface.co/AIDC-AI/Ovis2-34B) |
| [Kimi-k1.5](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.5/scores_mathvista_kimi-1.5.json) | 74.9% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-01-22 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.5/scores_mathvista_kimi-1.5.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Kimi-k1.5/meta.json); [original result](https://arxiv.org/abs/2501.12599) |
| [OpenAI o1](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OpenAI%20o1/scores_openai_gpt-o1_testmini.json) | 73.9% | Research o1; maximal test-time compute; vision enabled; exact checkpoint undisclosed; Source date 2024-09-12 (compiler field); The original source reports research o1, not o1-preview or released o1 | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OpenAI%20o1/scores_openai_gpt-o1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OpenAI%20o1/meta.json); [original result](https://openai.com/index/learning-to-reason-with-llms/) |
| [Llama 4 Maverick](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Llama%204%20Maverick/scores_mathvista_llama-4.json) | 73.7% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2025-04-05 (compiler field); Checkpoint identity and catalog date require review; source configuration retained | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Llama%204%20Maverick/scores_mathvista_llama-4.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Llama%204%20Maverick/meta.json); [original result](https://ai.meta.com/blog/llama-4-multimodal-intelligence/) |
| [Vision-R1-7B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Vision-R1-7B/scores_mathvista_Vision-R1.json) | 73.2% | Published method: Reason 🧠; Additional checkpoint and evaluation settings unverified; Source date 2025-03-09 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Vision-R1-7B/scores_mathvista_Vision-R1.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Vision-R1-7B/meta.json); [original result](https://arxiv.org/abs/2503.06749) |
| [Gemini 2.0 Flash](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%202.0%20Flash/scores_mathvista_gemini_2.0_flash.json) | 73.1% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2025-02-05 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%202.0%20Flash/scores_mathvista_gemini_2.0_flash.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%202.0%20Flash/meta.json); [original result](https://deepmind.google/technologies/gemini/flash/) |
| [QVQ-72B-Preview](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/QVQ-72B-Preview/scores_mathvista.json) | 71.4% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-12-24 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/QVQ-72B-Preview/scores_mathvista.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/QVQ-72B-Preview/meta.json); [original result](https://huggingface.co/Qwen/QVQ-72B-Preview) |
| [Qwen2VL-72B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen2VL-72B/scores_mathvista.json) | 70.5% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-12-24 (compiler field); Source metadata links a QVQ card; sibling checkpoint settings were not inherited | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen2VL-72B/scores_mathvista.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen2VL-72B/meta.json); [original result](https://huggingface.co/Qwen/QVQ-72B-Preview) |
| [Pixtral Large (124B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral%20Large%20(124B)/scores_openai_pixtral_12b_testmini.json) | 69.4% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-11-18 (compiler field); Source folder says Pixtral-Large 124B; score filename alone does not establish model identity | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral%20Large%20(124B)/scores_openai_pixtral_12b_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral%20Large%20(124B)/meta.json); [original result](https://mistral.ai/news/pixtral-large/) |
| [Grok-2](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2/scores_grok_2_testmini.json) | 69% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-08-13 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2/scores_grok_2_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2/meta.json); [original result](https://x.ai/blog/grok-2) |
| [Grok-2 mini](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2%20mini/scores_grok_2_mini_testmini.json) | 68.1% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-08-13 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2%20mini/scores_grok_2_mini_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-2%20mini/meta.json); [original result](https://x.ai/blog/grok-2) |
| [Claude 3.5 Sonnet](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203.5%20Sonnet/scores_claude3.5_sonnet_testmini.json) | 67.7% | Initial June 2024 Sonnet; zero-shot chain-of-thought; exact API snapshot undisclosed; Source date 2024-06-20 (compiler field); June 2024 result; source metadata says June 20 and the provider page says June 21 | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203.5%20Sonnet/scores_claude3.5_sonnet_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203.5%20Sonnet/meta.json); [original result](https://www.anthropic.com/news/claude-3-5-sonnet) |
| [LLaVA-OneVision](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-OneVision/scores_llava-onevision_testmini.json) | 67.5% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-08-06 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-OneVision/scores_llava-onevision_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-OneVision/meta.json); [original result](https://llava-vl.github.io/blog/2024-08-05-llava-onevision/) |
| [InternVL2-8B-MPO](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-8B-MPO/scores_testmini_InternVL2-8B-MPO.json) | 67.3% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-11-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-8B-MPO/scores_testmini_InternVL2-8B-MPO.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-8B-MPO/meta.json); [original result](https://internvl.github.io/blog/2024-11-14-InternVL-2.0-MPO/) |
| [Ovis1.6-Gemma2-9B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis1.6-Gemma2-9B/scores_Ovis1.6-Gemma2-9B_testmini.json) | 67.2% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2025-02-10 (compiler field); Source metadata links an Ovis2 card; sibling checkpoint settings were not inherited | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis1.6-Gemma2-9B/scores_Ovis1.6-Gemma2-9B_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Ovis1.6-Gemma2-9B/meta.json); [original result](https://huggingface.co/AIDC-AI/Ovis2-34B) |
| [InternVL2-Pro](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-Pro/scores_internvl2_pro.json) | 66.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-09-04 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-Pro/scores_internvl2_pro.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL2-Pro/meta.json); [original result](https://github.com/OpenGVLab/InternVL) |
| [Chimera-Reasoner-8B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Chimera-Reasoner-8B/scores_testmini_Chimera-Reasoner.json) | 66.2% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-12-05 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Chimera-Reasoner-8B/scores_testmini_Chimera-Reasoner.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Chimera-Reasoner-8B/meta.json); [original result](https://unimodal4reasoning.github.io/chimera_page/) |
| [TextGrad (GPT-4o)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TextGrad%20(GPT-4o)/scores_textgrad_gpt4-o_testmini.json) | 66.1% | TextGrad pipeline with GPT-4o; separate from a bare GPT-4o evaluation; Source date 2024-07-08 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TextGrad%20(GPT-4o)/scores_textgrad_gpt4-o_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TextGrad%20(GPT-4o)/meta.json); [original result](https://github.com/zou-group/textgrad) |
| [TBAC-VLR1-3B-preview](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TBAC-VLR1-3B-preview/scores_testmini.json) | 64.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2025-04-16 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TBAC-VLR1-3B-preview/scores_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/TBAC-VLR1-3B-preview/meta.json); [original result](https://huggingface.co/TencentBAC/TBAC-VLR1-3B-preview) |
| [Gemini 1.5 Pro (May 2024)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(May%202024)/scores_gemini_pro_testmini.json) | 63.9% | May 2024 source result; separate from the February 2024 configuration; Source date 2024-05-17 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(May%202024)/scores_gemini_pro_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(May%202024)/meta.json); [original result](https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf) |
| [GPT-4o](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4o/scores_openai_gpt-4o_testmini.json) | 63.8% | May 2024 source result; exact API snapshot and sampling settings unverified; Source date 2024-05-13 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4o/scores_openai_gpt-4o_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4o/meta.json); [original result](https://openai.com/index/hello-gpt-4o/) |
| [Phi-4-multimodal-ins](/models/phi-4-multimodal-instruct) | 62.4% | Microsoft internal platform; zero-shot vision prompts; original benchmark images; sampling settings undisclosed; Source date 2024-03-03 (compiler field); Source date 2024-03-03 conflicts with the 2025 release; retained as a compiler-field error | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-4-multimodal-ins/scores_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-4-multimodal-ins/meta.json); [original result](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) |
| [InternVL-Chat-V1.2-Plus](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2-Plus/scores_InternVL-Chat-V1.2_Plus_testmini.json) | 59.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-02-22 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2-Plus/scores_InternVL-Chat-V1.2_Plus_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2-Plus/meta.json); [original result](https://arxiv.org/abs/2312.14238) |
| [Gemini 1.5 Flash (May 2024)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Flash%20(May%202024)/scores_gemini_pro_testmini.json) | 58.4% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-05-17 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Flash%20(May%202024)/scores_gemini_pro_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Flash%20(May%202024)/meta.json); [original result](https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf) |
| [GPT-4T 2024-04-09](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4T%202024-04-09/scores_openai_gpt-4t_testmini.json) | 58.1% | April 9, 2024 source configuration; prompting and sampling settings unverified; Source date 2024-05-13 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4T%202024-04-09/scores_openai_gpt-4t_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4T%202024-04-09/meta.json); [original result](https://openai.com/index/hello-gpt-4o/) |
| [Pixtral-12B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral-12B/scores_openai_pixtral_12b_testmini.json) | 58% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-10-09 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral-12B/scores_openai_pixtral_12b_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Pixtral-12B/meta.json); [original result](https://arxiv.org/abs/2410.07073) |
| [InternLM-XComposer2-VL-7B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternLM-XComposer2-VL-7B/scores_testmini_InternLM-XComposer2-VL.json) | 57.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-01-22 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternLM-XComposer2-VL-7B/scores_testmini_InternLM-XComposer2-VL.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternLM-XComposer2-VL-7B/meta.json); [original result](https://github.com/InternLM/InternLM-XComposer) |
| [Gemini 1.0 Ultra](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Ultra/scores_gemini_ultra_testmini.json) | 53% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-12-06 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Ultra/scores_gemini_ultra_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Ultra/meta.json); [original result](https://arxiv.org/abs/2312.11805) |
| [Grok-1.5V](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-1.5V/scores_grok_1.5v_testmini.json) | 52.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-04-12 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-1.5V/scores_grok_1.5v_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Grok-1.5V/meta.json); [original result](https://x.ai/blog/grok-1.5v) |
| [Gemini 1.5 Pro (Feb 2024)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(Feb%202024)/scores_gemini_pro_testmini.json) | 52.1% | February 2024 source result; separate from the May 2024 configuration; Source date 2024-02-15 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(Feb%202024)/scores_gemini_pro_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.5%20Pro%20(Feb%202024)/meta.json); [original result](https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf) |
| [Claude 3 Opus](/models/claude-3-opus) | 50.5% | Zero-shot chain-of-thought; exact API snapshot undisclosed; Source date 2024-03-04 (compiler field); Named profile reference; the source configuration remains the evaluation identity | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Opus/scores_claude3_opus_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Opus/meta.json); [original result](https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf) |
| [GPT-4V (Playground)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4V%20(Playground)/scores_gpt4v_manual_1015.json) | 49.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-15 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4V%20(Playground)/scores_gpt4v_manual_1015.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/GPT-4V%20(Playground)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [Claude 3 Sonnet](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Sonnet/scores_claude3_sonnet_testmini.json) | 47.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-04 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Sonnet/scores_claude3_sonnet_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Sonnet/meta.json); [original result](https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf) |
| [InternVL-Chat-V1.2](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2/scores_InternVL-Chat-V1.2_testmini.json) | 47.7% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-02-22 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2/scores_InternVL-Chat-V1.2_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InternVL-Chat-V1.2/meta.json); [original result](https://arxiv.org/abs/2312.14238) |
| [Math-LLaVA-13B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Math-LLaVA-13B/scores_math_llava_13b_testmini.json) | 46.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-06-25 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Math-LLaVA-13B/scores_math_llava_13b_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Math-LLaVA-13B/meta.json); [original result](http://arxiv.org/abs/2406.17294) |
| [LLaVA-NeXT-34B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-34B/scores_LLaVA-NeXT-34B.json) | 46.5% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-01-30 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-34B/scores_LLaVA-NeXT-34B.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-34B/meta.json); [original result](https://llava-vl.github.io/blog/2024-01-30-llava-1-6/) |
| [Claude 3 Haiku](/models/claude-3-haiku) | 46.4% | Chain-of-thought prompting; exact API snapshot and shot count undisclosed; Source date 2024-03-04 (compiler field); Named profile reference; the source configuration remains the evaluation identity | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Haiku/scores_claude3_haiku_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Claude%203%20Haiku/meta.json); [original result](https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf) |
| [Gemini 1.0 Pro](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Pro/scores_gemini_pro_testmini.json) | 45.2% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-12-06 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Pro/scores_gemini_pro_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Pro/meta.json); [original result](https://arxiv.org/abs/2312.11805) |
| [Phi-3-Vision-128K-In](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3-Vision-128K-In/scores_phi3v_testmini.json) | 44.5% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-05-21 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3-Vision-128K-In/scores_phi3v_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3-Vision-128K-In/meta.json); [original result](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct) |
| [Phi-3.5-Vision 4.2B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3.5-Vision%204.2B/scores_openai_phi_3.5_vision_testmini.json) | 43.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-04-22 (compiler field); Source metadata links a Phi-3 report; exact Phi-3.5 vision configuration is unverified | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3.5-Vision%204.2B/scores_openai_phi_3.5_vision_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Phi-3.5-Vision%204.2B/meta.json); [original result](https://arxiv.org/abs/2404.14219) |
| [Mini-Gemini-HD (Hermes-2-Yi-34B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini-HD%20(Hermes-2-Yi-34B)/scores_Mini-Gemini-HD_Hermes-2-Yi-34B_testmini.json) | 43.3% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-27 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini-HD%20(Hermes-2-Yi-34B)/scores_Mini-Gemini-HD_Hermes-2-Yi-34B_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini-HD%20(Hermes-2-Yi-34B)/meta.json); [original result](https://arxiv.org/abs/2403.18814) |
| [Qwen-VL-Plus](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen-VL-Plus/scores_qwenvl_plus_pretrain.json) | 43.3% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-12-21 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen-VL-Plus/scores_qwenvl_plus_pretrain.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Qwen-VL-Plus/meta.json); [original result](https://github.com/QwenLM/Qwen-VL) |
| [SPHINX-MoE](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-MoE/scores_mathvista_SPHINX-MoE_2E.json) | 42.3% | Published method: MoE 🤖; Additional checkpoint and evaluation settings unverified; Source date 2024-01-12 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-MoE/scores_mathvista_SPHINX-MoE_2E.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-MoE/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [Mini-Gemini (Mixtral-8x7B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini%20(Mixtral-8x7B)/scores_Mini-Gemini_Mixtral-8x7B_testmini.json) | 41.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-27 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini%20(Mixtral-8x7B)/scores_Mini-Gemini_Mixtral-8x7B_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Mini-Gemini%20(Mixtral-8x7B)/meta.json); [original result](https://arxiv.org/abs/2403.18814) |
| [MM1-7B-MoE-Chat ](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-MoE-Chat%20/scores_mm1_testmini.json) | 40.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-MoE-Chat%20/scores_mm1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-MoE-Chat%20/meta.json); [original result](https://arxiv.org/abs/2403.09611) |
| [MiniCPM-V-2 (2.8B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MiniCPM-V-2%20(2.8B)/scores_minicpm-v-2_testmini.json) | 40.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-04-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MiniCPM-V-2%20(2.8B)/scores_minicpm-v-2_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MiniCPM-V-2%20(2.8B)/meta.json); [original result](https://github.com/OpenBMB/MiniCPM-V) |
| [MM1-30B-Chat](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-30B-Chat/scores_mm1_testmini.json) | 39.4% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-30B-Chat/scores_mm1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-30B-Chat/meta.json); [original result](https://arxiv.org/abs/2403.09611) |
| [SPHINX-Plus](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Plus/scores_mathvista_SPHINX-Plus.json) | 36.8% | Published method: MoE 🤖; Additional checkpoint and evaluation settings unverified; Source date 2024-01-12 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Plus/scores_mathvista_SPHINX-Plus.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Plus/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [SPHINX (V2)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V2)/scores_mathvista_soup_13999.json) | 36.7% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-11-17 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V2)/scores_mathvista_soup_13999.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V2)/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [MM1-7B-Chat ](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-Chat%20/scores_mm1_testmini.json) | 35.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-Chat%20/scores_mm1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-7B-Chat%20/meta.json); [original result](https://arxiv.org/abs/2403.09611) |
| [SPHINX-Intern2](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Intern2/scores_mathvista_SPHINX-Intern2.json) | 35.5% | Published method: MoE 🤖; Additional checkpoint and evaluation settings unverified; Source date 2024-01-12 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Intern2/scores_mathvista_SPHINX-Intern2.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Intern2/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [OmniLMM-12B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OmniLMM-12B/scores_InternVL-Chat-V1.2_Plus_testmini.json) | 34.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-02-01 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OmniLMM-12B/scores_InternVL-Chat-V1.2_Plus_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/OmniLMM-12B/meta.json); [original result](https://github.com/OpenBMB/OmniLMM) |
| [Multimodal Bard](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Multimodal%20Bard/scores_bard.json) | 34.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Multimodal%20Bard/scores_bard.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Multimodal%20Bard/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [LLaVA-NeXT-Vicuna-7B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-Vicuna-7B/scores_llava_testmini.json) | 34.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-01-30 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-Vicuna-7B/scores_llava_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-NeXT-Vicuna-7B/meta.json); [original result](https://llava-vl.github.io/blog/2024-01-30-llava-next/) |
| [PoT GPT-4 (Caption+OCR)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20GPT-4%20(Caption%2BOCR)/scores_gpt4_2shot_code_use_caption_ocr.json) | 33.9% | Caption and OCR inputs; published prompting setup; remaining tools and sampling settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20GPT-4%20(Caption%2BOCR)/scores_gpt4_2shot_code_use_caption_ocr.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20GPT-4%20(Caption%2BOCR)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [CoT ChatGPT (Caption+OCR)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20ChatGPT%20(Caption%2BOCR)/scores_chatgpt_2shot_solution_use_caption_ocr.json) | 33.2% | Caption and OCR inputs; published prompting setup; remaining tools and sampling settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20ChatGPT%20(Caption%2BOCR)/scores_chatgpt_2shot_solution_use_caption_ocr.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20ChatGPT%20(Caption%2BOCR)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [CoT Claude (Caption+OCR)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20Claude%20(Caption%2BOCR)/scores_claude_2shot_solution_use_caption_ocr.json) | 33.2% | Caption and OCR inputs; published prompting setup; remaining tools and sampling settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20Claude%20(Caption%2BOCR)/scores_claude_2shot_solution_use_caption_ocr.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20Claude%20(Caption%2BOCR)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [CoT GPT4 (Caption+OCR)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20GPT4%20(Caption%2BOCR)/scores_gpt4_2shot_solution_use_caption_ocr.json) | 33.2% | Caption and OCR inputs; published prompting setup; remaining tools and sampling settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20GPT4%20(Caption%2BOCR)/scores_gpt4_2shot_solution_use_caption_ocr.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/CoT%20GPT4%20(Caption%2BOCR)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [MM1-3B-MoE-Chat](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-MoE-Chat/scores_mm1_testmini.json) | 32.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-MoE-Chat/scores_mm1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-MoE-Chat/meta.json); [original result](https://arxiv.org/abs/2403.09611) |
| [MM1-3B-Chat](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-Chat/scores_mm1_testmini.json) | 32% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-03-14 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-Chat/scores_mm1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/MM1-3B-Chat/meta.json); [original result](https://arxiv.org/abs/2403.09611) |
| [Gemini 1.0 Nano 2](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%202/scores_gemini_nano2_testmini.json) | 30.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-12-06 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%202/scores_gemini_nano2_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%202/meta.json); [original result](https://arxiv.org/abs/2312.11805) |
| [LLaVA-1.5-13B](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-1.5-13B/scores_LLaVA-1.5-13B.json) | 27.6% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2024-01-30 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-1.5-13B/scores_LLaVA-1.5-13B.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA-1.5-13B/meta.json); [original result](https://llava-vl.github.io/blog/2024-01-30-llava-1-6/) |
| [SPHINX (V1)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V1)/scores_dialog_llava1.5noboxRef4v2Cwbflicker30k_llamaEns5_13B.json) | 27.5% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-11-09 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V1)/scores_dialog_llava1.5noboxRef4v2Cwbflicker30k_llamaEns5_13B.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX%20(V1)/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [Gemini 1.0 Nano 1](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%201/scores_gemini_nano1_testmini.json) | 27.3% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-12-06 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%201/scores_gemini_nano1_testmini.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Gemini%201.0%20Nano%201/meta.json); [original result](https://arxiv.org/abs/2312.11805) |
| [PoT ChatGPT (Caption+OCR)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20ChatGPT%20(Caption%2BOCR)/scores_chatgpt_2shot_code_use_caption_ocr.json) | 26.8% | Caption and OCR inputs; published prompting setup; remaining tools and sampling settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20ChatGPT%20(Caption%2BOCR)/scores_chatgpt_2shot_code_use_caption_ocr.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/PoT%20ChatGPT%20(Caption%2BOCR)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [SPHINX-Tiny](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Tiny/scores_mathvista_SPHINX-Tiny.json) | 26.4% | Published method: MoE 🤖; Additional checkpoint and evaluation settings unverified; Source date 2024-01-12 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Tiny/scores_mathvista_SPHINX-Tiny.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/SPHINX-Tiny/meta.json); [original result](https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX) |
| [LLaVA (LLaMA-2-13B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA%20(LLaMA-2-13B)/scores_llava_llama_2_13b.json) | 26.1% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA%20(LLaMA-2-13B)/scores_llava_llama_2_13b.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVA%20(LLaMA-2-13B)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [InstructBLIP (Vicuna-7B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InstructBLIP%20(Vicuna-7B)/scores_instructblip2.json) | 25.3% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InstructBLIP%20(Vicuna-7B)/scores_instructblip2.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/InstructBLIP%20(Vicuna-7B)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [LLaVAR](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVAR/scores_llavar.json) | 25.2% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVAR/scores_llavar.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaVAR/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [LLaMA-Adapter-V2 (7B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaMA-Adapter-V2%20(7B)/scores_mathvista_llama_adapter_v2.json) | 23.9% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaMA-Adapter-V2%20(7B)/scores_mathvista_llama_adapter_v2.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/LLaMA-Adapter-V2%20(7B)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [miniGPT4 (LLaMA-2-7B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/miniGPT4%20(LLaMA-2-7B)/scores_mathvista_minigpt4_llama2.json) | 23.1% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/miniGPT4%20(LLaMA-2-7B)/scores_mathvista_minigpt4_llama2.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/miniGPT4%20(LLaMA-2-7B)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [mPLUG-Owl (LLaMA-7B)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/mPLUG-Owl%20(LLaMA-7B)/scores_mplugowl_7b_ft.json) | 22.2% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/mPLUG-Owl%20(LLaMA-7B)/scores_mplugowl_7b_ft.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/mPLUG-Owl%20(LLaMA-7B)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |
| [IDEFICS (9B-Instruct)](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/IDEFICS%20(9B-Instruct)/scores_idefics_9b_instruct.json) | 19.8% | Published method: LMM 🖼️; Additional checkpoint and evaluation settings unverified; Source date 2023-10-03 (compiler field) | [Pinned aggregate](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/IDEFICS%20(9B-Instruct)/scores_idefics_9b_instruct.json); [source metadata](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/IDEFICS%20(9B-Instruct)/meta.json); [original result](https://arxiv.org/abs/2310.02255) |

Reference baselines, outside the model table: [Human Performance*](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Human%20Performance*/scores_human_performance.json): 60.3%; [Random Chance](https://raw.githubusercontent.com/mathvista/mathvista.github.io/a9daca33c6b3ae348b776e6586ec6b8a62656d29/data/results/Random%20Chance/scores_test_random_guess.json): 17.86%.

### Published task and skill scores

Missing values remain unreported. All values below are published accuracy percentages for the named source configuration.

- **DreamPRM (o4-mini):** figure question answering: 87.7%; geometry problem solving: 95.7%; math word problem: 83.9%; textbook question answering: 82.3%; visual question answering: 73.2%; algebraic reasoning: 94.1%; arithmetic reasoning: 80.8%; geometry reasoning: 95.5%; logical reasoning: 75.7%; numeric commonsense: 64.5%; scientific reasoning: 81.4%; statistical reasoning: 91.1%.
- **VL-Rethinker:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Step R1-V-Mini:** figure question answering: 80.3%; geometry problem solving: 89.9%; math word problem: 87.6%; textbook question answering: 77.2%; visual question answering: 63.1%; algebraic reasoning: 86.1%; arithmetic reasoning: 75.6%; geometry reasoning: 87.4%; logical reasoning: 54.1%; numeric commonsense: 56.3%; scientific reasoning: 77.9%; statistical reasoning: 87.7%.
- **Kimi-k1.6-preview-20250308:** figure question answering: 82.9%; geometry problem solving: 91.8%; math word problem: 82.3%; textbook question answering: 76%; visual question answering: 63.1%; algebraic reasoning: 87.2%; arithmetic reasoning: 73.1%; geometry reasoning: 89.1%; logical reasoning: 45.9%; numeric commonsense: 52.8%; scientific reasoning: 77%; statistical reasoning: 90.7%.
- **Doubao-pro-1.5:** figure question answering: 77.7%; geometry problem solving: 88.94%; math word problem: 86.02%; textbook question answering: 82.28%; visual question answering: 62.01%; algebraic reasoning: 89.67%; arithmetic reasoning: 73.45%; geometry reasoning: 88.29%; logical reasoning: 35.14%; numeric commonsense: 55.32%; scientific reasoning: 77.12%; statistical reasoning: 86.47%.
- **Ovis2_34B:** figure question answering: 75.46%; geometry problem solving: 84.62%; math word problem: 89.25%; textbook question answering: 72.78%; visual question answering: 62.01%; algebraic reasoning: 80.07%; arithmetic reasoning: 72.24%; geometry reasoning: 82.01%; logical reasoning: 37.84%; numeric commonsense: 54.86%; scientific reasoning: 73.77%; statistical reasoning: 85.05%.
- **Kimi-k1.5:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **OpenAI o1:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Llama 4 Maverick:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Vision-R1-7B:** figure question answering: 75.1%; geometry problem solving: 82.7%; math word problem: 80.1%; textbook question answering: 70.3%; visual question answering: 54.7%; algebraic reasoning: 78.6%; arithmetic reasoning: 65.2%; geometry reasoning: 79.1%; logical reasoning: 40.5%; numeric commonsense: 45.1%; scientific reasoning: 68.9%; statistical reasoning: 83.1%.
- **Gemini 2.0 Flash:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **QVQ-72B-Preview:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Qwen2VL-72B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Pixtral Large (124B):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Grok-2:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Grok-2 mini:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Claude 3.5 Sonnet:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **LLaVA-OneVision:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **InternVL2-8B-MPO:** figure question answering: 72.49%; geometry problem solving: 73.56%; math word problem: 69.89%; textbook question answering: 66.46%; visual question answering: 50.28%; algebraic reasoning: 70.11%; arithmetic reasoning: 57.51%; geometry reasoning: 71.55%; logical reasoning: 27.03%; numeric commonsense: 43.06%; scientific reasoning: 65.57%; statistical reasoning: 79.07%.
- **Ovis1.6-Gemma2-9B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **InternVL2-Pro:** figure question answering: 70.63%; geometry problem solving: 65.38%; math word problem: 76.88%; textbook question answering: 71.52%; visual question answering: 48.04%; algebraic reasoning: 66.55%; arithmetic reasoning: 62.32%; geometry reasoning: 63.6%; logical reasoning: 27.03%; numeric commonsense: 40.28%; scientific reasoning: 65.57%; statistical reasoning: 81.06%.
- **Chimera-Reasoner-8B:** figure question answering: 65.43%; geometry problem solving: 71.63%; math word problem: 74.73%; textbook question answering: 66.46%; visual question answering: 51.96%; algebraic reasoning: 68.33%; arithmetic reasoning: 58.92%; geometry reasoning: 70.29%; logical reasoning: 21.62%; numeric commonsense: 45.83%; scientific reasoning: 61.48%; statistical reasoning: 73.09%.
- **TextGrad (GPT-4o):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **TBAC-VLR1-3B-preview:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Gemini 1.5 Pro (May 2024):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **GPT-4o:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Phi-4-multimodal-ins:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **InternVL-Chat-V1.2-Plus:** figure question answering: 51.67%; geometry problem solving: 61.06%; math word problem: 79.57%; textbook question answering: 52.53%; visual question answering: 56.98%; algebraic reasoning: 54.45%; arithmetic reasoning: 63.17%; geometry reasoning: 61.09%; logical reasoning: 16.22%; numeric commonsense: 48.61%; scientific reasoning: 55.74%; statistical reasoning: 60.8%.
- **Gemini 1.5 Flash (May 2024):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **GPT-4T 2024-04-09:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Pixtral-12B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **InternLM-XComposer2-VL-7B:** figure question answering: 55.02%; geometry problem solving: 62.98%; math word problem: 73.66%; textbook question answering: 56.33%; visual question answering: 39.66%; algebraic reasoning: 56.58%; arithmetic reasoning: 52.41%; geometry reasoning: 62.34%; logical reasoning: 8.11%; numeric commonsense: 42.36%; scientific reasoning: 59.02%; statistical reasoning: 64.12%.
- **Gemini 1.0 Ultra:** figure question answering: 49.07%; geometry problem solving: 56.25%; math word problem: 53.76%; textbook question answering: 68.99%; visual question answering: 40.22%; algebraic reasoning: 58.36%; arithmetic reasoning: 45.89%; geometry reasoning: 55.65%; logical reasoning: 21.62%; numeric commonsense: 38.89%; scientific reasoning: 62.3%; statistical reasoning: 59.47%.
- **Grok-1.5V:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Gemini 1.5 Pro (Feb 2024):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Claude 3 Opus:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **GPT-4V (Playground):** figure question answering: 43.12%; geometry problem solving: 50.48%; math word problem: 57.53%; textbook question answering: 65.19%; visual question answering: 37.99%; algebraic reasoning: 53.02%; arithmetic reasoning: 49.01%; geometry reasoning: 51.05%; logical reasoning: 21.62%; numeric commonsense: 20.14%; scientific reasoning: 63.11%; statistical reasoning: 55.81%.
- **Claude 3 Sonnet:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **InternVL-Chat-V1.2:** figure question answering: 50.93%; geometry problem solving: 61.06%; math word problem: 30.65%; textbook question answering: 48.1%; visual question answering: 44.69%; algebraic reasoning: 52.31%; arithmetic reasoning: 36.54%; geometry reasoning: 58.16%; logical reasoning: 18.92%; numeric commonsense: 30.56%; scientific reasoning: 54.92%; statistical reasoning: 51.83%.
- **Math-LLaVA-13B:** figure question answering: 37.17%; geometry problem solving: 57.69%; math word problem: 56.45%; textbook question answering: 51.27%; visual question answering: 33.52%; algebraic reasoning: 53.02%; arithmetic reasoning: 40.23%; geometry reasoning: 56.49%; logical reasoning: 16.22%; numeric commonsense: 33.33%; scientific reasoning: 49.18%; statistical reasoning: 43.85%.
- **LLaVA-NeXT-34B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Claude 3 Haiku:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Gemini 1.0 Pro:** figure question answering: 47.58%; geometry problem solving: 40.38%; math word problem: 39.25%; textbook question answering: 61.39%; visual question answering: 39.11%; algebraic reasoning: 45.2%; arithmetic reasoning: 38.81%; geometry reasoning: 41%; logical reasoning: 10.81%; numeric commonsense: 32.64%; scientific reasoning: 54.92%; statistical reasoning: 56.81%.
- **Phi-3-Vision-128K-In:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Phi-3.5-Vision 4.2B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Mini-Gemini-HD (Hermes-2-Yi-34B):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Qwen-VL-Plus:** figure question answering: 54.65%; geometry problem solving: 38.46%; math word problem: 31.18%; textbook question answering: 55.06%; visual question answering: 34.08%; algebraic reasoning: 39.15%; arithmetic reasoning: 32.01%; geometry reasoning: 39.33%; logical reasoning: 18.92%; numeric commonsense: 26.39%; scientific reasoning: 59.02%; statistical reasoning: 56.15%.
- **SPHINX-MoE:** figure question answering: 49.81%; geometry problem solving: 31.25%; math word problem: 42.47%; textbook question answering: 46.84%; visual question answering: 39.66%; algebraic reasoning: 31.67%; arithmetic reasoning: 41.64%; geometry reasoning: 30.54%; logical reasoning: 16.22%; numeric commonsense: 27.08%; scientific reasoning: 50.82%; statistical reasoning: 50.83%.
- **Mini-Gemini (Mixtral-8x7B):** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **MM1-7B-MoE-Chat :** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **MiniCPM-V-2 (2.8B):** figure question answering: 53.16%; geometry problem solving: 25.96%; math word problem: 37.1%; textbook question answering: 44.3%; visual question answering: 39.11%; algebraic reasoning: 28.47%; arithmetic reasoning: 33.14%; geometry reasoning: 28.03%; logical reasoning: 10.81%; numeric commonsense: 39.58%; scientific reasoning: 48.36%; statistical reasoning: 51.83%.
- **MM1-30B-Chat:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **SPHINX-Plus:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **SPHINX (V2):** figure question answering: 54.65%; geometry problem solving: 16.35%; math word problem: 23.12%; textbook question answering: 41.77%; visual question answering: 43.02%; algebraic reasoning: 20.64%; arithmetic reasoning: 33.43%; geometry reasoning: 17.57%; logical reasoning: 24.32%; numeric commonsense: 21.53%; scientific reasoning: 43.44%; statistical reasoning: 51.5%.
- **MM1-7B-Chat :** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **SPHINX-Intern2:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **OmniLMM-12B:** figure question answering: 44.98%; geometry problem solving: 17.79%; math word problem: 26.88%; textbook question answering: 44.94%; visual question answering: 39.11%; algebraic reasoning: 23.13%; arithmetic reasoning: 32.29%; geometry reasoning: 20.92%; logical reasoning: 18.92%; numeric commonsense: 27.78%; scientific reasoning: 45.9%; statistical reasoning: 44.19%.
- **Multimodal Bard:** figure question answering: 26.02%; geometry problem solving: 47.12%; math word problem: 29.57%; textbook question answering: 48.73%; visual question answering: 26.82%; algebraic reasoning: 46.49%; arithmetic reasoning: 28.61%; geometry reasoning: 47.75%; logical reasoning: 13.51%; numeric commonsense: 14.89%; scientific reasoning: 47.46%; statistical reasoning: 33%.
- **LLaVA-NeXT-Vicuna-7B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **PoT GPT-4 (Caption+OCR):** figure question answering: 30.11%; geometry problem solving: 39.42%; math word problem: 30.65%; textbook question answering: 39.87%; visual question answering: 31.28%; algebraic reasoning: 37.37%; arithmetic reasoning: 31.73%; geometry reasoning: 41%; logical reasoning: 18.92%; numeric commonsense: 20.14%; scientific reasoning: 44.26%; statistical reasoning: 37.87%.
- **CoT ChatGPT (Caption+OCR):** figure question answering: 26.02%; geometry problem solving: 31.73%; math word problem: 35.48%; textbook question answering: 48.1%; visual question answering: 30.17%; algebraic reasoning: 32.38%; arithmetic reasoning: 32.29%; geometry reasoning: 33.05%; logical reasoning: 16.22%; numeric commonsense: 17.36%; scientific reasoning: 54.92%; statistical reasoning: 36.21%.
- **CoT Claude (Caption+OCR):** figure question answering: 27.51%; geometry problem solving: 29.33%; math word problem: 36.02%; textbook question answering: 49.37%; visual question answering: 29.05%; algebraic reasoning: 30.96%; arithmetic reasoning: 32.86%; geometry reasoning: 30.96%; logical reasoning: 16.22%; numeric commonsense: 17.36%; scientific reasoning: 50.82%; statistical reasoning: 37.21%.
- **CoT GPT4 (Caption+OCR):** figure question answering: 27.88%; geometry problem solving: 31.73%; math word problem: 31.18%; textbook question answering: 51.9%; visual question answering: 28.49%; algebraic reasoning: 33.45%; arithmetic reasoning: 30.88%; geometry reasoning: 32.22%; logical reasoning: 13.51%; numeric commonsense: 12.5%; scientific reasoning: 58.2%; statistical reasoning: 37.87%.
- **MM1-3B-MoE-Chat:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **MM1-3B-Chat:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **Gemini 1.0 Nano 2:** figure question answering: 28.62%; geometry problem solving: 23.56%; math word problem: 30.65%; textbook question answering: 41.77%; visual question answering: 31.84%; algebraic reasoning: 27.05%; arithmetic reasoning: 29.75%; geometry reasoning: 26.78%; logical reasoning: 10.81%; numeric commonsense: 20.83%; scientific reasoning: 40.16%; statistical reasoning: 33.55%.
- **LLaVA-1.5-13B:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **SPHINX (V1):** figure question answering: 23.42%; geometry problem solving: 23.08%; math word problem: 21.51%; textbook question answering: 39.87%; visual question answering: 34.08%; algebraic reasoning: 25.62%; arithmetic reasoning: 28.05%; geometry reasoning: 23.43%; logical reasoning: 16.22%; numeric commonsense: 17.36%; scientific reasoning: 40.16%; statistical reasoning: 23.59%.
- **Gemini 1.0 Nano 1:** figure question answering: 30.86%; geometry problem solving: 21.63%; math word problem: 23.66%; textbook question answering: 29.11%; visual question answering: 30.73%; algebraic reasoning: 23.84%; arithmetic reasoning: 25.5%; geometry reasoning: 21.34%; logical reasoning: 13.51%; numeric commonsense: 20.83%; scientific reasoning: 27.87%; statistical reasoning: 30.9%.
- **PoT ChatGPT (Caption+OCR):** figure question answering: 24.54%; geometry problem solving: 26.44%; math word problem: 23.66%; textbook question answering: 33.54%; visual question answering: 27.93%; algebraic reasoning: 27.76%; arithmetic reasoning: 26.06%; geometry reasoning: 28.03%; logical reasoning: 18.92%; numeric commonsense: 13.19%; scientific reasoning: 33.61%; statistical reasoning: 29.9%.
- **SPHINX-Tiny:** figure question answering: Not reported; geometry problem solving: Not reported; math word problem: Not reported; textbook question answering: Not reported; visual question answering: Not reported; algebraic reasoning: Not reported; arithmetic reasoning: Not reported; geometry reasoning: Not reported; logical reasoning: Not reported; numeric commonsense: Not reported; scientific reasoning: Not reported; statistical reasoning: Not reported.
- **LLaVA (LLaMA-2-13B):** figure question answering: 26.77%; geometry problem solving: 29.33%; math word problem: 16.13%; textbook question answering: 32.28%; visual question answering: 26.26%; algebraic reasoning: 27.31%; arithmetic reasoning: 20.06%; geometry reasoning: 28.83%; logical reasoning: 24.32%; numeric commonsense: 18.31%; scientific reasoning: 37.29%; statistical reasoning: 25.08%.
- **InstructBLIP (Vicuna-7B):** figure question answering: 23.05%; geometry problem solving: 20.67%; math word problem: 18.28%; textbook question answering: 32.28%; visual question answering: 35.2%; algebraic reasoning: 21.77%; arithmetic reasoning: 27.14%; geometry reasoning: 20.72%; logical reasoning: 18.92%; numeric commonsense: 20.42%; scientific reasoning: 33.05%; statistical reasoning: 23.1%.
- **LLaVAR:** figure question answering: 21.93%; geometry problem solving: 25%; math word problem: 16.67%; textbook question answering: 34.81%; visual question answering: 30.73%; algebraic reasoning: 24.2%; arithmetic reasoning: 22.1%; geometry reasoning: 23.01%; logical reasoning: 13.51%; numeric commonsense: 15.28%; scientific reasoning: 42.62%; statistical reasoning: 21.93%.
- **LLaMA-Adapter-V2 (7B):** figure question answering: 21.19%; geometry problem solving: 25.48%; math word problem: 11.29%; textbook question answering: 32.28%; visual question answering: 31.84%; algebraic reasoning: 26.33%; arithmetic reasoning: 20.4%; geometry reasoning: 24.27%; logical reasoning: 24.32%; numeric commonsense: 13.89%; scientific reasoning: 29.51%; statistical reasoning: 18.27%.
- **miniGPT4 (LLaMA-2-7B):** figure question answering: 18.59%; geometry problem solving: 25.96%; math word problem: 13.44%; textbook question answering: 30.38%; visual question answering: 30.17%; algebraic reasoning: 28.11%; arithmetic reasoning: 20.96%; geometry reasoning: 24.69%; logical reasoning: 16.22%; numeric commonsense: 16.67%; scientific reasoning: 25.41%; statistical reasoning: 17.94%.
- **mPLUG-Owl (LLaMA-7B):** figure question answering: 22.68%; geometry problem solving: 23.56%; math word problem: 10.22%; textbook question answering: 27.22%; visual question answering: 27.93%; algebraic reasoning: 23.62%; arithmetic reasoning: 19.17%; geometry reasoning: 23.87%; logical reasoning: 13.51%; numeric commonsense: 12.68%; scientific reasoning: 26.27%; statistical reasoning: 21.45%.
- **IDEFICS (9B-Instruct):** figure question answering: 21.56%; geometry problem solving: 21.15%; math word problem: 6.45%; textbook question answering: 25.95%; visual question answering: 24.02%; algebraic reasoning: 22.14%; arithmetic reasoning: 15.04%; geometry reasoning: 19.82%; logical reasoning: 18.92%; numeric commonsense: 9.86%; scientific reasoning: 24.58%; statistical reasoning: 18.15%.

## FAQ

### What does MathVista measure?

MathVista tests mathematical reasoning about images. This table shows published accuracy on the testmini split, with each source configuration kept separate. The results are display only and do not enter overall or category rankings.

### Which source row has the highest reported MathVista value?

DreamPRM (o4-mini) has the highest reported MathVista value at 85.20%. The compilation contains different evaluation setups; the highest reported value does not establish a comparable model ranking.

### How many models are evaluated on MathVista?

The June 9, 2025 contains 78 published model or configuration records.

### Does MathVista affect BenchLM's overall score?

Not directly. MathVista is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.