TextVQA

We show this table for reference; we do not rank on it.

TextVQA requires reading and reasoning about text in images. These two own-provider reports use the validation split, with metric and scorer left undisclosed in the original model cards.

Evaluation results published by Zyphra. Two original Zyphra TextVQA validation reports; metric/scorer not disclosed.

How to read these TextVQA reports

Read with care: the benchmark is not marked current.

Saturation Different source configurations: Separate original-provider cohorts are shown below. Undisclosed metrics and different report setups do not support a shared performance ranking or a pooled score.
Who produced the rows Mirrored table: Selected rows come from the original reports linked in each source cohort. BenchLM did not rerun these evaluations.
Freshness and weight Display only: Dated provider reports, refreshed reviewed pinned original-provider artifacts. Display only; it does not move rankings. Original upstream task; no task contents mirrored.

Row statuses come from the verification registry; saturation and freshness from the benchmark lifecycle. A provider self-report is displayable evidence, not an independent run.

Zyphra: ZAYA1-VL-8B model card

Zyphra harness based on VLMEvalKit; metric/scorer and run settings not disclosed.

Source repository modified: 2026-05-17; this is not an evaluation date.

Zyphra: ZAYA1-VL-8B model card: own-provider aggregate scores
Source model	Reported score	Metric/scorer	Split	Original provider source	Parameters (B)	Open / closed
ZAYA1-VL-8BSource column: ZAYA1-VL-8B(0.7B / 8B)	74.4	Metric/scorer not disclosed	validation	Zyphra reportPinned official model-card Performance table; own-model column only	Not reported	Unknown

Zyphra: Zamba2-VL-7B model card

Zyphra harness based on VLMEvalKit; metric/scorer and run settings not disclosed.

Source repository modified: 2026-05-24; this is not an evaluation date.

Zyphra: Zamba2-VL-7B model card: own-provider aggregate scores
Source model	Reported score	Metric/scorer	Split	Original provider source	Parameters (B)	Open / closed
Zamba2-VL-7B	81.0	Metric/scorer not disclosed	validation	Zyphra reportPinned official model-card Performance table; own-model column only	Not reported	Unknown

Only each provider's own model rows are included. Undisclosed metrics keep the source's printed number without an accuracy, ANLS, or percentage interpretation. Scorer revisions, individual run dates, exact evaluated checkpoints, image processing, and evaluated question counts remain unknown. Separate cohorts do not establish a shared protocol or a cross-cohort rank. These reports do not affect overall or category scores.

How we show these TextVQA reports

This page retains 2 selected rows across original provider reports about their own models. Comparison-model columns from those reports are excluded. Source model identities remain separate from catalog profiles.

Each cohort keeps its original split, printed score, source, and disclosed setup. The source revision or report publication date is separate from an evaluation date. Unknown settings remain unreported, and the scores do not enter overall or category rankings.

Snapshot

Own-provider rowsSeparate cohortsDisplay only

Zyphra ZAYA1-VL-8B model card Zyphra Zamba2-VL-7B model card Original benchmark task

About TextVQA

Year

2019

Format

Provider-reported validation scores; metric/scorer not disclosed

The source calls the split val. ZAYA1-VL-8B and Zamba2-VL-7B retain separate model-card cohorts. The named VLMEvalKit-based harness does not establish an exact scorer revision or run setup. Scores stay display only, without a percentage suffix or an inferred accuracy definition.

Towards VQA Models That Can Read Public benchmark source

Freshness and provenance

Version

Selected May 2026 Zyphra model cards; validation split

Refresh cadence

Reviewed pinned original-provider artifacts

Staleness state

Dated provider reports

Question availability

Original upstream task; no task contents mirrored

Dated provider reportsDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does TextVQA measure?

TextVQA requires reading and reasoning about text in images. These two own-provider reports use the validation split, with metric and scorer left undisclosed in the original model cards.

Can these source results be compared directly?

Separate original-provider cohorts are shown below. Undisclosed metrics and different report setups do not support a shared performance ranking or a pooled score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many source reports are shown?

This snapshot shows 2 selected source reports. Each row retains its original source and disclosed setup.

Do these TextVQA reports use the test split?

Both original provider cards label these rows TextVQA val, meaning validation. We retain that split and do not present them as test results. The cards leave scorer identity and evaluated question counts undisclosed, so the table preserves their printed numeric scores without claiming a particular VQA accuracy calculation.

Last updated: May 2026 source revisions; checked October 1, 2026 · retained from pinned original source reports

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 5,600+ readers.

One email each week. Unsubscribe anytime.