VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text
Organizations: Nanyang Technological University
Abstract
Visual-text compression renders long contexts as images and is engineered around how much of the text a vision-language model (VLM) can recover. Yet recovered text is not usable text. VLMs can transcribe the text in an image word for word and still fail to answer from it; given back their own transcript, they answer nearly as well as from the original text. We call this difference the read-use gap. Recognizing and using visual text have different resolution requirements: Qwen3-VL-8B's recognition saturates at 15-pixel characters while use catches up only at 24 pixels, and scaling from 2B to 32B raises recognition without narrowing the gap. Putting the evidence into words closes it: thinking VLMs recite the evidence in their thinking, and their own transcript, written without seeing the question, recovers most of what thinking adds. In a preregistered held-out test, the gap ranks how much eight VLMs gain from their own words over pixels (Spearman = 0.86), which recognition rates do not. We propose ALOUD, which spends visual resolution on finding and recognizing evidence and language tokens on using it. Without keeping the source text, it doubles the accuracy of a frozen Qwen3-VL-8B on long documents compressed 5x and 10x: at 5x, from 27.7% to 54.8%, near the original text's 58.5%, with a context smaller than the compressed pages themselves. For less than the cost of reading the original text, it leads at equal budget every method that does without it, at every level, and beats AGAR and VERA, which paste the original text back; at 5x it also beats LensVLM-9B, which is trained and keeps the original text. Visual-text compression is limited not by how much text a model can recover, but by how much it can use.
Figures & tables
| 5 | 10 | 15 | |||||||||||
| Group | Method | HQA | MSQ | NQ | Avg. | HQA | MSQ | NQ | Avg. | HQA | MSQ | NQ | Avg. |
| AGAR | 9.5 | 2.6 | 1.3 | 4.5 | 1.3 | 1.6 | 3.5 | 1.1 | 0.7 | 1.2 | 2.1 | 0.9 | |
| VERA (text) | 22.4 | 11.4 | 15.3 | 16.4 | 12.6 | 5.0 | 13.4 | 10.3 | 1.3 | 2.7 | 9.3 | 1.8 | |
| LensVLM-9B † | 31.1 | 20.4 | 22.0 | 24.5 | 28.4 | 14.8 | 27.2 | 23.5 | 18.6 | 8.0 | 17.1 | 14.5 | |
| Original text | BM25 | 34.5 | 19.8 | 28.9 | 27.8 | 40.7 | 22.3 | 33.7 | 32.2 | 41.3 | 18.7 | 35.2 | 31.7 |
| Aloud , every page | 34.1 | 18.0 | 27.3 | 26.5 | 34.2 | 12.4 | 23.0 | 23.2 | 22.4 | 6.9 | 21.2 | 16.8 | |
| Benchmark | Reader | As | Alone | Aloud |
|---|---|---|---|---|
| RULER, DPI 72 | Glyph | W | 73.2 | 79.8 6.6 |
| LongBench, DPI 72 | Glyph | W | 53.4 | 60.1 6.7 |
| LongBench, DPI 48 | Glyph | W | 39.4 | 51.7 12.3 |
| MMLongBench-Doc | Qwen3-VL-8B | P | 39.7 | 45.6 5.9 |
| Long-doc QA, | GPT-6.1 | W | 52.5 | 55.0 |
| Long-doc QA, | Gemini 4 | W | 75.7 | 82.3 6.6 |
| Observation | Methods | Evidence |
|---|---|---|
| Obs. 1 VLMs recognize evidence they cannot use | A.2 | B.1 , B.3 , B.6 |
| Obs. 2 Recognizing visual text needs fewer pixels than using it | A.2 | B.2 |
| Obs. 3 Thinking closes the gap by putting the answer into words | A.3 | B.4 , B.5 |
| Obs. 4 The answer reads the image before its characters become words | A.5 | B.6 |
| Obs. 5 The read–use gap ranks how much words beat pixels | A.6 | B.7 |
| Obs. 6 – Obs. 7 Aloud finds the evidence and leads at equal budget | A.7 , A.8 | B.8 , B.9 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| 5 | 10 | 15 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pages | BM25 | Scan | Fused | Gen. | BM25 | Scan | Fused | Gen. | BM25 | Scan | Fused | Gen. |
| 1 | 39.7 | 39.7 | 39.7 | 440 | 32.7 | 32.3 | 32.3 | 587 | 25.7 | 25.7 | 25.7 | 497 |
| 2 | 50.3 | 50.3 | 50.3 | 888 | 37.3 | 36.7 | 37.7 | 1,190 | 26.7 | 26.7 | 26.3 | 1,037 |
| 3 | 53.3 | 53.7 | 54.0 | 1,337 | 42.0 | 37.3 | 43.3 | 1,784 | 30.0 | 27.7 | 29.0 | 1,573 |
| 4 | 52.7 | 52.7 | 53.3 | 1,789 | 41.3 | 37.7 | 41.0 | 2,384 | 31.3 | 26.7 | 29.7 | 2,106 |
| 5 | 52.7 | 51.3 | 54.0 | 2,232 | 42.3 | 38.0 | 42.7 | 2,994 | 31.0 | 26.7 | 29.3 | 2,614 |
| Group | Reader | Recognized (%) | Text | Transcript | Image | Gap | 95% CI | All |
|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-2B | 62 | 57.0 | 52.7 | 41.4 | +11.3 | [+3.2, +19.4] | +4.0 | |
| Qwen3-VL-4B | 70 | 64.6 | 60.3 | 46.4 | +13.9 | [+6.2, +21.5] | +7.3 | |
| Qwen3-VL-8B | 84 | 70.8 | 66.4 | 55.7 | +10.7 | [+4.7, +16.6] | +7.7 | |
| Qwen3-VL-32B | 88 | 74.6 | 73.5 | 59.5 | +14.0 | [+8.7, +19.3] | +11.7 | |
| InternVL3.5-8B | 65 | 69.9 | 68.4 | 56.1 | +12.2 | [+5.1, +18.9] | +5.3 | |
| Instruction-tuned | Gemma-3-12B | 80 | 68.0 | 66.0 | 58.9 | +7.1 | [+1.2, +13.3] | +5.0 |
| Reader | Recognized | Text | Transcript | Image | Gap | 95% CI | All |
|---|---|---|---|---|---|---|---|
| GPT-6.1 | 93% (84) | 81.0 | 76.2 | 72.6 | +3.6 | [ 2.4, +10.7] | +4.4 |
| Gemini 4 | 98% (88) | 85.2 | 81.8 | 84.1 | 2.3 | [ 6.8, +2.3] | 2.2 |
| Evidence | 144 DPI | 72 DPI | 48 DPI | |
|---|---|---|---|---|
| Text | 0.0 | +7.2 | +25.8 | 93 |
| Layout | 13.9 | 0.0 | +9.4 | 32 |
| Table | 3.0 | +4.9 | +12.0 | 117 |
| Chart | 10.9 | +4.0 | +16.1 | 93 |
| Picture | 14.8 | 5.4 | 8.3 | 108 |
| All | 7.6 | +2.5 | +10.6 |
| Recognized | Recognized, prior wrong | |||||||
|---|---|---|---|---|---|---|---|---|
| Reader | Questions | Prior only | Gap | 95% CI | Gap | 95% CI | ||
| Qwen3-VL-2B | 300 | 12.7 | 186 | +12.4 | [+4.3, +20.4] | 155 | +16.1 | [+7.1, +24.5] |
| Qwen3-VL-4B | 300 | 17.7 | 209 | +14.8 | [+7.7, +22.0] | 166 | +16.9 | [+8.4, +25.3] |
| Qwen3-VL-8B | 280 | 21.1 | 236 | +9.3 | [+3.8, +14.8] | 181 | +11.0 | [+4.4, +17.7] |
| Qwen3-VL-32B | 150 | 22.7 | 134 | +10.4 | [+3.7, +17.9] | 101 | +15.8 | [+6.9, +24.8] |
| Control | Result |
|---|---|
| Synthetic records, no prior knowledge, asked in the evidence’s words | gap ; image 83.0, own transcript 98.4 (182 of 210 questions) |
| the same, question reworded | change in the gap |
| Reasoning freely before answering | on recognized questions (CI crosses zero) |
| Transcribing first | |
| Prefilling own transcript vs. as many dots | |
| Prefilling own transcript vs. another question’s |
| Thinking on | Thinking off | In the trace | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Reader | Image | Transcript | Gap | 95% CI | Image | Transcript | Gap | 95% CI | Quotes | Answer |
| GLM-4.1V-9B-Thinking | 69.9 | 68.4 | 1.5 | [ 8.2, +5.6] | 51.5 | 62.8 | +11.2 | [+4.1, +18.9] | 87% | 70% |
| Qwen3-VL-8B-Thinking | 63.8 | 69.8 | +6.0 | [ 0.5, +13.1] | 46.2 | 59.3 | +13.1 | [+5.5, +20.6] | 88% | 68% |
| GLM-4.1V-9B-Thinking | Qwen3-VL-8B-Thinking | ||||
| Prefilled thinking | Words | Acc. | Rec. | Acc. | Rec. |
| Off (empty) | 0 | 51.8 | 0% | 51.5 | 0% |
| Full trace | 100% | 67.9 | 100% | 67.5 | 100% |
| Answer kept | |||||
| Quotes only | 26 / 30% | 61.1 | 58% | 62.6 | 69% |
| Reasoning with answer | 75 / 70% | 67.4 | 97% | 65.6 | 88% |
| Comparison | GLM-4.1V-9B-Thinking | Qwen3-VL-8B-Thinking |
|---|---|---|
| Own transcript full trace | ||
| Answer only other quoted words | ||
| Answer only full trace | ||
| Answer only quotes only | ||
| Quotes without answer off | ||
| Reasoning without answer off |
| Prefilled thinking | GLM-4.1V-9B-Thinking | Qwen3-VL-8B-Thinking |
|---|---|---|
| Gold answer: answer given | 90.2% | 93.9% |
| Wrong entity from the evidence: entity given | 23% | 37% |
| accuracy, empty wrong entity | 52.1 39.6 | 52.5 35.0 |
| Questions its own full thinking got wrong (62 and 53) | ||
| Gold answer: answer given | 69.4% | 81.1% |
| Empty thought: gold answer given | 17.7% | 18.9% |
| All recognized | Not given away by candidates | |||||
| Reader | Gap | With cand. | Gap | With cand. | ||
| Qwen3-VL-2B | 174 | 65 | ||||
| Qwen3-VL-4B | 199 | 48 | ||||
| Qwen3-VL-8B | 240 | 54 | ||||
| InternVL3.5-8B | 183 | 40 | ||||
| Gemma-3-12B | 228 | 58 | ||||
| Start end | |||||
|---|---|---|---|---|---|
| Reader | Layers | Text | Pixels | Words | Read before words |
| Qwen3-VL-2B | 28 | 10 22 | 10 22 | 23 | 79% |
| Qwen3-VL-4B | 36 | 14 30 | 12 30 | 27 | 61% |
| Qwen3-VL-8B | 36 | 12 28 | 14 30 | 30 | 81% |
| Qwen3-VL-32B | 64 | 40 56 | 40 56 | 60 | 99% |
| Second look at 144 DPI | Second look at 48 DPI | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Reader | 15 px gap | 24 px gap | Pixels | Words | 95% CI | 6 px gap | Pixels | Words | 95% CI | ||
| Qwen3-VL-2B | +5.3 | +8.3 | 31.8 | 28.3 | 3.5 | [ 6.4, 0.3] | +20.0 | 17.0 | 19.6 | +2.6 | [+0.0, +5.4] |
| Qwen3-VL-4B | +7.9 | +10.2 | 41.3 | 39.4 | 1.9 | [ 4.7, +0.9] | +24.3 | 20.1 | 25.9 | +5.7 | [+3.0, +8.3] |
| Qwen3-VL-8B | +7.9 | +4.1 † | 45.1 | 43.4 | 1.7 | [ 4.5, +1.0] | +14.2 | 27.3 | 34.2 | +6.9 | [+4.0, +9.9] |
| Qwen3-VL-32B | +10.5 | +6.2 | 50.7 | 50.9 | +0.2 | [ 2.4, +3.0] | +17.8 | 30.7 | 37.8 | +7.1 | [+4.0, +10.2] |
| InternVL3.5-8B | +13.2 | +17.3 | 39.2 | 40.8 | +1.6 | [ 1.9, +5.0] | +11.2 | 35.9 | 34.9 | 1.0 | [ 4.3, +2.1] |
| Predictor and points | Test | |
| Registered | ||
| 15 px gap, eight readers at 144 DPI (Addendum 1) | exact one-sided | |
| document bootstrap (130 documents) | 95% CI | |
| Matched gap, 13 points over both densities (Addendum 2) | permutation | |
| Post hoc, same data | ||
| 15 px gap, four Qwen readers | ||
| 5 | 10 | 15 | ||||
| Method | Acc. | Rate | Acc. | Rate | Acc. | Rate |
| Uniform rendering | 27.7 | 5.9 | 18.3 | 11.7 | 16.5 | 16.4 |
| AGAR | 33.3 | 5.4 | 20.5 | 10.7 | 17.8 | 15.2 |
| VERA (text) | 48.0 | 4.3 | 40.7 | 4.9 | 30.2 | 5.6 |
| LensVLM-9B † | 56.5 | 4.1 | 51.7 | 5.7 | 37.2 | 7.9 |
| BM25 | 56.5 | 5.5 | 52.8 | 9.7 | 49.7 | 12.8 |
| 5 | 10 | 15 | ||||
| Step | Pre. | Gen. | Pre. | Gen. | Pre. | Gen. |
| Uniform rendering | 1,675 | 4 | 897 | 5 | 729 | 5 |
| Scan: all pages | 1,675 | – | 897 | – | 729 | – |
| Read aloud: found pages | 955 | 1,347 | 991 | 1,799 | 988 | 2,077 |
| Answer: kept passages | 1,276 | 5 | 881 | 6 | 676 | 6 |
| Aloud , total | 3,905 | 1,353 | 2,769 | 1,805 | 2,393 | 2,083 |
| 5 | 10 | 15 | |
| Pages six found lines | |||
| First pass, enlarged | 12.2 | 7.9 | 2.4 |
| Closer look, enlarged | 19.2 | 15.8 | 9.3 |
| cropped to text lines | 17.2 | 15.8 | 10.1 |
| read aloud | 21.7 | 19.9 | 7.3 |
| Pages read aloud, words only | |||
| Setting | Reader | W | P | Why |
|---|---|---|---|---|
| MMLongBench-Doc | Qwen3-VL-8B | 42.7 | 45.6 | native 144 DPI, about 24 px |
| Long-doc QA, | GPT-6.1 | 55.0 | 51.7 | rendered text |
| Long-doc QA, | Gemini 4 | 81.7 | 76.7 | rendered text |
Vision-Language Models
Vision-language models connect visual inputs with text for retrieval, question answering, generation, and grounded multimodal reasoning.
See all 764 papers