Know When to Zoom: How Small Is Too Small for Vision-Language Models?
Organizations: Carnegie Mellon University, Pittsburgh, PA, USA · Amazon Web Services AI Native, Pittsburgh, PA, USA
Abstract
Online vision-language models often miss small details in large images, because the service shrinks the image, or rejects it, before the model ever sees it. A car 20 pixels wide in an 8000 by 6000 drone photo reaches Claude about 6 pixels wide, and OpenAI's default rejects the photo outright. We measure how much detail survives for seven models from four vendors, using an eye exam of printed codes and 601 questions about small objects in real photos. The smallest text a model reads half the time grows from 6 pixels or less on a small image to a predicted 10 to 13 pixels on a 7680 by 4320 photo at default settings. Zooming wins much of the detail back. Splitting each photo into up to nine tiles raises accuracy by 11.1 to 14.5 percentage points for Claude and Gemini, and by avoiding rejections it lifts GPT-6.1 Sol from 50.7% to 83.7%. A two-step zoom, in which the model first names the region it needs, matches or beats tiles on Gemini and comes within 2.5 points of them on Claude, with about 40% of their image tokens. When a detail is lost, Claude and Gemini both say they cannot tell about half the time, but they fail differently in the rest: Gemini gives a confident wrong answer in 43% to 48% of these cases, Claude in only 5% to 9%. A check based on detail size, run before the call, misses most of the gain from tiles on Claude and Gemini, but Claude's own reply is a reliable trigger: tiling only after an unsure reply comes within 1.4 points of always tiling with 62% to 70% of the image tokens. We turn these findings into five rules for when an agent should zoom, and release the eye exam, every model answer and code at https://siddvoh.com/know-when-to-zoom.
Figures & tables
| Service and models | Setting | Rule | 12 MP photo, 4000 3000 | 8K photo, 7680 4320 |
| Claude Haiku 5.5, Sonnet 5.5 | native | long side 2576 px, 4,784 tokens | 2212 1659 | 2576 1449 |
| standard | long side 1568 px, 1,568 tokens | 1270 952 | 1456 819 | |
| Gemini 3.5 Flash-Lite, 3.8 Flash | low | at most 280 tokens | 266 tokens, about 1,300 px | 264 tokens, about 1,300 px |
| medium | at most 560 tokens | 540 tokens, about 1,700 px | 527 tokens, about 1,700 px | |
| high or unset | at most 1,120 tokens | 1,064 tokens, about 2,300 px | 1,100 tokens, about 2,600 px | |
| ultra high | at most 2,240 tokens | 2,160 tokens, about 3,400 px | 2,205 tokens, about 3,100 px |
| Canvas long side (px) | ||||||
| Model | Setting | 1024 | 2048 | 4096 | 8000 | H1 |
| Claude Haiku 5.5 | native | 4.7 | 4.8 | 8.1 | 14.4 | yes |
| standard | 4.4 | 6.8 | 12.5 | 24.8 | yes | |
| Claude Sonnet 5.5 | native | 4.2 | 4.4 | 6.5 | 12.0 | no |
| standard | 4.3 | 5.5 | 9.9 | 21.3 | no | |
| Gemini 3.5 Flash-Lite | high | 4.8 | 4.9 | 8.2 | 13.7 | no |
| Model | As is | Highest | 2 2 | 3 3 | Zoom | Check |
|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 59.2 | n/a | 73.2 | 73.7 | 73.5 | 59.4 |
| Gemini 3.5 Flash-Lite | 58.6 | 64.1 | 70.5 | 70.5 | 72.2 | 58.9 |
| GPT-6 Luna | 38.6 | n/a | 67.1 | 65.7 | 70.2 | 65.6 |
| Claude Sonnet 5.5 | 72.4 | n/a | 82.0 | 83.5 | 81.0 | 72.4 |
| Gemini 3.8 Flash | 68.1 | 71.7 | 78.5 | 80.5 | 83.0 | 67.9 |
| GPT-6.1 Sol | 50.7 | n/a | 83.5 | 83.7 | 82.2 | 83.0 |
Vision-Language Models
Vision-language models connect visual inputs with text for retrieval, question answering, generation, and grounded multimodal reasoning.
See all 774 papers