cs.CVOct 29, 2026
SaveTelling Is Not Seeing: Textual Personas Rarely Make Vision-Language Models Answer Like the People They Describe
Organizations: Netflix
Abstract
Vision-language models (VLMs) are increasingly prompted with personas, such as "You have low vision", to stand in for people in user studies. We ask whether a persona makes a model see like the person it describes. We compare persona answers with measured people with low vision, color blindness or hearing loss, and with simple rules that use only the facts the persona states. The persona never beats these rules, and it erases the real differences between groups and individuals. Describing blur leaves recognition unchanged in all 14 models we test, while blurring the photo lowers it as it does for people. Inside open models, the description changes neither the image information nor the answer probabilities. Yet the models know the effects they fail to show: they can estimate how blur and illusions affect people. Telling is not seeing: a persona makes a model say what it knows about people, not see what they see.
Figures & tables
Figure 1: Telling a model about blur does not make it see blur. Four sighted people and Gemini name objects in photos at eight blur levels. Top: Gemini is either shown the blurred photo, or given the clean photo and told about the blur by a persona that states the matching acuity. Bottom: shown the blur, Gemini loses accuracy as people do; told about it, Gemini stays at 98–99%. The photos illustrate the blur levels; the test photos are greyscale.
| Study: task | People (data) | The persona states | We compare with | Models |
| Primary tests: people whose own answers are recorded ( Sec. 4.1 ) | ||||
| Low-vision reading: read sentences whose print size is stated | 56 readers with low vision [ 54 ] | The reader’s letter-chart acuity, contrast sensitivity and diagnosis | The rule “reading limit = letter-chart acuity”; an image filter | 3 frontier, 4 open |
| Colour sorting (D-15): order 15 caps by hue | 39 colour-blind viewers (group scores) [ 50 ] | The type of colour blindness | No persona; the textbook order for the stated type; colours changed for that type | 3 frontier, 4 open a |
| Subtitle viewers: choose line breaks; rate clips; state preferences | 88–94 viewers in five groups [ 44 ] | The participant’s language, age, gender, English level and hearing | The same model without a persona | 3 frontier, 4 open b |
| Controlled test: sighted people view changed images ( Sec. 4.2 ) | ||||
| Blur: name the object in a photo | 4 observers with normal vision [ 12 ] | The acuity that matches the blur | The blurred photo, no persona | 14 |
Table 1: Overview of the five studies. Frontier models: Gemini-3.8-Flash, Claude-Opus-5.5, GPT-6-Sol. a Three of the four open models cannot do the task without a persona. b One open model cannot run the subtitle-speed clips.
Figure 2: A reading persona copies the stated letter-chart acuity, so it misses readers whose reading limit is different. (a) Stated letter-chart acuity against reading limit, per reader (logMAR). Black: measured limit. Red: Gemini with the persona. Dashed: acuity rule. (b) Mean error of the frontier models for three ways of telling the model about the reader; none beats the acuity rule (dashed, 95% interval).
Figure 3: Colour-blind personas give a partial textbook answer that fails for red-blind viewers; changed colours work. Each square is one of 20 cap layouts per type. Red: persona. Blue: no persona, colours changed as that type sees them. Filled: ordered like the named type; dark grey: like another type; open: like a normal viewer; hatched: no valid answer.
Figure 4: Subtitle personas answer like the model, not like the viewer. (a) Share of viewers who prefer two-line to three-line subtitles (95% intervals; all 19 deaf and hard-of-hearing viewers chose two lines). (b) A different task: for 30 two-line subtitles, viewers chose between a well-split and a poor line break. Each dot is one person (black) or that person’s persona (red). Diamonds: model without a persona. Right: spread across viewers (SD), and agreement, the share of frames on which the model chose the same break as the viewer, with no persona and with that viewer’s persona.
Figure 5: A text description of blur has no effect in any of 14 models; blurred photos lower accuracy in all of them. Model’s accuracy loss divided by people’s (1 = same; 0 = none; 95% intervals). Red: clean photo with the persona. Blue: blurred photo. Grey band: 0.80–1.25. Bold: as large as people’s after Holm correction.
Figure 6: Inside Qwen3-VL-8B, the blur persona leaves the image information intact. Linear probes of the hidden states on held-out photos, per layer (bands: 95% intervals). (a) The object is as decodable from the image tokens with the blur persona (red) as for the clean photo (black) or an unrelated persona of the same length (yellow, dashed); blurred photos (blue) remove more of it as the blur grows. (b) A probe that tracks real blur (= 1) at the last prompt token reads almost none of the stated blur, no more than for the unrelated persona.
Figure 7: Models know how often illusions fool people, but their own answers are rarely fooled. (a) Example images; numbers are human error rates. (b) Illusion effect, relative to people, when Gemini, Opus and Sol answer themselves (blue) and estimate people’s answers (purple). Grey band: 0.80–1.25. Bottom row: human effect. (c) Qwen2.5-VL-72B matches people on Müller-Lyer (0.99), but only outward fins mislead it, while both fin types mislead people.
More in this topic
VLM Evaluation
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
See all 999 papers