How VLMs Plan What's Next
Organizations: Northeastern University
Abstract
When a vision-language model describes an image, it names one object at a time and it rarely repeats itself, yet how the model plans this order is far from obvious. We characterize the mechanism a VLM uses to decide where to look before it looks. The mechanism has four main parts. First, binding heads match the relevant text tokens against the image and resolve them into pointers to the matched image region. Second, the same binding heads bind each written answer to the region it came from, and the model turns it into a progress signal: a cursor, the position in the image up to which the list has been written. Third, a select step chooses the nearest unwritten region after the cursor, or stops. Finally, gaze heads execute the chosen pointer and return a concept of what is drawn there, which language heads and the late MLPs turn into a word in the language of the question. We validate each part of this mechanism through causal patching in Qwen3-VL models on a synthetic visual retrieval task, and test our mechanisms against alternative hypotheses. We further show that the mechanism holds across input formats, natural photographs, and languages, and that its binding, relay, and gaze heads appear at every model size from 2B to 32B. This work allows us to investigate how a VLM turns a question and an image into a sequence of answers, layer by layer and token by token. More broadly, it opens a way to study how models plan over complex multi-step tasks with long rollouts. Our project is available at https://plan.baulab.info/
Figures & tables
Appendix figures & tables51 assets
Supplementary material from the paper’s appendix.
Appendix
| Experiment | Token | Patched or blocked | Target answers | Result |
|---|---|---|---|---|
| Residual patch by layer | colon | the residual vector, one layer at a time | K | K at layers 23-29; D from layer 30 |
| Residual patch at every token | every text token | the residual vector, one token and one layer at a time | logit of K | shape token at layers 16-20; colon token at 23-29 |
| Binding heads | shape | outputs of the 16 heads, layers 15-21 | K | 85.9 (random 0.1) |
| Query heads | shape | queries of the top 8 heads | P | 81.0 |
| Key heads | shape | the top 8 heads read the source’s image keys | R | 83.1 |
| Query and key heads | shape | queries and image keys of both sets | K | 95.0 |
| Experiment | Token | Patched or blocked | Target answers | Result |
|---|---|---|---|---|
| Binding on a list | shape | no patch: image attention of the binding heads and the 2 query and key heads outside them, and a probe on their summed output | the matching panels | 79.6 of image attention; probe 97.7 |
| Progress at the comma token | colon | the source’s comma-token residual, one layer at a time; the source wrote one letter, at the target’s first matching panel | its second answer | 91.0 at layer 20; 70.0 at 21 |
| Count, set, or position | colon | the same patch; the source wrote two letters, from panels 2 and 4 | its third match (count), its first (set), or the first after panel 4 (position) | count 77.8-90.8 at layers 13-18; position 98.4 at 20, 97.6 at 21; set 1.2 |
| Cursor set at the last answer token | last answer | the 16 binding heads read the keys of an image with the last written letter one panel to the right | the letter after the moved panel | 84.2 (random 0.0); the 32 answer-token heads 88.0 |
| first answer token | first answer | the same patch, with the first written letter moved | its own answer | 96.4-98.8 |
| Comma token blocked from the answer token | last comma | attention to the last answer token is blocked | the written letter again | 67.8 (colon token instead: 99.8 unchanged) |
| Experiment | Token | Patched or blocked | Target answers | Result |
|---|---|---|---|---|
| Gaze heads | colon | queries of the 32 heads, layers 19-34 | K | 74.0 (random 0.0) |
| Pointer through the gaze heads | shape, colon | binding-head patch at the shape token; gaze-head queries kept at the target’s original values | T | 98.4; with random heads kept instead, the pointer gets through in 86.5 |
| Only the gaze heads see the image | every token | attention to the image is blocked, except at the shape token and for the 32 gaze heads at the colon token | T | 95.1 (random 4.8) |
| Gaze heads, image values | colon | the gaze heads read the source’s image values | C | 93.7 |
| Gaze heads blocked | colon | the gaze heads’ attention to the image is blocked | T | 88.2; 2.6 when the tokens between the shape token and the colon token are blocked as well |
| Published Gaze Heads | colon | queries of the top 100; image values of the top 100 | K ; C | 64.2; 50.0 |
| Question | Instruction | Prefill | |
|---|---|---|---|
| English | What is the color of the {shape}? | Answer with a single word. | Answer: |
| French | Quelle est la couleur de la forme « {shape} » ? | Réponds par un seul mot. | Réponse : |
| Chinese | {shape}是什么颜色? | 请用一个词回答。 | 答案: |
| German | Welche Farbe hat die Form „{shape}“? | Antworte mit einem einzigen Wort. | Antwort: |
| Concept | English | French | Chinese | German |
|---|---|---|---|---|
| red | red | rouge | 红色 | rot |
| blue | blue | bleu (bleue) | 蓝色 | blau |
| green | green | vert (verte) | 绿色 | grün |
| yellow | yellow | jaune | 黄色 | gelb |
| purple | purple | violet (violette) | 紫色 | lila (violett) |
| black | black | noir (noire) | 黑色 | schwarz |
| Patch (token) | Moved | Letter | Gaze |
|---|---|---|---|
| Queries of 8 heads (shape token) | request | 81.0 2.5% | 73.3 2.8% |
| Image keys of 8 heads (shape token) | layout | 83.1 2.3% | 82.3 2.3% |
| Queries and image keys, 10 heads (shape token) | pointer | 95.0 1.5% | 93.0 1.6% |
| Outputs of 16 heads (shape token) | pointer | 85.9 2.4% | 81.2 2.5% |
| Values read by 16 heads (colon token), from the binding-head patch | pointer | 89.0 2.1% | 85.2 2.2% |
| Same, source’s own values | pointer | 87.5 2.0% | 87.8 1.9% |
| Patch | Selected heads |
|---|---|
| Queries | L19H27, L15H9, L19H24, L18H15, L15H11, L17H4, L15H10, L15H2 |
| Image keys | L19H27, L15H9, L19H24, L15H11, L18H15, L15H2, L21H11, L18H12 |
| Head outputs | L18H15, L21H11, L17H4, L19H27, L15H9, L19H24, L21H18, L15H11, L15H2, L17H7, L20H29, L21H8, L16H27, L20H15, L17H15, L18H11 |
| Values at the colon token (relay heads) | L22H10, L22H11, L35H25, L23H26, L25H15, L19H21, L34H28, L23H0, L20H2, L20H3, L23H3, L25H12, L28H22, L21H19, L22H3, L22H0 |
| Queries at the colon token (gaze heads) | L24H29, L34H7, L23H4, L24H31, L29H5, L24H23, L26H26, L29H6, L22H0, L22H11, L21H18, L24H5, L21H11, L33H31, L32H10, L29H11, L24H30, L32H8, L23H30, L28H0, L34H28, L32H11, L27H18, L29H12, L21H8, L24H13, L29H8, L30H6, L20H15, L29H10, L21H19, L19H3 |
| Queries, comic images | L15H2, L15H9, L15H11, L17H4, L18H15, L19H24, L19H27, L21H11 |
| Patch (expected) | Shapes (paper) | Color | Category | Two attributes | Words | Word asked |
|---|---|---|---|---|---|---|
| Binding queries (request) | 81.0 2.5 | 98.9 0.6 | 82.1 2.3 | 84.4 2.1 | 85.0 2.2 | 86.2 2.1 |
| Binding image keys (layout) | 83.1 2.3 | 99.8 0.2 | 97.5 1.0 | 81.5 2.4 | 80.9 2.4 | 86.1 2.1 |
| Queries and keys (pointer) | 95.0 1.5 | 100.0 0.0 | 99.9 0.1 | 93.9 1.5 | 87.8 2.1 | 97.0 1.0 |
| Binding outputs (pointer) | 85.9 2.4 | 96.9 1.1 | 96.9 1.1 | 93.8 1.5 | 75.4 2.8 | 85.8 2.2 |
| Relay values, from the binding-head patch (pointer) | 89.0 2.1 | 97.9 0.9 | 95.3 1.3 | 94.3 1.5 | 79.9 2.4 | 86.3 2.1 |
| target’s values restored (own) | 99.8 0.2 | 100.0 0.0 | 100.0 0.0 | 100.0 0.0 | 99.6 0.4 | 100.0 0.0 |
| Attention blocked | Letter | Gaze |
|---|---|---|
| Nothing blocked | 100.0 0.0% | 97.4 0.9% |
| The 10 query and key heads blocked from the image at the shape token | 85.3 2.2% | 81.6 2.5% |
| same number of random heads | 100.0 0.0% | 97.7 0.9% |
| The 16 binding heads and the 2 query and key heads outside them, blocked from the image at the shape token | 84.9 2.1% | 74.7 2.6% |
| Every head blocked from the image at the shape token | 40.4 3.1% | 35.9 3.0% |
| Every head blocked from the image, from the shape token to the token before the colon token | 32.6 2.8% | 30.0 2.7% |
| Patch | Heads of Sections 4 and 6 , unchanged | Random heads | Heads found here ( ) |
|---|---|---|---|
| First answer: values read at the colon token | 78.5% | 0.0% | 83.0 3.2% (16) |
| First answer: shape token outputs | 53.4% | 0.5% | 74.8 4.0% (32) |
| First answer: shape token image key reads | 9.9% | 0.0% | 10.8 2.8% (128) |
| First answer: colon token queries | 46.1% | 0.0% | 57.8 4.2% (128) |
| Last comma token: queries (next panel moves) | 88.2 2.8% | 0.0% | 86.0 2.9% (16) |
| Blocked | Own answer kept | Repeats the written letter |
|---|---|---|
| Nothing blocked | 100.0 0.0% | 0.0 0.0% |
| Shape token blocked from the image | 68.8 3.8% | 0.0 0.0% |
| Later tokens cannot attend to the shape token | 66.4 4.0% | 0.0 0.0% |
| control: later tokens cannot attend to the token before the shape token | 100.0 0.0% | 0.0 0.0% |
| Relay heads cannot attend to the text at the comma token | 65.6 4.0% | 0.0 0.0% |
| random heads cannot attend to the text at the comma token | 100.0 0.0% | 0.0 0.0% |
| Written so far | The comma token cannot attend to | Next unwritten panel | A panel already written |
|---|---|---|---|
| 45.4 4.7% | 52.8 4.6% | ||
| 80.5 3.8% | 6.2 2.3% | ||
| 42.5 4.6% | 57.0 4.5% | ||
| 76.2 3.9% | 23.5 3.7% | ||
| any letter but the last | 38.2 4.4% | 60.5 4.3% |
| Head set | Queries patched: source’s panel | Pointer moved, queries held: own kept | Source’s image values: source’s letter | Blocked from the image: own lost | Only these heads see: own kept |
|---|---|---|---|---|---|
| 32 gaze heads found by patching | 74.0 2.9% | 98.4 0.8% | 93.7 1.6% | 11.8 2.0% | 95.1 1.3% |
| same number of random heads | 0.0 0.0% | 5.9 1.6% | 0.0 0.0% | 0.2 0.2% | 4.8 1.2% |
| Published Gaze ranking, top 100 | 64.2 2.9% | 95.2 1.3% | 50.0 3.3% | 1.4 0.8% | 97.4 1.0% |
| Published Gaze ranking, top 200 | 64.9 3.1% | 96.2 1.1% | 88.6 2.1% | 1.1 0.8% | 98.6 0.8% |
| Heads from the blocking search | 0.0 0.0% | 8.3 2.0% | 0.0 0.0% | 2.1 0.9% | – |
| Head set | Queries patched: source’s panel | Source’s image values: source’s letter |
|---|---|---|
| 32 gaze heads (chosen by patching) | 74.0 2.9% | 93.7 1.6% |
| Gaze score on our task, top 32 | 63.3% | 64.8% |
| Gaze score on our task, top 100 | 65.3% | 93.4% |
| Published Gaze Heads, top 100 | 64.2 2.9% | 50.0 3.3% |
| 32 random heads | 0.0% | 0.0% |
| Head set | Heads | Layers | In published top 100 | In published top 200 | In re-measured top 100 | Expected by chance in a top 100 |
|---|---|---|---|---|---|---|
| Binding, query, and key heads | 18 | 15-21 | 3 | 6 | 6 | 1.6 |
| Relay heads | 16 | 19-35 | 4 | 5 | 3 | 1.4 |
| Gaze heads | 32 | 19-34 | 23 | 29 | 29 | 2.8 |
| Language heads | 16 | 19-35 | 0 | 0 | 0 | 1.4 |
| List task: outputs at the shape token | 32 | 14-21 | 5 | 11 | 13 | 2.8 |
| List task: image keys at the answer token | 32 | 15-34 | 8 | 20 | 19 | 2.8 |
| Experiment | Colored shapes | Objects in shapes |
|---|---|---|
| Answers in the question’s language, no patch | 100.0 0.0% | 99.0% |
| Language heads patched from the source, every generated token | 95.3 0.5% | 87.6% |
| first generated token only | 90.7% | 78.4% |
| copied source words | 0.0 0.0% | 1.3% |
| Random heads | 0.0 0.0% | 0.5% |
| Lowest-ranked heads | 0.0% | 0.3% |
| Patch | German | French | Chinese |
|---|---|---|---|
| 16 language heads zeroed | 99.6 | 88.4 | 0 |
| and 16 more heads found with those zeroed | 100 | 100 | 0 |
| and 64 more heads found with those zeroed | 100 | 100 | 90 |
| and 64 random heads | 92 | 85 | 0 |
| The 64 more heads zeroed alone | 0 | 12 | 0 |
| 16 heads zeroed, residual at layer 4 set to the English average | 100 | 100 | 31 |
| Patch | 2B | 4B | 8B | 32B |
|---|---|---|---|---|
| Binding queries: source’s request | 37.5% | 59.0% | 81.0 2.5% | 82.0% |
| Binding image keys: source’s layout | 71.0% | 82.0% | 83.1 2.3% | 94.0% |
| Queries and keys: source’s panel | 84.0% | 96.0% | 95.0 1.5% | 99.0% |
| Binding outputs: source’s panel | 86.0% | 93.5% | 85.9 2.4% | 87.0% |
| Relay values from the run with the binding-head patch: source’s panel | 84.5% | 95.5% | 89.0 2.1% | 84.0% |
| target’s values restored: own answer | 91.0% | 98.0% | 99.8 0.2% | 100.0% |
| Test | 2B (16 heads) | 4B (128 heads) | 8B (32 heads) | 32B (64 heads) |
|---|---|---|---|---|
| Queries patched: source’s panel | 63.5% | 75.5% | 74.0 2.9% | 90.0% |
| Pointer moved at the shape token, queries held: own kept | 82.0% | 92.0% | 98.4 0.8% | 100.0% |
| Source’s image values: source’s letter | 44.5% | 63.0% | 93.7 1.6% | 97.5% |
| Only these heads and the shape token see the image: own kept | 84.5% | 75.5% | 95.1 1.3% | 95.0% |
| These heads blocked from the image at the colon token: own kept | 25.5% | 95.5% | 88.2 2.0% | 100.0% |
| Random heads, queries patched | 0.0% | 0.0% | 0.0 0.0% | 0.0% |
| Reading of the copied comma state | 2B (28 layers) | 4B (36 layers) | 8B (36 layers) | 32B (64 layers) |
|---|---|---|---|---|
| Skip experiment: skips the shared first matching panel | 78 (17) | 74 (20) | 91.0 2.4 (20) | 99 (47) |
| Cursor experiment: “one written”, skip one | 87 to 99 (3 to 7) | 97 (7) | 92.4 to 98.4 (0 to 7) | 75 to 91 (34 to 40) |
| Cursor experiment: count of two | 100 (8 to 13) | 65.5 (20) | 90.8 2.5 (15) | 93 (42) |
| Cursor experiment: the visited panel itself | 41 (18) | 49 (22) | 1.6 1.1 (20 to 24) | 1 |
| Cursor experiment: first matching panel after the visited panel | 22 (20) | 53 (23) | 98.4 1.1 (20) | 100 (48) |
| Source’s letter, from layer | 22 | 30 | 30 | 53 |
| Heads that read the source’s image keys | Next answer follows the source |
|---|---|
| The 64 heads found on the 32B | 68.0% |
| 64 random heads from the same layers | 0.0% |
| The top 256 heads | 68.0% |
| All 4,096 heads | 68.0% |
| The 16 binding heads | 22.0% |
| The 64 heads without the binding heads | 0.0% |
| Experiment | 2B | 4B | 8B |
|---|---|---|---|
| Answers in the question’s language, no patch | 97.5% | 98.2% | 100.0 0.0% |
| Gaze heads’ summed output: cosine across languages / across images | 0.87 / 0.78 | 0.88 / 0.71 | 0.95 / 0.50 |
| 16 language heads replaced from the source: own concept, source’s language | 23.7% | 93.0% | 95.3 0.5% |
| copied source words | 55.7% | 2.6% | 0.0 0.0% |
| MLPs of the last three layers held | 0.4% | 10.5% | 0.9 0.2% |
| Heads zeroed: German and French runs to English | 76% | 54% | 94.0 1.5% |
| Qwen3-VL-8B | LLaVA-OV-7B | InternVL3-8B | Gemma-3-12B | Pixtral-12B | Idefics3-8B | |
|---|---|---|---|---|---|---|
| Language model | Qwen3 | Qwen2 | Qwen2.5 | Gemma 3 | Mistral | Llama 3 |
| No patch: single answer / list | 98.5 / 96.5 | 99.5 / 18.5 | 99.5 / 80.0 | 100.0 / 99.0 | 100.0 / 98.0 | 88.5 / 3.5 |
| (1) Pointer at the colon token | 80.0 (24/36) | 84.0 (22/28) | 65.5 (22/28) | 30.5 (34/48) | 68.0 (25/40) | 38.5 (17/32) |
| (2) Pointer at the shape token | 42.5 (20/36) | 69.5 (17/28) | 68.0 (17/28) | 32.0 (20/48) | 79.0 (16/40) | 31.0 (12/32) |
| (3) Shape token is read: answers lost | 77 | 70 | 80 | 73 | 75 | 69 |
| (4) Comma token skips a written panel | 96 (20/36) | not solvable | 79 (19/28) | 96 (23/48) | 87 (18/40) | not solvable |
| Patch | Row, paper | Prompt A | Prompt B | Grid | Column | Photographs |
|---|---|---|---|---|---|---|
| Binding queries: source’s request | 81.0 2.5% | 73.3% | 48.3% | 67.5% | 72.5% | 90.8% |
| Binding image keys: source’s layout | 83.1 2.3% | 83.3% | 61.7% | 75.0% | 75.0% | 98.3% |
| Queries and keys: source’s panel | 95.0 1.5% | 90.0% | 56.7% | 93.8% | 92.5% | 96.7% |
| Binding outputs: source’s panel | 85.9 2.4% | 80.0% | 53.3% | 93.8% | 85.0% | 96.7% |
| Relay values from the run with the binding-head patch: source’s panel | 89.0 2.1% | 79.2% | 58.3% | 93.8% | 86.2% | 98.3% |
| target’s values restored: own answer | 99.8 0.2% | 99.2% | 93.3% | 98.8% | 96.2% | 100.0% |
| Paper’s | One sentence after | Two sentences after | ||
|---|---|---|---|---|
| Patch | Tokens patched | prompt | the shape token | the shape token |
| Binding outputs | The shape token | 84.0% | 79.0% | 52.0% |
| The tokens after it | 4.0% | 8.0% | 26.0% | |
| Both | 87.0% | 88.0% | 87.0% | |
| Both, random heads | 1.0% | 0.0% | 0.0% | |
| Queries and keys | The shape token | 93.0% | 88.0% | 55.0% |
| Test | Row, paper | Prompt A | Prompt B | Grid | Column |
|---|---|---|---|---|---|
| Queries patched: source’s panel | 74.0 2.9% | 83.3% | 76.7% | 82.5% | 73.8% |
| Pointer moved, gaze queries held: own kept | 98.4 0.8% | 98.3% | 93.3% | 97.5% | 96.2% |
| Source’s image values: source’s letter | 93.7 1.6% | 94.2% | 95.0% | 90.0% | 90.0% |
| Only these heads see the image: own kept | 95.1 1.3% | 90.0% | 91.7% | 96.2% | 96.2% |
| Random heads: queries patched / only these see | 0.0 0.0% / 4.8 1.2% | 0.0% / 3.3% | 0.0% / 1.7% | 1.2% / 3.8% | 0.0% / 3.8% |
| First answer: binding outputs at the shape token: source’s first panel | 53.4% | 58.6% | – | 71.1% | 51.3% |
| Heads blocked from the instruction | Left-to-right answer |
|---|---|
| None | 0.0% |
| All 1,152 heads | 100.0% |
| The 16 heads found here | 98.4% |
| All heads except these 16 | 0.0% |
| 16 random heads from the same layers | 1.6% |
| The 16 heads, blocked from seven other question tokens | 0.0% |
| Test | Animals | Vehicles | Furniture | Objects | All |
|---|---|---|---|---|---|
| Comma token cannot attend to ’s tokens: names again | 89.2% | 70.6% | 82.2% | 75.2% | 79.6% |
| control: comma token cannot attend to the colon token: unchanged | 99.5% | 99.5% | 95.8% | 97.1% | 98.0% |
| ’s tokens blocked from ’s region: unchanged | 98.6% | 95.9% | 93.9% | 88.1% | 94.2% |
| ’s tokens blocked from the image: unchanged | 93.7% | 91.8% | 89.7% | 67.6% | 85.7% |
| Comma token blocked from the image: unchanged | 73.0% | 70.1% | 91.1% | 39.5% | 68.6% |
| ’s tokens and the comma token both blocked from the image: unchanged | 69.8% | 67.5% | 82.2% | 31.0% | 62.7% |
VLM Interpretability
Vision-language model interpretability explains how image and language representations shape predictions, evidence use, and multimodal reasoning.
See all 274 papers