Humans imagine scene changes under hypothetical motions, an ability central to spatial reasoning and robotic planning. However, this capability remains underexplored in vision-language models and world models. We formalize this capability as Counterfactual Spatial Imagination (CSI) and introduce FlowBench, a resource for training and evaluating both, with 428K training examples spanning simulated and real-world scenes, household environments and tabletop manipulation, and diverse question types. A scalable, fully automatic pipeline pairs each hypothetical motion with spatial questions and, in simulation, exact outcome frames and optical flow. For VLMs, proprietary models exhibit an imagination gap, performing worse when motion outcomes must be inferred than when observed, and open-source models perform near chance; fine-tuning Qwen3-VL-8B on FlowBench-Sim-Train raises its accuracy by 29.5 percentage points in simulation and 16.9 on real images, a strong sim-to-real transfer. For world models, training on FlowBench-Sim improves text-to-flow prediction for all four model families we test, raising the accuracy of Psi-0.6 from 52.1% to 61.7%. FlowBench thus serves as a training and evaluation resource for both. Code and datasets are released at https://github.com/liuyifan22/FlowBench.
Figures & tables
Figure 1: Counterfactual Spatial Imagination (CSI) is a core capability for acting in the world. Given the initial image and a hypothetical motion, a human imagines the outcome, whereas open-source VLMs such as Qwen3.5 reason in text and fail (top). Even proprietary models show an imagination gap (bottom left): on FlowBench-Real-Test, Claude-Fable-5.1 and GPT-6 Astra lose 5.0 and 3.1 points when they must imagine the outcome instead of seeing it; Qwen3-VL-8B loses 13.0. Training on FlowBench-Sim transfers to real scenes (bottom right): fine-tuning Qwen3-VL-8B on FlowBench-Sim-Train alone adds 29.5 points on Sim and 16.9 on Real.
Figure 2: FlowBench is diverse and comprehensive: simulated homes and real robot tabletops, nine question categories, and 428K training questions. FlowBench-Sim covers kitchens, living rooms, bedrooms, and bathrooms in about 10K procedurally generated houses (420K training questions; 714 test questions from 120 held-out AI2-THOR scenes). FlowBench-Real covers tabletop manipulation from five robot datasets (8,575 training and 1,083 test questions). Questions probe visibility, relative position, proximity, depth order, feasibility, and contact: eight categories in Sim and seven in Real, six of them shared (shares of the training and test split, respectively).
Figure 3: A scalable, fully automatic, and accurate pipeline for counterfactual data that others can adapt to new simulators and tasks. (a) Four stages: find informative views, plan answer-changing motions, execute them in the simulator, and annotate. Answers are computed by rules on simulator metadata, so they are exact; a VLM only adds reasoning and language variation. (b) The pipeline needs no human labeling, scales to 420K questions across 9.7K houses, and records depth, flow, masks, and reasoning for every example. (c) A training example. The model sees only (I0,tm,tq) ; coloured nouns refer to the marks of the same colour. The outcome image, flow, depth, metadata, and the reasoning chain cgt→c1→c0 (excerpts) are training annotations that supervise both VLMs and world models.
Model
Set.
Img.
Act.
CoT
FlowBench-Sim
FlowBench-Real
Avg
Cat.
Avg
Cat.
Base
1A
I0
✓
–
46.9
47.9
42.7
41.2
2A
I1
–
–
50.4
52.3
55.7
53.0
1B
I0
✓
✓
43.4
43.1
49.8
48.4
FT
1A
I0
✓
–
76.3
77.9
59.6
56.3
2A
I1
–
–
73.9
76.7
73.6
70.3
Table 2: Ablation experiments. Qwen3-VL-8B-Instruct, base and fine-tuned on FlowBench-Sim-Train (FT), in the three settings. Avg is per question and Cat. the mean of the category accuracies; mean over three answer-order shuffles.
Acc
rEPE move
Blank
Dir
Δ PSNR
Method
(%) ↑
mean ↓
med ↓
(%) ↓
(°) ↓
↑
Zero flow
0.0
1.000
1.000
100.0
–
0.00
Modular [ 44 , 53 , 10 ]
50.2
0.581
0.398
14.8
1.4
–
+ depth calibration
50.8 (+0.6)
0.552
0.358
15.0
1.4
–
FOFPred [ 52 ]
0.2
0.985
1.000
0.0
39.5
–
+ LoRA [ 31 ]
2.4 (+2.3)
0.915
0.958
0.0
25.9
–
Table 3: Training world models on FlowBench-Sim. Text-to-flow and text-conditioned frame generation on the FlowBench-Sim test split, zero-shot and after training on FlowBench-Sim-Train (ProcTHOR). Acc = percentage of samples with per-pixel rEPE move≤0.40 ; rEPE move = per-pixel relative endpoint error on ground-truth moving pixels (mean and median over samples); Blank = percentage of samples with no predicted motion ( < 1 px); Dir = 2-D angle between predicted and ground-truth motion where both move; Δ PSNR = PSNR of the generated frame on the change region minus that of copying frame 0, in each model’s own generation domain. The modular pipeline receives camera intrinsics, and the image editors the ground-truth frame-0 mask.
Figure 4: World-model-assisted VQA. Ψ -0.6 [ 6 ] first imagines the motion as optical flow and a future frame, decoding high- P(motion) patches first with text-enhanced guidance ( +2/−1 ), then reasons over the imagined outcome to answer. One model performs every step ( Ψ badges).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: FlowBench-Real. (a) Four-stage curation pipeline; answers come only from the measured 3D geometry, and a VLM only names objects and checks marks. (b) FlowBench-Real test set: 1,083 questions, one per image pair, over seven categories and five robot datasets. (c) A test question. The model sees only (I0,tm,tq) ; the after frame, masks, depth, tracks and distances are what the pipeline measured to derive the answer.
FlowBench-Sim
FlowBench-Real
Model
Set.
Img.
Eff.
Avg
Cat.
Avg
Cat.
GPT-6 Astra
1A
I0
low
54.6
57.6
88.4
87.6
1B
I0
max
60.2
64.4
89.9
89.2
2B
I1
max
56.3
60.9
93.0
92.0
GPT-5.6 Luna
1A
I0
low
47.6
52.2
65.0
63.2
2A
I1
low
42.7
47.7
60.4
60.9
Appendix
Table 4: GPT models when imagining the outcome from the before image I0 (1A, 1B) and when seeing the after image I1 (2A, 2B). Eff.: reasoning effort; 1A and 2A use the lowest, 1B and 2B the highest; 2B is the after-image counterpart of 1B. GPT-6 Astra answers 50% and GPT-5.6 Luna 20% of the questions of each benchmark, the same samples as Tab. 1 ; Astra was not run in 2A and Luna not at max effort. Avg is per question and Cat. the mean of the category accuracies. Bold: best per column; drops in grey.
FlowBench-Real
Chg.
Unchg.
Gap
Gap mix
GPT-6 Astra
1A
low
87.3
91.2
+3.8
+8.0
1B
max
88.9
92.5
+3.7
+6.3
2B
max
92.9
93.2
+0.3
+0.1
GPT-5.6 Luna
1A
low
67.3
59.0
− 8.3
− 5.3
2A
low
62.2
55.7
− 6.4
− 6.2
FlowBench-Sim
Chg.
Unchg.
Gap
Stale (chance)
Appendix
Table 5: Accuracy (%) on questions whose answer the motion changes (Chg.) or does not change (Unchg.). Gap: Unchg. − Chg., so a positive gap means lower accuracy when the scene changes. Gap mix pools only the categories that contain both kinds of question (Depth Order, Left Right, and Near Object on Real; on Sim, every category does). Stale: share of wrong answers on changed multi-choice questions that equal the before-motion answer, with its chance level. Same question samples as Tab. 4 ; Feasibility is excluded because it has no before state.
Category
n
Direct
Imagine
Δ
Closer To
94
38.3
27.7
− 10.6
Closest Three
112
12.5
8.9
− 3.6
Contact Wall
19
63.2
63.2
0.0
Count Near
110
50.0
31.8
− 18.2
Feasibility
106
76.4
76.4
0.0
Left Right
116
90.5
90.5
0.0
Appendix
Table 6: PSI-0.6 on FlowBench-Sim when answering directly from (I0,tm,tq) and when it first imagines the flow and the outcome frame (imagine, then answer), with the same checkpoint. Accuracy in %; drops in grey.
Figure 6: Visualization of FlowBench-Sim-Train samples: (a) Occlusion and (b) Count Near.
Figure 7: Visualization of FlowBench-Sim-Train samples: (a) Closest Three and (b) Contact Wall.
Figure 8: Visualization of FlowBench-Sim-Train samples: (a) Closer To and (b) Feasibility.
Figure 9: Visualization of FlowBench-Sim-Train samples: (a) Near Object and (b) Left Right.
Figure 10: Visualization of FlowBench-Real-Train samples: (a) Left Right from Bridge V2 and (b) Closest Three from Language-Table.
Figure 11: Visualization of FlowBench-Real-Train samples: (a) Near Object from Berkeley UR5 and (b) Depth Order from DROID, a no-change question.
Figure 12: Visualization of FlowBench-Real-Train samples: (a) Closer To from Bridge V2 and (b) Occlusion from DROID.
Figure 13: Visualization of FlowBench-Real-Train samples: (a) Count Near from DROID and (b) Left Right from RT-1.
Vision-language models (VLMs) have shown strong performance on static visual understanding, yet they still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion. Recent efforts address this limitation either by scaling spatial supervision with synthetic data or by coupling VLMs with world models at inference time. However, the former often lacks explicit modeling of motion-conditioned state transitions, while the latter incurs substantial computational overhead. In this work, we propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model. Given an initial observation and a parameterized camera trajectory, we use a view-consistent world model to synthesize geometrically aligned future views and derive structured supervision for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. We post-train the VLM with a two-stage recipe on a compact dataset generated by this pipeline and evaluate it on multiple spatial reasoning benchmarks. World2VLM delivers consistent improvements over the base model across diverse benchmarks, including SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube. It also outperforms the test-time world-model-coupled methods while eliminating the need for expensive inference-time generation. Our results suggest that world models can serve not only as inference-time tools, but also as effective training-time teachers, enabling VLMs to internalize spatial imagination in a scalable and efficient manner.
Wanyue Zhang, Wenxiang Wu, Wang Xu +6
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Harbin Institute of Technology +2
While Vision-Language Models (VLMs) have shown strong visual reasoning capabilities, their spatial reasoning abilities remain largely constrained to the observed images and text-oriented chain-of-thought. They often struggle to infer unobserved layouts, maintain cross-view consistency, and reason from alternative viewpoints when only limited egocentric observations are available. In this work, we study this problem as thinking with imagination, where a VLM actively acquires imagined visual evidence by interacting with a world simulator during reasoning. We propose Astra, an agentic spatial reasoning framework that empowers VLMs with action-conditioned visual imagination. Specifically, Astra couples Astra-VL, an RL-trained VLM policy, with Astra-WM, a Bagel-based world simulator that generates novel-view observations from context images and natural-language camera motions. To provide reliable imagined evidence, Astra-WM is trained with view consistency tuning to improve pose and content consistency across views. In the RL stage, we propose a world-simulator-in-the-loop two-phase RL curriculum to stabilize tool-use exploration and advance the model's ability to invoke the simulator only when imagined observations improve over direct answering. Experiments demonstrate that both the world simulator and the agentic policy are necessary: Astra-WM improves simulator-augmented Gemini-3-Flash on MMSI-Bench from 45.1 to 49.5, while Astra-VL improves the Qwen3-VL backbone from 29.8 to 38.8 on MMSI-Bench and from 36.8 to 42.7 on MindCube. These results show that imagined observations can provide useful spatial evidence, but effective world-model-augmented reasoning requires learning when, where, and how to imagine.
Chenming Zhu, Jingli Lin, Yilin Long +4
1The University of Hong Kong · 2Shanghai AI Laboratory · 3Shanghai Jiao Tong University +2
Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input. Yet real-world spatial intelligence demands predictive spatial reasoning: constructing a scene from observations, anticipating how an intervention changes it, and reasoning about the unseen outcome. We introduce SpaceCast-Bench, the first benchmark to directly and diagnostically evaluate this capability. Built around an observe-transform-infer framework, its 3,862 questions from 182 real-world scenes span 16 task types at three levels: static perception, local prediction, and global prediction, progressively requiring scene understanding, spatial state updating, and relational inference over unobserved outcomes. Evaluating 21 models exposes a stark gap: the strongest model reaches only 58.0% against 87.2% human performance, while spatially specialized models remain near random chance. Controlled analyses further reveal that bridge views are critical for integrating distributed observations, and that explicit 3D evidence benefits models more reliably than generated outcome images or videos. Fine-tuning on our programmatically generated data lifts Qwen3-VL-4B from 34.0% to 65.7% with macro-average gains across six out-of-domain benchmarks.