Dynamic spatial reasoning from monocular video enables vision-language models (VLMs) to understand how 3D environments evolve over time, supporting physical world understanding and embodied intelligence. Existing efforts primarily address observed-dynamics understanding, whereas agents in dynamic environments also require future-dynamics prediction beyond the observation boundary. Moreover, external geometric processing and textual descriptions can increase inference complexity or lose information when representing continuous dynamics. These limitations motivate a shared internal state that preserves observed dynamics and remains predictive beyond the input video. Accordingly, we present Pros4D, a framework for modeling evolving 4D worlds in latent space that unifies retrospective understanding of observed-dynamics with prospective reasoning for future-dynamics prediction. Specifically, Pros4D-Engine derives geometry-grounded supervision and constructs paired examples for both tasks, while Pros4D-Bench evaluates them across eight spatial task categories spanning dynamic cameras, objects, and scenes. At the model level, Pros4D compresses observed dynamics into latent observed states and autoregressively rolls them forward into latent future states. 4D Latent Alignment (4DLA) grounds these states through complementary visual-semantic and geometric supervision, while joint language supervision connects them to task reasoning. Experiments across benchmarks show consistent improvements over strong baselines in both capabilities.
Figures & tables
Figure 2: Overview of Pros4D-Engine . Given raw videos, Stage 1 grounds and verifies entities to obtain reliable object-mask alignments. Stage 2 recovers 3D geometry and constructs question-answer pairs for camera motion, object trajectories, and relation evolution. Stage 3 synthesizes 4D latent-state reasoning traces, pairing textual reasoning with observed- and future-state supervision.
Figure 3: 4D Latent Alignment (4DLA) . Visual-semantic and geometric supervision ground the interleaved observed and future states used by 4D-LSR. Alignment is used only during training. Both encoders stay frozen while the decoder and alignment head are trained.
Figure 4: Pros4D-Bench task composition across dynamic camera, object, and scene reasoning categories.
Retrospective
Prospective
Cam. Dir.
Cam.-Obj. Dist.
Obj. Dir.
Inter-Obj. Dist.
Obj. Speed
Obj. Dist.
Scene Rel.
Scene Traj.
Methods
Avg.
Temporal
Dynamic Camera
Dynamic Objects
Dynamic Scene
Spatial Understanding Models
SpaceR-7B [ 40 ]
46.9
49.5
42.6
86.3
39.1
41.4
24.5
53.6
29.9
33.9
19.4
VST-7B-RL [ 63 ]
41.1
44.2
35.8
74.9
37.9
32.6
25.2
45.8
23.3
31.9
23.0
Spatial-SSRL-7B [ 35 ]
39.3
43.2
32.6
73.4
32.8
35.0
25.8
30.0
31.4
34.7
18.3
Table 1: Pros4D-Bench results (accuracy, %) . Retrospective and Prospective evaluate observed-dynamics understanding and future-dynamics prediction from the same observed video prefix. Eight task columns cover dynamic camera, object, and scene reasoning across the two temporal subsets, with averages over all spatial tasks. Bold and underlined values mark the best and second best scores.
Methods
Avg.
Spatial Reasoning
Goal Reasoning
Interaction Reasoning
RS
OS
SU
MV
TS-G
TS-S
TS-GL
AU
IP
TU
Qwen2.5-VL-3B
30.48
37.09
80.11
23.23
13.47
37.46
40.63
2.10
24.05
26.81
46.94
4D-RGPT-8B
29.70
45.20
42.70
18.60
17.70
30.40
38.10
19.90
18.90
32.80
34.00
MLLM-4D-8B
46.30
70.30
98.50
22.60
20.00
39.70
70.50
20.90
37.90
41.00
46.20
Pros4D-3B
51.39 [-0.1em] (+20.91)
47.09 [-0.1em] (+10.00)
100.00 [-0.1em] (+19.89)
30.09 [-0.1em] (+6.86)
29.51 [-0.1em] (+16.04)
39.80 [-0.1em] (+2.34)
37.01 [-0.1em] (−3.62)
47.90 [-0.1em] (+45.80)
30.76 [-0.1em] (+6.71)
79.41 [-0.1em] (+52.60)
43.54 [-0.1em] (−3.40)
Table 5: OpenX-VQA results (accuracy, %) . The ten tasks cover spatial, goal, and interaction reasoning in robot manipulation videos. Parentheses in the Pros4D-3B row give percentage-point changes from Qwen2.5-VL-3B. Task abbreviations are defined in Appendix C.4 .
Time
LSR
Target Distance
Target Direction
Ego Speed
Target Speed
t0
14.67 (+3.18)
43.03 (+9.78)
35.70 (-0.49)
42.30 (+12.72)
35.70 (+5.14)
t1
44.74 (+2.93)
44.74 (+2.93)
44.74 (+2.93)
44.74 (+10.75)
38.63 (+4.64)
t2
42.30 (+2.94)
42.30 (+2.94)
42.30 (+2.94)
44.50 (+11.25)
35.94 (+6.11)
t3
39.85 (+5.62)
39.85 (+5.62)
39.85 (+5.62)
52.57 (+17.36)
34.23 (+4.65)
Table 6: STRIDE-QA-Bench results (success rate, %) . Cells report Pros4D-3B success rates, with percentage-point changes from Qwen2.5-VL-3B in parentheses. At each future horizon ( t1 – t3 ), the same answers meet the distance and direction criteria, so both rates equal the joint localization success rate (LSR, Appendix C.4 ).
Figure 5: Latent state analysis on Pros4D-8B. (a) Cosine similarity to future visual and geometric targets; GT-Neg. Margin is matched-minus-negative future similarity. (b) Pros4D-Bench accuracy (%) under alignment-loss ablations; CE denotes cross entropy. (c) Future-dynamics prediction accuracy drops relative to the unmodified model, measured in percentage points (pp).
Dominant axis of accumulated translation; six directions
Camera-Object Distance
∥ct−pt∥2
Robust distance trend; five change categories
Object Motion Direction
Object trajectory in the camera view
Robust view-plane displacement; eight directions or nearly stationary
Inter-Object Distance
∥pt−qt∥2
Robust distance trend; five change categories
Object Speed Change
Displacement per elapsed second
Smoothed speed changes; constant, monotonic, or two-phase trends
Object Travel Distance
Smoothed object trajectory
Robust endpoint separation, rounded to 0.1 m
Table 13: Task definitions for observed-dynamics understanding and future-dynamics prediction.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Table 15: Comparison of dynamic spatial datasets and benchmarks. Cam.-, Obj.-, and Scn.-Level denote camera, object, and scene coverage; Future Pred. indicates future-dynamics prediction.
Methods
Dyn-Bench
VLM4D
Avg.
Inter-Obj.
Obj.-Scene
Cam.-Obj.
Avg.
Real
Syn.
Spatial Understanding Models
SpaceR-7B [ 40 ]
56.5
54.1
71.9
47.8
47.4
49.2
41.8
Spatial-SSRL-7B [ 35 ]
48.4
45.5
68.9
36.7
52.5
52.2
53.3
SpatialReasoner [ 38 ]
54.5
50.1
68.0
48.4
42.4
42.8
41.3
SpatialLadder-3B [ 28 ]
52.3
48.0
69.7
43.5
42.4
44.0
37.5
Appendix
Table 19: Complete Dyn-Bench and VLM4D comparison (accuracy, %). Relation and source subsets follow the main tables.
Column
Task
Queried information
RS
Robot State
Arm or gripper configuration
OS
Object State
Reachability or manipulability
SU
Scene Understanding
Spatial layout and depth relations
MV
Multiple View
Correspondence across viewpoints
TS-G
Task State, grasp
Grasp state and stability
TS-S
Task State, success
Whether manipulation succeeded
Appendix
Table 20: OpenX-VQA task abbreviations. Scores are multiple-choice accuracy (%).
Figure 6: Camera-Object Distance templates for observed-dynamics understanding and 4D-LSR.
Figure 7: Camera-Object Distance templates for future-dynamics prediction and 4D-LSR.
Figure 8: Inter-Object Distance templates for observed-dynamics understanding and 4D-LSR.
Figure 9: Inter-Object Distance templates for future-dynamics prediction and 4D-LSR.
Figure 10: Scene Spatial Relationship templates for observed-dynamics understanding and 4D-LSR.
Figure 11: Scene Spatial Relationship templates for future-dynamics prediction and 4D-LSR.
Figure 12: Pros4D-Bench examples across eight spatial task categories in a shared scene, covering dynamic camera, object, and scene reasoning.
Figure 13: Camera-Object Distance on Pros4D-Bench. The comparison concerns the decreasing distance between the camera and the seated person.
Figure 14: Object Motion Direction on Pros4D-Bench. The queried person moves leftward in the camera view.
Figure 15: Object-scene relations on Dyn-Bench. The bus becomes increasingly occluded by trees as it moves through the scene.
Figure 16: Observer-relative motion on VLM4D. The queried player moves away from the camera.
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.