Interactive visual world models must distinguish observation from physical intervention. Camera motion reveals new surfaces, whereas intervention changes object motion, contact, and deformation. Current video world models are largely driven by appearance priors and often lose physical or spatial consistency over long horizons. We present TourPhysics, an online framework initialized from a single image and a declarative physical configuration. TourPhysics extends PhysOmni, our ACM Multimedia 2026 work, from finite physics-grounded video synthesis to persistent exploration and manipulation. TourPhysics combines deterministic simulation with video generation while assigning separate roles to simulator state, geometric evidence, generator controls, and appearance memory. For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated. Accepted observations publish the terminal state and update the appearance memory and subsequent generator controls, while the committed state and simulator geometry remain fixed throughout synthesis and retry. We further separate the simulator geometry used for projection and visibility from the relative depth used to condition the generator. A reference-anchored memory retrieves accepted static appearance through geometric cross-view correspondence and incorporates it through a bounded residual that reverts to the native path when no valid correspondence exists. On simulator-defined camera tours and object manipulations, TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines, preserves the input scene, and reduces appearance drift during long-horizon revisits.
Figures & tables
Figure 1: Given a single image I0 and a declarative configuration Θdecl , TourPhysics maintains a persistent scene model for camera exploration and object manipulation. For each action, a physical transition computes a fixed physical and camera trajectory; the system then synthesizes the corresponding observation, applies a quality gate, and atomically commits the simulator state together with the accepted appearance evidence. The simulator state and geometric record remain fixed during synthesis, and the committed state supports the long-horizon tour and manipulation shown on the right.
Figure 2: Overview of TourPhysics. (1) Scene Lifting (One-time Preprocessing) combines the input image with a declarative user configuration, including material models, force and velocity keyframes, and camera commands, to construct a persistent scene hypothesis comprising instance masks, a background plate, per-object 3D meshes, calibrated foreground and background geometry, a reference camera, and cached simulator assets. (2) Coupled Physical and Epistemic Transitions advances one action window. The orange Physical transition block fixes the multi-material physical and camera segment before synthesis; the read-only geometric record Gn supplies simulator-consistent depth, cameras, masks, and motion; and the observation model combines these controls with generator-facing depth DWngen , the real-image anchor I0 , and committed appearance memory Kn to produce YWn . A quality gate publishes the accepted state and appearance atomically; only after acceptance do reference-anchored memory and novel-view depth adaptation affect subsequent appearance controls. (3) Long-Horizon Tour-and-Manipulate Video (Output) chains the accepted windows into a persistent sequence that supports free-viewpoint exploration, physically faithful manipulation, and photorealistic long-horizon consistency, while the generated RGB never replaces simulator renderings or writes back to the physical state or simulator geometry.
Figure 3: Material families and solver routing used by the physical backend. The declarative configuration selects the material model and its parameters; the image is a schematic taxonomy, not a generated-video result.
Figure 4: Point-cloud visualization of the camera update. Columns show the initial, intermediate, and full-trajectory views; red arrows denote camera motion and blue points denote newly observed support.
Figure 5: Generator-facing depth update, following the reference-style routing diagram. The fixed geometric record constructs the base control; accepted tail evidence is aligned and filtered into D(En) , while Wn gates the newly exposed, under-supported region. The final control is formed by point-wise fusion; outside the mask, the base control is preserved exactly. The depth maps are illustrative visualizations of the update, not additional simulator geometry.
Figure 6: Reference-anchored appearance memory. Historical features become accessible only after their corresponding source window is committed. Simulator-consistent geometry determines whether a target location corresponds to reliable static support in an earlier view. In the diagram, Zsattn denotes the stored attention-value tensor and is distinct from the geometric visibility Vs . The selected historical read is blended with the native reference-conditioned output through a bounded confidence gate. Invalid correspondence recovers the native path exactly.
Method
Image
Text
Camera
Depth
Track
Mask
Physics script
Native / diagnostic duration
TourPhysics
✓
Scene
Action/pose
Int.
Int.
Int.
Declarative
25.1/30.1 s
LingBot-Cam
✓
Frozen
Pose
–
–
–
–
25.1/30.1 s
LingBot-Act
✓
Frozen
Discrete action
–
–
–
–
25.1/30.1 s
Wan2.1-I2V
✓
Frozen
Text
–
–
–
–
10.7 s / 25–30 s d
Sora-2
✓
Frozen
Text
–
–
–
–
12.2 s
MotionCtrl
✓
–
RT pose
–
–
–
–
1.4 s / 25–30 s d
Table 1: Evaluation inputs and temporal support. “Int.” denotes controls derived internally from the declared simulator state, “Text” under Camera denotes prompt-only specification, and d marks a uniformly time-stretched full-timeline diagnostic.
Table 2: Prescribed camera- and object-motion adherence. Camera metrics compare generated and declared trajectories; object metrics measure image-space agreement with paired simulator-rendered sequences. Generation inputs are listed in Table 1 .
Method
Collision Proxies
Deformation Proxies
VideoPhy-2 AutoEval (Auxiliary)
Contact Err. ↓
Response Acc. ↑
Overlap Exc. ↓
Shape Err. ↓
Area Drift ↓
Recovery Acc. ↑
SA ↑
PC ↑
Avg ↑
LingBot-Cam
6.9196 s
<0.01%
<10−4
0.0664
0.5136
50.00%
2.9444
4.4167
3.6806
LingBot-Act
7.8304 s
<0.01%
<10−4
0.0809
0.1757
<0.01%
2.9352
4.2315
3.5833
Wan2.1-I2V
5.1875 s
33.33%
<10−4
0.0866
0.2763
30.00%
2.9444
4.0185
3.4815
Sora-2
4.9107 s
33.33%
<10−4
0.1351
0.3369
33.33%
2.9630
4.2778
3.6204
MotionCtrl
7.0893 s
<0.01%
<10−4
0.1234
0.3539
N/A
2.9352
4.7963
3.8657
Table 3: Adherence to simulator-defined physical trajectories. Collision and deformation columns are tracked image-space proxies computed against paired simulator-rendered videos. VideoPhy-2 supplies an auxiliary perceptual diagnostic.
Figure 8: TourPhysics results across several scenarios. Key frames progress from left to right and show long-horizon appearance, camera motion, and physical interactions.
Variant
Scope
End-to-start drift ↓
Tail-to-head drift ↓
History error change ↓
Known-region change ↓
Ordinary temporal cache
72 sequences
26.33
20.38
–
–
Persistent sink cache
72 sequences
11.17
7.74
–
–
Appearance memory, development
4 identities
–
–
−8.25%
−2.99%
Appearance memory, held-out
14 identities
–
–
−2.39%
−0.90%
Table 4: Long-horizon observation and appearance-memory diagnostics. Cache drift is measured on paired long-video sequences. Memory changes are reported relative to the same generator with appearance memory disabled; negative values denote lower error.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Camera
Object Motion
ATE/PLR
T-Dir
R-Dir
Object Motion
LingBot-Cam
49/40
70/48/40
52/25/25
74/49/27
LingBot-Act
49/40
70/48/40
52/25/25
97/51/29
Wan2.1-I2V
49/40
70/48/40
52/25/25
165/92/49
Sora-2
49/40
57/45/37
1/1/1
186/97/53
MotionCtrl
49/40
70/48/40
52/25/25
179/95/51
Appendix
Table 5: Effective support for the camera- and object-motion protocols. Each cell lists counts in the order specified in the table note; “–” denotes an incompatible protocol.
Method
Collision Proxies
Deformation Proxies
VideoPhy-2 AutoEval (Auxiliary)
Contact Err.
Response Acc.
Overlap Exc.
Shape Err.
Area Drift
Recovery Acc.
SA
PC
Avg
LingBot-Cam
14/14/6
1/1/1
2/2/2
19/17/8
19/17/8
6/2/1
108/61
108/61
108/61
LingBot-Act
14/14/6
2/2/2
1/1/1
27/24/12
27/24/12
3/3/2
108/61
108/61
108/61
Wan2.1-I2V
14/14/6
3/3/2
6/6/5
31/26/14
31/26/14
11/5/2
108/61
108/61
108/61
Sora-2
14/14/6
7/7/3
8/8/4
31/26/14
31/26/14
4/3/2
108/61
108/61
108/61
MotionCtrl
14/14/6
8/8/3
4/4/3
31/26/14
31/26/14
–
108/61
108/61
108/61
Appendix
Table 6: Effective support for simulator-trajectory adherence and auxiliary VideoPhy-2 diagnostics. “–” denotes an incompatible protocol.
World simulators can provide safe and scalable environments for training Physical AI systems before real-world deployment. Large video generation models are emerging as a promising basis for such simulators because they can generate diverse and realistic visual futures. However, using them as world simulators requires physically faithful video continuations, namely, generated videos that preserve the physical state implied by the conditioning input, and evolve in ways consistent with basic physical principles. We propose PhyWorld, a video generation world model designed to produce temporally coherent and physically faithful scene continuations through two-stage post-training. In the first stage, we improve video-to-video continuation with flow matching fine-tuning, encouraging stable visual attributes and coherent motion dynamics across frames. In the second stage, we align generated dynamics with physical principles using Direct Preference Optimization (DPO) over physics preference pairs, guiding the model toward outputs with higher physical plausibility. To evaluate PhyWorld, we use both standard video-quality benchmarks and a dedicated physical-faithfulness benchmark with per-law scoring. Experiments show that PhyWorld improves video consistency, achieving an average score of 0.769 on VBench compared with 0.756 or below for state-of-the-art baselines. PhyWorld also improves physical plausibility, reaching an average score of 3.09 on our physical-faithfulness benchmark compared with 2.99 for the strongest baseline. These results suggest that post-training large video generation models with continuation and physics-preference signals can make them more effective world simulators for Physical AI.
Pu Zhao, Juyi Lin, Timothy Rupprecht +10
Northeastern University · University of Georgia · Tulane University +1
Recent generative video models achieve impressive visual quality but remain constrained by limited physical consistency and controllability. Existing video generation methods provide minimal physical control, and single-image-to-3D conversion approaches often suffer from object interpenetration. Furthermore, physics-based scene-level 3D generation methods exhibit spatial misalignment, stylized artifacts, and inconsistencies with the input data, restricting their use in realistic interactive video synthesis. We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video through holistic scene-level 3D reconstruction. By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity. Unlike prior methods, this formulation enables accurate scenelevel multi-object interactions and introduces richer, complex control types for advanced mechanicsbased manipulation. By decoupling simulation from rendering, TelePhysics bypasses latency-heavy priors, achieving real-time physical interaction previews paired while preserving photorealistic visual fidelity. Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability. The open-source code is available at https://github.com/xinzhang007/TelePhysics.
Xin Zhang, Yabo Chen, Yijie Fang +5
Fudan University · Institute of Artificial Intelligence, China Telecom (TeleAI)
We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/