Video world models aim to preserve scene structure and predict how dynamic objects evolve beyond visual observations. We present Kepler4D, a framework for future video generation through explicit 4D scene state evolution. Given a monocular video, Kepler4D constructs a shared 3D representation of background geometry, object motion histories, coarse spatial supports, and semantic context. Chain-of-Motion summarizes observed motion and uses a vision-language model to select structured speed and heading decisions and decide whether to bound object-center height from below. A deterministic rollout converts these decisions into future object trajectories for inspection and editing before synthesis. We render the evolving proxies into geometric controls for a pretrained video generator, separating coarse object motion from the synthesis of appearance and articulation. Experiments on real-world videos demonstrate that Kepler4D enables controllable object motion and plausible future rollout while preserving scene consistency.
Figures & tables
Figure 1: Kepler4D enables future video generation through explicit trajectory evolution and control in a 4D scene state. Our method organizes static geometry, foreground motion histories, and semantic context into the scene state, predicting scene-aware 3D trajectories to guide generation. It supports auto matic future evolution, language-guided trajectory edit ing, and view-consistent rendering across viewpoints, producing plausible future motion while preserving the reconstructed scene structure.
Figure 2: Overview of Kepler4D. Given an observed video, Kepler4D constructs a structured 4D scene state comprising static scene geometry, foreground motion histories, and semantic context. Chain-of-Motion reasons over the observed motion and scene context to infer structured motion decisions, which a deterministic 3D rollout expands into future object trajectories. These trajectories are rendered with static geometry into depth maps, object control signals, and masks to guide a pretrained video generator. This design separates explicit object motion prediction from video synthesis, enabling controllable generation of future videos while preserving the scene structure.
Figure 3: Static novel view synthesis on DL3DV. We compare Kepler4D with GEN3C, TrajectoryCrafter, and SPMem on indoor and outdoor scenes under target camera motion. Each row shows an input view, the target camera path, the held-out reference view, and the generated results. In these examples, Kepler4D preserves the scene layout and appearance details across viewpoint changes.
Reconstruction Fidelity
Generation Quality
Efficiency
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
Imag. ↑
MotSm. ↑
BgC. ↑
Overall ↑
Mem. (GB)
Time (min)
GEN3C Ren et al. (2025)
11.67
0.2963
0.5706
63.99
0.6991
0.9863
0.9314
0.8041
54.73
43
TrajectoryCrafter Yu et al. (2025)
11.90
0.3114
0.5791
74.23
0.6709
0.9712
0.9372
0.8005
37.81
10.4
SPMem Wu et al. (2025)
13.19
0.3611
0.5273
56.92
0.5819
0.9776
0.9134
0.8175
45.57
9.7
Ours
13.55
0.3675
0.3995
41.37
0.7606
0.9745
0.9448
0.8322
62.46
5.2
Table 1: Static view synthesis on DL3DV ( Ling et al., 2024 ) . We compare Kepler4D with three baselines using reference fidelity, FID, and VBench ( Huang et al., 2024 ) metrics. We also report peak GPU memory and per-clip inference time under the evaluated model and inference configurations.
Figure 4: Automatic future evolution. All methods use the input video without externally supplied object trajectories or motion commands. Methods with camera control are given the same target camera path. Kepler4D depicts the runner moving forward and vaulting over the railing, as in the reference. TrajectoryCrafter shows a viewpoint change without a vault. SPMem shows apparent intersections between the runner and railing. VerseCrafter shows a lateral deviation before the vault.
Reconstruction Fidelity
Generation Quality
Motion Fidelity
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
Imag. ↑
DynDeg ↑
Overall ↑
Flow-EPE ↓
RotErr ↓
TrajectoryCrafter Yu et al. (2025)
14.349
0.3593
0.5145
214.13
0.6595
0.1622
0.7064
17.409
7.5400
SPMem Wu et al. (2025)
14.641
0.3837
0.5102
265.13
0.5870
0.7838
0.7832
20.138
18.549
VerseCrafter Zheng et al. (2026)
14.090
0.3649
0.4556
303.80
0.6896
0.6486
0.7773
15.400
8.9610
Ours
14.921
0.3924
0.4561
202.64
0.6970
0.7838
0.8029
15.220
5.0200
Table 2: Automatic evolution on DAVIS ( Pont-Tuset et al., 2017 ) . All methods receive the same observed video and camera path. We report reconstruction fidelity, generation, and motion fidelity.
Figure 5: Language-guided controllable future generation. Given the same input video, the user specifies the intended motion through a high-level language command. Kepler4D infers an object-level trajectory from the command, shown in the second column. Baselines receive the same language command but not the inferred trajectory or control maps. With explicit trajectory control, Kepler4D better realizes the intended object motion while preserving scene layout and consistency.
Spatial mask used by the video-conditioning interface
Appendix
Table A2: Control streams supplied to the pretrained synthesis interface. The combined background/foreground visualization is diagnostic and distinct from the separate conditioning streams.
Dimension
Measured property
Aesthetic Quality
Visual aesthetic quality
Imaging Quality (Imag.)
Image quality, including sharpness and artifacts
Motion Smoothness (MotSm.)
Smoothness of temporal transitions
Subject Consistency
Consistency of foreground visual identity
Background Consistency (BgC.)
Consistency of background appearance
Dynamic Degree (DynDeg)
Presence of sufficiently strong motion
Appendix
Table A3: Interpretation of the VBench dimensions used for video evaluation in the main paper.
Variant
Text guidance
CoM trajectory guidance
Qwen prompt
Observed scene description
Absent
CoM w/o traj.
Predicted motion description
Not rendered into foreground controls
CoM w/ traj.
Predicted motion description
Rendered future object trajectories
Appendix
Table A4: Text and trajectory guidance supplied by the three tested motion-conditioning variants.
Figure 6: Schematic illustration of Chain-of-Motion inputs and outputs. CoM uses observed motion and semantic scene context to select speed and heading actions, decide whether to apply a lower bound to object-center height, and produce a motion description. The structured decisions guide deterministic 3D trajectory rollout, while the description provides text conditioning for video synthesis. The fields illustrate the interface and are not numerical logs from a benchmark sequence.
Information
Role
Endpoint images
Visual evidence for object identity and scene context
Object identifier and label
Associate masks, history, decisions, and controls
Observed motion summary
Numerical evidence for speed and heading decisions
Scene description
Context for a plausible continuation
Speed, heading, and height-bound decisions
Inputs to deterministic trajectory updates
Motion description
Semantic guidance for video synthesis
Appendix
Table A5: Roles of observed information and motion decisions in the structured CoM interface.
Understanding and predicting motion is a fundamental component of visual intelligence. Although modern video models exhibit strong comprehension of scene dynamics, exploring multiple possible futures through full video synthesis remains prohibitively inefficient. We model scene dynamics orders of magnitude more efficiently by directly operating on a long-term motion embedding that is learned from large-scale trajectories obtained from tracker models. This enables efficient generation of long, realistic motions that fulfill goals specified via text prompts or spatial pokes. To achieve this, we first learn a highly compressed motion embedding with a temporal compression factor of 64x. In this space, we train a conditional flow-matching model to generate motion latents conditioned on task descriptions. The resulting motion distributions outperform those of both state-of-the-art video models and specialized task-specific approaches.
Nick Stracke, Kolja Bauer, Stefan Andreas Baumann +3
We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/
We propose UniDynamics, a diffusion-based framework for future 4D dynamic scenes (RGB, depth, and optical flow) generation from a single event-RGB pair, without requiring long histories or control priors as in existing methods, while explicitly modeling future motion fields. The core idea is to leverage event streams to offer an alternative motion prior for single-RGB extrapolation, and to enforce geometric and motion constraints throughout generation via multimodal modeling. Specifically, we design an Event Latent Enhancement (ELE) module to align and enhance event latents into diffusion-injectable conditioning features, providing robust initial motion priors and reliable texture/structure cues. We further introduce a Perceptual Dynamics Space (PDS) embedded in the multi-scale U-Net, which decouples and adaptively interacts depth and flow while continuously feeding back constraints to appearance features, improving geometric-motion consistency for physically plausible and spatiotemporally coherent prediction. Experiments on VKitti2 and DSEC demonstrate state-of-the-art performance, producing high-quality, temporally coherent, and 4D-consistent future predictions, especially under challenging high-speed motion blur.