Modeling physical dynamics, including how objects move, interact, and change state, is central to video world models for embodied AI. We present StrucPhysVideo, a family of video world models that bridges physics-focused data curation with language- and action-conditioned prediction of scene evolution. Our data pipeline combines motion-aware video segmentation, quality and content filtering, and physical relevance verification with structured annotations of objects, materials, and temporally localized interactions. By disentangling camera motion from object behavior and explicitly describing contact, deformation, and state transitions, the pipeline provides supervision grounded in observable physical events. Building on these data, we introduce StrucPhysVideo-TI2V, a sparse Mixture-of-Experts (MoE) text-image-to-video model trained with a curriculum that progressively emphasizes physical dynamics while retaining general-domain video data. StrucPhysVideo-TI2V achieves state-of-the-art performance on Physics-IQ Verified, scoring 45.5% and outperforming Cosmos3-Super-Image2Video by 2.8 percentage points. Caption ablations across backbones further demonstrate the effectiveness of physics-focused supervision. We further extend StrucPhysVideo-TI2V to StrucPhysVideo-IA2V, an interactive image-action-to-video world model that predicts visual outcomes from robot end-effector commands. Action conditioning, causal autoregressive generation, and few-step distillation enable incremental robot rollouts with only four denoising steps. Together, StrucPhysVideo advances physical dynamics modeling from image- and language-conditioned video prediction toward action-driven interaction.
Figures & tables
Figure 1: Physics-IQ Verified benchmark Results [ 1 ] . StrucPhysVideo achieves the highest Physics-IQ Verified score among all compared models in the text-image-to-video setting. * denotes our reproduction, and results for all other models are taken from the benchmark snapshot dated 16 September 2026 [ 2 ] .
Figure 2: Representative StrucPhysVideo generations under both the text-image-to-video (TI2V) and image-action-to-video (IA2V) settings, spanning general-domain, physical-world, and embodied scenarios.
Figure 3: StrucPhysVideo-TI2V overall architecture. The frozen Qwen3-VL-32B encoder jointly encodes the text prompt and reference image, while the frozen VAE encodes the target video. The first clean video latent is retained as a reference and the remaining latents are corrupted along the flow-matching path. Multimodal condition tokens and video tokens interact through joint self-attention in the StrucPhysVideo Transformer. Each block combines timestep-modulated attention with routed and shared experts, and the model predicts the latent velocity field used to supervise future-frame generation.
Figure 4: StrucPhysVideo-IA2V architecture. A causal action encoder maps raw action commands into latent action tokens and pooled contexts aligned with each video latent frame. In each DiT block, the pooled context modulates the video queries through AdaLN, while the corresponding action window provides the keys and values for cross-attention. A gated residual integrates action-conditioned features into the video stream.
Figure 5: Physical data pipeline. Source videos undergo Spatiotemporal Video Splitting, Technical Quality Filtering, and Content Purity Filtering. Physical Video Captioning first performs physical verification and generates a structured caption, then produces physical tags in a second model call. The resulting clips and annotations provide candidates for final dataset selection.
Figure 6: Examples of visual defects addressed by Technical Quality Filtering: solid-color frames, severe exposure defects, and persistent blur. Input-validity and temporal-quality checks are described in the text.
Figure 7: Face and Talking-Head Filtering rejects presenter-dominated clips while retaining clips with distant operators and human–object interactions.
Figure 8: VLM Text Review distinguishes text on physical objects from post-production overlays. Clips containing authentic scene text are retained for physical verification.
Figure 9: Physical verification in the first annotation call. Top row: rejected clips dominated by static product staging and decorative hand gestures. Bottom row: retained clips containing physical interactions. Retained clips receive complete structured captions and proceed to Closed-Set Tag Generation.
Rule of thirds, centered, symmetrical, diagonal, leading lines
Table 1: General video attributes, vocabulary sizes, and representative values.
Figure 10: Two-level taxonomy of physical phenomena. Five top-level categories contain 26 predefined leaf phenomena. An additional Other category covers physical interactions outside the vocabulary and has no predefined leaves.
Figure 11: Multi-label physical annotations for car washing, boiling water, and lathe cutting. Each example contains six frames and the associated phenomenon tags, grouped by top-level category. A filled star marks the primary phenomenon; outlined tags indicate additional phenomena.
Figure 12: Curation from source videos to annotated clips. Spatiotemporal Video Splitting produces candidate clips; Technical Quality Filtering and Content Purity Filtering select clips for Physical Video Captioning. Physical Verification selects physical clips, which receive structured captions and closed-set tags.
Figure 13: Action-conditioned data curation. Dataset-specific gates and adapters produce a unified EpisodeIR. Canonical versions define the training representations, with semantic checks during production and quality checks and independent readback before delivery of the corresponding training datasets.
Figure 14: Qualitative comparison among ground truth, Wan2.2-5B [ 4 ] , LingBot-Video-30B [ 6 ] , and StrucPhysVideo (Ours), with five frames shown from the start to the end of each sequence. In Case 1, the red boxes highlight substantial deformation of the track geometry, while the blue boxes indicate that the ball fails to roll along the shape of the track. In contrast, our model generates physically plausible ball-rolling motion that faithfully follows the track geometry. In Case 2, the blue boxes highlight the loss of the subject’s key facial features, whereas our model effectively preserves the subject’s identity and distinctive appearance.
Figure 15: Qualitative StrucPhysVideo-TI2V results for general-domain video generation. Each row shows five temporally ordered frames from one generated video, spanning people, vehicles, landscapes, and camera motion.
Figure 16: Qualitative StrucPhysVideo-TI2V results for physical-world video generation. Each row shows five temporally ordered frames from one generated video depicting a physical phenomenon or object interaction.
Figure 17: Qualitative StrucPhysVideo-TI2V results for embodied scenarios. Each row shows five temporally ordered frames from a generated robot manipulation sequence.
Figure 18: Caption ablation on Physics-IQ Verified. Wan2.2-5B [ 4 ] and LingBot-Video-30B [ 6 ] are evaluated under three settings: the reproduced baseline, incorporation of raw captions, and incorporation of physics-focused captions. Physics-focused captions yield the highest Verified Score for both backbones. Only Physics-IQ Verified scores are reported. ∗ denote scores reproduced by us.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Motion MAE ↓
Traj. Acc. ↑
Depth Acc. ↑
Interaction ↑
DreamDojo-AgiBot
20.86
0.8277
0.1825
0.03509
0.7905
0.9736
3.366
StrucPhysVideo-IA2V
22.12
0.8365
0.1602
0.03368
0.8330
0.9854
3.470
Table 2: Quantitative performance on 5K video clips. Metrics exclude the reference frame. PSNR is in dB. Traj. Acc. and Depth Acc. are normalized to [0,1] , with higher values indicating better agreement.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Motion MAE ↓
Traj. Acc. ↑
Depth Acc. ↑
Interaction ↑
A2World
17.71
0.7110
0.2105
0.04316
-
-
-
DreamDojo-AgiBot
22.14
0.8415
0.1687
0.03338
0.7853
0.9685
3.365
StrucPhysVideo-IA2V
22.89
0.8571
0.1550
0.03172
0.8287
0.9827
3.470
Table 3: Quantitative comparison on 256×256 head-view grid and frames 1–20 after the reference frame. A2World additionally receives initial wrist views and observed-pose-derived controls. Qualitative comparison is provided in Figure 19 .
Figure 19: AgiBot head-view rollouts at 0, 1, 2, 3, and 4 seconds. Each group shows GT, A2World, DreamDojo-AgiBot, and ours. All start from the same head image. All three models produce arm motion, but the generated trajectories and local robot geometry differ from ground truth. A2World also exhibits visible appearance distortion in these examples.
Action dropout
Text dropout
Teacher CFG setting
Motion MAE ↓
Traj. Acc. ↑
0%
0%
w/o CFG
0.03537
0.7913
15%
0%
w/o CFG
0.03455
0.7896
15%
0%
Action CFG sa=2
0.03449
0.8270
15%
0%
Action CFG sa=3
0.03517
0.8284
15%
0%
Action CFG sa=5
0.03583
0.8195
15%
5%
Action CFG sa=2
0.03432
0.8272
Table 4: Effects of teacher dropout and guidance on final distilled students. Dropout rates refer to bidirectional action-injection training. The reported action CFG scale is used both by the AR teacher to generate causal ODE distillation trajectories and by the bidirectional teacher to compute the real score during DMD. Bold and underlined values indicate the best and second-best values in each metric column, respectively.
Reliable physical reasoning from video requires understanding how objects move, interact, and respond to interventions. Existing vision-language models (VLMs) often struggle to interpret these dynamics and reason reliably about future and counterfactual outcomes. We introduce PhysMind, a training-free agentic framework that constructs one reusable, question-agnostic executable world per video. PhysMind recovers a temporally consistent dynamic scene through object segmentation, mesh reconstruction, and 6D pose tracking, then fits analytic continuous-time dynamics and latent physical parameters without unrolling a time-stepped simulator. Given a question, it inspects, continues, or edits the world and answers from the resulting trajectories and interactions. Relative to direct chain-of-thought (CoT) reasoning with the same VLM, PhysMind improves accuracy by 38.23 points on CLEVRER and 8.08 points on Physion++. On counterfactual questions, it exceeds the strongest evaluated VLM baseline, GPT-5.5, by 19.25 points.
Chen Yang, Shenxiang Zeng, Haoyang Zhao +6
1Tsinghua University · 2The University of Hong Kong
Physical world modeling requires predicting how interactions change a scene, not merely generating coherent motion. We propose STRIKE, a framework that separates visual state transition learning from dense video generation. We construct event-aligned supervision by extracting observed states from training videos and pairing them with transition descriptions and temporal offsets. An image-based transition model learns to predict the next scene configuration from the current image, a local transition specification, and elapsed time. At inference, a pretrained vision-language planner predicts time transition specifications, and recursive application of the learned transition model produces a sequence of future visual states. A separately trained dynamic model then generates the complete rollout conditioned on these states and their temporal locations. Experiments on Physics-IQ Verified, PhyGenBench, Pisa-Experiments, and RoboTwin2.0 show improvements of STRIKE over the corresponding video-backbone baselines in benchmark measures of physical consistency and manipulation-video fidelity. These results support learned visual state transitions as an effective intermediate representation for physical world modeling.
Wenbin Teng, Tianshuo Xu, Depu Meng +5
Applied Intuition · University of Southern California · University of California, Berkeley
Video-based world models offer a powerful paradigm for embodied simulation and planning, yet state-of-the-art models often generate physically implausible manipulations - such as object penetration and anti-gravity motion - due to training on generic visual data and likelihood-based objectives that ignore physical laws. We present ABot-PhysWorld, a 14B Diffusion Transformer model that generates visually realistic, physically plausible, and action-controllable videos. Built on a curated dataset of three million manipulation clips with physics-aware annotation, it uses a novel DPO-based post-training framework with decoupled discriminators to suppress unphysical behaviors while preserving visual quality. A parallel context block enables precise spatial action injection for cross-embodiment control. To better evaluate generalization, we introduce EZSbench, the first training-independent embodied zero-shot benchmark combining real and synthetic unseen robot-task-scene combinations. It employs a decoupled protocol to separately assess physical realism and action alignment. ABot-PhysWorld achieves new state-of-the-art performance on PBench and EZSbench, surpassing Veo 3.1 and Sora v2 Pro in physical plausibility and trajectory consistency. We will release EZSbench to promote standardized evaluation in embodied video generation.