PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment

Zhexiao Xiong; Yizhi Song; Liu He; Wei Xiong; Yu Yuan; Feng Qiao; Nathan Jacobs

cs.CVMar 14, 2026

PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment

Authors: Zhexiao Xiong, Yizhi Song, Liu He, Wei Xiong, Yu Yuan, Feng Qiao, Nathan Jacobs

Organizations: Washington University in St. Louis · Purdue University · NVIDIA

Abstract

Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing models often generate temporally incoherent content that violates basic physical intuition, significantly limiting their practical applicability. We propose PhysAlign, an efficient framework for physics-coherent image-to-video (I2V) generation that explicitly addresses this limitation. To overcome the critical scarcity of physics-annotated videos, we first construct a fully controllable synthetic data generation pipeline based on rigid-body simulation, yielding a highly-curated dataset with accurate, fine-grained physics and 3D annotations. Leveraging this data, PhysAlign constructs a unified physical latent space by coupling explicit 3D geometry constraints with a Gram-based spatio-temporal relational alignment that extracts kinematic priors from video foundation models. Extensive experiments demonstrate that PhysAlign significantly outperforms existing VDMs on tasks requiring complex physical reasoning and temporal stability, without compromising zero-shot visual quality. PhysAlign shows the potential to bridge the gap between raw visual synthesis and rigid-body kinematics, establishing a practical paradigm for genuinely physics-grounded video generation. The project page is available at https://physalign.github.io/PhysAlign.

Figures & tables

Figure 2: PhysAlign framework. Our data generation pipeline leverages physical simulator (i.e. Blender) to generate synthetic videos with 3D physical ground truth. Our method aligns the DiT [ 32 ] latent features with both (i) physical knowledge feature by V-JEPA2 [ 1 ] , and (ii) 3D geometric feature encoded from synthetic ground truth (e.g., depth). This unified alignment internalizes both physical laws and visual fidelity for I2V generation task.

Figure 3: Comparison of our result with other baseline models on WISA-test set. Results show that our method shows better understanding of the physics law, which demostrates our method’s strong generalization ability to act as real-world simulator. Zoom-in for details.

PIS	Ref	CogVideoX-5B	Hunyuan I2V	Wan2.2	Ours
$a_{x}$	0.701	0.350	0.571	0.520	0.632
$a_{y}$	0.715	0.385	0.604	0.517	0.648
$v_{x}$	0.790	0.494	0.704	0.679	0.746
$v_{y}$	0.827	0.467	0.746	0.661	0.798
$\Delta l$	0.777	0.227	0.592	0.546	0.641

Table 1: Comparison of Physical Invariance Score (PIS, acceleration metrics) on Synthetic test set. Higher is better.

PIS	CogVideoX-5B	HunyuanI2V	Wan2.2	Ours
$a_{x}$	0.444	0.568	0.444	0.604
$a_{y}$	0.456	0.558	0.481	0.611
$v_{x}$	0.678	0.708	0.669	0.775
$v_{y}$	0.663	0.688	0.634	0.739
$\Delta l$	0.372	0.416	0.317	0.451

Table 2: Comparison of Physical Invariance Score (PIS, acceleration metrics) on WISA Test Set. Higher is better.

Method	I2V-Metrics		Quality Metrics
Method	i2v_subject	i2v_background	subj_consis.	bkg_consis.	motion_smooth.	dynamic_degree	aesthetic	imaging
CogVideoX-5B	0.816	0.870	0.772	0.876	0.985	0.540	0.406	0.636
Hunyuan I2V	0.842	0.891	0.814	0.912	0.993	0.340	0.453	0.635
Wan2.2 I2V	0.879	0.910	0.826	0.922	0.978	0.676	0.411	0.516
Ours	0.911	0.928	0.886	0.955	0.996	0.730	0.467	0.655
Reference Video	0.931	0.985	0.913	0.971	0.996	0.820	0.460	0.673

Table 3: VBench-I2V evaluation on the paired-subject set. Higher is better. I2V-Metrics: i2v_subject (subject consistency between input image and video), i2v_background (background consistency). Quality Metrics: subj_consis. (temporal subject consistency), bkg_consis. (temporal background consistency), motion_smooth. (motion smoothness), dynamic_degree (motion amplitude), aesthetic (aesthetic quality), imaging (imaging quality).

Method	I2V-Metrics		Quality Metrics
Method	i2v_subject	i2v_background	subj_consis.	bkg_consis.	motion_smooth.	dynamic_degree	aesthetic	imaging
CogVideoX-5B	0.953	0.953	0.918	0.941	0.988	0.337	0.454	0.618
Hunyuan I2V	0.951	0.956	0.915	0.942	0.994	0.404	0.452	0.606
Wan2.2 I2V	0.963	0.969	0.925	0.936	0.988	0.607	0.451	0.624
Ours	0.975	0.981	0.934	0.951	0.995	0.635	0.458	0.643

Table 4: VBench-I2V evaluation on the WISA-test set. Higher is better.

Figure 4: Visualization of the synthetic data generated though our data generation pipeline.

PhysicIQ (%)	CogVideoX-5B	Wan2.2	Wan2.2I2V+LoRA	Ours
Overall Score	32.3	29.6	34.5	38.1

Table 5: Comparison of PhysicIQ score . Higher is better.

PIS	Ref Video	Lora I2V	w/o FA	w/o 3D-A	Ours-Full
$a_{x}$	0.701	0.531	0.597	0.611	0.632
$a_{y}$	0.715	0.548	0.602	0.615	0.648
$v_{x}$	0.790	0.705	0.716	0.709	0.746
$v_{y}$	0.827	0.689	0.768	0.762	0.798
$\Delta l$	0.777	0.575	0.609	0.601	0.641

Table 6: Ablation Study of Physical Invariance Score (PIS) on Blender Test Set. Higher is better. FA: Feature Alignment, 3D-A: 3D Alignment.

PIS	LoRA	w/o FA.	w/o 3D-A.	Ours-Full
$a_{x}$	0.518	0.559	0.522	0.604
$a_{y}$	0.505	0.578	0.549	0.611
$v_{x}$	0.748	0.748	0.739	0.775
$v_{y}$	0.669	0.679	0.662	0.739
$\Delta l$	0.369	0.402	0.396	0.451

Table 7: Ablation study of Physical Invariance Score (PIS) on the WISA-Test set. Higher is better. FA: Feature Alignment, 3D-A: 3D Alignment.

	CogVideoX	Hunyuan	Wan2.2	Ours
Motion Physics Quality	1.671	1.543	2.557	3.236
Identity Physics Quality	1.757	1.679	2.536	3.343
Overall Video Quality	1.771	1.607	2.643	3.286

Table 8: User study results. We evaluate the user’s preference from three aspects: Motion Physics Quality, Identity Physics Quality and Overall Video Quality. Higher is better.

Method	I2V-Metrics		Quality Metrics
	i2v_subject	i2v_background	subj_consis.	bkg_consis.	motion_smooth.	dynamic_degree	aesthetic	imaging
Train on WISA	0.965	0.971	0.927	0.944	0.991	0.612	0.452	0.627
DINOv3	0.972	0.978	0.928	0.944	0.993	0.617	0.455	0.633
Ours	0.975	0.981	0.934	0.951	0.995	0.635	0.458	0.643

Table 9: Additional ablation study on VBench-I2V evaluation on the WISA-test set. Higher is better.

PIS	Train on WISA	DINOv3	Ours
$a_{x}$	0.538	0.592	0.604
$a_{y}$	0.521	0.598	0.611
$v_{x}$	0.753	0.764	0.775
$v_{y}$	0.674	0.725	0.739
$\Delta l$	0.399	0.436	0.451

Table 10: Additional ablation Physical Invariance Score(PIS) on WISA Test Set. Higher is better.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: The visualization of our user study page.

Figure 6: Additional visualizations of synthetic samples generated by our data construction pipeline.

Physical Consistency in Video Generation

Physical consistency in video generation studies how to make and evaluate videos whose geometry, motion, interactions, and state changes obey real-world laws.

See all 186 papers

Physical Simulator In-the-Loop Video Generation
Mar 6, 2026Lin Geng Foo, Mark He Huang, Alexandros Lattas +3Physical Consistency in Video Generation Video Generation
Max Planck Institute for Informatics, Saarland Informatics Campus · Saarbrücken Research Center for Visual Computing, Interaction and Artificial Intelligence · Singapore University of Technology and Design +2
PhyCo: Learning Controllable Physical Priors for Generative Motion
Apr 30, 2026Sriram Narayanan, Ziyu Jiang, Srinivasa Narasimhan +1Physics-Informed Generative Modeling Physical Consistency in Video Generation
1Carnegie Mellon University · 2NEC Labs America · 3UC San Diego
MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modeling
Apr 3, 2026Shubo Lin, Xuanyang Zhang, Wei Cheng +3Physical Consistency in Video Generation Video Diffusion Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · StepFun +2

Mar 6, 2026J/K move · Enter open · S save

Physical Simulator In-the-Loop Video Generation

Lin Geng Foo, Mark He Huang, Alexandros Lattas, Stylianos Moschoglou+2

Max Planck Institute for Informatics, Saarland Informatics Campus · Saarbrücken Research Center for Visual Computing, Interaction and Artificial Intelligence · Singapore University of Technology and Design+2

Physical Consistency in Video Generation Video Generation Video Diffusion Models Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Recent advances in diffusion-based video generation have achieved remarkable visual realism but still struggle to obey basic physical laws such as gravity, inertia, and collision. Generated objects often move inconsistently across frames, exhibit implausible dynamics, or violate physical constraints, limiting the realism and reliability of AI-generated videos. We address this gap by introducing Physical Simulator In-the-loop Video Generation (PSIVG), a novel framework that integrates a physical simulator into the video diffusion process. Starting from a template video generated by a pre-trained diffusion model, PSIVG reconstructs the 4D scene and foreground object meshes, initializes them within a physical simulator, and generates physically consistent trajectories. These simulated trajectories are then used to guide the video generator toward spatio-temporally physically coherent motion. To further improve texture consistency during object movement, we propose a Test-Time Texture Consistency Optimization (TTCO) technique that adapts text and feature embeddings based on pixel correspondences from the simulator. Comprehensive experiments demonstrate that PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity. Project Page: https://vcai.mpi-inf.mpg.de/projects/PSIVG/

Abstract

Physical Consistency in Video Generation

Explore similar work

Physical Simulator In-the-Loop Video Generation

PhyCo: Learning Controllable Physical Priors for Generative Motion

MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modeling