OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance

Cong Wang; Hanxin Zhu; Xiao Tang; Jiayi Luo; Xin Jin; Long Chen; Zhibo Chen

cs.CVMar 19, 2026

OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance

Authors: Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Zhibo Chen

Organizations: the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences · the School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy · School of Information Science and Technology, University of Science and Technology of China ·

Abstract

Recent progress in video generation has led to substantial improvements in visual fidelity, yet ensuring physically consistent motion remains a fundamental challenge. Intuitively, this limitation can be attributed to the fact that real-world object motion unfolds in three-dimensional space, while video observations provide only partial, view-dependent projections of such dynamics. To address these issues, we propose OrthoPhys, a two-stage framework that leverages orthogonal-view geometry guidance to enforce physical plausibility. Instead of directly generating unstructured 2D videos, our first stage generates synchronized, four-view orthogonal videos of the foreground dynamics. By incorporating a geometry-enhanced attention mechanism across these orthogonal views, this stage effectively enforces 3D spatial coherence and implicitly grounds the motion in physical attributes. In the second stage, these physically consistent orthogonal foregrounds serve as rigid guidance to synthesize the final complete video, seamlessly learning the interaction between foreground dynamics and the background context. To support this orthogonal-view training paradigm, we construct PhysMV, a dataset containing 40K scenes, each consisting of four orthogonal viewpoints, resulting in a total of 160K video sequences. Extensive experiments demonstrate that OrthoPhys significantly improves physical realism and spatial-temporal coherence over existing video generation methods. Project page: https://anonymous.4open.science/w/Phys4D/.

Figures & tables

Figure 1: Comparison with prior paradigms . (a) Data-driven generative models. (b) Physics-engine-based methods. (c) Our physics-aware two-stage framework utilizing foreground multi-view videos.

Figure 2: Pipeline of OrthoPhys . OrthoPhys generates physically plausible videos via a two-stage pipeline: 1) generating physics-aware orthogonal-view foreground videos, 2) generating a plausible video with the guidance of foreground motion.

Figure 3: Overview of Phys4View . This stage of OrthoPhys incorporates physics-aware attention to model physical motion and introduces a geometry-enhanced cross-view attention module to generate spatially and temporally consistent orthogonal-view videos.

Figure 4: Overview of VideoSyn . The framework extracts foreground motion cues from a pre-generated physics-aware video and leverages them to guide the full video generation process.

Figure 5: Qualitative comparisons. The top row shows the given image, text prompt and corresponding physics settings. For the definition of velocity and acceleration directions, the positive x-axis is defined as horizontal left, the positive y-axis as vertical upward, and the positive z-axis as perpendicular to the paper plane inward.

Method	WorldScore			VBench
Method	Photo↑	Motion↑	3D Consist↑	Motion↑	Subject↑	Flickering↑	Image↑
CogVideoX-I2V-5B [ 4 ]	64.72	67.39	74.43	0.993	0.886	0.989	0.636
Wan2.2-TI2V-5B [ 5 ]	65.97	64.90	63.86	0.994	0.926	0.991	0.653
Force Prompting [ 40 ]	74.38	72.53	50.97	0.994	0.938	0.992	0.641
OmniPhysGS [ 31 ]	58.67	90.65	37.24	0.995	0.929	0.995	0.378
PhysGen [ 41 ]	90.80	82.45	84.27	0.995	0.956	0.994	0.648

Table 1: Quantitative comparisons on WorldScore and VBench. Bold : Best. Underline : Second Best.

Method	GPT-4o			Human Study
Method	Physical↑	Photo↑	Semantic↑	Physical↑	Photo↑	Semantic↑
CogVideoX-5B [ 4 ]	0.612	0.858	0.617	2.80	3.14	3.14
Wan2.2-5B [ 5 ]	0.609	0.877	0.630	3.24	3.41	3.46
ForcePrompt [ 40 ]	0.615	0.875	0.611	2.25	2.41	2.74
OmniPhysGS [ 31 ]	0.622	0.723	0.402	2.19	2.55	1.92
PhysGen [ 41 ]	0.547	0.808	0.550	2.17	2.07	2.76

Table 2: GPT-4o and human evaluation results . The left block reports GPT-4o scores, and the right block reports results from a fair blind user study with 59 participants. Both use three metrics: physical realism (Physical), photorealism (Photo), and semantic consistency (Semantic).

Figure 6: Qualitative comparison of orthogonal-view video synthesis results .

Method	View1			View2
Method	Motion↑	Subject↑	Image↑	Motion↑	Subject↑	Image↑
DG4D [ 45 ]	0.986	0.892	0.268	0.987	0.883	0.260
$\text{Diffusion}^{2}$ [ 46 ]	0.970	0.637	0.371	0.979	0.666	0.362
L4GM [ 47 ]	0.994	0.898	0.368	0.992	0.899	0.305
Ours	0.995	0.912	0.389	0.994	0.901	0.368

Table 3: Evaluation of foreground orthogonal-view video synthesis.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Physical↑	Semantic↑
CogVideoX-5B	3.06	3.33
Wan2.2-5B	2.91	3.45
ForcePrompt	2.72	3.27
OmniPhysGS	2.21	3.53
PhysGen	2.69	3.63
PhysGen3D	2.73	3.57

Appendix

Table 5: Quantitative comparisons on VideoPhy-2 .

Method	PSNR↑	Traj. RMSE↓	Contact Timing Err.↓	Def. Trend Err.↓
DG4D	18.32	0.1203	19.59	36.51
Ours	22.24	0.0148	18.03	11.11

Appendix

Table 6: Quantitative evaluation on the controlled synthetic evaluation set . PSNR is reported as a reference reconstruction metric, while Traj. RMSE, Contact Timing Err., and Def. Trend Err. are proxy physical metrics.

Figure 8: Qualitative ablation results . We compare the baseline with variants that incorporate individual components, including Phys-Attn, Cross-View Attn, and Temporal-Attn. Each row corresponds to one setting.

Figure 9: Ablation study on conditional videos . The left column shows the input single image, and the three right columns each display one representative frame from the generated RGB video.

Method	Motion↑	Image↑	Physical↑
Ours (w/o mask prior)	0.991	0.646	0.609
Ours (w/o orthogonal-view prior)	0.992	0.637	0.625
Ours (w/o gated fusion)	0.995	0.651	0.648
Ours (w/o depth-confidence weighting)	0.994	0.642	0.643
Ours	0.996	0.655	0.691

Appendix

Table 7: Ablation on finer-grained design choices in OrthoPhys .

Method	Motion↑	Image↑	Physical↑
Ours (w/o orthogonal-view constraints)	0.983	0.636	0.628
Ours	0.996	0.655	0.691

Appendix

Table 8: Ablation study on the two-stage design .

Method	Motion↑	Image↑	Physical↑
Ours (w/o cross-view)	0.994	0.594	0.636
Ours (only Cross-Attn)	0.994	0.640	0.678
Ours (w/ Geometry-enhanced Attn)	0.996	0.655	0.691

Appendix

Table 9: Ablation study on geometry-enhanced cross-view attention .

Young’s Modulus	1e5	3e5	1e6	3e6	1e7
Peak Deformation (norm.)	1.000	0.992	0.985	0.981	0.976

Appendix

Table 10: Sensitivity analysis for Young’s modulus .

Density Factor	0.25x	0.5x	1.0x	2.0x	4.0x
Peak Deformation (norm.)	1.000	1.006	1.011	1.019	1.026

Appendix

Table 11: Sensitivity analysis for density .

Method	Motion↑	Image↑	Physical↑
Ours (w/ GPT-4o)	0.996	0.655	0.691
Ours (w/ expert)	0.996	0.658	0.694

Appendix

Table 12: Ablation on GPT-4o-estimated versus expert-provided physical attributes .

Figure 10: Prompt used for GPT-4o evaluation.

Figure 11: Four-view video generation results . Views 1 through 4 correspond to the front, left, rear, and right viewpoints, respectively.

Figure 12: Dataset demonstration . Our dataset contains various objects and motion patterns.

Figure 13: Qualitative visualization results of our method and the baselines .

Figure 14: Qualitative visualization results of our method and the baselines .

Figure 15: Qualitative visualization results of our method and the baselines .

Figure 16: Qualitative visualization results of our method and the baselines .

3D-Aware Video Generation

See all 23 papers

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation
Sep 8, 2026Cong Wang, Hanxin Zhu, Yonglin Tian +5Optical Flow Physical Consistency in Video Generation
State Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Zhongguancun Academy, Beijing 100094, China +2
PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Grounding
Nov 25, 2025Haoze Zhang, Tianyu Huang, Zichen Wan +4Physical Consistency in Video Generation Physical Reasoning
1Harbin Institute of Technology, China
Physical Simulator In-the-Loop Video Generation
Mar 6, 2026Lin Geng Foo, Mark He Huang, Alexandros Lattas +3Physical Consistency in Video Generation Video Generation
Max Planck Institute for Informatics, Saarland Informatics Campus · Saarbrücken Research Center for Visual Computing, Interaction and Artificial Intelligence · Singapore University of Technology and Design +2

Sep 8, 2026J/K move · Enter open · S save

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation

Cong Wang, Hanxin Zhu, Yonglin Tian, Jiayi Luo+4

State Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Zhongguancun Academy, Beijing 100094, China+2

Optical Flow Physical Consistency in Video Generation Controllable Video Generation

Open paper AskPDF arXiv

Video generation models have recently attracted substantial attention for their ability to generate visually compelling videos, yet ensuring physically consistent and plausible dynamics still remains a fundamental challenge, driving a growing line of research on physical realism in video generation. To address this challenge, motivated by the fact that physical regularities are primarily encoded in motion patterns, we propose PhysFlow, a novel two-stage framework for improving the physical plausibility of generated videos by decomposing video generation into motion-aware optical flow generation followed by motion-conditioned appearance synthesis. Specifically, PhysFlow consists of a physics-aware optical-flow video generator called PA-Flow and a flow-guided video generator called FlowRender. During the first stage, PA-Flow employs a physics-aware attention module to model how motion attributes and material properties influence global motion and local deformation, respectively, and generates an optical flow video as an explicit representation of motion. In the second stage, FlowRender leverages the decoupled motion representation as guidance to synthesize realistic textures and appearances, ultimately producing the final physically plausible video. To further support model training with explicit physical supervision, we construct PhysVideo, a physics-based video dataset generated with a physics engine and 3D-GS rendering, containing 10K foreground objects and 50K realistic video sequences with annotations of motion and material properties. Extensive experiments demonstrate that our proposed PhysFlow generates videos with superior physical plausibility while maintaining high visual fidelity compared with existing methods.

Abstract

3D-Aware Video Generation

Explore similar work

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation

PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Grounding

Physical Simulator In-the-Loop Video Generation