Video generation models are increasingly viewed as world simulators, which requires not only plausible motion but dynamics that respond correctly to specified physical parameters. Existing models, datasets and benchmarks, however, largely target physical plausibility, leaving this physical parameter responsiveness insufficiently explored. We introduce PhysResponse, a benchmark that measures it at two levels: scene-conditioned simulator inversion recovers the effective gravity, friction and initial speed of generated motion, and event evaluation tests outcomes near and across critical transitions. Evaluating ten image-to-video models, we find that even the lowest parameter errors exceed those of physically correct videos by over an order of magnitude, that models rarely reproduce both outcomes of a transition pair, and that the two levels expose complementary failures that visual metrics such as PSNR miss. To supply the missing training signal, we construct 72,000 simulated videos of 20 multi-object interaction types with controlled parameters; fine-tuning on them lowers parameter errors, even on held-out interaction types, and transfers to real videos and an external physics benchmark. Continued training with a numerical encoder further lowers all three errors on held-out types. PhysResponse provides a foundation for world simulators that follow physical conditions, not just appearances.
Figures & tables
Figure 2: Speed decides the outcome in simulation but often not in generation. In simulation only the faster book tips the can onto the shelf, but PhysAlign, even told so, leaves it on the table at both speeds. ✓ / ✗ : match/contradict the simulation.
Figure 3: Constructing PhysResponse. Scenes are assembled from household assets and four backgrounds, edited one physical condition at a time, simulated in Genesis and Isaac and rendered from three views; panel 2 brackets two reference transitions, in the initial speed of T17 and the channel width of T05. Depth and masks are derived from RGB.
Figure 4: Twenty everyday interaction types. Arrows: motion; letters: parameters evaluated; ∗ held out of training.
Scene
Recovery
Work
Videos
Set param.
Multi
Any
Multi
Ctrl.
Flips
Train
Physics-IQ [ 30 ]
396
R
✗
✓
✗
✗
✗
✗
✗
PhyGenBench [ 27 ]
160
P
✗
(✓)
✗
✗
✗
✗
✗
VideoPhy-2 [ 2 ]
3.9K
P
✗
✓
✗
✗
✗
✗
✗
WorldBench [ 45 ]
748
R+S
μ,η†
✓
✓
✗
✗
✗
✗
PhysInOne [ 64 ]
2M
S
μ,e,ρ,…
✓
✗
✗
✗
✗
✓
Table 1: Datasets and evaluation capabilities. Recovery: parameters recovered from generated videos; Ctrl.: one parameter varied; flips: opposite reference outcomes. R/S: real/simulated; P: prompts; proc.: on demand; (✓): partial; † by material; § geometry only. e,ρ,η,k,m,F : restitution, density, viscosity, stiffness, mass, force.
Figure 5: PhysResponse evaluation overview. Core100 conditions and prompts produce videos for two complementary measurements. Object tracks support simulator-based parameter recovery; reference transitions and visual event observations support NearAcc, FarAcc and PairAcc. Measurability screening identifies the evidence available to each branch, with human validation of the observations.
Figure 6: Recovering gravity from a generated video. (a) MiniMax-H3, Moon gravity: tracks (white) match the replay at g=11.8 m/s 2 (gold), not at the prompted g (grey). (b) Excess fit loss, square-root axis; shaded: tolerance δ ; thin grey: a lower bound and an unidentifiable estimate.
Visual and motion consistency
Kinematics
Effective parameter error
Human
Model
L2 err. ↓
S-IoU ↑
PSNR ↑
SSIM ↑
Displ. → 1
Order ↑
g↓
μ↓
v↓
Meas. ↑
Condition frame repeated
12.59%
0.000
37.8
0.977
0.00
0%
–
–
–
–
CogVideoX-1.5
13.59%
0.057
22.9
0.844
0.03
8%
114.5%
166.0%
93.0%
23%
Cosmos-Predict2
13.60%
0.071
20.8
0.800
0.06
8%
89.2%
76.8%
81.9%
40%
Cosmos3-Nano
17.40%
0.153
29.7
0.935
0.66
23%
81.8%
77.0%
53.5%
45%
HunyuanVideo-1.5
14.90%
0.200
27.4
0.927
0.10
26%
86.5%
77.0%
63.9%
58%
Table 2: No model follows the prompted physics. All four prompt formats pooled (per format: supplement); errors aggregated as in Sec. 4.2 . Condition frame: first frame repeated, not ranked. Bold/underlined: best/second.
Visual and motion consistency
Kinematics
Effective parameter error
Human
Model
L2 err. ↓
S-IoU ↑
PSNR ↑
SSIM ↑
Displ. → 1
Order ↑
g↓
μ↓
v↓
Meas. ↑
Wan2.2-TI2V-5B
16.10%
0.051
23.2
0.819
0.06
13%
94.9%
130.3%
85.8%
35%
PR-S
13.84%
0.305
32.1
0.949
0.59
20%
69.1%
70.6%
45.6%
80%
PR-D
14.69%
0.292
32.0
0.948
0.72
25%
69.1%
87.6%
67.3%
78%
Table 5: Fine-tuning on PhysResponse lowers all three parameter errors. Wan2.2-TI2V-5B on the 100 cases; bold: best, underlined: second. Text-only versions: Supp. Table 34; event results: Table 6 .
Standard (S)
Detailed (D)
Model
NearAcc ↑
FarAcc ↑
PairAcc ↑
Coverage ↑
VPSR ↑
NearAcc ↑
FarAcc ↑
PairAcc ↑
Coverage ↑
VPSR ↑
Wan2.2-TI2V-5B
35.4
25.0
0.0
36.3
0.0
40.9
55.6
8.3
70.6
5.9
PR-S
42.8
61.0
15.0
96.1
14.4
50.4
60.4
10.2
96.1
9.8
PR-D
33.0
58.0
14.3
96.1
13.7
54.7
60.2
4.8
96.1
4.6
Table 6: Event response after fine-tuning (%). Standard and Detailed prompts; baseline and metrics as in Table 4 . Text-only results: Supp. Table 17. Bold/underlined: best/second.
Model
PMF ↑
PSNR ↑
SSIM ↑
LPIPS ↓
FVD ↓
Wan2.2-TI2V-5B
2.058
12.5
0.404
0.503
1334.3
PhysInOne LoRA †
2.835
15.8
0.622
0.337
893.4
PhysInOne SFT †
2.880
15.7
0.685
0.305
772.3
PR-S
3.144
17.8
0.738
0.239
794.7
PR-D
3.148
17.8
0.740
0.238
786.2
Table 7: Fine-tuning on PhysResponse transfers to PhysInOne. 100 training-split scenes, seed 0; † fine-tuned by the PhysInOne authors on this split. Bold/underlined: best/second.
Figure 7: Matched fine-tuning example. T17_C03, Standard prompt: GT, Wan2.2-TI2V-5B, PR-S and PR-D. Right insets show fitted gravity and contact outcomes; the Wan2.2-TI2V-5B result is marked as excluded.
Physical consistency in video generation studies how to make and evaluate videos whose geometry, motion, interactions, and state changes obey real-world laws.
Generative world models are increasingly used for video generation, where learned simulators are expected to capture the physical rules that govern real-world dynamics. However, evaluating whether generated videos actually follow these rules remains challenging. Existing physics-focused video benchmarks have made important progress, but they still face three key challenges, including the coarse evaluation frameworks that hide law-specific failures, response biases and fatigue that undermine the validity of annotation judgments, and automated evaluators that are insufficiently physics-aware or difficult to audit. To address those challenges, we introduce PhyGround, a criteria-grounded benchmark for evaluating physical reasoning in video generation. The benchmark contains 250 curated prompts, each augmented with an expected physical outcome, and a taxonomy of 13 physical laws across solid-body mechanics, fluid dynamics, and optics. Each law is operationalized through observable sub-questions to enable per-law diagnostics. We evaluate eight modern video generation models through a large-scale, quality-controlled human study, grounded on social science lab experiment design. A total of 459 annotators provided 5,796 complete annotations and over 37.4K fine-grained labels; after quality control, the retained annotations exhibited high split-half model-ranking correlations (Spearman's rho > 0.90). To support reproducible automated evaluation, we release PhyJudge-9B, an open physics-specialized VLM judge. PhyJudge-9B achieves substantially lower aggregate relative bias than Gemini-3.1-Pro (3.3% vs. 16.6%). We release prompts, human annotations, model checkpoints, and evaluation code on the project page https://phyground.github.io/.
Juyi Lin, Arash Akbari, Yumei He +13
1Northeastern University · 2Tulane University · University of Washington +1
Video generation models are increasingly used as world simulators for storytelling, simulation, and embodied AI. As these models advance, a key question arises: do generated videos obey the physical laws of the real world? Existing evaluations largely rely on automated metrics or coarse human judgments such as preferences or rubric-based checks. While useful for assessing perceptual quality, these methods provide limited insight into when and why generated dynamics violate real-world physical constraints. We introduce Physion-Eval, a large-scale benchmark of expert human reasoning for diagnosing physical realism failures in videos generated by five state-of-the-art models across egocentric and exocentric views, containing 10,990 expert reasoning traces spanning 22 fine-grained physical categories. Each generated video is derived from a corresponding real-world reference video depicting a clear physical process, and annotated with temporally localized glitches, structured failure categories, and natural-language explanations of the violated physical behavior. Using this dataset, we reveal a striking limitation of current video generation models: in physics-critical scenarios, 83.3% of exocentric and 93.5% of egocentric generated videos exhibit at least one human-identifiable physical glitch. We hope Physion-Eval will set a new standard for physical realism evaluation and guide the development of physics-grounded video generation. The benchmark is publicly available at https://huggingface.co/datasets/PhysionLabs/Physion-Eval.
World simulators can provide safe and scalable environments for training Physical AI systems before real-world deployment. Large video generation models are emerging as a promising basis for such simulators because they can generate diverse and realistic visual futures. However, using them as world simulators requires physically faithful video continuations, namely, generated videos that preserve the physical state implied by the conditioning input, and evolve in ways consistent with basic physical principles. We propose PhyWorld, a video generation world model designed to produce temporally coherent and physically faithful scene continuations through two-stage post-training. In the first stage, we improve video-to-video continuation with flow matching fine-tuning, encouraging stable visual attributes and coherent motion dynamics across frames. In the second stage, we align generated dynamics with physical principles using Direct Preference Optimization (DPO) over physics preference pairs, guiding the model toward outputs with higher physical plausibility. To evaluate PhyWorld, we use both standard video-quality benchmarks and a dedicated physical-faithfulness benchmark with per-law scoring. Experiments show that PhyWorld improves video consistency, achieving an average score of 0.769 on VBench compared with 0.756 or below for state-of-the-art baselines. PhyWorld also improves physical plausibility, reaching an average score of 3.09 on our physical-faithfulness benchmark compared with 2.99 for the strongest baseline. These results suggest that post-training large video generation models with continuation and physics-preference signals can make them more effective world simulators for Physical AI.
Pu Zhao, Juyi Lin, Timothy Rupprecht +10
Northeastern University · University of Georgia · Tulane University +1