Video diffusion models are able to generate samples with high-quality appearance and textural qualities. However, content produced by these methods is known to exhibit significant physical errors. This is because, in a classical video generation setting, all physical reasoning must take place simultaneously and exclusively within the diffusion process.We propose to resolve these issues by redistributing the task of physical reasoning onto a language model-backed keyframe generation module, which allows for explicit reasoning in language, and fine-grained control over the appearance of the video at important moments in the video. We then condition a video generation model with these keyframes, reducing the role of the diffusion model to the simpler task of interpolating between physically-correct intermediate states, rather than inferring an entire sequence with diffusion. Results demonstrate that our approach significantly outperforms existing approaches, achieving state-of-the-art performance on the Physics-IQ Verified benchmark and PhyGenBench.
Figures & tables
Figure 1. Our approach generates keyframes to serve as stronger conditioning for a video model, leading to an approach which outperforms existing methods both qualitatively and quantitatively.
Figure 2. Overview of the proposed method. We design a keyframe module which reasons about the physical evolution of a given scene and produces keyframes with timestamps and an expanded caption. These are then used as the conditioning signal for Wan2.2, resulting in videos with more accurate physical plausibility.
Figure 3. Keyframe module detail. A VLM planning agent orchestrates a recursive loop producing keyframes. The first frame is passed to an image editor with an instruction from the VLM to produce the appearance of the frame later in time. The VLM reviews this frame and, if deemed correct, the frame is added to a stack of keyframes. The VLM repeats this process, now passing the top image to the image editor to add to the keyframe stack. Once all keyframes are produced, the model finishes, returning an expanded caption describing the motion of objects in the video with the keyframe stack.
Figure 4. Conditioning model detail. Keyframes are inserted into the typical conditioning stack of blank frames at locations designated in Equation 3 . This is then passed to the VAE, and is subsequently treated as normal conditioning in the Wan pipeline. The conditioning is concatenated with a mask indicating which keyframes are filled, and with the latent noise before being passed to the transformer. The mask and latent conditioning is appended before each denoising step.
Figure 5. Qualitative results from our approach. The top row shows an example from Physics-IQ Verified and the bottom row an example from PhyGenBench. These results highlight the diverse placement of keyframes in different videos, and the physical quality of the resulting generated videos.
Model
Physics-IQ Verified
1
K2V+Wan2.2-14B †
55.0 ± 0.8
2
K2V+Wan2.2-5B †
50.0 ± 1.3
3
Cosmos3-Super
42.7 ± 0.8
4
MiniMax H3 (FL2VA)
39.8 ± 0.4
5
Cosmos3-Nano
37.3 ± 0.9
6
MiniMax H3 Max
36.2 ± 0.7
Table 1. Physics-IQ Verified, image-to-video track. Higher is better and 100 is the maximum possible score. Methods marked with † are ours.
Figure 6. Physics-IQ Verified score against cost per video for the image-to-video models of Table 1 , with costs as listed on the Physics-IQ Verified leaderboard ( Anates Labs, 2026 ) on 8 October 2026. Costs follow the leaderboard’s normalization, which scales generation cost to 24 fps and 1280-pixel-wide output and then adds any language-model cost; for K2V this is the leaderboard’s Wan2.2 generation cost plus our planning cost from Table 4 . The dashed line traces the cost score frontier, and vertical bars show the standard deviations of Table 1 .
Figure 7. Left: Compared to vanilla Wan2.2, the quality of our generations (measured by mean squared error above physical variance) does not rise over the 5 seconds of a video. Beneath, keyframe density shows that our approach distributes keyframes relatively evenly throughout videos on Physics-IQ Verified, with spikes at 0s and 5s, as these keyframes are required for each video. Right: The time taken to produce keyframes shows that the language model spends compute efficiently, spending more time on scenes which require more careful planning and less time on simpler scenes.
Figure 8. Qualitative comparison. Top: Physics-IQ Verified (centre view), where ours, Wan2.2-14B and Cosmos3-Super are conditioned on the same input frame. Bottom: PhyGenBench, where ours, Wan2.2-14B and NEWTON 1 are given the same text prompt and no image. Frames are shown at 1 s intervals. Cosmos3-Super is image-to-video only and so does not appear for PhyGenBench. Further Physics-IQ comparisons are given in Figure 11 .
Method
Mech.
Optics
Therm.
Mat.
Avg.
1
K2V+Wan2.2-14B †
0.583
0.707
0.589
0.625
0.633
2
K2V+Wan2.2-5B †
0.600
0.713
0.578
0.608
0.633
3
LTX-Video-2B ( HaCohen et al., 2024 ) + NEWTON
0.500
0.647
0.522
0.542
0.560
4
Wan2.2-TI2V-5B
0.550
0.580
0.533
0.500
0.544
5
Wan2.2-T2V-A14B
0.500
0.707
0.433
0.400
0.527
6
LTX-Video-2B ( HaCohen et al., 2024 )
0.508
0.580
0.478
0.450
0.510
Table 2. PhyGenBench, text-to-video. Scores are PCA from PhyGenEval, averaged over the two judge tracks as the benchmark specifies; higher is better and 1.0 is the ceiling. Methods marked † are ours.
Figure 9. Left: Conditioning Wan2.2-5B on N equally spaced ground truth keyframes, for N between 1 and 6, raises the Physics-IQ Verified score from 27.7 to 67.4. Right: Editing the final keyframe with the same image edit model used in our pipeline yields a new video, here with the inner petals recoloured.
Physics-IQ
Δ
K2V (full)
56.0
—
Planning
no agentic loop
54.3
−1.7
open-weight models (Qwen)
40.2
−15.8
Conditioning
endpoints only
48.6
−7.5
Table 3. Ablations on Physics-IQ Verified, centre perspective. Δ is measured against the full method at the top.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Model
Physics-IQ
PhyGenBench
K2V
Planning agent
Gemini 3.8 Flash
6.1 calls, 0.161
10.1 calls, 0.165
Keyframe edits
Gemini 3.1 Flash Image
1.7 images, 0.143
2.9 images, 0.258
Video generation
Wan2.2-14B
1 video, 0.165
1 video, 0.165
Total
0.47
0.59
NEWTON 1
Appendix
Table 4. Cost per video in US dollars at list prices on 8 October 2026. Counts are means per video. The NEWTON total excludes its video verifier, whose usage was not metered.
Figure 10. Further results. Top four: Physics-IQ Verified (centre view, run 1), where the input is the benchmark’s conditioning frame and description, and keyframes are placed where each scene’s physics requires. Bottom four: PhyGenBench, where the input is the text prompt alone and the opening frame at t=0 is generated by our agent.
Figure 11. Further qualitative comparisons on Physics-IQ Verified (centre view), continuing Figure 8 , against Wan2.2-14B and Cosmos3-Super from the same input frame.
Physical consistency in video generation studies how to make and evaluate videos whose geometry, motion, interactions, and state changes obey real-world laws.
Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation. Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback. This combination enables a generative model to produce physically consistent and controllable outputs through variations in physical attributes-without any simulator or geometry reconstruction at inference. On the Physics-IQ benchmark, PhyCo significantly improves physical realism over strong baselines, and human studies confirm clearer and more faithful control over physical attributes. Our results demonstrate a scalable path toward physically consistent, controllable generative video models that generalize beyond synthetic training environments.
Generative video models achieve high visual fidelity but often violate basic physical principles, limiting reliability in real-world settings. Prior attempts to inject physics rely on conditioning: frame-level signals are domain-specific and short-horizon, while global text prompts are coarse and noisy, missing fine-grained dynamics. We present PhysVid, a physics-aware local conditioning scheme that operates over temporally contiguous chunks of frames. Each chunk is annotated with physics-grounded descriptions of states, interactions, and constraints, which are fused with the global prompt via chunk-aware cross-attention during training. At inference, we introduce negative physics prompts (descriptions of locally relevant law violations) to steer generation away from implausible trajectories. On VideoPhy, PhysVid improves physical commonsense scores by ≈33% over baseline video generators, and by up to ≈8% on VideoPhy2. These results show that local, physics-aware guidance substantially increases physical plausibility in generative video and marks a step toward physics-grounded video models.
Video generation models achieve high visual quality but often struggle to generate physics-aware videos. Unlike rigid-body motion, which can be described by explicit trajectories or formulas, complex deformation dynamics remain challenging to synthesize. We observe that a lack of physical reasoning for localizing dynamic areas allows irrelevant regions to dilute the model's attention, leading to generation failure. In this paper, we propose DeforM, a reasoning-guided image-to-video generation framework that directs the model's focus toward physics-critical regions. To reason about and localize these critical regions, we introduce a VLM-guided physical reasoning module, DeforM-Reason, to identify target objects and generate spatial-temporal masks. For physical guidance, we develop two alternative strategies: DeforM-Free for training-free mechanism analysis and DeforM-Injection as a powerful training-based generator. Experimental results demonstrate that DeforM improves the realism of generated deformation scenarios, outperforming baseline models in both visual quality and physical consistency.
Yunyi Li, Yu Qiao, Yaohui Wang +1
Shanghai Artificial Intelligence Laboratory, Shanghai, China