K2V: Generative Keyframing For Physically Plausible Video Generation
Organizations: University of Cambridge, Cambridge, UK · Toyota Motor Europe, Brussels, Belgium
Abstract
Video diffusion models are able to generate samples with high-quality appearance and textural qualities. However, content produced by these methods is known to exhibit significant physical errors. This is because, in a classical video generation setting, all physical reasoning must take place simultaneously and exclusively within the diffusion process.We propose to resolve these issues by redistributing the task of physical reasoning onto a language model-backed keyframe generation module, which allows for explicit reasoning in language, and fine-grained control over the appearance of the video at important moments in the video. We then condition a video generation model with these keyframes, reducing the role of the diffusion model to the simpler task of interpolating between physically-correct intermediate states, rather than inferring an entire sequence with diffusion. Results demonstrate that our approach significantly outperforms existing approaches, achieving state-of-the-art performance on the Physics-IQ Verified benchmark and PhyGenBench.
Figures & tables
| Model | Physics-IQ Verified | |
|---|---|---|
| 1 | K2V+Wan2.2-14B † | 55.0 0.8 |
| 2 | K2V+Wan2.2-5B † | 50.0 1.3 |
| 3 | Cosmos3-Super | 42.7 0.8 |
| 4 | MiniMax H3 (FL2VA) | 39.8 0.4 |
| 5 | Cosmos3-Nano | 37.3 0.9 |
| 6 | MiniMax H3 Max | 36.2 0.7 |
| Method | Mech. | Optics | Therm. | Mat. | Avg. | |
|---|---|---|---|---|---|---|
| 1 | K2V+Wan2.2-14B † | 0.583 | 0.707 | 0.589 | 0.625 | 0.633 |
| 2 | K2V+Wan2.2-5B † | 0.600 | 0.713 | 0.578 | 0.608 | 0.633 |
| 3 | LTX-Video-2B ( HaCohen et al., 2024 ) + NEWTON | 0.500 | 0.647 | 0.522 | 0.542 | 0.560 |
| 4 | Wan2.2-TI2V-5B | 0.550 | 0.580 | 0.533 | 0.500 | 0.544 |
| 5 | Wan2.2-T2V-A14B | 0.500 | 0.707 | 0.433 | 0.400 | 0.527 |
| 6 | LTX-Video-2B ( HaCohen et al., 2024 ) | 0.508 | 0.580 | 0.478 | 0.450 | 0.510 |
| Physics-IQ | ||
| K2V (full) | 56.0 | — |
| Planning | ||
| no agentic loop | 54.3 | |
| open-weight models (Qwen) | 40.2 | |
| Conditioning | ||
| endpoints only | 48.6 | |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Component | Model | Physics-IQ | PhyGenBench |
|---|---|---|---|
| K2V | |||
| Planning agent | Gemini 3.8 Flash | 6.1 calls, 0.161 | 10.1 calls, 0.165 |
| Keyframe edits | Gemini 3.1 Flash Image | 1.7 images, 0.143 | 2.9 images, 0.258 |
| Video generation | Wan2.2-14B | 1 video, 0.165 | 1 video, 0.165 |
| Total | 0.47 | 0.59 | |
| NEWTON 1 | |||
Physical Consistency in Video Generation
Physical consistency in video generation studies how to make and evaluate videos whose geometry, motion, interactions, and state changes obey real-world laws.
See all 202 papers