Overview

Video prediction studies how to infer what a scene will look like or how it will change, supporting tasks such as autonomous driving, robot control, and physical simulation. Methods range from recurrent CNNs and spatiotemporal transformers to autoregressive and diffusion-based models, often predicting compact latent states or using motion, geometry, events, and physical dynamics as guidance. Evaluation spans frame-forecasting datasets such as Moving MNIST and KTH, driving scenes, and action-conditioned manipulation settings. A central challenge is maintaining coherent, physically and causally consistent predictions over long horizons while controlling drift, inference cost, and performance under changes in viewpoint or environment.

Latest papers

CardsList

76 papers10 papers in the last four weeks, down 33% on the four weeks before. 0.1% of all new papers.

  1. Latent World Modeling for Point-Cloud Video Pretraining

    Nov 15, 2026Zhuoyue Zhang, Jian Liu, Jihua Zhu +1Self-Supervised Point Cloud LearningLatent World Models

  2. Ego Guidance: Enhancing Robot Policy Learning with Action-Free Human Egocentric Videos

    Nov 10, 2026Lin Long, Tianren Ma, Zhong Chu +5Robot Policy LearningRobot Learning from Video

  3. JEPAhead: From Inpainting to Causal Forecasting for Urban Driving

    Nov 5, 2026Daniel G. P. Petrini, Davi Pincinato, Fabio Grassiotto +4Video PredictionSpatiotemporal Forecasting

  4. Scaling Action-Free Video Latent Prediction for Transferable Representations in End-to-End Autonomous Driving

    Nov 2, 2026Xingtai Gui, Xubin Zhou, Yucheng Zhou +4Video Representation LearningEnd-to-End Autonomous Driving

  5. Video World Models Need Predictable Latents

    Oct 26, 2026Mattia Scardecchia, Sherry YangVideo World ModelsWorld Model Learning

  6. Motion Equivariant Convolutional LSTM: When Motion Becomes a Gauge Symmetry

    Oct 20, 2026Vahid Hosseinzadeh, T. Anderson Keller, Michael PuthawalaEquivariant Neural NetworksConvolutional LSTM

  7. PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents

    Oct 14, 2026Siqi Yang, Yimu Wang, Ziqi Cai +6World Model LearningAction-Conditioned World Models

  8. Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction

    Oct 13, 2026Anton Arapin, Aditya Singh, Yixin Wang +2Aleatoric UncertaintyAction-Conditioned World Models

  9. ST-GRU PredRNN-V2: A GRU-Based Reformulation of PredRNN-V2 for Video Prediction

    Oct 13, 2026Augusta van Haren, Hanna Hoogen, Luca Pattavina +1Video PredictionRecurrent Neural Networks

  10. LynxScenes: A Large-Scale Real-World Benchmark for Outdoor Wheel-Legged Robot Perception, Navigation, Imagination and Reconstruction

    Oct 13, 2026Jun Tian, Hao Wu, Ruo Chen +5Robot Navigation BenchmarksRobot Navigation

  11. IMPULSE: Push First, Then Predict in V-JEPA 2 Feature Space

    Oct 12, 2026Tingan Jin, Chung-Hsien ChouVideo PredictionPhysics-Informed Forecasting

  12. Remembering What Matters for Long-Horizon Robotic World Models

    Oct 10, 2026Hao Wu, Tianyu Shao, Shengju Qian +10Video World ModelsVideo Prediction

  13. Video Prediction Policy 2: Predict Better, Act Better

    Oct 7, 2026Yanjiang Guo, Haodong Yan, Zhide Zhong +15Visuomotor Policy LearningWorld Action Models

  14. VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction

    Oct 5, 2026Qiutong Chen, Yuchan Guo, Zhenlong Yuan +10Video PredictionCausal Reasoning in Videos

  15. Learning Conditional Source Distribution via Flow Reversal for Temporal Flow Matching

    Oct 4, 2026Kuan-Hsun Tu, Hsuan-Chi Liu, Jia-Wei Liao +3Flow MatchingVideo Prediction

  16. UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation

    Oct 2, 2026Daikun Liu, Xin Zhan, Teng Wang +24D Scene GenerationVideo Prediction

  17. Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models

    Oct 1, 2026Efstathios Karypidis, Spyros Gidaris, Nikos KomodakisVideo PredictionPredictive Representation Learning

  18. MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception

    Sep 27, 2026Yuhao Li, Louie Hong Yao, Tianyi Shi +4Spatiotemporal ReasoningPositional Encoding

  19. CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model

    Sep 19, 2026Ziming Xu, Shuang Liang, Ruobing Han +10Causal World ModelsAction-Conditioned World Models

  20. JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting

    Sep 18, 2026Trinh Tra Giang Nguyen, Thanh Nguyen Vo, Nguyen Hoai Thuong Bui +1Video PredictionTraffic Forecasting

  21. Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models

    Sep 15, 2026Rwiddhi Chakraborty, Yinong Oliver Wang, Cheng Zhang +6Video-Language Model EvaluationVideo Question Answering

  22. Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation

    Sep 3, 2026Yingmao Miao, Pengfei Zhang, Chaoran Xu +5Long Video GenerationVideo Generation