Video Prediction

Video prediction studies how to infer what a scene will look like or how it will change, supporting tasks such as autonomous driving, robot control, and physical simulation. Methods range from recurrent CNNs and spatiotemporal transformers to autoregressive and diffusion-based models, often predicting compact latent states or using motion, geometry, events, and physical dynamics as guidance. Evaluation spans frame-forecasting datasets such as Moving MNIST and KTH, driving scenes, and action-conditioned manipulation settings. A central challenge is maintaining coherent, physically and causally consistent predictions over long horizons while controlling drift, inference cost, and performance under changes in viewpoint or environment.