World Action Models
Overview
World Action Models (WAMs) learn policies alongside predictions of future observations or task-relevant states, aiming to make control account for physical consequences. Approaches range from joint video-and-action diffusion and inverse-dynamics models to compact latent, geometric, object-centric, tactile, and motion representations, trained with robot demonstrations, human video, or synthetic data. On manipulation benchmarks such as LIBERO and RoboTwin and driving tests such as NAVSIM, many methods use future prediction as training supervision or compact guidance instead of generating full video at every step. A central challenge is ensuring imagined futures are causally aligned with actions and useful under distribution shifts, while keeping inference fast and light enough for closed-loop deployment.
Latest papers
223 papers73 papers in the last four weeks, up 115% on the four weeks before. 0.4% of all new papers.