cs.LGJan 3, 2026

Flow Equivariant World Models: Structured Memory for Dynamic Environments

Authors: Hansen Jin Lillemark, Benhao Huang, Fangneng Zhan, Yilun Du, Thomas Anderson Keller

Organizations: Kempner Institute, Harvard University · CSE, UC San Diego · ML, Carnegie Mellon University · SEAS, Harvard University.

Abstract

Embodied systems experience the world as 'a symphony of flows': a combination of many continuous streams of sensory input coupled to self-motion, interwoven with the dynamics of external objects. These sensory streams and the underlying dynamics of the world obey smooth, time-parameterized symmetries which existing world models ignore. Without a memory that respects this structure, partial observability presents a major obstacle to existing methods: each observation reveals only a fraction of the world, while unobserved regions continue to evolve. In this work, we introduce Flow Equivariant World Modeling, a framework that leverages time-parameterized symmetries within a latent memory for stable and accurate dynamics prediction over long horizons. The latent memory shifts and transforms equivariantly with self-motion and inferred external object motion, keeping information about out-of-view regions aligned as time progresses. We demonstrate the advantage of this framework over state-of-the-art diffusion, memory-augmented, and recurrent world model architectures on 2D and 3D partially observed video world modeling benchmarks. More broadly, our results suggest that predictive representations become more powerful when they are organized in line with the temporal and dynamical structure of the world they model. Project page: https://flowequivariantworldmodels.github.io/

Figures & tables

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining

    Oct 29, 2026Yuhao Lu, Jiongxu Chen, Hexiang Wei +9Video World ModelsMultimodal Trajectory Prediction

  2. Flow Matching in Feature Space for Stochastic World Modeling

    Jun 27, 2026Francois Porcher, Nicolas Carion, Karteek Alahari +1Flow MatchingWorld Model Learning