Representation Learning for Spatiotemporal Physical Systems

Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning, but their capability to learn physics and generate physically realistic dynamics remains hitherto untested. In this work, we introduce SemiGroup-JEPA (SG-JEPA), which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model via action-conditioning and jointly training an encoder and predictor through an autoregressive latent rollout. To evaluate the model's ability to generalize out of distribution, we design dynamical tasks under different gravitational fields that, despite obeying the same physical law, exhibit qualitatively different dynamics, ranging from floating motion in weak gravitational fields to rapid bouncing in strong ones. In contrast to DINO-WM, SG-JEPA reduces open-loop prediction error by up to 2 times on two-dimensional datasets, and increases control success rate up to 2.5 times for three-dimensional robotic datasets, for which we train independent diffusion policies. To explain this advantage, we develop a linear feature model that separates local law-conditioned error from its recursive amplification under rollout. Guided by this model, we find that back-propagating the multi-step rollout loss into the representation trains the encoder to keep the features that the predictor can carry forward, and that those are the features the dynamics depend on, so most of the gain comes from the encoder learning better features rather than from the predictor learning better dynamics. See project page at https://sg-jepa.github.io.

Multivariate forecasting in physical systems requires models that predict coupled temporal variables while preserving meaningful state evolution. Deep forecasters can fit temporal correlations, and physics-informed models can regularize predictions with scientific constraints, but these directions are often connected only at the decoded-output level. As a result, the hidden predictive state that generates future trajectories may remain statistically useful but physically unstructured. We introduce Phys-JEPA, a physics-informed joint-embedding predictive architecture for multivariate time-series forecasting. Phys-JEPA learns a latent world model in which predictive states are decomposed into physical and residual components, and physical consistency is imposed directly on latent states and latent transitions rather than only on decoded forecasts. This formulation uses known physical variables to organize the representation space while retaining residual capacity for unresolved dynamics. On Jena Climate 2009--2016, Phys-JEPA reduces aggregate MSE from 0.12482 to 0.12273 and temperature MSE from 0.01892 to 0.01831 at H=24. On Traffic, full Phys-JEPA improves aggregate MSE over the supervised baseline across all tested horizons, reducing H=192 MSE from 0.800784 to 0.773873. On Electricity, the best variant depends on horizon: static latent consistency is strongest at H=24 and H=48, while full Phys-JEPA gives the best aggregate and target-variable MSE at H=192. These initial results suggest that moving physics-informed learning from output space to latent predictive state space is a promising direction for interpretable temporal world models.

Although machine learning can be used to predict the evolution of physical systems from data, a formulation that learns only the system state at each time leaves the temporal and dynamical structure of the solution to be resolved within a broad hypothesis space. We introduce Learning$^2$, a representation-level framework that structures this space by coupling a primary representation to a second representation through a known physical transformation. The resulting cross-representation constraint restricts the effective hypothesis space and provides an ante-hoc, physically interpretable criterion for excluding solutions that satisfy the primary representation alone. We instantiate Learning$^2$ through EuLaNet, an Eulerian--Lagrangian representation for fluid dynamics. Given the velocity state $u(\mathbf{x},t)$, EuLaNet constructs its induced Lagrangian flow map $X(\mathbf{a},t)$ through $\dot{X}(\mathbf{a},t)=u(X(\mathbf{a},t),t)$, from which material transport and finite-time deformation are derived. The resulting representation couples the predicted state to the dynamical consequences it induces, providing a second consistency criterion beyond state-level agreement. We formalize this construction through an effective hypothesis space $\mathcal{H}_{L^2}\subseteq\mathcal{H}$ and define the conditions under which a consequence representation provides discriminative constraints on candidate solutions. EuLaNet is implemented as a model-independent representation module, separating the physical constraint from the downstream learning architecture. This construction provides an ante-hoc mechanism for physically interpretable constraint in scientific learning and offers a basis for developing and evaluating broader classes of Learning$^2$ architectures. The implementation is open-sourced to support the development and extension of the architecture across scientific domains.