Predictive Representation Learning

Follow this topic Ask about this topic

Predictive representation learning studies how models can build useful internal states by predicting masked, future, or cross-view representations rather than reproducing every input detail. Common approaches include joint-embedding predictive architectures such as JEPA, contrastive predictive coding, masked latent prediction, and models that forecast temporal dynamics; they are used in vision, speech, health data, and robotics. Researchers evaluate learned features with transfer tasks, linear probes, forecasting benchmarks, and measures of control or planning performance. A central open tension is that predictability alone may discard rare but important information, so preventing collapse and aligning representations with causal, clinical, or action-relevant structure remain key challenges.

Latest papers

176 papers40 papers in the last four weeks, up 111% on the four weeks before. 0.2% of all new papers.

Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning
Jun 5, 2026Paarth Gulati, Ilya NemenmanContrastive Learning Representation Learning
Department of Physics, and Initiative in Theory and Modeling of Living Systems, Emory University, Atlanta, GA
What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction
Jun 5, 2026Jewon Yeom, Hanseul Kim, Jeongjae Park +3Video Representation Learning Predictive Representation Learning
Graduate School of Data Science, Seoul National University
Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
Jun 2, 2026Yonghao Yu, Lang Huang, Runyi Li +2Temporal Consistency in Video Generation Autoregressive Diffusion
The University of Tokyo · National Institute of Informatics · Peking University
GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
Jun 2, 2026Fulong Ma, Daojie Peng, Wenjun Yue +4World Models for Robotics Next-Action Prediction
HKUST(GZ) · OC · HKU +4
Exploratory Experience Shapes the Geometry of Predictive Representations
May 27, 2026Kseniia Shilova, Abdelrahman Sharafeldin, Advay Balakrishnan +1Computational Cognitive Modeling Predictive Coding
School of Mathematics Georgia Institute of Technology Atlanta, GA 30332 · College of Computing Georgia Institute of Technology Atlanta, GA 30332
Learn from your own latents and not from tokens: A sample-complexity theory
May 26, 2026Daniel J. Korchinski, Alessandro Favero, Matthieu WyartUnsupervised Learning Self-Supervised Learning
Department of Physics & Institute of Physics Johns Hopkins University & EPFL Baltimore, MD USA & Lausanne, VD Switzerland
On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series
May 25, 2026Sharmita Dey, Diego Paez-GranadosSelf-Supervised Time Series Representation Learning Gait Analysis
ETH Zurich, Zurich, Switzerland · Swiss Paraplegic Research, Nottwil
Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
May 23, 2026Mathis Immertreu, Achim Schilling, Thomas Kinfe +1Predictive Representation Learning Representation Geometry in Language Models
Cognitive Computational Neuroscience Group, Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nürnberg (FAU), Germany · Neuromodulation and Neuroprosthetics, University Hospital Mannheim, University Heidelberg, Germany · Neuroscience Lab, University Hospital Erlangen, Germany +1
Towards a General Intelligence and Interface for Wearable Health Data
May 21, 2026Girish Narayanswamy, Maxwell A. Xu, A. Ali Heydari +37Self-Supervised Pre-Training Clinical Outcome Prediction
1Google Research · University of Washington · University of Illinois Urbana-Champaign +2
PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment
May 17, 2026Michael Arbel, Basile Terver, Jean PonceUnsupervised Learning Visual Representation Learning
Univ. Grenoble Alpes, Inria · CNRS, Grenoble INP, LJK · Ecole Normale Supérieure / PSL +2
Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework
May 17, 2026Yumeng Yao, Jingzhi Dong, Haowen Gu +4Point Cloud Learning 3D ViTs
Nanjing University of Science and Technology · Hangzhou City University · East China Normal University +1
Constrained latent state modeling: A unifying perspective on representation learning under competing constraints
May 15, 2026Gwenolé QuellecRepresentation Learning Predictive Representation Learning
LaTIM UMR 1101, Inserm, Brest, 29200, France.
Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model
May 15, 2026Tianqiu Zhang, Muyang Lyu, Xiao Liu +1Computational Cognitive Modeling World Model Learning
Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Center of Quantitative Biology, School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University. · HHMI Janelia.
CSI-JEPA: Towards Foundation Representations for Ubiquitous Sensing with Minimal Supervision
May 13, 2026Xuanhao Luo, Zhizhen Li, Yuchen LiuSelf-Supervised Pre-Training Self-Supervised Time Series Representation Learning
North Carolina State University, USA
TRACE: Temporal Routing with Autoregressive Cross-channel Experts for EEG Representation Learning
May 12, 2026Fan Ma, Qier An, Peng Chen +6Representation Learning Self-Supervised Pre-Training
Department of Biomedical Informatics and Data Science, Yale University
Forecast-aware Gaussian Splatting for Predictive 3D Representation in Language-Guided Pick-and-Place Manipulation
May 11, 2026Kaixin Jia, Jiacheng Xu3D Gaussian Splatting 3D Scene Prediction
KTH Royal Institute of Technology, Stockholm, Sweden.
Clin-JEPA: A Multi-Phase Co-Training Framework for Joint-Embedding Predictive Pretraining on EHR Patient Trajectories
May 11, 2026Yixuan Yang, Mehak Arora, Ryan Zhang +10Self-Supervised Pre-Training Healthcare
Duke University, Durham, NC, USA
MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning
May 10, 2026Valliappan Chidambaram Adaikkappan, Sai Rajeswar, Pietro Mazzaglia +1Offline RL Predictive Representation Learning
Mila, McGill University · Qualcomm Research
Towards Effective Theory of LLMs: A Representation Learning Approach
May 10, 2026Muhammed Ustaomeroglu, Guannan QuLLM Interpretability Mechanistic Interpretability
Carnegie Mellon University Pittsburgh, PA 15213
3D MRI Image Pretraining via Controllable 2D Slice Navigation Task
May 7, 2026Yu Wang, Qingchao ChenVisual Representation Learning Self-Supervised Pre-Training
Beijing University of Posts and Telecommunications · Peking University
AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modeling
May 7, 2026Francisco Giral, Abhijeet Vishwasrao, Andrea Arroyo Ramo +8Joint-Embedding Predictive Architecture PDE Surrogate Modeling
Universidad Politécnica de Madrid · University of Michigan · Universitat Politècnica de València +3
The Predictive-Causal Gap: An Impossibility Theorem and Large-Scale Neural Evidence
May 6, 2026Kejun LiuCausal Representation Learning Predictive Representation Learning
State Key Laboratory of Bioinspired Interface Material Science, Institute of Nano & Functional Materials, Soochow University, Suzhou 215123, China
Understanding Self-Supervised Learning via Latent Distribution Matching
May 5, 2026Fabian A Mikulasch, Friedemann ZenkeRepresentation Learning Self-Supervised Learning
Friedrich Miescher Institute for Biomedical Research, 4056 Basel, Switzerland · Faculty of Science, University of Basel, 4033 Basel, Switzerland
Text-Conditional JEPA for Learning Semantically Rich Visual Representations
May 5, 2026Chen Huang, Xianhang Li, Vimal Thilak +2Joint-Embedding Predictive Architecture Predictive Representation Learning
Apple, Cupertino, United States
Latent State Design for World Models under Sufficiency Constraints
May 3, 2026Keon Woo KimRepresentation Learning Predictive Representation Learning
Om Labs
CGM-JEPA: Learning Consistent Continuous Glucose Monitor Representations via Predictive Self-Supervised Pretraining
May 1, 2026Hada Melino Muhammad, Zechen Li, Flora Salim +1Cross-Modal Representation Learning Self-Supervised Time Series Representation Learning
University of New South Wales · Google Research
Why Self-Supervised Encoders Want to Be Normal
Apr 30, 2026Yuval DombRepresentation Collapse Self-Supervised Learning
Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data
Apr 27, 2026K. Michael Martini, Eslam Abdelaleem, Paarth Gulati +1Dynamical Systems Information Bottleneck
Department of Physics, Emory University, Atlanta, GA, USA · Schools of Physics and Psychology, Georgia Institute of Technology, Atlanta, GA, USA · Initiative in Theory and Modeling of Living Systems, Emory University, Atlanta, GA, USA +1
A General Representation-Based Approach to Multi-Source Domain Adaptation
Apr 26, 2026Ignavier Ng, Yan Li, Zijian Li +3Multi-Source Domain Adaptation Unsupervised Domain Adaptation
Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence
Beyond Patient Invariance: Learning Cardiac Dynamics via Action-Conditioned JEPAs
Apr 24, 2026Jose Geraldo Fernandes, Luiz Facury, Pedro Robles Dutenhefner +1Predictive Representation Learning Healthcare
Department of Computer Science Universidade Federal de Minas Gerais Belo Horizonte, Brazil

Jun 5, 2026J/K move · Enter open · S save

Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning

Paarth Gulati, Ilya Nemenman

Department of Physics, and Initiative in Theory and Modeling of Living Systems, Emory University, Atlanta, GA

Contrastive Learning Representation Learning Predictive Representation Learning Shortcut Learning Temporal Representation Learning

Open paper AskPDF arXiv

Self-supervised methods that learn representations and predict dynamics fully in the latent space, such as JEPA, have been shown to confuse slowly varying noise with the dynamical signals they aim to capture. Specifically, when noise features remain approximately constant within each trajectory, contrastive predictive objectives preferentially encode these features instead of the true latent variables governing the system. The learned representation then becomes dominated by trajectory-specific noise, so downstream performance degrades with noise strength and does not improve even as the number and duration of training trajectories increase. We argue that this failure is a property of the objective itself, shared by a long line of contrastive predictive objectives that sample negatives across trajectories. To illustrate this generality, we study the failure mode and its remedy in two settings: a standard SimCLR-style JEPA on a synthetic moving-dot dataset, and DySIB, a recently introduced method designed for extracting physically interpretable representations of dynamics, on movies of a rigid-body pendulum. When negatives are instead sampled within a single trajectory, the slow noise can no longer distinguish frames within that trajectory, removing the predictive shortcut. Training one encoder simultaneously on many such trajectories then forces it to encode the variables relevant for the dynamics, with longer trajectories yielding better representations even for strong slow noise. Our results point toward principles for designing contrastive predictive objectives in dynamical representation learning, especially for physical systems with noisy experimental observations.