Predictive Representation Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 156% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 116

All topics

CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining
Jul 5, 2026Jingyu Song, Yi Liu, Katherine A. SkinnerCamera-Radar Fusion World Model Learning
Department of Robotics, University of Michigan, Ann Arbor, MI 48109 USA
Mask-based Predictive Representations for Reinforcement Learning
Jul 5, 2026Kai ZhaoRepresentation Learning Predictive Representation Learning
School of Systems Science, Beijing Normal University · Beijing, China
SiamJEPA: On the Role of Siamese Student Encoders in JEPA
Jul 4, 2026Makoto YamadaJoint-Embedding Predictive Architecture Siamese Neural Networks
Okinawa Institute of Science and Technology
Latent Clarity: Bridging World-Model Kinematics to Semantic Manifolds for Video Anomaly Anticipation
Jul 3, 2026Abu Anas Ibn SamadWorld Models Cross-Modal Alignment
Sigmind.ai Dhaka 1215, Bangladesh
Fourier Preconditioning for Neural Feature Learning
Jul 2, 2026Preston Pitzer, Anish Pradhan, Harpreet S. DhillonFrequency-Domain Feature Learning Predictive Representation Learning
Rhythm-Structured Predictive Learning for Remote Photoplethysmography
Jun 30, 2026Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo +1Remote Photoplethysmography Mamba
VNU University of Engineering and Technology, Hanoi, Vietnam
OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL
Jun 29, 2026Karl El Hajal, Mathew Magimai. -DossPredictive Representation Learning Self-Supervised Speech Representation Learning
Idiap Research Institute, Switzerland · EPFL, Switzerland
FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks
Jun 29, 2026Marek Polewczyk, Maximilian Schambach, Marco Spinaci +2Tabular Foundation Models Tabular Anomaly Detection
1SAP SE · 2SAP France
Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study
Jun 29, 2026Ayan PendharkarRepresentation Learning Joint-Embedding Predictive Architecture
ScaleAware-JEPA: Latent Representation for Discovery in Multiscale Physical Fields
Jun 29, 2026Guang-Xing LiRepresentation Learning Joint-Embedding Predictive Architecture
Contrastive Predictive Coding with Compression for Enhanced Channel State Feedback in Wireless Networks
Jun 26, 2026Ahmed Y. Radwan, Fahad Syed Muhammad, Matthew Baker +1Wireless Communications Predictive Representation Learning
Department of Electrical Engineering and Computer Science, York University, Toronto, ON M3J 1P3, Canada · Nokia Networks, 12 rue Jean Bart, Paris-Saclay, Massy, 91300, France · Nokia UK, Broers Building, 21 JJ Thomson Avenue, Cambridge, CB3 0FA, United Kingdom
Semantic Allocation in Ordered Bottlenecks: Predictive Residual Inference for Visual Representation Learning
Jun 23, 2026Erik Ayari, Manuel Traub, Martin V. ButzRepresentation Learning Visual Representation Learning
Neuro-Cognitive Modeling Group, University of Tübingen, Tübingen, Germany
Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging
Jun 12, 2026Haoxu Huang, Long Chen, Jingyun Chen +8Medical Imaging Foundation Models Joint-Embedding Predictive Architecture
New York University, Center for Data Science, New York, NY, 10001, USA · NYU Grossman School of Medicine, Department of Radiology, New York, NY, 10016, USA · State University of New York at Binghamton, School of Computing, Binghamton, NY 13902, USA +7
FLaRA: Predicting Future Latent Representations for Accident Anticipation
Jun 12, 2026Lorenzo Caselli, Tomaso Trinci, Tommaso Bianconcini +4Autonomous Driving Traffic Accident Anticipation
Department of Information Engineering, University of Florence, Italy. · 2Verizon Connect, Florence, Italy.
RePAIR: Predictive Self-Supervised Representation Learning in Chess
Jun 10, 2026Christoph Koller, Johannes Fürnkranz, Timo BertramMasked Autoencoders Joint-Embedding Predictive Architecture
JKU Linz, Austria
DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction
Jun 8, 2026Reazul Hasan Russel, Mingwei Tang, Rostam Shirani +18Learning to Rank User Modeling
AI at Meta
Momentum-Guided Semantic Forecasting (MoFore) for Self-Supervised Video Representation Learning
Jun 8, 2026Qinwu XuVideo Understanding Video Representation Learning
∗This work was independently developed during Early Feb –Mid April 2026.
Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning
Jun 5, 2026Paarth Gulati, Ilya NemenmanContrastive Learning Representation Learning
Department of Physics, and Initiative in Theory and Modeling of Living Systems, Emory University, Atlanta, GA
What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction
Jun 5, 2026Jewon Yeom, Hanseul Kim, Jeongjae Park +3Video Representation Learning Predictive Representation Learning
Graduate School of Data Science, Seoul National University
Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
Jun 2, 2026Yonghao Yu, Lang Huang, Runyi Li +2Video Diffusion Models Autoregressive Diffusion
The University of Tokyo · National Institute of Informatics · Peking University
GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
Jun 2, 2026Fulong Ma, Daojie Peng, Wenjun Yue +4World Models for Robotics Next-Action Prediction
HKUST(GZ) · OC · HKU +4
Exploratory Experience Shapes the Geometry of Predictive Representations
May 27, 2026Kseniia Shilova, Abdelrahman Sharafeldin, Advay Balakrishnan +1Computational Cognitive Modeling Predictive Coding
School of Mathematics Georgia Institute of Technology Atlanta, GA 30332 · College of Computing Georgia Institute of Technology Atlanta, GA 30332
Learn from your own latents and not from tokens: A sample-complexity theory
May 26, 2026Daniel J. Korchinski, Alessandro Favero, Matthieu WyartUnsupervised Learning Self-Supervised Learning
Department of Physics & Institute of Physics Johns Hopkins University & EPFL Baltimore, MD USA & Lausanne, VD Switzerland
On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series
May 25, 2026Sharmita Dey, Diego Paez-GranadosSelf-Supervised Time Series Representation Learning Gait Analysis
ETH Zurich, Zurich, Switzerland · Swiss Paraplegic Research, Nottwil
Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
May 23, 2026Mathis Immertreu, Achim Schilling, Thomas Kinfe +1Predictive Representation Learning Representation Geometry in Language Models
Cognitive Computational Neuroscience Group, Friedrich-Alexander-Universität Erlangen–Nürnberg (FAU), Germany. · Mannheim Center for Neuromodulation and Neuroprosthetics (MCNN), University Hospital Mannheim, University Heidelberg, Germany.
Towards a General Intelligence and Interface for Wearable Health Data
May 21, 2026Girish Narayanswamy, Maxwell A. Xu, A. Ali Heydari +37Self-Supervised Pre-Training Clinical Outcome Prediction
1Google Research · University of Washington · University of Illinois Urbana-Champaign +2
PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment
May 17, 2026Michael Arbel, Basile Terver, Jean PonceUnsupervised Learning Visual Representation Learning
Univ. Grenoble Alpes, Inria · CNRS, Grenoble INP, LJK · Ecole Normale Supérieure / PSL +2
Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework
May 17, 2026Yumeng Yao, Jingzhi Dong, Haowen Gu +43D ViTs Self-Supervised Pre-Training
Nanjing University of Science and Technology · Hangzhou City University · East China Normal University +1
Constrained latent state modeling: A unifying perspective on representation learning under competing constraints
May 15, 2026Gwenolé QuellecRepresentation Learning Predictive Representation Learning
LaTIM UMR 1101, Inserm, Brest, 29200, France.
Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model
May 15, 2026Tianqiu Zhang, Muyang Lyu, Xiao Liu +1Computational Cognitive Modeling World Model Learning
Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Center of Quantitative Biology, School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University. · HHMI Janelia.

Jul 5, 2026, cs.CVJ/K move · Enter open · S save

CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining

Jingyu Song, Yi Liu, Katherine A. Skinner

Department of Robotics, University of Michigan, Ann Arbor, MI 48109 USA

Camera-Radar Fusion World Model Learning Autonomous Driving Predictive Representation Learning

Open paper AskPDF arXiv

Camera-radar (CR) fusion is a practical sensing configuration for autonomous driving, but existing models are typically trained with task-specific supervision, limiting reusable representation learning. We present CRISP, a spatiotemporal CR backbone pretrained through forecasting-based representation learning. Given historical multi-view images and radar sweeps, CRISP learns a unified bird's-eye-view (BEV) representation by predicting future LiDAR point clouds. LiDAR is used only as privileged supervision during pretraining; the deployed model requires only camera and radar. To make forecasting-based pretraining effective for CR fusion, CRISP introduces an enhanced radar encoder, radar-enhanced temporal self-attention, and multimodal feature rendering with modality innovation gating. These components inject radar range and Doppler cues into BEV temporal propagation and allow BEV tokens to selectively incorporate camera and radar evidence. Experiments on nuScenes show that CRISP improves long-horizon point cloud forecasting and transfers effectively to downstream tasks, including 3D detection, tracking, online mapping, motion forecasting, future occupancy prediction, and planning, suggesting that predictive CR pretraining is a promising path toward scalable driving representations under practical sensor configurations. The project website is https://umfieldrobotics.github.io/CRISP.

Momentum

Related topics

Latest papers 116