Representation Collapse

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 27

All topics

CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization
Sep 30, 2026Morgan Byrd, Robert Wright, Sehoon HaRepresentation Learning Representation Collapse
Georgia Institute of Technology, Atlanta, GA, 30308, USA · Georgia Tech Research Institute, Atlanta, GA, 30308, USA
Collapse, Not Invariance: Diagnosing Auxiliary Objectives in Speech Anti-Spoofing
Sep 30, 2026Ksenia Lysikova, Kirill Borodin, Maxim Maslov +1Audio Deepfake Detection Representation Collapse
Lab260, Yerevan, Armenia · BitmanagerAI, Dubai, UAE · MTUCI, Moscow, Russia
Stable Transformers for Graph Generation
Sep 30, 2026Luca Miglior, Alessio Gravina, Davide BacciuRepresentation Collapse Graph Generation
Department of Computer Science, University of Pisa, Italy
Generative Residual Factorization
Sep 28, 2026Letian Gong, Yuzhou HongRepresentation Collapse Latent Variable Models
Zhejiang University of Science and Technology · Zhejiang Sci-Tech University
Three Necessary Principles for Self-Supervised Visual Representation Learning
Aug 8, 2026Nikos Giakoumoglou, Paschalis Giakoumoglou, Tania StathakiContrastive Learning Representation Collapse
Imperial College London · CERTH, ITI
SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation
Jul 27, 2026Yu Cui, Yi Xu, Jiahao Wang +6Representation Collapse Transformer
Zhejiang University Hangzhou, China · Alibaba Group Beijing, China
The JEPA Paradox in Language: The Geometry of Linguistic Alternatives
Jul 26, 2026Anh Trac Duc Dinh, Khang Nhat Hoang VoRepresentation Collapse Masked Language Modeling
Center for AI Research (CAIR), VinUniversity, Hanoi, Vietnam · Faculty of Computer Science and Engineering Ho Chi Minh City University of Technology (HCMUT), VNU-HCM Ho Chi Minh City, Vietnam · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates
Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification
Jul 16, 2026Patricia Medina, Hy P. G. LamRepresentation Collapse Latent Dynamics Modeling
Department of Mathematics New York City College of Technology, CUNY Brooklyn, NY, USA · The Graduate Center, CUNY New York, NY, USA · Department of Mathematical Sciences Worcester Polytechnic Institute Worcester, MA, USA
How to Tame Grokking: Representation Geometry as a Control Signal
Jul 13, 2026Maksim A KazanskiiNeural Network Generalization Spectral Regularization
Independent Researcher
Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models
Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Audio Representation Learning Representation Learning
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · ASUS AICS, Taipei, Taiwan
All Routes Lead to Collapse
Jun 21, 2026K. R. BalasubramanianRepresentation Collapse Attention Sinks
Independent Research
Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration
Jun 2, 2026Achraf Ben AhmedRemote Photoplethysmography Representation Collapse
PlesmoSense SARL
VISReg: Variance-Invariance-Sketching Regularization for JEPA training
Jun 1, 2026Haiyu Wu, Randall Balestriero, Morgan LevineRepresentation Collapse Joint-Embedding Predictive Architecture
Altos Labs · Brown University
Representation Collapse in Sequential Post-Training of Large Language Models
May 28, 2026Yichen Liu, Mingyu Chen, Hao Wang +7Continual Learning for LLMs Representation Collapse
Hangzhou Dianzi University · Zhejiang Gongshang University · Ningbo University +1
Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation
May 22, 2026Guoming Li, Shangyu Zhang, Junwei Pan +7Representation Collapse Recommender Systems
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Tencent Inc. Shenzhen, China
How Well Do Large Language Models Capture Human Personality?
May 12, 2026Aanisha Bhattacharyya, Yaman Kumar Singla, Rajiv Ratn Shah +2LLM Evaluation Representation Collapse
Adobe Media and Data Science Research (MDSR) · IIIT-Delhi · SUNY at Buffalo
Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets
May 11, 2026Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1Class-Imbalanced Learning Contrastive Learning
Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute +1
Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models
May 10, 2026Kai Zhao, Dongliang Nie, Yuchen Lin +4Representation Learning Representation Collapse
1Shanghai University · 2The University of Manchester · 3Nankai University
Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis
May 8, 2026Ozgu Goksu, Nicolas PugeaultRepresentation Collapse Non-IID Federated Learning
School of Computing Science University of Glasgow
Mean Mode Screaming: Mean--Variance Split Residuals for 1000-Layer Diffusion Transformers
May 7, 2026Pengqi LuDeep Learning Optimization Representation Collapse
Beijing, China
Lost in State Space: Probing Frozen Mamba Representations
Apr 30, 2026Bhagyashree Wagh, Akash SinghRepresentation Geometry Representation Collapse
University of Washington · University of Florida
Why Self-Supervised Encoders Want to Be Normal
Apr 30, 2026Yuval DombRepresentation Collapse Self-Supervised Learning
Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index
Apr 28, 2026Alexander KalinowskiRepresentation Collapse Neural Representation Geometry
SUNY Empire
Complexity of Linear Regions in Self-supervised Deep ReLU Networks
Apr 27, 2026Mufhumudzi Muthivhi, Terence L. van ZylRepresentation Collapse Self-Supervised Visual Representation Learning
University of Johannesburg, Johannesburg, South Africa
Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
Apr 26, 2026Giansalvo CirrincioneTransformer Interpretability Representation Collapse
aLaboratoire LTI, Université de Picardie Jules Verne, Chemin du Thil, 80025 Amiens, France
Hypersolid: Emergent Vision Representations via Short-Range Repulsion
Jan 29, 2026Esteban Rodríguez-Betancourt, Edgar Casasola-MurilloUnsupervised Learning Representation Learning
Posgrado en Computación e Informática, Universidad de Costa Rica, San José, Costa Rica · Escuela de Ciencias de la Computación e Informática, Universidad de Costa Rica, San José, Costa Rica
You Do Not Fully Utilize Transformer's Representation Capacity
Feb 13, 2025Gleb Gerasimov, Yaroslav Aksenov, Nikita Balagansky +2Representation Collapse Memory-Augmented Language Models
1T-Tech · 2Moscow Institute of Physics and Technology · 3HSE University

Sep 30, 2026, cs.ROJ/K move · Enter open · S save

CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization

Morgan Byrd, Robert Wright, Sehoon Ha

Georgia Institute of Technology, Atlanta, GA, 30308, USA · Georgia Tech Research Institute, Atlanta, GA, 30308, USA

Representation Learning Representation Collapse Latent World Models Vision-Based Robot Control

Open paper AskPDF arXiv

Controlling an agent with vision requires being able to separate useful information from irrelevant background information. JEPA-style latent world models seem like a natural approach for this, as they do not perform pixel-level reconstruction; however, they are still sensitive to these distractor signals and experience latent collapse. In this work, we introduce Controllability Factorized JEPA (CF-JEPA), a JEPA-style world model which splits the latent space into controllable and uncontrollable subspaces. This factorization allows us to capture all the distractor information into the uncontrollable region, while we use the control-relevant latent information for our task. With this, we show comparable performance across 2D and 3D control tasks under nominal conditions and improved performance under distracted conditions, where CF-JEPA is the only model that does not experience latent collapse. We also validate our model under distracted conditions for a simulated robot task, highlighting the practical application of such a scheme.

Momentum

Related topics

Latest papers 27