Overview

Representation collapse occurs when a model's features become too similar or occupy too few dimensions, making it harder to distinguish inputs or preserve useful information. It is studied in self-supervised vision and JEPA world models, as well as in transformers, recommendation, reinforcement learning, and continual learning. Researchers diagnose it with effective rank, feature variance, angular dispersion, and spectral or geometric analyses, and try to prevent it using covariance or isotropy regularization, contrastive objectives, and architectural changes to routing and residual connections. A central challenge is balancing diversity with useful invariance: high rank or good probe accuracy alone does not ensure that representations retain information needed for retrieval, transfer, or control.

Latest papers

CardsList

54 papers5 papers in the last four weeks, down 44% on the four weeks before. 0.0% of all new papers.

  1. Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning

    Nov 17, 2026M. Hadi Sepanj, Benyamin Ghojogh, Saed Moradi +1Self-Supervised LearningSelf-Supervised Visual Representation Learning

  2. SADL: On the Demand-Supply Geometry of Representation Collapse

    Nov 16, 2026Junhan Sun, Yigui Feng, Mingzhe Liu +7Representation CollapsePredictive Representation Learning

  3. Breaking feature collapse in self-supervised time series encoders

    Nov 16, 2026Özgün Turgut, Daniel RueckertSelf-Supervised Time Series Representation LearningRepresentation Collapse

  4. A phase diagram for training and inference in language models

    Nov 15, 2026Zhuo-Yang Song, Zeyu Li, Qing-Hong Cao +2Representation Geometry in Language ModelsTransformer Training Dynamics

  5. The Sense Paradigm for World Models

    Nov 15, 2026Haidong HuLatent World ModelsWorld Model-Based Planning

  6. Self-Supervised Collapse Prevention with Score-Based Entropy Maximization

    Nov 8, 2026Adarsh Kappiyath, Silpa Vadakkeeveetil Sreelatha, Xiatian Zhu +3Self-Supervised Visual Representation LearningRepresentation Collapse

  7. Motion-guided Contrastive Regularization For Latent World Models

    Nov 4, 2026Yiwen Chen, Qiqi Hou, Jisoo Jeong +3World Models for RoboticsLatent World Models

  8. LocaLeJEPA: Maximum Entropy Representations under Geometric Priors

    Oct 20, 2026Kartik Tandon, Alessio Borgi, Pavel Snopov +8Representation LearningJoint-Embedding Predictive Architecture

  9. Representations Have Trajectories: Characterizing Temporal Representation Dynamics in Neural Networks

    Oct 17, 2026Shivaprasad B. Gowda, Sparsh MittalLow-Rank Representation LearningNeural Representation Geometry

  10. Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures

    Oct 1, 2026José Lucas De Melo Costa, Seong Woo Ahn, Fabrice Popineau +2Joint-Embedding Predictive ArchitectureRepresentation Collapse

  11. CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization

    Sep 30, 2026Morgan Byrd, Robert Wright, Sehoon HaRepresentation LearningRepresentation Collapse

  12. Collapse, Not Invariance: Diagnosing Auxiliary Objectives in Speech Anti-Spoofing

    Sep 30, 2026Ksenia Lysikova, Kirill Borodin, Maxim Maslov +1Audio Deepfake DetectionRepresentation Collapse

  13. Stable Transformers for Graph Generation

    Sep 30, 2026Luca Miglior, Alessio Gravina, Davide BacciuRepresentation CollapseGraph Generation

  14. Generative Residual Factorization

    Sep 28, 2026Letian Gong, Yuzhou HongRepresentation CollapseLatent Variable Models

  15. The Price of Locality: Why Forward-Forward Underperforms Backpropagation?

    Sep 27, 2026Zhaoxian Wu, Haichuan Liu, Tianyi ChenForward-Forward LearningNeural Network Optimization

  16. AeroLat: Channel-Aware Latent Space Semantic Communication for Decentralized UAV Swarms

    Sep 15, 2026Rajdeep Ghosh, Goparaju Venkata Seshachala Sree Vatsava, Sudip MisraSemantic CommunicationWireless Communications

  17. When Graph-JEPA Learns the Wrong Thing: Diagnosing and Repairing Category-Conditional Collapse

    Aug 20, 2026Gollam Rabby, Sören AuerRepresentation CollapseJoint-Embedding Predictive Architecture

  18. Three Necessary Principles for Self-Supervised Visual Representation Learning

    Aug 8, 2026Nikos Giakoumoglou, Paschalis Giakoumoglou, Tania StathakiContrastive LearningRepresentation Collapse

  19. SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation

    Jul 27, 2026Yu Cui, Yi Xu, Jiahao Wang +6Representation CollapseTransformer

  20. The JEPA Paradox in Language: The Geometry of Linguistic Alternatives

    Jul 26, 2026Anh Trac Duc Dinh, Khang Nhat Hoang VoRepresentation CollapseMasked Language Modeling

  21. Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification

    Jul 16, 2026Patricia Medina, Hy P. G. LamRepresentation CollapseLatent Dynamics Modeling

  22. Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models

    Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Audio Representation LearningRepresentation Learning

  23. All Routes Lead to Collapse

    Jun 21, 2026K. R. BalasubramanianRepresentation CollapseAttention Sinks

  24. Representation Collapse in Sequential Post-Training of Large Language Models

    May 28, 2026Yichen Liu, Mingyu Chen, Hao Wang +7Continual Learning for LLMsRepresentation Collapse