Representation Collapse

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets
Sep 29, 2026Gaspard Botté, Séverin Baroudi, Samir Sadok +5Discrete Speech Representations Speech Encoder
Concordia University · Mila – Qu´ebec AI Institute · Univ Toulon, Aix Marseille Univ, CNRS, LIS +4
Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction
Sep 28, 2026Burc GokdenInference Dynamics Renormalization Group
Fromthesky Research Labs LLC, Oregon, USA
Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure
Sep 20, 2026Shuyang XiangRotary Position Embedding Hierarchical
Comfort by Construction: Adaptive, Comfort-Bounded Action Spaces for Learned Driving Policies
Sep 14, 2026Anna Rothenhäusler, Daniel Jost, Raghu Rajan +4Autonomous Driving Simulation Human Driving
University of Freiburg, Freiburg, Germany · Bosch Center for Artificial Intelligence, Stuttgart, Germany · Coburg University of Applied Sciences, Coburg, Germany
GEOSTEER: Geodesic Optimization for Activation Steering in Large Language Models
Sep 11, 2026Xuan Cuong Ngo, Hao Vo, Ngan LeLinear Activation Steering Model Activations
University of Arkansas, Fayetteville, Arkansas, USA
Marginal Fidelity Does Not Establish User Simulation in Demographic Synthetic Survey Panels: Response Contracts, Support Collapse and Conditioning Failure
Sep 7, 2026Alexander DoudkinSurvey Elicitation
Minds AI Labs, Inc.
Spectral-Target Physical Latent Structuring for JEPA-Style World Models
Sep 2, 2026Penghao Zhu, Salvatore Penachio, Kaustav Mukherjee +1Latent World Models World Models
Kaliber Labs San Francisco, CA 94103, USA
$β$ -VAEs as Effective Theories: Tolerance-Dependent Dimension
Aug 11, 2026Johannes HirnRepresentation Collapse Nonlinearity
Image Processing Laboratory (IPL), Universitat de València, Paterna, València 46980, Spain
SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors
Aug 4, 2026Yongchao HuangJoint-Embedding Predictive Architectures Latent Dynamics
BiKAN: Restoring Collapsed Basis of Binary Kolmogorov--Arnold Networks
Aug 2, 2026Kazi Ahmed Asif Fuad, Lizhong ChenKolmogorov-Arnold Networks Orthogonality
Department of EECS Oregon State University Corvallis, OR 97331
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
Jul 31, 2026Atul Anand, Sourav ChattarajInstruction-Tuned Models Instruction
Temporally Centered SIGReg Improves Multi-Task LeWorldModel Learning: From Analysis to Method
Jul 29, 2026Chang Liu, Fei Suo, Yanzhou Jin +3Multi--Task Learning Latent World Models
Graduate School of Engineering, The University of Tokyo, Tokyo, Japan
ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games
Jul 29, 2026Zhenfeng Gan, ZiTong Zeng, Jiajun Cheng +3Latent World Models Long-Horizon Planning
Is Progressive Disclosure All You Need for Long-Context Agents?
Jul 20, 2026Yifeng He, Yinzhe Zhao, Jicheng Wang +1Disclosures Representation Collapse
University of California, Davis · Zhejiang University · The University of Hong Kong
Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse
Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarRectified Linear Unit Networks Stochastic Gradient Descent
Department of Statistics & Data Science University of California, Los Angeles Los Angeles, CA 90095, USA · CISPA Helmholtz Center for Information Security · Departments of Mathematics and Statistics & Data Science University of California, Los Angeles; and Max Planck Institute for Mathematics in the Sciences
RaMem: Contextual Reinstatement for Long-term Agentic Memory
Jun 22, 2026Wei Yang, Bryce Kan, Shixuan Li +5Agentic Memory Long-Term
University of Southern California
All Routes Lead to Collapse
Jun 21, 2026K. R. BalasubramanianRepresentation Collapse Collapse
Independent Research
Does Text Actually Help? Uncovering and Resolving Text Collapse in Multimodal Time Series Forecasting
Jun 17, 2026Huu Hiep Nguyen, Minh Hoang Nguyen, Dung Nguyen +1Time Series Forecasting Representation Collapse
Applied Artificial Intelligence Initiative Deakin University Geelong, Australia
PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalization
Jun 11, 2026Hao Jiang, Xin Li, Annan Wang +4Representation Collapse Efficient Long-Context Inference
1Nanyang Technological University · 2Peking University · 3Independent Researcher
Overcoming Rank Collapse in Feedback Alignment
Jun 9, 2026Gauthier Boeshertz, Razvan Pascanu, Claudia ClopathBackpropagation Orthogonality
Department of Bioengineering Imperial College London · Mila
Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks
Jun 9, 2026Parviz Haggi-Mani, Irina RishResidual Networks Renormalization Group
Universit´e de Montr´eal · Mila - Quebec AI Institute
Unlocking Latent Value: Taxonomy-Guided Recovery of High-Performing Data from Low-Tier Web Corpora
Jun 5, 2026Neeraj Varshney, Sanket Lokegaonkar, Nasser Zalmout +3Taxonomy Data-Curation
Amazon
Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning
Jun 4, 2026Ayushman Trivedi, Bhavika MelwaniCatastrophic Forgetting Continual Learning
Independent Researchers
Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration
Jun 2, 2026Achraf Ben AhmedRemote Photoplethysmography Cardiovascular Risk Prediction
PlesmoSense SARL
Generating Reports or Repeating Templates? Measuring and Mitigating Template Collapse in 3D CT Report Generation
May 29, 2026Tom Maye-Lasserre, Yitong Li, Bailiang Jian +3Radiology Report Generation Report
Technical University of Munich (TUM), Germany · TUM Hospital, Germany · Munich Center for Machine Learning (MCML), Germany
Representation Collapse in Sequential Post-Training of Large Language Models
May 28, 2026Yichen Liu, Mingyu Chen, Hao Wang +7Post-Training Representation Collapse
Hangzhou Dianzi University · Zhejiang Gongshang University · Ningbo University +1
Semantic Flow Regularization: Teaching LLMs to Generate Diverse Yet Coherent Responses
May 27, 2026Kerui Peng, Feifei Li, Xingyu Fan +1Large Language Model Generation Responses
WeChat, Tencent Inc., Beijing, China
What Makes a Medical Checker Trainable? Diagnosing Signal Collapse and Reward Hacking in Checker-Guided RAG for Biomedical QA
May 25, 2026Yuelyu Ji, Min Gu Kwak, Hang Zhang +3Clinical Reasoning Training Natural Language Inference
University of Pittsburgh, Pittsburgh, PA, USA
Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents
May 25, 2026Dong-Hee Kim, Reuben Tan, Donghyun KimVisual Reasoning Chain-of-Thought Reasoning
Department of Artificial Intelligence, Korea University, Seoul, South Korea · 2Microsoft Research, Redmond, USA.
A Mechanistic Study of Tabular Foundation Models
May 20, 2026Marin Biloš, James T. Wilson, Anderson Schneider +1Tabular Foundation Models Inductive Bias
Machine Learning Research, Morgan Stanley
A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits
May 19, 2026Yuyang Zhang, Yifu Zhang, Xuehai Zhou +1Out-Of-Distribution LLM Reasoning Strategies
McGill University · Mila - Quebec AI Institute · Université de Montréal
Persona-Model Collapse in Emergent Misalignment
May 13, 2026Davi Bastos Costa, Renato VicenteEmergent Misalignment Large Language Model Alignment
TELUS Digital Research Hub · Center for Artificial Intelligence and Machine Learning · Institute of Mathematics, Statistics and Computer Science +1
Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
May 10, 2026Jeonseong KimDiffusion Language Models Iterative Denoising Methods
Layer Collapse in Diffusion Language Models
May 7, 2026Alexander Conzelmann, Albert Catalan-Tatjer, Shiwei LiuDiffusion Language Models Large Language Model Compression
Tübingen AI Center · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen
Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs
May 6, 2026Aofan Liu, Jingxiang MengLarge Language Models Fail Collapse
1Peking University. · University of Chicago.
When Attention Collapses: Residual Evidence Modeling for Compositional Inference
May 4, 2026Niklas HoubaIncomplete Evidence Representation Collapse
Institute for Particle Physics and Astrophysics ETH Zurich
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Emotion Recognition Multimodal Large Language Models
School of Future Technology, South China University of Technology, Guangzhou, China
Escaping Mode Collapse in LLM Generation via Geometric Regulation
May 1, 2026Xin Du, Kumiko Tanaka-IshiiLarge Language Model Generation Collapse
Department of Communications and Computer Engineering, Waseda University, Tokyo, Japan · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai, China · Department of Computer Science and Engineering, Waseda University, Tokyo, Japan
Why Self-Supervised Encoders Want to Be Normal
Apr 30, 2026Yuval DombSelf-Supervised Representations Information Bottleneck
Auditing Frontier Vision-Language Models for Trustworthy Medical VQA: Grounding Failures, Format Collapse, and Domain Adaptation
Apr 30, 2026Xupeng Chen, Binbin Shi, Chenqian Le +5Medical Vision-Language Models Medical Visual Question Answering
New York University, New York, USA · Tsinghua University, Beijing, China · Columbia University, New York, USA +1
Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
Apr 26, 2026Giansalvo CirrincioneMulti-Head Attention Transformer Architectures
aLaboratoire LTI, Université de Picardie Jules Verne, Chemin du Thil, 80025 Amiens, France
Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs
Jan 10, 2026Yutong Song, Jiang Wu, Shaofan Yuan +5Large Language Model Personalization Personalization
University of California, Irvine · Independent Researcher · TikTok
Pushing Toward the Simplex Vertices: A Simple Remedy for Code Collapse in Smoothed Vector Quantization
Sep 26, 2025Takashi MoritaResidual Vector Quantization Quantizer
Academy of Emerging Sciences, Chubu University · Institute for Advanced Research, Nagoya University
You Do Not Fully Utilize Transformer's Representation Capacity
Feb 13, 2025Gleb Gerasimov, Yaroslav Aksenov, Nikita Balagansky +2Transformer Architectures Language Modeling
1T-Tech · 2Moscow Institute of Physics and Technology · 3HSE University

Sep 29, 2026, eess.ASJ/K move · Enter open · S save

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets

Gaspard Botté, Séverin Baroudi, Samir Sadok, Francesco Paissan+4

Concordia University · Mila – Qu´ebec AI Institute · Univ Toulon, Aix Marseille Univ, CNRS, LIS+4

Discrete Speech Representations Speech Encoder Self-Supervised Learning Exponential Moving Average Representation Collapse

Open paper AskPDF arXiv

Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed prediction targets. We challenge this necessity with GLaS-JEPA, a framework that directly predicts the current encoder's continuous representations at masked positions, without contrastive learning, discrete targets, or separate EMA target encoders. We prevent representation collapse using SIGReg representation-space regularization, eliminating the need for engineered target-generation mechanisms. Pretrained on 960 hours of LibriSpeech, our 57M-parameter model achieves a 6.89% WER on frozen-encoder SUPERB ASR and a 25.87% CER on slot filling, outperforming the best non-distilled sub-90M baselines by 43.1% and 22.0%, respectively. These results demonstrate that highly competitive speech representations can emerge from a radically simplified training recipe.

Momentum

Related topics

Latest papers 44