Neural Collapse

Follow this topic Ask about this topic

Neural Collapse describes how late-training features can become compact within each class, arrange class means in a regular simplex-like pattern, and align classifier weights with those means. Researchers study when this geometry emerges using theoretical feature models, training-dynamics analyses, and diagnostics across optimizers, architectures, and data regimes. Related methods build prototype-based or contrastive losses around this geometry, with applications including long-tailed classification, few-shot learning, and test-time adaptation. A central open tension is whether collapse is a general property of trained networks or is altered or prevented by factors such as optimization, class imbalance, task structure, and the need to retain contextual information.

Latest papers

21 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning
Oct 17, 2026Berk SezerContinual Learning Class-Incremental Learning
Middle East Technical University, Türkiye
A Closed Form Depth-Width Law for Neural Collapse in Deep Linear Networks
Oct 15, 2026Pankhil Gawade, Kirsten Fischer, Guy Wolf +1Neural Representation Geometry Neural Scaling Laws
Institute of AI for Health, Helmholtz Munich · School of Computation, Information and Technology, Technical University of Munich · Université de Montréal +3
Reliable Neural Collapse Approximation for Open-World Test-Time Adaptation
Aug 20, 2026Jia-Qi Lin, Yuangang Pan, Chang-Dong Wang +3Test-Time Adaptation Neural Collapse
Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632 · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China, Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China, and Guangdong Province Key Laboratory of Computational Science, Guangzhou, China · School of Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai, China
Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergence
Jul 14, 2026Blanca Cano-Camarero, Ángela Fernández-Pascual, José R. DorronsoroGeometric Representation Learning Neural Collapse
1Departamento de Ingeniería Informática, Universidad Autónoma de Madrid, Calle Francisco Tomás y Valiente, 11, 28049, Madrid, España
Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse
Jul 12, 2026Truong Xuan Khanh, Phan Thanh DucNeural Representation Geometry Neural Collapse
H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam
Neural Collapse Is Forbidden: Information Floors in Language Models
Jul 10, 2026Bruno AbrahaoNeural Collapse Representation Geometry in Language Models
NYU Shanghai · Leonard N. Stern School of Business, New York University
Ordinal Neural Collapse as a Representation Prior for Visual Navigation
Jun 25, 2026E-In Son, Jung-Taak Kim, Seung-Woo SeoRepresentation Learning Robot Navigation
Dept. of ECE & ASRI, Seoul National University, Korea
Structure Before Collapse: Transient semantic geometry in next-token prediction
Jun 25, 2026Yize Zhao, Isabel Papadimitriou, Christos ThrampoulidisAutoregressive Language Modeling Next-Token Prediction
Department of Electrical and Computer Engineering, The University of British Columbia · Department of Linguistics, The University of British Columbia
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Jun 8, 2026Hu Tan, Kuo Gai, Shihua ZhangNeural Representation Geometry Classification
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China; and Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Hangzhou 310024, China
Optical-Guided Neural Collapse for SAR Few-Shot Class Incremental Learning
Jun 3, 2026Fan Zhang, Sijin Zheng, Fei Ma +4Class-Incremental Learning Remote Sensing
School of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China · School of Information Science and Technology and the Interdisciplinary Research Center for Artificial Intelligence, Beijing University of Chemical Technology, Beijing 100029, China · Beihang University, Beijing 100191, China +1
The role of class encoding in neural collapse
May 29, 2026Bastien Massion, Roy Makhlouf, Estelle MassartNeural Representation Geometry Multiclass Classification
ICTEAM, UCLouvain Louvain-la-Neuve, Belgium
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
May 21, 2026Connall Garrod, Jonathan P. Keating, Christos ThrampoulidisDeep Linear Networks Implicit Bias
Mathematical Institute, University of Oxford · Department of Electrical and Computer Engineering, University of British Columbia
Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
May 19, 2026Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1Prototype-Based Classification Hyperspherical Representation Learning
University of Athens · NSCR Demokritos · University of Cyprus +2
Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets
May 11, 2026Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1Class-Imbalanced Learning Contrastive Learning
Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute +1
Flag Varieties: A Geometric Framework for Deep Network Alignment
May 11, 2026Jingchuan Xiao, Xinyi Sui, Cihan RuanNeural Representation Geometry Neural Collapse
Department of Mathematics and Computer Studies, Mary Immaculate College, Ireland · Department of Computer Science and Engineering, Santa Clara University, USA
How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
May 3, 2026Xiaoxuan Ma, Yixuan Yang, Song Li +1Class-Imbalanced Learning Multi-Label Classification
Department of Mathematics, Zhejiang University, Hangzhou, China.
Neural Collapse Dynamics: Depth, Activation, Regularisation, and Feature Norm Threshold
Mar 31, 2026Anamika Paul RupaNeural Collapse Neural Network Training Dynamics
Department of Electrical Engineering and Computer Science, Howard University, Washington, DC, 20059, USA
Neural collapse in the orthoplex regime
Mar 21, 2026James Alcala, Rayna Andreeva, Vladimir A. Kobzar +4Neural Collapse Neural Representation Geometry
Department of Mathematics, University of Southern California, Los Angeles, CA · School of Informatics, University of Edinburgh, Edinburgh, Scotland · Department of Mathematics, The Ohio State University, Columbus, OH +3
Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning
Mar 3, 2026Achleshwar Luthra, Yash Salunkhe, Tomer GalantiNeural Collapse Transfer Learning
Department of Computer Science & Engineering, Texas A&M University.
Optimizer choice matters for the emergence of Neural Collapse
Feb 18, 2026Jim Zhao, Tin Sum Cheng, Wojciech Masarczyk +1Neural Collapse Neural Network Training Dynamics
University of Basel · Warsaw University of Technology IDEAS Research Institute
Equilibrium contrastive learning for imbalanced image classification
Feb 10, 2026Sumin Roh, Harim Kim, Ho Yun Lee +1Class-Imbalanced Learning Supervised Contrastive Learning
Department of Electrical and Computer Engineering (ECE), Sungkyunkwan University (SKKU), Suwon 16419, South Korea · Department of Radiology, Samsung Medical Center (SMC), Seoul 06351, South Korea · Department of Radiology, SMC, Seoul 06351, South Korea +3

Oct 17, 2026J/K move · Enter open · S save

An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning

Berk Sezer

Middle East Technical University, Türkiye

Continual Learning Class-Incremental Learning Neural Collapse Class-Imbalanced Learning

Open paper AskPDF arXiv

Analytic continual learning (ACL) methods such as ACIL and GACL fit a ridge-regression classifier head on frozen features via the Sherman-Morrison-Woodbury identity, achieving exemplar-free, exact updates. A separate line of work, Progressive Neural Collapse (ProNC), shows standard continual classifiers benefit from a progressively expanding Simplex Equiangular Tight Frame (ETF) target. It is natural to ask whether fusing the two ideas, exact analytic updates and ETF targets, would yield a better closed-form continual learner. We show this is not the case as the fusion turns out to be identical to plain one-hot ACIL. Further, Simplex ETFs are not nested across class counts, so the literal "progressively expand the ETF" construction stops yielding an ETF as new tasks arrive, and it introduces catastrophic forgetting (

4\times

that of a classic ACIL head). We therefore separate an ETF's two defining properties, equiangularity and equinormality, and apply each on its own to the classifier weights rather than the targets, where the invariance above does not hold. Equinorm post-processing of the fitted weights recovers both tail-class and overall average accuracy across the long-tailed regime from 5:1 to 100:1 imbalance (e.g. +32.0 tail-accuracy points and +16.0 average-accuracy points at 100:1 on CIFAR-100/ViT), transferring in sign across two backbones and two datasets, at a mild cost of 5.4 average-accuracy points on CIFAR-100/ViT on near-balanced streams (below

\approx 3{:}1

). Equiangular geometry, by contrast, is exactly inert on the target side but actively harmful on the weight side, usually losing to plain one-hot. Hence, the generally useful part of Neural Collapse geometry in Analytic Continual Learning is equinorm post-processing of the weights, becoming more useful as class imbalance grows; and equiangularity is at best a no-op and induces catastrophic forgetting at worst.