Neural Collapse

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 33

All topics

Generalization in Neural Networks Through the Lens of Magnitude Potential
Oct 1, 2026Sahel Torkamani, Henry Gouk, Rik SarkarNeural Collapse Neural Network
School of Informatics University of Edinburgh
Feature Selective Model Collapse in Diffusion Models: Total Replacement versus Fixed-Budget Training
Oct 1, 2026Hanna Malet, Gabriel TuriniciDiffusion Models Collapse
CEREMADE CNRS Université Paris Dauphine - PSL Paris 75116 FRANCE · Cornell University Ithaca, NY, USA
Hidden not Deleted: How Networks Suppress Entangled Features
Sep 23, 2026Akash Samanta, Manish Pratap Singh, Debasis ChaudhuriConcept Erasure Entanglement
Techno India University Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT Chennai, India
Information Geometric Self-Organization at the Edge of Stability in High-Capacity Kernel Associative Memories
Sep 15, 2026Akira TamamoriHopfield Networks Neural Collapse
Faculty of Information Science, Aichi Institute of Technology, Aichi, 470-0392, Japan
Neural-Collapse-guided Task-Free Continual Anomaly Detection
Sep 3, 2026Xiaotong Kong, Chaoyang Song, Ziai Zhou +3Graph Anomaly Detection Unsupervised Detection
Key Laboratory of Measurement and Control of CSE, Ministry of Education, School of Automation, Southeast University, Nanjing 210096, China
Clustered Attractor Manifolds and Dynamical Condensation in Self-Attention
Aug 9, 2026Qucheng Gao, Zuyi Yang, Xiao ChenAttractors Linear Attention
Department of Physics, Boston College, Chestnut Hill, Massachusetts 02467, USA · Department of Physics, Tsinghua University, Beijing 100084, China
Simplifying Neural Networks During Training
Jul 30, 2026Lorenzo Sciandra, Samuele Fonio, Roberto EspositoNeural Collapse Deep Network
Department of Computer Science, University of Turin, Turin, Italy
More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility
Jul 28, 2026Yanli Yan, Yuanzheng Li, Yong Zhao +2Neural Collapse Decisions
Huazhong University of Science and Technology Wuhan, China
Perturbative-NeuSA: A Structured Spectral Framework for Time-Dependent PDEs
Jul 27, 2026Xianli Zhu, Jia YinNeural Partial Differential Equation Solvers Partial Differential Equations
School of Mathematical Sciences, Fudan University Shanghai, China
Test Case Prioritization for DNNs via Neural Collapse Instability
Jul 22, 2026Chunyu Liu, Mingyuan Li, Yang Li +4Neural Collapse Prioritization
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China · National Engineering Research Center of Disaster Backup and Recovery, Beijing University of Posts and Telecommunications, China · Information Security Center, Beijing University of Posts and Telecommunications, China
Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse
Jul 12, 2026Truong Xuan Khanh, Phan Thanh DucNeural Collapse Fingerprint
H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam
Neural Collapse Is Forbidden: Information Floors in Language Models
Jul 10, 2026Bruno AbrahaoNeural Collapse Language Modeling
NYU Shanghai · Leonard N. Stern School of Business, New York University
Ordinal Neural Collapse as a Representation Prior for Visual Navigation
Jun 25, 2026E-In Son, Jung-Taak Kim, Seung-Woo SeoVision-Language Navigation Neural Collapse
Dept. of ECE & ASRI, Seoul National University, Korea
Structure Before Collapse: Transient semantic geometry in next-token prediction
Jun 25, 2026Yize Zhao, Isabel Papadimitriou, Christos ThrampoulidisNeural Collapse Next-Token Prediction
Department of Electrical and Computer Engineering, The University of British Columbia · Department of Linguistics, The University of British Columbia
Parallel Manifold Steering: Efficient Adaptation of Large Associative Memories via Residual Energy Shaping
Jun 23, 2026Kanishk AwadhiyaDense Associative Memory Transformer Architectures
Independent Researcher
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Jun 8, 2026Hu Tan, Kuo Gai, Shihua ZhangNeural Collapse Neural Representations
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China; and Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Hangzhou 310024, China
Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning
Jun 4, 2026Ayushman Trivedi, Bhavika MelwaniCatastrophic Forgetting Continual Learning
Independent Researchers
Optical-Guided Neural Collapse for SAR Few-Shot Class Incremental Learning
Jun 3, 2026Fan Zhang, Sijin Zheng, Fei Ma +4Synthetic Aperture Radar Class-Incremental Learning
School of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China · School of Information Science and Technology and the Interdisciplinary Research Center for Artificial Intelligence, Beijing University of Chemical Technology, Beijing 100029, China · Beihang University, Beijing 100191, China +1
The role of class encoding in neural collapse
May 29, 2026Bastien Massion, Roy Makhlouf, Estelle MassartNeural Collapse Collapse
ICTEAM, UCLouvain Louvain-la-Neuve, Belgium
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
May 21, 2026Connall Garrod, Jonathan P. Keating, Christos ThrampoulidisNeural Collapse Low-Rank Structure
Mathematical Institute, University of Oxford · Department of Electrical and Computer Engineering, University of British Columbia
Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
May 19, 2026Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1Learnable Prototypes Neural Collapse
University of Athens · NSCR Demokritos · University of Cyprus +2
Conflict-Free Replicated Data Types for Neural Network Model Merging: A Two-Layer Architecture Enabling CRDT-Compliant Model Merging Across 26 Strategies
May 16, 2026Ryan GillespieContinual Model Merging Neural Collapse
Independent researcher
MahaVar: OOD Detection via Class-wise Mahalanobis Distance Variance under Neural Collapse
May 14, 2026Donghwan Kim, Hyunsoo YoonOut-Of-Distribution Detection In-Distribution
Department of Industrial Engineering Yonsei University Seoul, South Korea
State-Space NTK Collapse Near Bifurcations
May 12, 2026James Hazelden, Eric Shea-BrownNeural Tangent Kernel Bifurcation
University of Washington
Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models
May 12, 2026Boyang Guo, Liang Li, Lin Peng +3Vision-Language Model Adaptation Neural Collapse
Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China +2
Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning
May 12, 2026Huiyu Yi, Zhiming Xu, Dunwei Tu +3Class-Incremental Learning Neural Collapse
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Computer Science, Nanjing University, Nanjing, China
Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets
May 11, 2026Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1Contrastive Learning Imbalanced Classification
Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute +1
Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View
May 11, 2026Jinping Wang, Zixin Tong, Zhiwu Xie +1Balanced Learning Class Imbalance
CSMT, Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University
Flag Varieties: A Geometric Framework for Deep Network Alignment
May 11, 2026Jingchuan Xiao, Xinyi Sui, Cihan RuanNeural Collapse Subspace
Department of Mathematics and Computer Studies, Mary Immaculate College, Ireland · Department of Computer Science and Engineering, Santa Clara University, USA
The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks
May 7, 2026Taehun Cha, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1Feature Learning Deep Network
Department of Mathematics Korea University · Computer Science and Engineering UC San Diego · MIT Mathematics Broad Institute of MIT and Harvard
How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
May 3, 2026Xiaoxuan Ma, Yixuan Yang, Song Li +1Multi-Label Classification Neural Collapse
Department of Mathematics, Zhejiang University, Hangzhou, China.
Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index
Apr 28, 2026Alexander KalinowskiNeural Collapse Topology
SUNY Empire
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1Catastrophic Forgetting Large Language Model Memory
Stanford University

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Generalization in Neural Networks Through the Lens of Magnitude Potential

Sahel Torkamani, Henry Gouk, Rik Sarkar

School of Informatics University of Edinburgh

Neural Collapse Neural Network Memorization Lens

Open paper AskPDF arXiv

Explaining generalization and training dynamics in neural networks remains a challenge, and various approaches have been developed to study different aspects of these phenomena. In this paper, we introduce the idea of {\em magnitude potential} -- a quantity based on the theory of metric magnitude -- that reflects how well an arbitrary point is represented by a given set. We find that this basic quantity can be applied to examine various features in neural generalization. The ratio between the magnitude potential with respect to a class and with respect to the entire data, computed at the logit layer, is informative of the representation of the point. In experiments, these ratios for individual training points are found to be correlated with the Feldman memorization scores. Magnitude potential ratios aggregated across points detect structural changes in the decision boundaries and provide a geometric indicator of grokking in modular arithmetic. Although the magnitude potential ratio and neural collapse are both closely associated with intra-class and inter-class geometric structure, the magnitude potential ratio remains informative even when neural collapse is explicitly suppressed.

Momentum

Related topics

Latest papers 33