cs.LGNov 5, 2026

Neural Collapse on Imbalanced Datasets and Open-Set Scenario

Authors: Qiufu Li, Huibing Xiao, Linlin Shen, Yongdong Huang

Organizations: School of Artificial Intelligence; National Engineering Laboratory for Big Data System Computing Technology; Guangdong Provincial Key Laboratory of Intelligent Information Processing; Intelligent Computing Center, Shenzhen University, Shenzhen 518060, China · Institute of Image Understanding Research, North Minzu University, Yinchuan 750021, China

Abstract

Neural collapse (NC) describes the elegant geometric structure that sample features and classifier vectors exhibit when a classification model's loss reaches its minimum. Class imbalance, however, generally distorts standard NC. In this paper, we theoretically reformulate NC on imbalanced datasets and in open-set settings. Our analysis is based on an optimization problem over class-wise averaged losses, rather than the global averaged losses used in prior NC studies. We prove that when class-wise averaged losses attain their minima, standard NC is achieved on balanced or imbalanced datasets, for both cross-entropy (CE) and binary cross-entropy (BCE) losses. We further extend the analysis to an open-set-inspired scenario, where some classes contain no training samples yet are still assigned classifier vectors in the losses. We find that in this scenario, when class-wise averaged losses reach their minima, standard NC still emerges on the known classes with training samples, while the classifier vectors of unknown classes without training samples overlap with each other and become orthogonal to those of the known classes. Finally, we analyze Softmax- and Sigmoid-based contrastive losses re-balanced by inverse sample frequency, proving that at their minima on balanced or imbalanced datasets, sample features of different classes likewise form standard NC.

Figures & tables

Explore similar work

CardsList
  1. Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets

    May 11, 2026Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1Class-Imbalanced LearningContrastive Learning

  2. Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

    May 19, 2026Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1Prototype-Based ClassificationHyperspherical Representation Learning

  3. Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View

    May 11, 2026Jinping Wang, Zixin Tong, Zhiwu Xie +1Long-Tailed ClassificationAdaptive Loss Weighting