Full Collapse Is Suboptimal: Controlled Partial Neural Collapse for Long-Tailed Recognition

Hyeontaek Hwang; Cheonghwan Hur; Daeyoung Kim

cs.CVOct 27, 2026

Full Collapse Is Suboptimal: Controlled Partial Neural Collapse for Long-Tailed Recognition

Authors: Hyeontaek Hwang, Cheonghwan Hur, Daeyoung Kim

Organizations: KAIST · RTST Inc.

Abstract

Recent long-tailed recognition (LTR) methods incorporate neural collapse (NC) and align features to a fixed Equiangular Tight Frame (ETF) as a uniform target for all classes. However, forcing tail classes to fully collapse over-contracts their samples and weakens generalization. To address this, we propose a controlled partial neural collapse, realized through two coordinated controls on a backbone: the Class-Adaptive Sparse Activation (CASA) classifier and the ETF-constrained adaptive margin (ETF-AM) loss. Rather than forcing a uniform full collapse, this regime allows head classes to collapse along the ETF while tail classes retain within-class diversity. This prevents the tail from being forced into full collapse, which is suboptimal under imbalance. To set the dimensionality of collapse, CASA induces class-related node specialization through feature-conditioned gating so that tail classes occupy their own feature directions. For the clearance axis, ETF-AM applies a frequency-adaptive margin, with smaller margins for head classes and larger for tail classes. On CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018, our method achieves the best overall accuracy over prior methods. In the empirical analysis, within-class spread increases monotonically from head to tail classes, consistent with tail classes retaining variability rather than fully collapsing.

Figures & tables

Figure 1: Controlled partial neural collapse via two coordinated mechanisms. Full collapse over-concentrates tail classes at their ETF vertices (center); our method addresses this by controlling the dimensionality of collapse via instance-conditioned node selection (CASA, right; Sec. 3.1 ) and the training-to-inference clearance per class (ETF-AM, left; Sec. 3.2 ).

Figure 2: The overall architecture of the CASA classifier, which selects a subset of nodes through instance-conditioned node-level gating.

Method	Many	Medium	Few	All
CE (Baseline)	64.60	36.90	11.90	38.50
CB [ 5 ]	–	–	–	39.60
LDAM [ 4 ]	–	–	–	42.04
BALMS [ 40 ]	–	–	–	50.80
VS+SAM [ 39 ]	56.40	48.80	31.70	46.60
GCL [ 26 ]	–	–	–	48.70

Table 1: Top-1 accuracy on the test set of CIFAR100-LT (

\rho=100

). Baseline numbers are taken from the original papers; “-” denotes results not reported in the original papers. Bold marks the best and underline the second-best result.

\dagger

indicates result with ResNet34

Method	Many	Medium	Few	All
CE (Baseline)	65.90	37.50	7.70	44.40
LDAM [ 4 ]	–	–	–	51.30
ALA [ 59 ]	64.10	49.90	34.70	53.30
WB [ 1 ]	62.50	50.40	41.50	53.90
RBL [ 37 ]	64.80	49.60	34.20	53.30
NC-DRW-cRT [ 29 ]	65.60	51.20	35.40	54.20

Table 2: Top-1 accuracy on the test set of ImageNet-LT with ResNeXt50. Baseline numbers are taken from the original papers; “–” denotes results not reported in the original papers. Bold marks the best and underline the second-best result.

Method	Many	Medium	Few	All
CE (Baseline)	45.70	27.30	8.20	30.20
OLTR [ 30 ]	44.70	37.00	25.30	35.90
D-LWS [ 20 ]	40.60	39.10	28.60	37.60
BALMS [ 40 ]	41.20	39.80	31.60	38.70
DisAlign [ 56 ]	40.40	42.40	30.10	39.30
LADE [ 17 ]	42.80	39.00	31.20	38.80

Table 3: Top-1 accuracy on the test set of Places-LT with ResNet152. Baseline numbers are taken from the original papers; “–” denotes results not reported in the original papers. Bold marks the best and underline the second-best result.

Method	Many	Medium	Few	All
CE (Baseline)	75.70	66.90	61.70	65.80
cRT [ 20 ]	–	–	–	65.20
LWS [ 20 ]	–	–	–	65.90
MiSLAS [ 61 ]	–	–	–	71.60
GCL [ 26 ]	–	–	–	72.00
WB [ 1 ]	71.00	70.30	69.40	70.00

Table 4: Top-1 accuracy on iNaturalist-2018 with ResNet50. Baseline numbers are taken from the original papers; “–” denotes results not reported in the original papers. Bold marks the best and underline the second-best result.

		Vanilla		Ours
Dataset	Backbone	#param	FLOPs	#param	FLOPs
CIFAR100-LT	ResNet34	21.3M	2.3G	21.6M	2.3G
ImageNet-LT	ResNeXt50	25.0M	8.6G	29.2M	8.6G
Places-LT	ResNet152	60.2M	23.2G	64.4M	23.2G
iNaturalist	ResNet50	40.2M	8.2G	44.4M	8.2G

Table 5: Parameter count and FLOPs of the proposed method, and the Vanilla model. (The number of parameters and FLOPs are measured with ptflops )

Method	FDR (Train)	FDR (Test)	Many	Medium	Few	Avg.
Full model	$9.94{\times}10^{4}$	$1.28{\times}10^{2}$	74.71	55.77	31.65	54.73
Ablation on key components
w/o ETF-AM	$3.36{\times}10^{4}$	$1.23{\times}10^{2}$	74.88	53.34	26.74	52.42
w/o Gating layer	$9.46{\times}10^{4}$	$1.24{\times}10^{2}$	74.20	54.10	30.60	53.60
Capacity-matched gating and margin baselines
2-layer MLP	$3.84\times 10^{4}$	$1.05\times 10^{2}$	70.30	45.00	29.50	48.80

Table 6: Component ablation on CIFAR100-LT: FDRs and Top-1 accuracy. Bold marks the best.

Dataset	Loss	Many	Medium	Few
CIFAR100-LT	ETF-AM	1.00	1.40	1.76
CIFAR100-LT	CE	1.00	1.18	1.33
iNaturalist-2018	ETF-AM	1.00	1.38	1.40
iNaturalist-2018	CE	1.00	1.04	1.01

Table 7: Within-class spread

s_{c}

, averaged within each frequency split and normalized to the Many-class mean. In both datasets, the spread increases monotonically from head to tail, and ETF-AM maintains a larger tail spread than the CE baseline.

Figure 3: Epoch-wise cosine similarity (a) and symmetric KL divergence (b) of generated masks on CIFAR100-LT. Within-class masks grow more similar over training, whereas between-class masks separate.

Figure 4: Histograms of symmetric KL divergence of generated mask (before thresholding on the training time) between sample pairs. (a) shows divergences computed from training set samples, and (b) displays divergences from test set samples. All histograms are plotted on a logarithmic scale.

	Cosine Similarity		Avg. KL Divergence
	Inter	Intra	Inter	Intra
Train	0.48	0.96	1.115	0.001
Test	0.72	0.81	0.430	0.046

Table 8: Cosine similarity and average KL divergence of generated masks between inter-class and intra-class on CIFAR100-LT.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	CIFAR100-LT	ImageNet-LT	Places-LT	iNaturalist2018
Data Settings
# Classes	100	1000	365	8142
Batch Size	64	128	128	512
Optimizer Parameters
Total Epochs	320	200	30	200
Optimizer	SGD	SGD	SGD	SGD

Appendix

Table 9: Hyperparameters used in experiments.

\Gamma

is a fixed value, except for iNaturalist2018, which cannot have an ETF structure.

Method	FDR (Tr)	FDR (Te)	Many	Med	Few	Avg.
$\vartheta{=}0.5,\varpi{=}0.1$	$9.94\times 10^{4}$	${1.28\times 10^{2}}$	74.71	55.77	31.65	54.73
w/o ETF-AM	$3.36\times 10^{4}$	$1.23\times 10^{2}$	74.88	53.34	$26.74$	52.42
w/o Gating	$9.46\times 10^{4}$	$1.24\times 10^{2}$	74.20	54.10	30.60	53.60
$\vartheta{=}0.9,\varpi{=}0.1$	$9.65\times 10^{4}$	$1.24\times 10^{2}$	73.00	55.48	31.29	53.56
$\vartheta{=}0.7,\varpi{=}0.1$	$9.76\times 10^{4}$	$1.24\times 10^{2}$	73.52	55.26	31.48	54.10
$\vartheta{=}0.3,\varpi{=}0.1$	$9.80\times 10^{4}$	$1.25\times 10^{2}$	73.85	54.22	31.73	54.22

Appendix

Table 10: Extended ablation study on CIFAR100-LT: FDR on train and test sets, and Top-1 Accuracy for Many, Medium, Few, and overall Average classes.

Method	FDR (Tr)	FDR (Te)	Many	Med	Few	Avg.
$\vartheta{=}0.5,\varpi{=}0.1$	$9.19{\times}10^{2}$	$7.25{\times}10^{2}$	65.20	55.35	38.87	56.90
w/o ETF-AM	$5.20{\times}10^{2}$	$6.16{\times}10^{2}$	65.17	52.19	27.52	53.83
w/o Gating	$8.89{\times}10^{2}$	$7.02{\times}10^{2}$	64.54	52.87	30.73	54.34
$\vartheta{=}0.9,\varpi{=}0.1$	$8.42{\times}10^{2}$	$6.11{\times}10^{2}$	64.05	54.81	36.76	55.91
$\vartheta{=}0.7,\varpi{=}0.1$	$8.54{\times}10^{2}$	$6.42{\times}10^{2}$	62.25	53.63	34.33	54.32
$\vartheta{=}0.3,\varpi{=}0.1$	$8.64{\times}10^{2}$	$6.62{\times}10^{2}$	63.24	54.73	32.91	55.03

Appendix

Table 11: Extended Ablation study on ImageNet-LT with ResNeXt50: FDR on train and test sets, and Top-1 Accuracy for Many, Medium, Few, and overall Average classes. Bold marks the best.

Figure 5: Hyperparameter ( $\vartheta$ and $\varpi$ ) heatmap on CIFAR100-LT. (a) Average accuracy on the test set, (b) FDR score on the test set.

Dataset	Train	Test / Val.
CIFAR100-LT	$-0.602$	$-0.249$
iNaturalist-2018	$-0.120$	$-0.124$

Appendix

Table 12: Log–log OLS slope of class-wise within-class spread

s_{c}

against the training count

n_{c}

. All slopes are negative on both splits.

$\lambda$	Train $s_{c}$ ( $\times 10^{-3}$ )	Test $s_{c}$	Few Acc. (%)
$0.0$	$4.045{\pm}0.33$	$0.3325{\pm}0.0033$	$33.34{\pm}1.29$
$0.1$	$0.991{\pm}0.04$	$0.2925{\pm}0.0048$	$32.15{\pm}0.76$
$0.2$	$0.855{\pm}0.03$	$0.2748{\pm}0.0045$	$30.62{\pm}1.45$
$0.3$	$1.440{\pm}0.74$	$0.2670{\pm}0.0014$	$29.66{\pm}1.68$

Appendix

Table 13: Three-seed sweep of the Few-class contraction penalty with ResNet-34 on CIFAR100-LT. Each entry reports the mean

\pm

sample SD.

Method	FDR (Tr)	FDR (Te)	Many	Med.	Few	Avg.
Soft / Soft	$6.97{\times}10^{4}$	$1.21{\times}10^{2}$	77.53	52.03	25.10	52.35
Soft / Hard	$6.97{\times}10^{4}$	$1.21{\times}10^{2}$	76.18	50.06	24.55	51.03
Hard / Hard	$9.94{\times}10^{4}$	$1.28{\times}10^{2}$	75.34	52.01	29.45	53.41
Hard / Soft (Ours)	$9.94{\times}10^{4}$	$1.28{\times}10^{2}$	74.71	55.77	31.65	54.73

Appendix

Table 14: Mask mode at training and inference on CIFAR100-LT. FDR is determined by the training mode alone. Bold marks the best; underline the second-best.

	Intra-class	Inter-class
Train	0.986	0.254
Test	0.383	0.015

Appendix

Table 15: Residual cosine similarity after removing the dataset-mean mask, CIFAR100-LT.

Figure 6: Confusion matrix for CIFAR100-LT (

\rho=100

). Classes are sorted in descending order by the number of training samples per class.

Panel of Figure 7 — Figure 7: Generated mask similarity on the CIFAR100-LT test set for success and failure cases (‘streetcar’ (11 training samples) vs. ‘bus’ (262) and ‘whale’ (6) vs. ‘dolphin’ (119)): (a) the proposed method, (b) the proposed method w/o ETF-AM loss.

Long-Tailed Classification

Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.

See all 55 papers

A Strong Balanced-Softmax Classifier-Retraining Baseline for Long-Tailed Recognition
Jul 10, 2026Juan Terven, Diana Margarita Córdova Esparza, Julio Alejandro Romero Gonzalez +4Class-Imbalanced Learning Long-Tailed Classification
CICATA Querétaro Instituto Politecnico Nacional Mexico · Facultad de Informática Universidad Autónoma de Querétaro Mexico
VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition
Sep 4, 2026Jiangang Zhu, Zheng Wang, Bin Zhu +2Long-Tailed Classification Ensemble Learning
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China +2
Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models
May 12, 2026Boyang Guo, Liang Li, Lin Peng +3Long-Tail Learning VLM Adaptation
Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China +2

Jul 10, 2026J/K move · Enter open · S save

A Strong Balanced-Softmax Classifier-Retraining Baseline for Long-Tailed Recognition

Juan Terven, Diana Margarita Córdova Esparza, Julio Alejandro Romero Gonzalez, Edgar Arturo Chávez Urbiola+3

CICATA Querétaro Instituto Politecnico Nacional Mexico · Facultad de Informática Universidad Autónoma de Querétaro Mexico

Class-Imbalanced Learning Long-Tailed Classification

Open paper AskPDF arXiv

Long-tailed recognition methods often modify losses, margins, or representations to reduce the dominance of frequent classes. We ask whether, after Balanced Softmax training, the remaining tail error can be reduced by retraining only the classifier. We evaluate BS-cRT, a two-stage procedure that trains a backbone and cosine classifier with Balanced Softmax, freezes the backbone, and updates only the classifier on balanced episodic batches. The second stage keeps the empirical-prior Balanced Softmax objective and uses raw cosine logits at inference. Across CIFAR-100-LT, CIFAR-10-LT, ImageNet-LT, and Places-LT, this classifier-only step consistently improves Few-shot accuracy over the matched Balanced Softmax checkpoint. At imbalance factor 100, Few-shot gains are +5.15 points on CIFAR-100-LT and +5.83 on CIFAR-10-LT; on ImageNet-LT and Places-LT, gains are +6.92 and +9.78 points, respectively, with a Top-1/Few-shot trade-off on ImageNet-LT. We also analyze Counterfactual Boundary Risk Minimization (CBRM), a boundary-probe extension using prototype-based features near decision boundaries. CBRM identifies two failure modes: scaled-logit cosine margins destabilize training, and corrected hardest-negative probes remain head-class anchored. The results support BS-cRT as a practical classifier-side baseline and indicate that boundary supervision must account for class frequency.

Abstract

Long-Tailed Classification

Explore similar work

A Strong Balanced-Softmax Classifier-Retraining Baseline for Long-Tailed Recognition

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models