An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning

Berk Sezer

cs.LGOct 17, 2026

An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning

Authors: Berk Sezer

Organizations: Middle East Technical University, Türkiye

Abstract

Analytic continual learning (ACL) methods such as ACIL and GACL fit a ridge-regression classifier head on frozen features via the Sherman-Morrison-Woodbury identity, achieving exemplar-free, exact updates. A separate line of work, Progressive Neural Collapse (ProNC), shows standard continual classifiers benefit from a progressively expanding Simplex Equiangular Tight Frame (ETF) target. It is natural to ask whether fusing the two ideas, exact analytic updates and ETF targets, would yield a better closed-form continual learner. We show this is not the case as the fusion turns out to be identical to plain one-hot ACIL. Further, Simplex ETFs are not nested across class counts, so the literal "progressively expand the ETF" construction stops yielding an ETF as new tasks arrive, and it introduces catastrophic forgetting ( $4\times$ that of a classic ACIL head). We therefore separate an ETF's two defining properties, equiangularity and equinormality, and apply each on its own to the classifier weights rather than the targets, where the invariance above does not hold. Equinorm post-processing of the fitted weights recovers both tail-class and overall average accuracy across the long-tailed regime from 5:1 to 100:1 imbalance (e.g. +32.0 tail-accuracy points and +16.0 average-accuracy points at 100:1 on CIFAR-100/ViT), transferring in sign across two backbones and two datasets, at a mild cost of 5.4 average-accuracy points on CIFAR-100/ViT on near-balanced streams (below $\approx 3{:}1$ ). Equiangular geometry, by contrast, is exactly inert on the target side but actively harmful on the weight side, usually losing to plain one-hot. Hence, the generally useful part of Neural Collapse geometry in Analytic Continual Learning is equinorm post-processing of the weights, becoming more useful as class imbalance grows; and equiangularity is at best a no-op and induces catastrophic forgetting at worst.

Figures & tables

Figure 1: Imbalance-ratio sweep,

r\in\{1,\dots,200\}

, 5 seeds per ratio, CIFAR-100 T=10. Left: tail-class accuracy. Right: overall average accuracy. The crossover in overall accuracy lies between

r=2

and

r=5

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Per-task accuracy on classes seen so far, CIFAR-100 T=10, ViT. onehot and etf_exact are identical at every task. etf_stale ties at task 0 and then collapses as target staleness accumulates with drift.

dataset	backbone	$T$	head	avg. acc.	avg. incr. acc.	forgetting
CIFAR-100	ViT	10	onehot / etf_exact	$0.6856$	$0.7802$	$0.0949$
CIFAR-100	ViT	10	etf_stale	$0.0100$	$0.1936$	$0.3927$
TinyImageNet	ViT	10	onehot / etf_exact	$0.6675$	$0.7354$	$0.0829$
TinyImageNet	ViT	10	etf_stale	$0.0050$	$0.1773$	$0.3557$
CIFAR-100	ResNet-50	20	onehot / etf_exact	$0.6933$	$0.7924$	$0.1044$
CIFAR-100	ResNet-50	20	etf_stale	$0.0100$	$0.1149$	$0.2498$

Appendix

Table 1: Main CIL results, all heads fit in lockstep on identical features.

$m$	welch	random	truncated_etf
4	$0.0182\pm 0.0011$	$0.0271\pm 0.0048$	$0.0192\pm 0.0023$
8	$0.0445\pm 0.0015$	$0.0426\pm 0.0037$	$0.0403\pm 0.0015$
16	$0.0631\pm 0.0046$	$0.0642\pm 0.0028$	$0.0614\pm 0.0038$
32	$0.0780\pm 0.0047$	$0.0783\pm 0.0054$	$0.0754\pm 0.0027$

Appendix

Table 2: Compressed-frame sweep, CIFAR-100 T=10, ViT backbone, mean

\pm

sample stdev over 5 seeds.

$r$	onehot avg.	equinorm avg.	avg. $\Delta$	onehot tail	equinorm tail	tail $\Delta$
1	$0.686\pm 0.000$	$0.631\pm 0.000$	$-0.054$	$0.693\pm 0.000$	$0.639\pm 0.000$	$-0.054$
2	$0.666\pm 0.002$	$0.632\pm 0.004$	$-0.034$	$0.598\pm 0.016$	$0.609\pm 0.039$	$+0.011$
5	$0.603\pm 0.005$	$0.625\pm 0.008$	$+0.022$	$0.458\pm 0.020$	$0.583\pm 0.050$	$+0.125$
10	$0.542\pm 0.008$	$0.611\pm 0.006$	$+0.069$	$0.341\pm 0.023$	$0.552\pm 0.042$	$+0.211$
25	$0.463\pm 0.007$	$0.588\pm 0.010$	$+0.124$	$0.203\pm 0.022$	$0.501\pm 0.046$	$+0.298$
50	$0.415\pm 0.006$	$0.566\pm 0.008$	$+0.151$	$0.126\pm 0.017$	$0.452\pm 0.037$	$+0.326$

Appendix

Table 3: Full imbalance-ratio sweep, CIFAR-100/ViT, mean

\pm

sample stdev over 5 seeds. equinorm trails onehot only at

r\leq 2

and leads on both average and tail-class accuracy from

r{=}5

onward.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.686\pm 0.000$	$0.631\pm 0.000$	$0.580\pm 0.000$	$\mathbf{0.692\pm 0.000}$
2	$0.666\pm 0.002$	$0.632\pm 0.004$	$0.564\pm 0.011$	$\mathbf{0.686\pm 0.001}$
5	$0.603\pm 0.005$	$0.625\pm 0.008$	$0.507\pm 0.024$	$\mathbf{0.674\pm 0.001}$
10	$0.542\pm 0.008$	$0.611\pm 0.006$	$0.450\pm 0.024$	$\mathbf{0.659\pm 0.002}$
25	$0.463\pm 0.007$	$0.588\pm 0.010$	$0.381\pm 0.018$	$\mathbf{0.627\pm 0.006}$
50	$0.414\pm 0.006$	$0.566\pm 0.008$	$0.337\pm 0.015$	$\mathbf{0.597\pm 0.006}$

Appendix

Table 4: Overall average accuracy, CIFAR-100 / ViT, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.668\pm 0.000$	$0.601\pm 0.000$	$0.576\pm 0.000$	$\mathbf{0.677\pm 0.000}$
2	$0.644\pm 0.003$	$0.600\pm 0.004$	$0.545\pm 0.008$	$\mathbf{0.673\pm 0.001}$
5	$0.575\pm 0.004$	$0.596\pm 0.009$	$0.461\pm 0.011$	$\mathbf{0.663\pm 0.002}$
10	$0.507\pm 0.005$	$0.588\pm 0.010$	$0.397\pm 0.007$	$\mathbf{0.651\pm 0.002}$
25	$0.428\pm 0.011$	$0.575\pm 0.012$	$0.329\pm 0.009$	$\mathbf{0.625\pm 0.003}$
50	$0.381\pm 0.008$	$0.558\pm 0.009$	$0.298\pm 0.012$	$\mathbf{0.599\pm 0.005}$

Appendix

Table 5: Overall average accuracy, TinyImageNet / ViT, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.693\pm 0.000$	$0.680\pm 0.000$	$\mathbf{0.696\pm 0.000}$	$0.686\pm 0.000$
2	$0.680\pm 0.002$	$0.673\pm 0.002$	$\mathbf{0.680\pm 0.002}$	$0.680\pm 0.002$
5	$0.638\pm 0.002$	$0.660\pm 0.004$	$0.630\pm 0.005$	$\mathbf{0.666\pm 0.004}$
10	$0.593\pm 0.006$	$0.645\pm 0.004$	$0.581\pm 0.008$	$\mathbf{0.650\pm 0.004}$
25	$0.524\pm 0.005$	$0.619\pm 0.007$	$0.511\pm 0.007$	$\mathbf{0.622\pm 0.008}$
50	$0.473\pm 0.003$	$0.594\pm 0.008$	$0.463\pm 0.005$	$\mathbf{0.594\pm 0.009}$

Appendix

Table 6: Overall average accuracy, CIFAR-100 / ResNet-50, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.726\pm 0.000$	$0.711\pm 0.000$	$\mathbf{0.727\pm 0.000}$	$0.717\pm 0.000$
2	$0.721\pm 0.001$	$0.710\pm 0.001$	$\mathbf{0.721\pm 0.001}$	$0.716\pm 0.001$
5	$0.696\pm 0.003$	$0.703\pm 0.002$	$0.693\pm 0.003$	$\mathbf{0.711\pm 0.002}$
10	$0.667\pm 0.004$	$0.697\pm 0.003$	$0.659\pm 0.004$	$\mathbf{0.705\pm 0.002}$
25	$0.618\pm 0.005$	$0.687\pm 0.004$	$0.603\pm 0.006$	$\mathbf{0.694\pm 0.002}$
50	$0.573\pm 0.006$	$0.676\pm 0.003$	$0.555\pm 0.010$	$\mathbf{0.681\pm 0.002}$

Appendix

Table 7: Overall average accuracy, TinyImageNet / ResNet-50, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

cell	$r{=}2$	$r{=}5$	$r{=}200$
CIFAR-100 / ViT	$-0.034$	$+0.022$	$+0.160$
TinyImageNet / ViT	$-0.044$	$+0.022$	$+0.188$
CIFAR-100 / ResNet-50	$-0.007$	$+0.022$	$+0.133$
TinyImageNet / ResNet-50	$-0.011$	$+0.007$	$+0.154$

Appendix

Table 8: Cluster gap

\min(\texttt{equinorm},\texttt{etf\_project})-\max(\texttt{onehot},\texttt{equiangular})

: negative means equinorm status does not yet explain the ranking; positive means the two equinorm heads both outrank both non-equinorm heads. Positive for every ratio from

r{=}5

through

r{=}200

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 301 papers

AIR: Analytic Imbalance Rectifier for Continual Learning
Aug 19, 2024Di Fang, Yinan Zhu, Zhiping Lin +3Class-Imbalanced Learning Class-Incremental Learning
School of Future Technology, South China University of Technology, Guangzhou, China · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China
SPARCL: Spectral Partitioned Analytic Continual Learning
Aug 21, 2026James Hartley, Zeropy Surio, Daniel Whitmore +2Continual Learning Class-Incremental Learning
University of Sheffield
Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View
May 11, 2026Jinping Wang, Zixin Tong, Zhiwu Xie +1Long-Tailed Classification Adaptive Loss Weighting
CSMT, Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University

Aug 19, 2024J/K move · Enter open · S save

AIR: Analytic Imbalance Rectifier for Continual Learning

Di Fang, Yinan Zhu, Zhiping Lin, Cen Chen+2

School of Future Technology, South China University of Technology, Guangzhou, China · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China

Class-Imbalanced Learning Class-Incremental Learning Continual Learning Online Continual Learning Incremental Learning

Open paper AskPDF arXiv

Continual learning (CL) agents incrementally learn from sequentially arriving data and adapt to the dynamic, ever-changing nature of real-world environments. However, many existing CL methods suffer performance degradation in evolving, imbalanced data streams due to limited adaptation to changing class frequencies or ineffective use of mixed data from new and previously observed classes. To deal with these challenges, we propose an analytic imbalance rectifier (AIR) algorithm for real-world CL. AIR is an online exemplar-free approach with a frozen backbone as the feature extractor and a closed-form incremental classifier whose weight equals the joint-learning weight for the same class-weighted ridge objective. AIR addresses class imbalance with an analytic reweighting module (ARM) that calculates a reweighting factor for each class in the loss function to equalize total sample weights across classes. Under long-tailed class-incremental learning, AIR leads 28 baselines in aggregate accuracy and exemplar-free methods in aggregate macro F1, gaining 3.21% accuracy and 2.14% macro F1 over the respective strongest exemplar-free baselines. Under the Si-Blurry setting with recurring classes, AIR leads 15 exemplar-based and exemplar-free baselines, gaining 2.32% aggregate accuracy and 1.27% aggregate macro F1 over the strongest baseline. One-sided paired tests support positive mean absolute gains in these four comparisons (Holm-adjusted p<0.006).

Abstract

Continual Learning

Explore similar work

AIR: Analytic Imbalance Rectifier for Continual Learning

SPARCL: Spectral Partitioned Analytic Continual Learning

Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View