An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning

Berk Sezer

cs.LGOct 17, 2026

An Impossibility Result for ETF Target Geometry, and an Equinorm Remedy for Imbalanced Analytic Continual Learning

Authors: Berk Sezer

Organizations: Middle East Technical University, Türkiye

Abstract

Analytic continual learning (ACL) methods such as ACIL and GACL fit a ridge-regression classifier head on frozen features via the Sherman-Morrison-Woodbury identity, achieving exemplar-free, exact updates. A separate line of work, Progressive Neural Collapse (ProNC), shows standard continual classifiers benefit from a progressively expanding Simplex Equiangular Tight Frame (ETF) target. It is natural to ask whether fusing the two ideas, exact analytic updates and ETF targets, would yield a better closed-form continual learner. We show this is not the case as the fusion turns out to be identical to plain one-hot ACIL. Further, Simplex ETFs are not nested across class counts, so the literal "progressively expand the ETF" construction stops yielding an ETF as new tasks arrive, and it introduces catastrophic forgetting ( $4\times$ that of a classic ACIL head). We therefore separate an ETF's two defining properties, equiangularity and equinormality, and apply each on its own to the classifier weights rather than the targets, where the invariance above does not hold. Equinorm post-processing of the fitted weights recovers both tail-class and overall average accuracy across the long-tailed regime from 5:1 to 100:1 imbalance (e.g. +32.0 tail-accuracy points and +16.0 average-accuracy points at 100:1 on CIFAR-100/ViT), transferring in sign across two backbones and two datasets, at a mild cost of 5.4 average-accuracy points on CIFAR-100/ViT on near-balanced streams (below $\approx 3{:}1$ ). Equiangular geometry, by contrast, is exactly inert on the target side but actively harmful on the weight side, usually losing to plain one-hot. Hence, the generally useful part of Neural Collapse geometry in Analytic Continual Learning is equinorm post-processing of the weights, becoming more useful as class imbalance grows; and equiangularity is at best a no-op and induces catastrophic forgetting at worst.

Figures & tables

Figure 1: Imbalance-ratio sweep,

r\in\{1,\dots,200\}

, 5 seeds per ratio, CIFAR-100 T=10. Left: tail-class accuracy. Right: overall average accuracy. The crossover in overall accuracy lies between

r=2

and

r=5

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Per-task accuracy on classes seen so far, CIFAR-100 T=10, ViT. onehot and etf_exact are identical at every task. etf_stale ties at task 0 and then collapses as target staleness accumulates with drift.

dataset	backbone	$T$	head	avg. acc.	avg. incr. acc.	forgetting
CIFAR-100	ViT	10	onehot / etf_exact	$0.6856$	$0.7802$	$0.0949$
CIFAR-100	ViT	10	etf_stale	$0.0100$	$0.1936$	$0.3927$
TinyImageNet	ViT	10	onehot / etf_exact	$0.6675$	$0.7354$	$0.0829$
TinyImageNet	ViT	10	etf_stale	$0.0050$	$0.1773$	$0.3557$
CIFAR-100	ResNet-50	20	onehot / etf_exact	$0.6933$	$0.7924$	$0.1044$
CIFAR-100	ResNet-50	20	etf_stale	$0.0100$	$0.1149$	$0.2498$

Appendix

Table 1: Main CIL results, all heads fit in lockstep on identical features.

$m$	welch	random	truncated_etf
4	$0.0182\pm 0.0011$	$0.0271\pm 0.0048$	$0.0192\pm 0.0023$
8	$0.0445\pm 0.0015$	$0.0426\pm 0.0037$	$0.0403\pm 0.0015$
16	$0.0631\pm 0.0046$	$0.0642\pm 0.0028$	$0.0614\pm 0.0038$
32	$0.0780\pm 0.0047$	$0.0783\pm 0.0054$	$0.0754\pm 0.0027$

Appendix

Table 2: Compressed-frame sweep, CIFAR-100 T=10, ViT backbone, mean

\pm

sample stdev over 5 seeds.

$r$	onehot avg.	equinorm avg.	avg. $\Delta$	onehot tail	equinorm tail	tail $\Delta$
1	$0.686\pm 0.000$	$0.631\pm 0.000$	$-0.054$	$0.693\pm 0.000$	$0.639\pm 0.000$	$-0.054$
2	$0.666\pm 0.002$	$0.632\pm 0.004$	$-0.034$	$0.598\pm 0.016$	$0.609\pm 0.039$	$+0.011$
5	$0.603\pm 0.005$	$0.625\pm 0.008$	$+0.022$	$0.458\pm 0.020$	$0.583\pm 0.050$	$+0.125$
10	$0.542\pm 0.008$	$0.611\pm 0.006$	$+0.069$	$0.341\pm 0.023$	$0.552\pm 0.042$	$+0.211$
25	$0.463\pm 0.007$	$0.588\pm 0.010$	$+0.124$	$0.203\pm 0.022$	$0.501\pm 0.046$	$+0.298$
50	$0.415\pm 0.006$	$0.566\pm 0.008$	$+0.151$	$0.126\pm 0.017$	$0.452\pm 0.037$	$+0.326$

Appendix

Table 3: Full imbalance-ratio sweep, CIFAR-100/ViT, mean

\pm

sample stdev over 5 seeds. equinorm trails onehot only at

r\leq 2

and leads on both average and tail-class accuracy from

r{=}5

onward.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.686\pm 0.000$	$0.631\pm 0.000$	$0.580\pm 0.000$	$\mathbf{0.692\pm 0.000}$
2	$0.666\pm 0.002$	$0.632\pm 0.004$	$0.564\pm 0.011$	$\mathbf{0.686\pm 0.001}$
5	$0.603\pm 0.005$	$0.625\pm 0.008$	$0.507\pm 0.024$	$\mathbf{0.674\pm 0.001}$
10	$0.542\pm 0.008$	$0.611\pm 0.006$	$0.450\pm 0.024$	$\mathbf{0.659\pm 0.002}$
25	$0.463\pm 0.007$	$0.588\pm 0.010$	$0.381\pm 0.018$	$\mathbf{0.627\pm 0.006}$
50	$0.414\pm 0.006$	$0.566\pm 0.008$	$0.337\pm 0.015$	$\mathbf{0.597\pm 0.006}$

Appendix

Table 4: Overall average accuracy, CIFAR-100 / ViT, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.668\pm 0.000$	$0.601\pm 0.000$	$0.576\pm 0.000$	$\mathbf{0.677\pm 0.000}$
2	$0.644\pm 0.003$	$0.600\pm 0.004$	$0.545\pm 0.008$	$\mathbf{0.673\pm 0.001}$
5	$0.575\pm 0.004$	$0.596\pm 0.009$	$0.461\pm 0.011$	$\mathbf{0.663\pm 0.002}$
10	$0.507\pm 0.005$	$0.588\pm 0.010$	$0.397\pm 0.007$	$\mathbf{0.651\pm 0.002}$
25	$0.428\pm 0.011$	$0.575\pm 0.012$	$0.329\pm 0.009$	$\mathbf{0.625\pm 0.003}$
50	$0.381\pm 0.008$	$0.558\pm 0.009$	$0.298\pm 0.012$	$\mathbf{0.599\pm 0.005}$

Appendix

Table 5: Overall average accuracy, TinyImageNet / ViT, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.693\pm 0.000$	$0.680\pm 0.000$	$\mathbf{0.696\pm 0.000}$	$0.686\pm 0.000$
2	$0.680\pm 0.002$	$0.673\pm 0.002$	$\mathbf{0.680\pm 0.002}$	$0.680\pm 0.002$
5	$0.638\pm 0.002$	$0.660\pm 0.004$	$0.630\pm 0.005$	$\mathbf{0.666\pm 0.004}$
10	$0.593\pm 0.006$	$0.645\pm 0.004$	$0.581\pm 0.008$	$\mathbf{0.650\pm 0.004}$
25	$0.524\pm 0.005$	$0.619\pm 0.007$	$0.511\pm 0.007$	$\mathbf{0.622\pm 0.008}$
50	$0.473\pm 0.003$	$0.594\pm 0.008$	$0.463\pm 0.005$	$\mathbf{0.594\pm 0.009}$

Appendix

Table 6: Overall average accuracy, CIFAR-100 / ResNet-50, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

$r$	onehot	equinorm	equiangular	etf_project
1	$0.726\pm 0.000$	$0.711\pm 0.000$	$\mathbf{0.727\pm 0.000}$	$0.717\pm 0.000$
2	$0.721\pm 0.001$	$0.710\pm 0.001$	$\mathbf{0.721\pm 0.001}$	$0.716\pm 0.001$
5	$0.696\pm 0.003$	$0.703\pm 0.002$	$0.693\pm 0.003$	$\mathbf{0.711\pm 0.002}$
10	$0.667\pm 0.004$	$0.697\pm 0.003$	$0.659\pm 0.004$	$\mathbf{0.705\pm 0.002}$
25	$0.618\pm 0.005$	$0.687\pm 0.004$	$0.603\pm 0.006$	$\mathbf{0.694\pm 0.002}$
50	$0.573\pm 0.006$	$0.676\pm 0.003$	$0.555\pm 0.010$	$\mathbf{0.681\pm 0.002}$

Appendix

Table 7: Overall average accuracy, TinyImageNet / ResNet-50, all four heads, full imbalance-ratio sweep (mean

\pm

sample stdev, 5 seeds). Best per row in bold.

cell	$r{=}2$	$r{=}5$	$r{=}200$
CIFAR-100 / ViT	$-0.034$	$+0.022$	$+0.160$
TinyImageNet / ViT	$-0.044$	$+0.022$	$+0.188$
CIFAR-100 / ResNet-50	$-0.007$	$+0.022$	$+0.133$
TinyImageNet / ResNet-50	$-0.011$	$+0.007$	$+0.154$

Appendix

Table 8: Cluster gap

\min(\texttt{equinorm},\texttt{etf\_project})-\max(\texttt{onehot},\texttt{equiangular})

: negative means equinorm status does not yet explain the ranking; positive means the two equinorm heads both outrank both non-equinorm heads. Positive for every ratio from

r{=}5

through

r{=}200

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 301 papers

Aug 19, 2024

AIR: Analytic Imbalance Rectifier for Continual Learning

Continual learning (CL) agents incrementally learn from sequentially arriving data and adapt to the dynamic, ever-changing nature of real-world environments. However, many existing CL methods suffer performance degradation in evolving, imbalanced data streams due to limited adaptation to changing class frequencies or ineffective use of mixed data from new and previously observed classes. To deal with these challenges, we propose an analytic imbalance rectifier (AIR) algorithm for real-world CL. AIR is an online exemplar-free approach with a frozen backbone as the feature extractor and a closed-form incremental classifier whose weight equals the joint-learning weight for the same class-weighted ridge objective. AIR addresses class imbalance with an analytic reweighting module (ARM) that calculates a reweighting factor for each class in the loss function to equalize total sample weights across classes. Under long-tailed class-incremental learning, AIR leads 28 baselines in aggregate accuracy and exemplar-free methods in aggregate macro F1, gaining 3.21% accuracy and 2.14% macro F1 over the respective strongest exemplar-free baselines. Under the Si-Blurry setting with recurring classes, AIR leads 15 exemplar-based and exemplar-free baselines, gaining 2.32% aggregate accuracy and 1.27% aggregate macro F1 over the strongest baseline. One-sided paired tests support positive mean absolute gains in these four comparisons (Holm-adjusted p<0.006).

Di Fang, Yinan Zhu, Zhiping Lin +3

School of Future Technology, South China University of Technology, Guangzhou, China · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China

Class-Imbalanced Learning Class-Incremental Learning Continual Learning Online Continual Learning Incremental Learning

Ask PDF

Aug 21, 2026

SPARCL: Spectral Partitioned Analytic Continual Learning

Analytic continual learning has emerged as a strong exemplar-free alternative to gradient-based class-incremental learning because it replaces iterative optimization with closed-form ridge updates. Yet the usual forgetting narrative, centered on stochastic gradient overwriting, does not explain why analytic methods still drift on old classes despite exact recursive solvers. We identify the culprit as spectral interference: the joint ridge classifier for all tasks shares the inverse autocorrelation operator

(R+λI)^{-1}

, so incoming task samples that load onto old dominant eigendirections dilute the spectrum and perturb old-class logits even when old labels are never revisited. Based on this view, we propose SPARCL, a spectral partitioned analytic continual learner that decomposes the running autocorrelation into a high-energy core and a residual complement, freezes old-class classifier components in the core subspace, and updates only the residual block through recursive least squares with an optional residual random-projection expansion. This yields a simple closed-form update with a provable invariance guarantee for the core contribution of old logits. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A under a frozen ViT-B/16 protocol, SPARCL closes most of the gap from classical analytic learners to strong representation matchers, while remaining complementary to sparse feature-decorrelation approaches such as Fly-CL.

James Hartley, Zeropy Surio, Daniel Whitmore +2

University of Sheffield

Continual Learning Class-Incremental Learning Spectral Methods

Ask PDF

May 11, 2026

Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View

Loss reweighting is a widely used strategy for long-tailed classification, but existing reweighting strategies often rely on heuristics and rarely define a well-specified target. Inspired by Neural Collapse (NC), the ideal simplex Equiangular Tight Frame (ETF) terminal geometry suggests equal per-class average loss as a reasonable target for reweighting. Based on the ideal equal loss objective, we consider loss reweighting as an inverse problem and propose an inverse-view reweighting strategy that infers class weights dynamically to match this ideal objective. Empirically, NC metrics suggest our method can effectively reduce the loss imbalance coefficient and closer alignment with NC geometry while consistently outperforming strong long-tailed baselines on different datasets.

Jinping Wang, Zixin Tong, Zhiwu Xie +1

CSMT, Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University

Long-Tailed Classification Adaptive Loss Weighting

Ask PDF

Abstract

Continual Learning

Explore similar work

AIR: Analytic Imbalance Rectifier for Continual Learning

SPARCL: Spectral Partitioned Analytic Continual Learning

Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View