SPARCL: Spectral Partitioned Analytic Continual Learning

James Hartley; Zeropy Surio; Daniel Whitmore; Hannah Clarke; Thomas Reed

cs.LGAug 21, 2026

SPARCL: Spectral Partitioned Analytic Continual Learning

Authors: James Hartley, Zeropy Surio, Daniel Whitmore, Hannah Clarke, Thomas Reed

Organizations: University of Sheffield

Abstract

Analytic continual learning has emerged as a strong exemplar-free alternative to gradient-based class-incremental learning because it replaces iterative optimization with closed-form ridge updates. Yet the usual forgetting narrative, centered on stochastic gradient overwriting, does not explain why analytic methods still drift on old classes despite exact recursive solvers. We identify the culprit as spectral interference: the joint ridge classifier for all tasks shares the inverse autocorrelation operator $(R+λI)^{-1}$ , so incoming task samples that load onto old dominant eigendirections dilute the spectrum and perturb old-class logits even when old labels are never revisited. Based on this view, we propose SPARCL, a spectral partitioned analytic continual learner that decomposes the running autocorrelation into a high-energy core and a residual complement, freezes old-class classifier components in the core subspace, and updates only the residual block through recursive least squares with an optional residual random-projection expansion. This yields a simple closed-form update with a provable invariance guarantee for the core contribution of old logits. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A under a frozen ViT-B/16 protocol, SPARCL closes most of the gap from classical analytic learners to strong representation matchers, while remaining complementary to sparse feature-decorrelation approaches such as Fly-CL.

Figures & tables

Figure 1: Motivation for SPARCL. In analytic CIL, forgetting is not caused by SGD overwriting but by spectral interference . As new-task samples overlap with previously dominant eigendirections of the feature autocorrelation matrix, the inverse operator used by the joint ridge solution changes, diluting old modes and shifting old-class logits. SPARCL isolates a stable high-energy core and confines subsequent plasticity to the residual subspace.

Figure 2: SPARCL overview. A frozen backbone extracts features; recursive statistics $(R_{t},C_{t})$ are maintained as in ACIL. SPARCL then partitions the eigenspectrum of $R$ into a frozen high-energy core and a plastic residual band, optionally expands residual capacity with random projections, and solves a block analytic update that freezes old-class core coefficients.

Method	CIFAR-100	CUB-200	ImageNet-R	ImageNet-A
L2P [ 20 ]	87.81	77.55	76.20	49.02
DualPrompt [ 19 ]	87.39	79.72	74.05	56.88
EASE [ 27 ]	92.88	89.71	81.55	65.18
RanPAC [ 11 ]	94.15	92.58	83.10	67.41
F-OAL [ 32 ]	92.05	91.02	80.75	64.12
Fly-CL [ 36 ]	93.95	93.71	83.28	67.85

Table 1: Overall accuracy

\bar{A}

(%) on frozen ViT-B/16. Best in each column is bold ; best among classical analytic methods (ACIL–REAL) is underlined .

Method	CIFAR-100	CUB	ImageNet-R	ImageNet-A
RanPAC	85.1	34.2	66.9	29.3
F-OAL	56.4	2.1	8.6	1.1
Fly-CL	5.4	0.4	0.2	0.2
SPARCL	11.8	2.0	3.5	1.3

Table 2: Post-extraction training time per task

\tau_{\mathrm{post}}

(s) on ViT-B/16.

Variant	CIFAR-100	ImageNet-R	Forget	Drift
w/o partition	90.05	78.52	8.8	0.46
w/o core freeze	92.18	80.92	7.0	0.35
w/o residual RP	93.62	82.48	5.7	0.25
$\tau=0.85$	93.70	82.62	5.8	0.27
$\tau=0.90$	94.22	83.35	5.0	0.21
$\tau=0.95$	94.52	83.85	4.5	0.18

Table 3: Ablations on CIFAR-100 and ImageNet-R under the same protocol as Table 1 .

Figure 3: Mechanism analysis. SPARCL stabilizes leading eigenvalues, reduces old-logit drift versus ACIL/REAL, and keeps most adaptation in the residual band; Fly-CL instead decorrelates prototypes in an expanded sparse code.

Figure 4: Session-wise average accuracy and forgetting on CIFAR-100. SPARCL tracks RanPAC/Fly-CL more closely than prior analytic methods after mid-stream sessions, where spectral overlap accumulates.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Avg Acc	Forget
ACIL	90.40	9.2
DS-AL	91.55	8.1
REAL	92.85	6.6
RanPAC	94.05	5.1
Fly-CL	94.15	4.9
SPARCL	94.58	4.4

Appendix

Table 4: CIFAR-100 with longer streams (

T{=}20

, 5 classes/session).

$m$	0	64	128	256
Avg Acc	71.2	71.7	72.1	72.0
Forget	6.1	5.5	5.2	5.3

Appendix

Table 5: Residual RP width

m

on ImageNet-100.

Method	CIFAR-100	CUB	ImageNet-R	ImageNet-A
RanPAC	84.42	33.83	67.71	28.86
F-OAL	57.13	2.04	8.80	1.05
Fly-CL	5.38	0.35	0.21	0.15

Appendix

Table 6: Published

\tau_{\mathrm{post}}

(s) from Fly-CL (ViT-B/16).

Method	1	2	3	4	5	6	7	8	9	10
ACIL	90.1	84.2	79.6	76.1	73.4	71.2	69.5	68.1	67.0	66.2
REAL	90.4	85.1	81.0	78.0	75.6	73.8	72.3	71.1	70.2	69.4
RanPAC	91.0	86.4	83.0	80.4	78.3	76.7	75.4	74.4	73.6	72.9
SPARCL	91.1	86.8	83.7	81.4	79.5	78.0	76.8	75.9	75.2	74.6

Appendix

Table 7: Per-session average accuracy (%) on CIFAR-100.

Method	Mild order	Hard order
ACIL	49.1	45.0
GACL	52.4	48.3
REAL	53.8	50.1
SPARCL	58.9	56.0

Appendix

Table 8: ImageNet-R under mild vs. hard class orders.

Method	Extra params / stats	Exemplars
iCaRL / DER++	classifier + buffer	yes
L2P / DualPrompt	prompts	no
ACIL	$R,C$ ( $d\times d$ , $d\times c$ )	no
SPARCL	ACIL stats + $U_{k}$ + optional $Q$	no

Appendix

Table 9: Relative memory vs. storing a 2000-image exemplar buffer.

$\lambda\setminus\tau$	0.85	0.90	0.95	0.98
0.1	93.85	94.10	94.28	93.95
1.0	93.70	94.22	94.52	94.10
10	92.95	93.40	93.68	93.25

Appendix

Table 10:

\lambda

–

\tau

grid on CIFAR-100 (overall accuracy).

Refresh every	Avg Acc	Forget	Rel. time
1 session	94.58	4.4	1.45
2 sessions	94.52	4.5	1.18
4 sessions	94.20	4.9	1.09
never (base only)	93.05	6.2	1.05

Appendix

Table 11: Refresh period vs. wall-clock (relative to ACIL=

1.0

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 294 papers

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
Jul 24, 2026Quyen Tran, Hai Nguyen, Quan Dao +4Class-Incremental Learning Spectral Regularization
1Rutgers University · 2Tuft University · 3HUST +1
SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning
Oct 14, 2026Teif Aldaajani, Huda Hakami, Maha A. ThafarRegularization-Based Continual Learning Incremental Learning
Department of Computer Science, College of Computers and Information Technology, Taif University, Taif, 21944, Saudi Arabia .
AIR: Analytic Imbalance Rectifier for Continual Learning
Aug 19, 2024Di Fang, Yinan Zhu, Zhiping Lin +3Class-Imbalanced Learning Class-Incremental Learning
School of Future Technology, South China University of Technology, Guangzhou, China · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China

Jul 24, 2026J/K move · Enter open · S save

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

Quyen Tran, Hai Nguyen, Quan Dao, Zhuowei Li+3

1Rutgers University · 2Tuft University · 3HUST+1

Class-Incremental Learning Spectral Regularization Long-Tailed Classification Continual Learning

Open paper AskPDF arXiv

Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression (

L_2

) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix

Δ

that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.

Abstract

Continual Learning

Explore similar work

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning

AIR: Analytic Imbalance Rectifier for Continual Learning