RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA

Wen-Hsuan Liu; Amara Okonkwo; Tom Saliencro; Rohan Desai; Maya Lindqvist

cs.LGNov 2, 2026

RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA

Authors: Wen-Hsuan Liu, Amara Okonkwo, Tom Saliencro, Rohan Desai, Maya Lindqvist

Organizations: University of California, Irvine · University of Washington

Abstract

Parameter-efficient continual learning on pre-trained Vision Transformers largely relies on a costly premise: preventing catastrophic forgetting requires either spawning a dedicated LoRA branch per session---causing adapter parameters to scale linearly with task horizon---or applying sparse Mixture-of-Experts gating symmetrically to both forward and backward passes, which sacrifices representational rank at test time. We argue that reading consolidated knowledge and writing new task credit are fundamentally asymmetric operations. Because visual features in a shared low-rank space distribute across rank-1 atoms rather than localizing in disjoint task silos, forward inference should read all atoms at full rank capacity, whereas backward optimization should restrict plasticity to a sparse subset of input-aligned channels. We introduce RASP (Read-All, Sparse-Plasticity LoRA), which adapts a single stationary-size adapter across sequential tasks. At each step, a scene-modulated cosine router selects the top- $q$ unlocked atoms to receive gradients, blocks updates on the remaining $r-q$ channels, and penalizes Frobenius Gram coherence between active and non-updated output directions. Across sessions, the router's empirical activation trace and each atom's exact spectral norm form a zero-overhead geometric consolidation index $ω_j = \sqrt{\tilde{c}_j \tildeσ_j}$ that drives Cauchy step attenuation and boundary anchor locking. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A, RASP matches or surpasses branch-expanding LoRA learners under a strictly constant parameter footprint.

Figures & tables

Figure 1: Motivation for RASP. (a) Across all four benchmarks, decoupling full-rank forward reading from sparse backward writing consistently outperforms both unmasked shared updates (

q=r

) and symmetric top-

q

gating, with trace-spectral consolidation providing an additional boost. (b) On CIFAR-100, unmasked shared training suffers severe old-task logit drift; symmetric gating curbs drift at the expense of forward rank; RASP suppresses

84.0\%

of cumulative logit drift while preserving full forward capacity. (c) At

T=10

sessions under a unified ViT-B/16 protocol, RASP occupies the Pareto frontier of final accuracy versus total adapter parameters.

Figure 2: Overview of the RASP pipeline. Stage 1 (Left): Every input batch traverses all

r

rank-1 atoms in the forward pass without truncation; a cosine probe modulated by an image-level scene gate

\phi(\bar{x})

and a usage-equalizing prior scores the unlocked pool

\mathcal{P}_{t-1}

. Stage 2 (Middle): Only the top-

q

unlocked channels

\mathcal{A}_{t}

receive backward gradients, while a Frobenius Gram coherence penalty

\mathcal{R}_{\mathrm{dec}}

pushes active synthesis columns orthogonal to non-updated and locked columns. Stage 3 (Right): Cumulative routing trace

\tilde{c}_{j}

and rank-1 spectral norm

\tilde{\sigma}_{j}

form the consolidation index

\omega_{j}=\sqrt{\tilde{c}_{j}\tilde{\sigma}_{j}}

, driving continuous Cauchy step attenuation and boundary anchor locking.

Prompt-tuning families
				CIFAR-100		CUB-200		ImageNet-R		ImageNet-A
Method	Venue	$\Delta$ Params	Total@ $T_{10}$	FAA	CAA	FAA	CAA	FAA	CAA	FAA	CAA
L2P [ 35 ]	CVPR’22	+0	0.04M	82.79	88.45	62.28	73.89	66.54	72.81	33.84	45.91
DualPrompt [ 34 ]	ECCV’22	+0	0.04M	85.61	90.36	66.08	77.96	68.47	72.64	33.79	46.83
CODA-Prompt [ 28 ]	CVPR’23	+0.31M	3.07M	86.65	91.12	74.83	83.06	71.52	75.86	42.79	53.61
Adapter & prototype-surgery families

Table 1: Comparison with state-of-the-art rehearsal-free CIL methods across four 10-session benchmarks on ViT-B/16-IN21K. “

\Delta

Params” denotes additional adapter parameters allocated per session, and “Total@

T_{10}

” is the cumulative adapter overhead after 10 sessions ( † :

0.25

M for

r=50

0.30

M for

r=64

0.35

M for

r=80

, including the fixed

0.10

M scene gate). All cells are evaluated under a unified 3-seed protocol. Bold and underline indicate best and second-best among LoRA-based methods.

Benchmark	$r$	$q$ ( $q/r$ )	lr	Epochs	$\kappa$
CIFAR-100	50	6 (12.0%)	$5\mathrm{e}{-4}$	20	0.05
CUB-200	50	15 (30.0%)	$5\mathrm{e}{-4}$	20	0.05
ImageNet-R	64	18 (28.1%)	$1\mathrm{e}{-3}$	30	0.08
ImageNet-A	80	50 (62.5%)	$5\mathrm{e}{-4}$	20	0.05

Table 2: Benchmark-specific operating configurations for RASP. Total rank

r

, active plasticity quota

q

, base learning rate, training epochs per session, and Cauchy attenuation gain

\kappa

Figure 3: Mechanistic analysis of RASP. (a, b) Effect of plasticity ratio

q/r

\Delta

FAA (relative to

q=r

) and Average Forgetting across four benchmarks; stars mark the operating configurations in Table 2 . The optimal

q/r

shifts rightward as domain divergence from ImageNet-21K increases. (c) Spearman rank correlation

\rho_{s}

of consolidation indices

\omega^{(t)}

against session

t-1

and final session

T

, showing rapid stabilization after session 3. (d) In the 2D trace-spectral plane

(\tilde{c}_{j},\tilde{\sigma}_{j})

, hyperbolic contours of

\omega_{j}=\sqrt{\tilde{c}_{j}\tilde{\sigma}_{j}}

isolate true high-trace, high-energy anchors (crimson diamonds) while rejecting single-cue outliers (amber triangles).

Method	Venue	Total@ $T_{20}$	FAA	CAA	$F_{20}$ $\downarrow$
InfLoRA [ 15 ]	CVPR’24	3.06M	71.08	77.34	6.42
PLAN [ 32 ]	ICCV’25	3.06M	71.15	77.98	6.18
LoRA-P&M [ 24 ]	NeurIPS’25	3.06M	76.94	82.85	4.12
SplitLoRA [ 23 ]	ICLR’26	3.06M	78.79	84.52	3.45
FM-LoRA [ 41 ]	CVPRW’25	0.25M	74.62	80.41	4.86
RASP (Ours)	–	0.30M	78.16	83.64	3.18

Table 3: Long-horizon scalability on ImageNet-R (

T=20

sessions, 10 classes/session). “Total@

T_{20}

” is the total adapter parameter count after session 20, and

F_{20}

denotes final forgetting.

Configuration	FAA	Forget $\downarrow$	FAA	Forget $\downarrow$
	CIFAR-100		CUB-200
1. Unmasked shared LoRA ( $q=r$ )	89.48	5.68	79.06	6.81
2. Symmetric sparse (Fwd & Bwd)	89.64	5.01	79.52	6.24
3. Read-All Fwd + Sparse Bwd	89.89	4.86	80.34	5.95
4. $+$ Scene modulator $\phi(\bar{x})$ only	89.82	4.80	80.61	5.84
5. $+$ Gram decoupling $\mathcal{R}_{\mathrm{dec}}$ only	89.79	4.88	80.52	5.89

Table 4: Stage-1 architectural ablation on CIFAR-100 and CUB-200. Every row includes trace-spectral consolidation in Stage 2; Row 7 replaces null-space probe initialization on unlocked channels

\mathcal{P}_{t-1}

with Kaiming normal initialization.

Figure 4: Trajectories, resilience, and system cost. (a) On CIFAR-100, RASP matches the top expansion method across all 10 sessions at constant size. (b) On CUB-200, shared forward reading yields a

+8.48

pp final lead over branch-isolated SplitLoRA. (c) Silencing the top-

m

task-associated atoms after training collapses isolated branches (

-23.84\%

m=10

) but costs

<1\%

on RASP, confirming distributed encoding. (d) Training time and peak VRAM on CIFAR-100.

Part A: Protection mechanisms
	ImageNet-A		ImageNet-R
Consolidation Variant	FAA	Forget $\downarrow$	FAA	Forget $\downarrow$
Stage-1 Read-All/Sparse-Bwd only	57.24	5.68	78.42	2.94
$+$ Boundary anchor locking $\mathcal{L}_{t}$	57.62	5.47	78.91	2.72
$+$ Cauchy step attenuation	58.02	5.18	79.26	2.58
$+$ Lock $u_{j}$ only (re-init $v_{j}$ )	57.63	5.44	78.84	2.76

Table 5: Stage-2 consolidation ablations on ImageNet-A and ImageNet-R. Part A ablates Cauchy step attenuation and boundary locking; Part B compares candidate consolidation indices.

Method	Time (s/ep) $\downarrow$	VRAM $\downarrow$	Params@ $T_{1}$	Params@ $T_{10}$
InfLoRA [ 15 ]	241.8	13.24 GB	0.15M	1.53M
LoRA-P&M [ 24 ]	156.4	8.15 GB	0.15M	1.53M
SplitLoRA [ 23 ]	140.6	7.52 GB	0.15M	1.53M
RASP (Ours)	121.9	3.41 GB	0.25M	0.25M

Table 6: Computational and memory efficiency on CIFAR-100 (single NVIDIA A100 GPU, batch size 64). Adapter parameter counts for RASP include the shared LoRA (

153

K) and fixed scene gate (

98

K).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Domain Characteristic	Classes	Train / Test Images	Sessions $T$	Classes / Sess.	Rank $r$
CIFAR-100 [ 11 ]	Standard natural objects	100	50,000 / 10,000	10	10	50
CUB-200-2011 [ 30 ]	Fine-grained bird species	200	5,994 / 5,794	10	20	50
ImageNet-R [ 7 ]	Multi-style artistic renditions	200	24,000 / 6,000	10 / 20	20 / 10	64
ImageNet-A [ 8 ]	Natural adversarial examples	200	5,981 / 1,499	10	20	80

Appendix

Table 7: Summary of continual learning benchmarks and incremental session protocols.

Plasticity Ratio Band ( $q/r$ )	CIFAR-100 ( $r=50$ )	CUB-200 ( $r=50$ )	ImageNet-R ( $r=64$ )	ImageNet-A ( $r=80$ )
Ultra-sparse ( $\sim$ 8–12.5%)	90.41 / 4.71 ( $q=6$ )	79.62 / 5.31 ( $q=5$ )	77.96 / 2.29 ( $q=8$ )	54.82 / 6.14 ( $q=10$ )
Low-moderate ( $\sim$ 20–25%)	90.09 / 5.01 ( $q=12$ )	80.68 / 5.46 ( $q=10$ )	79.08 / 2.36 ( $q=14$ )	56.04 / 5.81 ( $q=20$ )
Mid-range ( $\sim$ 28–38%)	89.92 / 5.19 ( $q=18$ )	81.36 / 5.62 ( $q=15$ )	79.68 / 2.43 ( $q=18$ )	56.98 / 5.39 ( $q=30$ )
High-moderate ( $\sim$ 50–63%)	89.76 / 5.38 ( $q=25$ )	80.44 / 6.21 ( $q=25$ )	78.73 / 3.06 ( $q=32$ )	58.52 / 4.96 ( $q=50$ )
Near-dense ( $\sim$ 70–75%)	89.61 / 5.54 ( $q=38$ )	79.79 / 6.52 ( $q=35$ )	78.15 / 3.52 ( $q=48$ )	58.09 / 5.24 ( $q=60$ )
Unmasked ( $100\%,q=r$ )	89.48 / 5.68 ( $q=50$ )	79.06 / 6.81 ( $q=50$ )	77.62 / 3.91 ( $q=64$ )	56.84 / 5.91 ( $q=80$ )

Appendix

Table 8: Full plasticity-quota sweep across all four benchmarks (FAA % / Forgetting

F_{10}

%). Bold entries mark the default operating configurations from Table 2 .

Method	$\bar{a}_{\mathrm{diag}}$ $\uparrow$	$\mathrm{FWT}_{\mathrm{rel}}$ $\uparrow$	$F_{10}$ $\downarrow$	FAA $\uparrow$	$\bar{a}_{\mathrm{diag}}$ $\uparrow$	$\mathrm{FWT}_{\mathrm{rel}}$ $\uparrow$	$F_{10}$ $\downarrow$	FAA $\uparrow$
	CUB-200 (Fine-Grained)				ImageNet-A (Adversarial)
InfLoRA [ 15 ]	85.12	$+0.14$	5.84	79.92	55.42	$+0.08$	6.85	49.26
SplitLoRA [ 23 ]	81.46	$-0.32$	9.54	72.88	62.84	$+0.18$	5.42	57.98
FM-LoRA [ 41 ]	83.95	$+1.12$	6.79	77.84	59.68	$+0.64$	5.81	54.45
Symmetric Top- $q$ (Fwd+Bwd)	84.62	$+1.28$	6.24	79.52	61.95	$+0.72$	5.21	57.26
RASP (Ours)	86.42	$+$ 2.64	5.62	81.36	62.98	$+$ 1.42	4.96	58.52

Appendix

Table 9: Decomposition of immediate session plasticity (

\bar{a}_{\mathrm{diag}}

), relative forward transfer (

\mathrm{FWT}_{\mathrm{rel}}

), and forgetting (

F_{10}

) on CUB-200 and ImageNet-A.

Figure 5: Depth-wise routing diagnostics and attenuation profiles. (a) Usage-equalized routing prevents early entropy collapse across ViT-B/16 blocks while permitting semantic specialization in late sessions. (b) Routing trace

\tilde{c}_{j}

dominates in shallow blocks and spectral norm

\tilde{\sigma}_{j}

dominates in deep blocks; their geometric mean

\omega_{j}

equalizes layer-wise consolidation pressure. (c) Comparison of Cauchy, Linear, and Gaussian step-attenuation profiles (parentheses show ImageNet-A FAA).

Attenuation Schedule $g_{u_{j}}\leftarrow\psi(\omega_{j})\,g_{u_{j}}$	CIFAR-100	CUB-200	ImageNet-R	ImageNet-A
No attenuation ( $\kappa=0$ , boundary lock only)	90.08	80.79	78.91	57.62
Linear decay $\psi(\omega_{j})=\max(0,\,1-\kappa\,\omega_{j})$	90.19	80.96	79.31	57.88
Gaussian decay $\psi(\omega_{j})=\exp(-\kappa\,\omega_{j}^{2})$	90.04	80.72	79.12	57.49
Cauchy attenuation $\psi(\omega_{j})=(1+\kappa\,\omega_{j})^{-1}$ ( $\kappa=0.02$ )	90.26	81.14	79.44	58.19
Cauchy attenuation $\psi(\omega_{j})=(1+\kappa\,\omega_{j})^{-1}$ ( $\kappa\in\{0.05,0.08\}$ , Default )	90.41	81.36	79.68	58.52
Cauchy attenuation $\psi(\omega_{j})=(1+\kappa\,\omega_{j})^{-1}$ ( $\kappa=0.25$ )	89.95	80.81	79.18	57.94

Appendix

Table 10: Comparison of step-attenuation functional forms and gain

\kappa

across all four benchmarks (FAA %).

Boundary Locking Rule	CIFAR-100	CUB-200	ImageNet-R	ImageNet-A
$m_{\mathrm{lock}}=0$ (Cauchy attenuation only, no hard lock)	90.12	80.94	79.26	58.02
$m_{\mathrm{lock}}=\mathrm{clip}(\lfloor 0.05\|\mathcal{P}\|\rfloor,1,1)$ with guard	90.28	81.18	79.49	58.24
$m_{\mathrm{lock}}=\mathrm{clip}(\lfloor 0.05\|\mathcal{P}\|\rfloor,1,3)$ with guard ( Default )	90.41	81.36	79.68	58.52
$m_{\mathrm{lock}}=\mathrm{clip}(\lfloor 0.05\|\mathcal{P}\|\rfloor,1,3)$ + 1-session warm-up delay	90.34	81.31	79.72	58.46
$m_{\mathrm{lock}}=\mathrm{clip}(\lfloor 0.10\|\mathcal{P}\|\rfloor,1,8)$ with guard	89.90	80.64	79.05	57.74
Uncapped $5\%$ locking without starvation guard	90.02	80.85	79.21	58.12

Appendix

Table 11: Ablation of boundary anchor locking quota

m_{\mathrm{lock}}

, starvation guard (

|\mathcal{P}_{t-1}|\geq q+2

), and warm-up delay (FAA %).

Figure 6: Long-horizon scaling and session-by-task accuracy matrices. (a) On the 20-session ImageNet-R stream, RASP maintains a constant $0.30$ M footprint while tracking SplitLoRA ( $3.06$ M) within $0.63$ pp. (b, c) Lower-triangular accuracy matrices $a_{t,j}$ on CUB-200 ( $T=10$ ): RASP preserves old-session columns (vertical stability) and improves late-session diagonal learning (horizontal transfer) compared to branch-isolated SplitLoRA.

Method	$A_{1}$	$A_{2}$	$A_{3}$	$A_{4}$	$A_{5}$	$A_{6}$	$A_{7}$	$A_{8}$	$A_{9}$	$A_{10}$ (FAA)	CAA
L2P [ 35 ]	94.12	91.40	89.52	88.14	87.02	86.01	85.12	84.21	83.40	82.79	88.45
CODA-Prompt [ 28 ]	95.84	93.92	92.65	91.74	90.98	90.24	89.42	88.54	87.61	86.65	91.12
InfLoRA [ 15 ]	95.72	93.54	92.04	90.92	90.01	89.12	88.14	87.15	86.18	85.24	90.61
SD-LoRA [ 37 ]	96.14	94.72	93.71	92.94	92.21	91.52	90.74	89.85	88.94	88.05	92.58
LoRA-P&M [ 24 ]	96.32	94.96	94.02	93.28	92.58	91.91	91.14	90.28	89.39	88.49	92.92
SplitLoRA [ 23 ]	96.72	95.54	94.74	94.04	93.42	92.84	92.24	91.54	90.92	90.24	93.66

Appendix

Table 12: Session-by-session cumulative accuracy

A_{t}

(%) across

t=1,\dots,10

on CIFAR-100 (ViT-B/16-IN21K).

Method	$A_{1}$	$A_{2}$	$A_{3}$	$A_{4}$	$A_{5}$	$A_{6}$	$A_{7}$	$A_{8}$	$A_{9}$	$A_{10}$ (FAA)	CAA
CODA-Prompt [ 28 ]	91.12	87.72	85.32	83.52	81.82	80.22	78.82	77.42	76.02	74.83	83.06
InfLoRA [ 15 ]	93.42	91.12	89.42	88.02	86.72	85.32	83.92	82.52	81.12	79.92	88.21
SD-LoRA [ 37 ]	92.18	89.34	87.48	85.94	84.48	83.04	81.64	80.28	78.91	77.54	85.63
SplitLoRA [ 23 ]	90.42	87.12	85.02	83.32	81.72	80.02	78.32	76.52	74.62	72.88	83.18
FM-LoRA [ 41 ]	92.32	89.52	87.72	86.22	84.82	83.42	82.02	80.62	79.12	77.84	85.96
RASP (Ours)	92.92	90.52	88.82	87.32	86.02	84.82	83.82	82.92	82.12	81.36	85.68

Appendix

Table 13: Session-by-session cumulative accuracy

A_{t}

(%) across

t=1,\dots,10

on CUB-200-2011 (ViT-B/16-IN21K).

Benchmark & Metric	Seed 2024	Seed 2025	Seed 2026	Mean	Std
CIFAR-100 FAA (%)	90.45	90.37	90.41	90.41	0.04
CIFAR-100 CAA (%)	93.44	93.28	93.36	93.36	0.08
CUB-200 FAA (%)	81.58	81.14	81.36	81.36	0.22
CUB-200 CAA (%)	86.02	85.34	85.68	85.68	0.34
ImageNet-R FAA (%)	79.54	79.82	79.68	79.68	0.14
ImageNet-R CAA (%)	83.37	83.67	83.52	83.52	0.15

Appendix

Table 14: Three-seed breakdown (

\text{seed}\in\{2024,2025,2026\}

) for RASP across all four benchmarks.

Scene Gate Width $d_{g}$	CIFAR	CUB	IN-R	IN-A	Hyperparameter Setting	CIFAR	CUB	IN-R	IN-A
$d_{g}=16$ (0.03M)	90.16	80.92	79.34	58.14	$\lambda_{\mathrm{dec}}=0.0$ (no Gram penalty)	89.82	80.61	79.04	57.86
$d_{g}=32$ (0.05M)	90.31	81.19	79.52	58.36	$\lambda_{\mathrm{dec}}=0.1$	90.24	81.12	79.46	58.29
$d_{g}=64$ (0.10M, Default )	90.41	81.36	79.68	58.52	$\lambda_{\mathrm{dec}}=0.3$ ( Default )	90.41	81.36	79.68	58.52
$d_{g}=128$ (0.20M)	90.36	81.28	79.61	58.45	$\lambda_{\mathrm{dec}}=1.0$	90.08	80.84	79.22	58.04
Usage Prior $\mu_{\mathrm{eq}}=0.0$	89.94	80.74	79.12	57.91	Usage Prior $\mu_{\mathrm{eq}}=0.1$ ( Default )	90.41	81.36	79.68	58.52

Appendix

Table 15: Sensitivity to scene-gate bottleneck width

d_{g}

, Gram decoupling weight

\lambda_{\mathrm{dec}}

, and usage-prior weight

\mu_{\mathrm{eq}}

(FAA %).

Continual LoRA

See all 27 papers

BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning
Nov 2, 2026Tom Saliencro, Wen-Hsuan Liu, Rohan Desai +2Exemplar-Free Class-Incremental Learning Continual LoRA
University of California, Irvine · University of Washington
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
Oct 3, 2026Yuxuan Li, Fanhu Zeng, Hao TangContinual LoRA Continual Learning
School of Computer Science Peking University · Institute of Automation Chinese Academy of Sciences
Parameter Efficient Continual Learning for Sparse Event-Based Transformers
Aug 27, 2026Vaishnavi Nagabhushana, Kartikay Agrawal, Ayon BorthakurOnline Continual Learning Class-Incremental Learning
SustainAI Lab, MFSDS&AI IIT Guwahati

Nov 2, 2026J/K move · Enter open · S save

BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning

Tom Saliencro, Wen-Hsuan Liu, Rohan Desai, Amara Okonkwo+1

University of California, Irvine · University of Washington

Exemplar-Free Class-Incremental Learning Continual LoRA Low-Rank Adaptation

Open paper AskPDF arXiv

Exemplar-free continual learning on pre-trained Vision Transformers increasingly relies on Low-Rank Adaptation (LoRA), yet prevailing methods govern the low-rank update space through strict subspace isolation. By projecting each incoming task into the approximate null space of historical token activations, isolation-based learners equate any cross-task feature overlap with catastrophic interference. We show that this orthogonality premise induces a structural bottleneck we term spectral starvation: when down-projection bases are locked before task optimization, the gradient norm reaching the trainable up-projection is upper-bounded by the energy of current-task features captured by the chosen basis. Because natural visual tasks share low- and mid-level covariance structure, historical null tails contain barely

5.5\%

of incoming task variance, starving new-class plasticity while blocking positive consolidation along shared directions. We propose BISECT (Bilevel Spectral Decomposition), a continual LoRA framework that partitions a fixed rank-

r

budget into two orthogonal functional blocks: an invariant carrier spanned by the dominant modes of the joint historical-and-current Gram matrix, and a novelty complement extracted via generalized Rayleigh whitening followed by orthogonal complement projection. After single-task training, BISECT folds both blocks back into the frozen backbone using a closed-form precision gate that scales each rank-1 invariant atom by its current-to-joint Rayleigh ratio while preserving novelty atoms at unit gain. Across eight incremental settings on ImageNet-R, ImageNet-A, CIFAR-100, CUB-200, and DomainNet, BISECT outperforms prompt, adapter, and LoRA baselines with zero extra inference parameters or FLOPs, and remains complementary to sparse analytic expansion.

Abstract

Continual LoRA

Explore similar work

BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning

PaLoRA: Paced Low-Rank Adaptation for Continual Learning

Parameter Efficient Continual Learning for Sparse Event-Based Transformers