Reliable Replay through Spatial Coherence in Online Continual Learning

Haixiang Sun; Jiefu Zhang; Yinghao He; Yang Xu; Vaneet Aggarwal; Bharat Bhargava; Andrew L. Liu

stat.MLSep 27, 2026

Reliable Replay through Spatial Coherence in Online Continual Learning

Authors: Haixiang Sun, Jiefu Zhang, Yinghao He, Yang Xu, Vaneet Aggarwal, Bharat Bhargava, Andrew L. Liu

Organizations: Purdue University West Lafayette, IN 47907, USA

Abstract

Continually adapting models to new tasks requires retaining earlier knowledge under limited memory and computation. Experience replay addresses this challenge, but priorities based on individual loss increases overlook how related memories respond to the same update and can overemphasize isolated responses. We introduce SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings. SPHERE uses a representation kernel to aggregate signed prospective loss changes, attenuating unsupported spikes while retaining coherent increases. It then formulates allocation as entropy-regularized transport, redistributing uniform source mass toward supported high-risk regions while penalizing long-distance transfers. We derive replay coefficients from the transport objective's sensitivity to the original loss changes and blend them with uniform replay to maintain baseline rehearsal. Our analysis establishes conditions under which kernel aggregation improves risk estimation and bounds transport-value inflation due to residual noise and smoothing bias. Experiments demonstrate that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-generation reinforcement learning with incomplete test rewards.

Figures & tables

Figure 1: Isolated Spike and Coherent Forgetting.

	Buffer size 300				Buffer size 3000
Method	FP $\uparrow$	Worst Forget $\downarrow$	Tail Forget $\downarrow$	Min. $\uparrow$	FP $\uparrow$	Worst Forget $\downarrow$	Tail Forget $\downarrow$	Min. $\uparrow$
No replay	.415 $\pm$ .146	.811 $\pm$ .156	.693 $\pm$ .173	.130	.415 $\pm$ .146	.811 $\pm$ .156	.693 $\pm$ .173	.130
ER	.661 $\pm$ .075	.245 $\pm$ .149	.200 $\pm$ .137	.423	.641 $\pm$ .071	.329 $\pm$ .143	.269 $\pm$ .112	.479
Loss-prop	.689 $\pm$ .045	.188 $\pm$ .061	.142 $\pm$ .043	.538	.618 $\pm$ .050	.325 $\pm$ .085	.269 $\pm$ .076	.502
MIR-prop	.675 $\pm$ .045	.188 $\pm$ .083	.152 $\pm$ .072	.570	.635 $\pm$ .042	.296 $\pm$ .111	.237 $\pm$ .079	.561
MIR-top	.621 $\pm$ .023	.244 $\pm$ .055	.211 $\pm$ .060	.588	.633 $\pm$ .066	.375 $\pm$ .205	.285 $\pm$ .155	.500

Table 1: Replay performance with limited memory and replay. Values are mean

\pm

SD except Min.

Metric	$S=I$	SPHERE	Gain
Final acc. $\uparrow$	64.90	69.32	+4.42
Incremental acc. $\uparrow$	66.08	68.54	+2.47
New task acc. $\uparrow$	70.34	72.43	+2.08
Mean forgetting $\downarrow$	8.88	6.18	+2.70
Worst forgetting $\downarrow$	24.75	16.85	+7.90
Tail forgetting $\downarrow$	20.59	13.54	+7.05

Table 2: Matched neighborhood-block ablation (%).

Variant	FP $\uparrow$	Worst $\downarrow$	Tail $\downarrow$	Min. $\uparrow$
Full SPHERE	.690 $\pm$ .021	.180 $\pm$ .053	.143 $\pm$ .035	.661
Consensus only	.680 $\pm$ .021	.189 $\pm$ .054	.154 $\pm$ .030	.647
No transport	.640 $\pm$ .063	.275 $\pm$ .129	.227 $\pm$ .110	.478
Permuted geometry	.666 $\pm$ .056	.213 $\pm$ .082	.188 $\pm$ .078	.497
Matched MIR	.682 $\pm$ .048	.192 $\pm$ .113	.161 $\pm$ .087	.517
No uniform floor	.673 $\pm$ .034	.230 $\pm$ .082	.183 $\pm$ .063	.608

Table 3: Component controls with scarce memory and replay. Values are mean

\pm

SD except Min.

	CIFAR-100 ( $K=2{,}000$ )				TinyImageNet ( $K=4{,}000$ )
Method	Clean	20%	40%	60%	Clean	20%	40%	60%
ER	19.6 $\pm$ 3.4	14.3 $\pm$ 1.1	10.2 $\pm$ 1.1	5.1 $\pm$ 0.8	16.3 $\pm$ 1.2	12.0 $\pm$ 0.6	7.4 $\pm$ 0.6	3.3 $\pm$ 0.4
ER + SPHERE	20.9 $\pm$ 1.8	15.7 $\pm$ 1.2	12.2 $\pm$ 0.8	8.0 $\pm$ 0.7	15.3 $\pm$ 1.4	13.5 $\pm$ 0.9	10.1 $\pm$ 0.6	6.1 $\pm$ 0.7
MIR-prop	19.8 $\pm$ 1.4	15.0 $\pm$ 2.3	10.2 $\pm$ 1.6	6.2 $\pm$ 0.8	17.1 $\pm$ 0.5	11.6 $\pm$ 1.1	8.0 $\pm$ 0.8	4.7 $\pm$ 0.2
DER++	15.3 $\pm$ 4.1	10.9 $\pm$ 4.1	9.0 $\pm$ 2.0	5.6 $\pm$ 1.0	9.8 $\pm$ 1.0	7.4 $\pm$ 0.9	4.8 $\pm$ 0.7	3.4 $\pm$ 0.3
ER-ACE	20.4 $\pm$ 3.4	16.8 $\pm$ 2.1	12.9 $\pm$ 1.5	8.1 $\pm$ 1.1	17.9 $\pm$ 1.7	14.4 $\pm$ 0.8	9.6 $\pm$ 0.8	5.5 $\pm$ 0.4

Table 4: Replay across hosts under symmetric label noise.

Method	Class-IL $\uparrow$	Task-IL $\uparrow$	AAA $\uparrow$	avgF $\downarrow$	worst-class $\uparrow$	CVaR ${}_{0.2}\downarrow$
ER	33.4 $\pm$ 4.5	76.1 $\pm$ 5.1	51.4 $\pm$ 4.8	42.9 $\pm$ 5.6	5.6 $\pm$ 4.5	72.5 $\pm$ 5.7
ER-ACE	36.1 $\pm$ 4.5	77.9 $\pm$ 6.5	53.2 $\pm$ 2.1	25.3 $\pm$ 5.5	11.1 $\pm$ 7.2	54.8 $\pm$ 8.9
GDumb	36.8 $\pm$ 0.9	77.0 $\pm$ 2.8	28.1 $\pm$ 0.7	20.2 $\pm$ 7.4	8.4 $\pm$ 5.4	51.2 $\pm$ 19.0
DER++	28.2 $\pm$ 4.3	80.7 $\pm$ 4.8	48.4 $\pm$ 4.7	57.5 $\pm$ 9.3	1.3 $\pm$ 1.7	81.3 $\pm$ 7.0
MIR	32.1 $\pm$ 4.4	75.6 $\pm$ 4.1	48.5 $\pm$ 3.5	36.9 $\pm$ 7.0	5.5 $\pm$ 3.6	71.1 $\pm$ 6.9
X-DER	28.1 $\pm$ 8.2	75.1 $\pm$ 8.0	44.9 $\pm$ 5.3	25.0 $\pm$ 9.9	0.0 $\pm$ 0.0	60.0 $\pm$ 16.1

Table 5: CIFAR-10 at 20% noise,

K=500

(mean

\pm

SD).

Metric	SPHERE	$S=I$
Final acc $\uparrow$	46.61 $\pm$ 2.19	45.42 $\pm$ 2.99
Stage-average acc $\uparrow$	59.01 $\pm$ 3.00	58.93 $\pm$ 3.30
Mean forgetting $\downarrow$	35.10 $\pm$ 5.90	36.55 $\pm$ 7.71
Worst forgetting $\downarrow$	62.66 $\pm$ 9.63	69.85 $\pm$ 12.42
Top-2 forgetting $\downarrow$	59.53 $\pm$ 8.81	61.84 $\pm$ 10.69

Table 6: Neighborhood-block ablation on clean Split CIFAR-10.

Method	Final $\uparrow$	Old $\uparrow$	Worst $\uparrow$	$F\downarrow$	BWT $\uparrow$
Uniform replay	42.75 $\pm$ 2.92	43.56	40.00	4.00	-2.78
MIR	44.50 $\pm$ 1.63	45.78	38.67	4.56	-3.89
No Transport	42.67 $\pm$ 1.70	41.78	39.67	6.56	-5.56
Permuted geometry	48.58 $\pm$ 1.88	45.78	41.67	3.56	-2.89
SPHERE	49.08 $\pm$ 0.80	46.89	43.67	3.33	-1.56

Table 7: Code-generation results.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Setting
Representation	Current T5-Large with its LoRA adapters; last decoder layer, mean-pooled over target-label tokens (1024 dimensions), then $\ell_{2}$ -normalized. Refreshed with the candidate pool.
Distance	$C_{ij}=\\|h_{i}-h_{j}\\|_{2}/\nu$ , where $\nu=\mathrm{median}_{k<l}\\|h_{k}-h_{l}\\|_{2}$ within the pool.
Allocation	$\epsilon_{s}=0.2$ , $\epsilon_{T}=0.5$ ; $\tau=c_{\beta}\,\mathrm{std}([S\delta]_{+})/\epsilon_{T}$ with $c_{\beta}=1$ ; $\alpha=0.5$ .
Optimization	AdamW learning rate $10^{-3}$ for training; one virtual SGD step on LoRA parameters with $\eta_{v}=10^{-2}$ .
Memory and scoring	$K\in\{300,3000\}$ ; up to 512 distinct candidates, refreshed every four replay events.
Replay	Four examples per update, without replacement; unit-mass mean replay loss, $\lambda=1$ .

Appendix

Table 8: Instruction-tuning configuration. Distances and allocation scales are computed on the scored candidate pool.

Order	Task sequence
0	mnli, cb, wic, copa, qqp, boolq, rte, imdb, yelp, amazon, sst2, dbpedia, agnews, multirc, yahoo
1	multirc, boolq, wic, mnli, cb, copa, qqp, rte, imdb, sst2, dbpedia, agnews, yelp, amazon, yahoo
2	yelp, amazon, mnli, cb, copa, qqp, rte, imdb, sst2, dbpedia, agnews, yahoo, multirc, boolq, wic

Appendix

Table 9: Task orders for instruction-tuning evaluation.

Dataset	Host	$k_{s}$	$k_{T}$	$T_{\mathrm{frac}}$	Interval	$\eta$
CIFAR-10, MNIST	ER	5	25	0.5	4	0.1
CIFAR-10	ER-ACE	10	50	1.0	4	0.1
CIFAR-10	X-DER	5	50	0.5	8	0.03
CIFAR-100	ER, ER-ACE	5	25	0.5	4	0.1
CIFAR-100	X-DER	5	25	0.5	4	0.03
TinyImageNet	ER	5	25	0.5	4	0.1

Appendix

Table 10: Vision allocation settings. The last two columns give the rescoring interval in stream steps and the virtual-step size.

\alpha=0.5

; ER and ER-ACE use

\lambda=1

Method	Clean	20%	40%	60%	80%
Stream-time replay
ER	87.3 $\pm$ 0.8	65.3 $\pm$ 4.0	46.9 $\pm$ 5.4	31.3 $\pm$ 5.8	19.2 $\pm$ 2.9
ER + SPHERE	86.3 $\pm$ 1.1	70.8 $\pm$ 6.7	54.1 $\pm$ 4.9	31.5 $\pm$ 5.5	19.7 $\pm$ 3.1
MIR	88.2 $\pm$ 0.6	61.6 $\pm$ 6.6	45.7 $\pm$ 5.0	31.5 $\pm$ 5.5	20.6 $\pm$ 2.5
DER++	90.7 $\pm$ 0.6	77.5 $\pm$ 6.0	62.1 $\pm$ 4.6	38.0 $\pm$ 3.5	20.7 $\pm$ 3.0
ER-ACE	88.8 $\pm$ 1.1	75.7 $\pm$ 9.4	66.2 $\pm$ 2.8	43.7 $\pm$ 2.9	21.3 $\pm$ 2.8

Appendix

Table 11: Split MNIST Class-IL accuracy (%,

K=500

; mean

\pm

SD).

Stream-time replay
Method	Clean	20%	40%	60%
ER	43.4 $\pm$ 5.7	33.4 $\pm$ 4.5	25.1 $\pm$ 2.9	17.4 $\pm$ 2.7
ER + SPHERE	47.6 $\pm$ 3.8	35.8 $\pm$ 2.7	27.6 $\pm$ 1.6	19.9 $\pm$ 1.8
MIR-prop	44.6 $\pm$ 5.2	32.1 $\pm$ 4.4	24.7 $\pm$ 2.1	18.1 $\pm$ 1.2
DER++	43.7 $\pm$ 5.1	28.2 $\pm$ 4.3	23.3 $\pm$ 4.8	18.2 $\pm$ 3.3
ER-ACE	39.8 $\pm$ 4.5	36.1 $\pm$ 4.5	27.9 $\pm$ 3.4	20.3 $\pm$ 2.3

Appendix

Table 12: Final Class-IL accuracy on Split CIFAR-10 (%, memory 500). Values are mean

\pm

SD. Best means are bold within each protocol.

40% symmetric noise
Method	Class-IL $\uparrow$	Task-IL $\uparrow$	AAA $\uparrow$	avgF $\downarrow$	Worst-class $\uparrow$	CVaR ${}_{0.2}\downarrow$
ER	25.1 $\pm$ 2.8	71.6 $\pm$ 3.4	45.3 $\pm$ 3.6	49.7 $\pm$ 5.1	2.6 $\pm$ 1.5	81.1 $\pm$ 3.5
ER-ACE	27.9 $\pm$ 3.4	72.7 $\pm$ 4.1	47.0 $\pm$ 3.3	28.4 $\pm$ 4.5	8.3 $\pm$ 6.0	59.4 $\pm$ 8.6
ER-ACE + SPHERE	31.3 $\pm$ 2.4	74.3 $\pm$ 5.4	50.2 $\pm$ 3.0	18.4 $\pm$ 6.4	16.4 $\pm$ 3.6	52.2 $\pm$ 8.0
DER++	23.3 $\pm$ 4.8	76.8 $\pm$ 4.7	39.7 $\pm$ 4.9	60.6 $\pm$ 9.5	2.0 $\pm$ 4.6	86.9 $\pm$ 7.5
MIR-prop	24.7 $\pm$ 2.1	69.3 $\pm$ 3.8	38.7 $\pm$ 2.4	37.9 $\pm$ 5.2	2.5 $\pm$ 2.5	72.8 $\pm$ 5.3

Appendix

Table 13: CIFAR-10 retention (%,

K=500

; mean

\pm

SD). Bold compares ER-ACE with its SPHERE integration.

Control	Operation
$S=I$	Use the identity for aggregation and omit its pullback; retain clipping, transport with $C$ , and the uniform floor.
Consensus only	Replace $q^{\ast}(v)$ by $\operatorname{softmax}_{T}(v)$ , with $T$ chosen to match the ESS of the full method’s weights. Set $w^{\mathrm{SC}}=S^{\top}(\operatorname{softmax}_{T}(v)\odot\chi)$ , and retain the uniform mixture and normalization in Eq. ( 83 ).
No transport	Normalize the supported field directly, $\mathcal{N}(v)$ , without transport or the $S^{\top}$ pullback.
Permuted geometry	At every pool refresh, randomly permute feature rows before constructing $C$ and $S$ , breaking their alignment with the memories while retaining the full allocation procedure.
Matched MIR	Use $\operatorname{softmax}_{T}([\delta]_{+})$ with $T$ chosen to match the ESS of the full method’s weights.
No uniform floor	Set $\alpha=1$ ; remove only the uniform mixture from the full allocation rule.

Appendix

Table 14: Instruction-tuning controls in Tables 5.1 and 5.1 . The two transport-free controls use different allocation rules.

Explore similar work

Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining
Sep 30, 2026Lukas Thede, Shengzhuang Chen, Stefan Winzeck +3Replay Pretraining
University of Tübingen, Tübingen AI Center · Helmholtz Munich · Munich Center for Machine Learning (MCML) +3
Rethinking Transfer in Continual Learning: A Replay-Based Realisation
Jul 17, 2026Yang Meng, Zhenya Liu, Zhuokai Zhao +1Replay-Based Continual Learning Continual Learning
Department of Computer Science, University of Chicago
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Replay-Based Continual Learning Continual Learning
Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining

Lukas Thede, Shengzhuang Chen, Stefan Winzeck, Matthias Bethge+2

University of Tübingen, Tübingen AI Center · Helmholtz Munich · Munich Center for Machine Learning (MCML)+3

Replay Pretraining Rapid Adaptation Forgetting Curriculum

Open paper AskPDF arXiv

Continued pretraining enables language models to adapt to new domains and knowledge, but often at the cost of forgetting previously acquired capabilities. Replay can mitigate this trade-off, but fixed replay mixtures allocate training independently of the model's actual retention needs. We introduce Replay on Demand (RoD), which instead derives the replay allocation from the model's learning dynamics. RoD jointly prioritizes adaptation samples by their remaining learning potential and replay samples by their observed forgetting. Their competition for a shared training budget yields an online curriculum that determines what to train on at each step. Across models, scales, and adaptation domains, RoD reaches or improves upon the adaptation-forgetting frontier of tuned fixed-replay baselines and model merging without prescribing a replay allocation in advance. Replay concentrates on sources that are more vulnerable to forgetting and dynamically increases and redistributes as forgetting emerges during training. Together, our results show that replay can be allocated online from the model's evolving state, targeting what is needed, when it is needed.

Abstract

Explore similar work

Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining

Rethinking Transfer in Continual Learning: A Replay-Based Realisation

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning