Online continual learning aims to continuously train neural networks on a non-stationary data stream in a single pass. Rehearsal-based methods alleviate catastrophic forgetting by replaying stored samples, but the imbalance between current and previous data often leads to predictions biased toward current classes. Many approaches have been proposed to overcome the forgetting problem by correcting the bias; however, they still need to be improved in an online fashion. By decomposing the dot-product logits into an angle factor and a norm factor, our analysis reveals that the angle factor efficiently captures novel knowledge but is particularly susceptible to prediction bias, whereas the norm factor helps preserve historical knowledge. Based on these observations, we propose Dual-Logit Experience Replay (DLER), which exploits dual-logits for both model optimization and memory management. Specifically, DLER learns current samples using cosine logits and incorporates additional dot-product logits when replaying previous samples to mitigate prediction bias. To strengthen dual-logit learning under limited replay memory, an auxiliary supervised contrastive objective promotes angular compactness within each class, extending corrections learned from limited replay samples to other samples of the same class. Meanwhile, we introduce a memory update strategy that preferentially retains samples for which dot-product prediction assigns a higher probability to the correct class than cosine prediction, aligning sample selection with dual-logit training. Extensive experiments on three datasets demonstrate the effectiveness of DLER, particularly under limited memory budgets.
Figures & tables
Fig. 1: Illustration of our work. (a) The dot-product logits can be decomposed into a norm factor and an angle factor; (b) Existing studies, which learn all samples by calculating logits in the same way; (c) Different from existing studies, the model in our work learns current samples only by the angle factor, while replays previous samples by both the angle factor and the norm factor. (d) By leveraging the norm factor, the model can correctly classify samples.
Fig. 2: Average accuracy rate on observed learning stages on three datasets when the memory buffer size is 1000.
Components
Buffer size
Method
Dual-logits
α
SC
BM
Advantage
400
1000
2000
4000
ER
5.2 ± 0.6
7.6 ± 0.5
10.1 ± 0.9
12.7 ± 1.1
Dual-logit training (Fixed)
✓
8.0 ± 0.6
10.9 ± 0.7
14.2 ± 0.8
16.1 ± 0.8
Dual-logit training
✓
✓
9.7 ± 0.4
13.4 ± 0.5
16.2 ± 0.7
18.9 ± 0.8
Dual-logit training + SC
✓
✓
✓
10.0 ± 0.5
13.9 ± 0.3
17.2 ± 0.5
19.6 ± 0.7
Dual-logit training + SC + CB
✓
✓
✓
✓
10.2 ± 0.5
14.5 ± 0.3
17.4 ± 0.8
19.7 ± 0.5
TABLE II: Ablation study on Split TinyImageNet with different memory buffer sizes.
Fig. 3: Average accuracy rate of previous classes, current classes, and their average on Split MiniImageNet with a 1000 memory buffer size.
Fig. 4: Improvement of existing methods with dual-logits training on three datasets while the buffer size is 1000.
Fig. 5: The performance of dual-logits training on Split CIFAR100 (buffer size=1000) with different values of α .
Method Setting
All Classes
Current Classes
Previous Classes
ER
15.6±2.6
50.5
11.7
ER-ACE
18.9±0.9
17.3
19.1
OBC
18.3±0.7
17.5
18.4
Dual-logit training (feature)
18.6±1.0
30.6
17.2
Dual-logit training (predictor)
19.6±1.5
34.7
18.0
Dual-logit training (both)
20.1±1.2
31.8
18.8
TABLE III: Final accuracy (%) on Split CIFAR100 with a buffer of 1,000 samples after adding a linear layer to the feature extractor.
Fig. 6: Conditional coverage and previous-class accuracy of dual-logits training and its enhancement with SC on Split TinyImageNet.
Metric
High-score
Low-score
Spatial JS divergence
0.2256
0.0463
Feature norm
15.7856
12.8475
Ground-truth cosine probability
0.1173
0.0260
Gradient norm (all)
7.3114
52.7980
Gradient norm (backbone)
7.0487
51.4370
Gradient conflict rate
41.5%
44.6%
TABLE IV: Comparison of the highest- and lowest-scoring training samples within the 96 selected classes on Split TinyImageNet.
Logit configuration
Accuracy (%)
Weight norm
Weight statistics
Bias
Configuration
Learning
Replaying
Testing
Aall
Ap
Ac
∥w∥p
∥w∥c
μWp±σWp
μWc±σWc
μbp
μbc
1 (ER)
Dot
Dot
Dot
27.3
17.7
36.9
2.08
1.86
0.0044±0.0274
−0.0038±0.0221
−0.22
0.22
2
Dot
Dot
Cos
24.1
12.7
35.6
2.08
1.86
0.0044±0.0274
−0.0038±0.0221
−0.22
0.22
3 (LUCIR)
Cos
Cos
Cos
26.9
14.9
38.9
2.00
1.64
0.0398±0.0235
0.0471±0.0146
−0.00
0.00
4
Cos
Cos
Dot
28.2
28.8
27.6
2.00
1.64
0.0398±0.0235
0.0471±0.0146
−0.00
0.00
5
Dot
Cos
Dot
19.6
0.0
39.1
1.37
2.09
0.0197±0.0115
0.0219±0.0270
−1.03
1.03
TABLE VI: Comparison of logit configurations for training and inference on Split CIFAR100 with a memory buffer of 5,000 samples. Aall , Ap , and Ac denote the final accuracies (%) on all, previous, and current classes, respectively. ∥w∥ denotes the mean class-weight vector norm; μW and σW denote the mean and standard deviation of weight entries; and μb denotes the mean bias. Subscripts p and c indicate previous and current classes. Dot and Cos denote dot-product and cosine logits, respectively.
Fig. 7: The analysis results of different ways during the learning process on Split CIFAR100 when the memory buffer size is 5000.
Fig. 8: Examples of high-score and low-score samples and their class activation maps obtained using dot-product and cosine logits.
Catastrophic forgetting, the tendency of neural networks to forget previously learned knowledge when learning new tasks, has been a major challenge in continual learning (CL). To tackle this challenge, CL methods have been proposed and shown to reduce forgetting. Furthermore, CL models deployed in mission-critical settings can benefit from uncertainty awareness by calibrating their predictions to reliably assess their confidences. However, existing uncertainty-aware continual learning methods suffer from high computational overhead and incompatibility with mainstream replay methods. To address this, we propose idempotent experience replay (IDER), a novel approach based on the idempotent property where repeated function applications yield the same output. Specifically, we first adapt the training loss to make model idempotent on current data streams. In addition, we introduce an idempotence distillation loss. We feed the output of the current model back into the old checkpoint and then minimize the distance between this reprocessed output and the original output of the current model. This yields a simple and effective new baseline for building reliable continual learners, which can be seamlessly integrated with other CL approaches. Extensive experiments on different CL benchmarks demonstrate that IDER consistently improves prediction reliability while simultaneously boosting accuracy and reducing forgetting. Our results suggest the potential of idempotence as a promising principle for deploying efficient and trustworthy continual learning systems in real-world applications.Our code is available at https://github.com/YutingLi0606/Idempotent-Continual-Learning.
Zhanwang Liu, Yuting Li, Haoyuan Gao +4
School of Computer Science, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Lehigh University +1
Online Continual Learning (OCL) requires models to learn from non-stationary data streams under a strict single-pass constraint, making them highly susceptible to catastrophic forgetting. While existing studies have explored various strategies from data, architecture, and optimization perspectives, the optimizer often directly uses generic approaches such as SGD and AdamW. In this work, we reveal a new link between optimization dynamics and OCL needs by recasting Polyak-Ruppert averaging as the engine of "plasticity" and Primal averaging as the anchor of "stability". This inspires our novel optimizer, SPIN (Stability-Plasticity INterpolation), which explicitly decouples plasticity and stability by interpolating between a fast-moving "plasticity" sequence and an adaptive "stability" sequence. We theoretically demonstrate that SPIN implicitly employs an inverse Hessian approximation, providing crucial Tikhonov regularization to damp noisy, single-sample Hessian estimates and mitigate forgetting. More importantly, SPIN can be seamlessly integrated into existing OCL methods by taking resulted gradients as input and replacing standard optimizers. Extensive experiments show that using SPIN instead of standard optimizers yields consistent and significant performance gains across multiple OCL approaches, including the challenging rehearsal-free setting and under strict memory constraints.
Continually adapting models to new tasks requires retaining earlier knowledge under limited memory and computation. Experience replay addresses this challenge, but priorities based on individual loss increases overlook how related memories respond to the same update and can overemphasize isolated responses. We introduce SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings. SPHERE uses a representation kernel to aggregate signed prospective loss changes, attenuating unsupported spikes while retaining coherent increases. It then formulates allocation as entropy-regularized transport, redistributing uniform source mass toward supported high-risk regions while penalizing long-distance transfers. We derive replay coefficients from the transport objective's sensitivity to the original loss changes and blend them with uniform replay to maintain baseline rehearsal. Our analysis establishes conditions under which kernel aggregation improves risk estimation and bounds transport-value inflation due to residual noise and smoothing bias. Experiments demonstrate that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-generation reinforcement learning with incomplete test rewards.