Regularisation-based class-incremental learning (CIL) methods control the stability-plasticity trade-off through fixed coefficients tuned offline, independently of the continual learning objective and blind to phase-specific variations in data volume, task difficulty, and class similarity. We argue this is a fundamental mismatch and propose REDAR: Reinforcement-Driven Adaptive Regularisation for Class Incremental Learning, which treats regularisation strength selection as a sequential decision problem. At each incremental phase, a lightweight reinforcement learning (RL) agent observes a compact state of current performance metrics and incoming data statistics, and outputs the distillation coefficients for that phase. To respect CIL's constraint on historical data access, the policy is trained entirely on pseudo-CIL tasks constructed from the initial phase, then deployed on the incremental sequence without further adaptation. Experiments on CIFAR-100 and Tiny-ImageNet demonstrate that REDAR reduces forgetting and improves forward transfer over fixed and heuristic baselines, and that policies trained on short pseudo-task horizons transfer effectively to longer incremental runs.
Figures & tables
Figure 1: Overview of REDAR . (a) Reinforcement-driven regularisation is formulated as a finite-horizon Markov Decision Process (MDP), M=(S,A,T,R,γ) , where the state si=(Ai−1,Fi−1,CRi,ISi) summarises the current learning dynamics, the action ai=(αi,βi) specifies the phase-specific regularisation coefficients, and the reward ri is the validation accuracy after CIL training. (b) The CIL model jointly optimises cross-entropy, feature distillation, logit distillation, and margin ranking losses while learning from replay memory and incoming samples. (c) The policy is trained offline on pseudo-CIL tasks using a distributed actor–learner architecture with prioritised experience replay before being deployed on the target incremental sequence.
CIFAR-100 (N=2)
CIFAR-100 (N=5)
CIFAR-100 (N=10)
Method
ACC ↑
FRG ↓
FWT ↑
ACC ↑
FRG ↓
FWT ↑
ACC ↑
FRG ↓
FWT ↑
JT (Upper Bound)
72.87
-
-
72.87
-
-
72.87
-
-
iCaRL [ 23 ]
60.32
28.54
-
51.96
55.29
-
45.40
68.26
-
LUCIR [ 9 ]
66.94
24.55
40.18
63.93
31.92
49.42
61.24
32.13
47.68
WA [ 35 ]
67.2
-
-
64.32
29.85
-
57.77
28.88
-
PodNet [ 5 ]
67.51
30.02
-
61.8
38.99
-
59.22
31.35
-
Table 1: Performance comparison on average accuracy ACC , average forgetting FRG and forward transfer FWT on CIFAR-100.
Tiny-ImageNet (N=5)
Tiny-ImageNet (N=10)
Method
ACC ↑
FRG ↓
FWT ↑
ACC ↑
FRG ↓
FWT ↑
JT (Upper Bound)
68.78
-
-
68.78
-
-
iCaRL [ 23 ]
37.51
31.89
-
42.45
32.77
-
EEIL [ 3 ]
47.4
25.56
-
45.2
25.91
-
LUCIR [ 9 ]
56.76
15.04
56.48
53.41
27.60
68.12
WA [ 35 ]
49.28
40.85
-
38.29
37.24
-
Table 2: Performance comparison on average accuracy ACC , average forgetting FRG and forward transfer FWT on Tiny-ImageNet.
Figure 2: Top: Phase-wise accuracy on CIFAR-100 and Tiny-ImageNet under the N=5 incremental setting. Accuracy is reported as the mean over all classes seen after each phase. Bottom: coefficients ( α , β ) used to train each phase.
Figure 3: Cross-setting transfer evaluation on CIFAR-100. Each cell reports ACC (%) for a policy trained on Ntrain phases and deployed on Ntest phases. Diagonal entries (pink outline) correspond to standard in-distribution evaluation.
Method
ACC ↑
FRG ↓
FWT ↑
Fixed (0.1)
63.93
31.92
49.42
Random Selection
59.84
30.28
43.16
REDAR (Ours)
64.43
28.02
51.77
Table 3: Benefit of varying hyperparameters.
State Representation
Metrics
CR
ACC
FRG
IS
ACC
FRG
FWT
✓
✓
✗
✗
62.48
37.39
45.42
✓
✓
✓
✗
59.97
45.46
39.96
✓
✓
✗
✓
62.49
37.44
48.82
✓
✓
✓
✓
64.43
28.02
51.77
Table 4: Ablation results of the state representation