Dynamic gain neuromodulation attenuates stability gaps during continual joint training

Alejandro Rodriguez-Garcia; Anindya Ghosh; Srikanth Ramaswamy

cs.LGJul 18, 2025

Dynamic gain neuromodulation attenuates stability gaps during continual joint training

Authors: Alejandro Rodriguez-Garcia, Anindya Ghosh, Srikanth Ramaswamy

Organizations: Neural Circuits Laboratory, Biosciences Institute, Newcastle University, Newcastle upon Tyne, NE2 4HH, United Kingdom

Abstract

Recent work in continual learning has highlighted the stability gap -- a temporary performance drop on previously learned tasks when new ones are introduced. This phenomenon reflects a mismatch between rapid adaptation and strong retention at task boundaries, underscoring the need for optimization mechanisms that balance plasticity and stability over abrupt distribution changes. While optimizers such as momentum-SGD and Adam introduce implicit multi-timescale behavior, they still exhibit pronounced stability gaps. Importantly, these gaps persist even under ideal joint training, making it crucial to study them in this setting to isolate their causes from other sources of forgetting. Motivated by how noradrenergic (neuromodulatory) bursts transiently increase neuronal gain under uncertainty, we introduce a dynamic gain scaling mechanism as a two-timescale optimization technique that balances adaptation and retention by transiently increasing the effective update magnitude while dynamically reparameterizing the weights governing the forward pass, thereby empirically mitigating transition-induced curvature amplification. Across domain- and class-incremental MNIST, CIFAR, and mini-ImageNet benchmarks under task-agnostic joint training, dynamic gain scaling effectively attenuates stability gaps while maintaining competitive accuracy, improving robustness at task transitions.

Figures & tables

Figure 1: Schematic of the stability gap. a) Conceptual illustration highlighting the distinction between the stability gap and catastrophic forgetting. While catastrophic forgetting arises from an inaccurate approximation of the joint loss, the stability gap emerges from a deviation of the optimization trajectory from the path of non- or minimally-increasing loss, leading to a transient drop in old-task performance. b) Old-task accuracy during sequential learning on Split CIFAR-10 under stochastic gradient descent with 0.9 momentum. The sharp performance drop at the task boundary quantifies the stability gap that arises from the deviated trajectory.

Figure 2: Simple proof of principle showing how gain boosts approximate a two-timescale optimizer. a) Schematic of gain-induced effective weight decomposition under noradrenergic neuromodulation. Phasic noradrenergic signals transiently increase neuronal gain, thereby “virtually” splitting the effective weight into a fast-slow scheme with

w_{\text{fast}}=\left[g(t)-g_{0}\right]w(t)

and

w_{\text{slow}}=g_{0}\,w(t)

. b) Left panel. Simplified model

y(t)=W_{\mathrm{eff}}(t)\,x

with constant input

x

predicting a target

T(t)

under mean squared error loss

L=\tfrac{1}{2}[T(t)-y(t)]^{2}

. Comparison of gain-modulated (orange), fast-slow (blue) and slow (purple) weight methods. The unshaded region corresponds to optimization toward target

T_{1}

, and the shaded region to optimization toward target

T_{2}

. The dashed vertical line marks the time of peak neuronal gain. Right panel. Effective-coordinate flattening induced by gain neuromodulation at the highest neuronal gain. In the effective-weight space, gain boosts reparameterize the loss, effectively reducing its curvature and scaling its gradient step. Dots illustrate each method’s state at the peak gain time, and arrow lengths indicate the gradient-step magnitudes.

Figure 3: Gain modulation attenuates transition-induced instability in Split MNIST. a) Samples of data around the first task transition, $T_{1\rightarrow 2}$ , where the cumulative training distribution expands from digits $\{0,1\}$ to $\{0,1,2,3\}$ . b) Test accuracy on Task 1 after the first task transition. c) Neuronal gain dynamics $g(t)$ for NGM-SGD, showing the phasic gain response following the task transition. d) Task 1 test loss increase after the task switch. e) Effective-coordinate curvature ratio $R_{\lambda}(t)=|\lambda_{\mathrm{eff}}(t)|/|\lambda_{\mathrm{eff}}(0)|$ , where $|\lambda_{\mathrm{eff}}(0)|$ denotes the effective curvature measured immediately before the transition. The horizontal dashed line indicates $R_{\lambda}(t)=1$ , corresponding to the pre-transition curvature. The vertical dashed lines in (b–d) mark the task switch at iteration zero. Curves show the mean, and shaded regions show the standard deviation across five independent runs. The curvature ratio is represented as the geometric mean $\pm 1$ standard deviation in log-space. f–g) Transition-level stability measures across the complete five-task sequence. f) Stability gap, computed as the maximum relative reduction in previous-task accuracy following each task switch (see Table 2 ). g) Previous-task loss spike, computed as the maximum post-transition loss minus the loss immediately before the switch. Bars and error bars denote the mean and standard deviation across five independent runs. Color coding denotes the different optimizers: our method (NGM-SGD, Algorithm 1 ) in orange, momentum-SGD (MSGD) in blue, Adam in green, and vanilla SGD in gray.

Figure 4: Effect of ablating gain modulation on Split MNIST. Comparison of the full NGM-SGD method (Algorithm 1 , orange) with three targeted ablations: Tonic gain (gold), which removes entropy-driven phasic modulation (

\eta=0

); Instantaneous gain (brown), which removes temporal integration of the gain dynamics (

\gamma=0

); and Entropy-LR (dark red), which applies the entropy-dependent signal directly to the learning rate (

\alpha(t)=\alpha_{0}q(t)^{2}

). a) Test accuracy on Task 1 as the model is incrementally trained across all benchmark tasks. b) Cumulative test accuracy over all tasks observed up to the current training stage. c) End-of-task accuracy on the newly introduced task following each task switch. d) Trade-off between the average stability gap across task transitions and the final cumulative accuracy at the end of training. Curves represent the mean

\pm

standard deviation (shaded area) over five runs with different random seeds. Bars and error bars denote the corresponding mean

\pm

standard deviation. In panel (d), markers denote the mean values, while horizontal and vertical error bars indicate the propagated standard deviation of the average stability gap and the standard deviation of the final cumulative accuracy, respectively. Vertical dashed lines in panels (a–b) indicate task transitions. Horizontal dotted lines in panel a illustrate the min-ACC per task.

		avg-ACC ( $\uparrow$ )	avg-min-ACC ( $\uparrow$ )	WC-ACC ( $\uparrow$ )	avg-SG ( $\downarrow$ )
Split MNIST	NGM-SGD (ours)	99.015 $\pm$ 0.047	97.570 $\pm$ 0.465	97.696 $\pm$ 0.373	0.017 $\pm$ 0.005
	MSGD	98.971 $\pm$ 0.086	72.518 $\pm$ 0.403	77.628 $\pm$ 0.329	0.267 $\pm$ 0.004
	Adam	99.088 $\pm$ 0.108	71.047 $\pm$ 3.753	76.537 $\pm$ 3.003	0.283 $\pm$ 0.038
	SGD	98.975 $\pm$ 0.099	95.887 $\pm$ 2.887	96.293 $\pm$ 2.311	0.034 $\pm$ 0.029
Split CIFAR-10	NGM-SGD (ours)	90.630 $\pm$ 1.576	79.485 $\pm$ 3.875	80.880 $\pm$ 3.349	0.134 $\pm$ 0.043
	MSGD	89.434 $\pm$ 2.134	52.828 $\pm$ 3.898	58.348 $\pm$ 3.521	0.425 $\pm$ 0.044

Table 1: Main quantitative metrics. For all benchmarks we report (across tasks) the average final accuracy (avg-ACC), average minimum accuracy (avg-min-ACC), average stability gap drop (avg-SG), and the worst-case accuracy (WC-ACC). Highlighted values indicate the best results.

Figure 5: Stability gaps across class- and domain-incremental learning. a) Class-incremental results on Split MNIST, Split CIFAR-10, and Split mini-ImageNet. b) Domain-incremental results on Rotated MNIST and Domain CIFAR-100. Within each block, the top row shows Task 1 test accuracy as the model is trained incrementally across all benchmark tasks, the middle row shows the corresponding Task 1 test loss, and the bottom row shows the evolution of neuronal gain. Curves and shaded regions represent the mean $\pm$ standard deviation over five runs with different random seeds, and dots indicate the minimum Task 1 accuracy reached within each task. NGM-SGD is shown in orange, momentum-SGD in blue, Adam in green, and vanilla SGD in gray. Accuracy panels are vertically restricted to emphasize the stability gaps; consequently, some performance drops may appear truncated.

Figure 6: Effect of resetting optimizer inertia at task transitions. Stability gaps for MSGD and Adam, with and without resetting their accumulated optimizer states at each task switch, across Split MNIST, Split CIFAR-10, Split mini-ImageNet, Rotated MNIST, and Domain CIFAR-100. Bars and error bars represent the mean

\pm

standard deviation over five runs with different random seeds. Orange dotted segments indicate the corresponding NGM-SGD stability gap at each transition.

Figure 7: Gain across tasks. Plots of the mean neuronal gain in the first three tasks for class-incremental (top) and domain-incremental (bottom) experiments. We used a linear fit and obtained the slopes

2.16\times 10^{-1}

for Split MNIST,

4.20\times 10^{-1}

for Split CIFAR-10,

1.36\times 10^{-1}

for Split mini-ImageNet,

-6.63\times 10^{-2}

for Rotated MNIST, and

-1.59\times 10^{-1}

for Domain CIFAR-100.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

		SG( $T_{1\to 2}$ )	SG( $T_{2\to 3}$ )	SG( $T_{3\to 4}$ )	SG( $T_{4\to 5}$ )
Split MNIST	NGM-SGD (ours)	0.028 $\pm$ 0.015	0.012 $\pm$ 0.006	0.014 $\pm$ 0.008	0.013 $\pm$ 0.004
	MSGD	1.000 $\pm$ 0.000	0.032 $\pm$ 0.013	0.018 $\pm$ 0.006	0.019 $\pm$ 0.009
	Adam	0.883 $\pm$ 0.133	0.172 $\pm$ 0.062	0.052 $\pm$ 0.032	0.025 $\pm$ 0.013
	SGD	0.100 $\pm$ 0.115	0.009 $\pm$ 0.005	0.007 $\pm$ 0.006	0.019 $\pm$ 0.012
Split CIFAR-10	NGM-SGD (ours)	0.133 $\pm$ 0.097	0.072 $\pm$ 0.060	0.072 $\pm$ 0.094	0.260 $\pm$ 0.090
	MSGD	0.476 $\pm$ 0.007	0.321 $\pm$ 0.089	0.116 $\pm$ 0.117	0.786 $\pm$ 0.095

Appendix

Table 2: Stability gap metrics. For all joint training benchmarks, we report the per-transition stability gaps (mean

\pm

std). Highlighted values indicate the best results.

LR		T1	T2	T3	T4	T5
1e-4	NGM-SGD (ours)	83.12 $\pm$ 1.35	74.14 $\pm$ 1.75	75.68 $\pm$ 2.38	75.83 $\pm$ 2.17	54.95 $\pm$ 5.02
	MSGD	80.40 $\pm$ 1.09	74.89 $\pm$ 0.74	77.15 $\pm$ 1.00	77.33 $\pm$ 0.67	61.42 $\pm$ 1.78
	Adam	92.07 $\pm$ 0.67	84.16 $\pm$ 1.89	87.53 $\pm$ 0.87	84.55 $\pm$ 2.65	75.02 $\pm$ 5.24
	SGD	55.99 $\pm$ 4.80	64.75 $\pm$ 2.75	65.12 $\pm$ 1.73	60.76 $\pm$ 1.95	55.75 $\pm$ 2.35
1e-3	NGM-SGD (ours)	91.09 $\pm$ 1.46	85.89 $\pm$ 2.82	85.84 $\pm$ 1.50	85.77 $\pm$ 1.44	70.84 $\pm$ 4.72
	MSGD	91.36 $\pm$ 1.35	85.31 $\pm$ 1.77	85.76 $\pm$ 1.22	86.24 $\pm$ 1.12	73.72 $\pm$ 2.82

Appendix

Table 3: Hyperparameter search: Final accuracies (ACC) for all tasks in the Split CIFAR-10 benchmark comparing different learning rates.

$\gamma$	$\eta$	avg-ACC ( $\uparrow$ )	avg-SG ( $\downarrow$ )
0.80	0.1	90.134 $\pm$ 1.292	0.182 $\pm$ 0.038
0.80	0.2	90.920 $\pm$ 1.396	0.182 $\pm$ 0.062
0.80	0.3	91.678 $\pm$ 0.875	0.215 $\pm$ 0.053
0.80	0.4	91.220 $\pm$ 1.068	0.176 $\pm$ 0.047
0.80	0.5	90.198 $\pm$ 1.471	0.170 $\pm$ 0.049
0.90	0.1	91.098 $\pm$ 1.263	0.159 $\pm$ 0.054

Appendix

Table 4: Hyperparameter search: Average final accuracies (avg-ACC) and average stability gaps (avg-SG) in the Split CIFAR-10 benchmark comparing different combinations of the

(\gamma,\eta)

parameters in the NGM-SGD method.

Figure 8: Hyperparameter sweep on Split CIFAR-10. a) Scores on best accuracy with the learning rate sweep for all methods. b) Scores on best accuracy and least stability gap with the hyperparameter sweep of $(\gamma,\eta)$ over the NGM-SGD method.

Figure 9: Gain dynamics: a) Relationship between the average output entropy of each dataset and the corresponding gain scale. b) Gain trajectories (Task 1) comparing NGM-SGD with EMA-based baselines for the domain- and class-incremental benchmarks. EMA (+ baseline) settings match only the special case

\eta=1-\gamma

, whereas NGM-SGD allows independent control of decay and entropy-driven modulation.

		avg-ACC ( $\uparrow$ )	avg-min-ACC ( $\uparrow$ )	WC-ACC ( $\uparrow$ )	avg-SG ( $\downarrow$ )
Split MNIST	NGM-SGD (ours)	99.015 $\pm$ 0.047	97.570 $\pm$ 0.465	97.696 $\pm$ 0.373	0.017 $\pm$ 0.005
	SGD	98.975 $\pm$ 0.099	95.887 $\pm$ 2.887	96.293 $\pm$ 2.311	0.034 $\pm$ 0.029
	Adam	99.088 $\pm$ 0.108	71.047 $\pm$ 3.753	76.537 $\pm$ 3.003	0.283 $\pm$ 0.038
	Adam Reset	99.075 $\pm$ 0.147	86.563 $\pm$ 6.869	88.921 $\pm$ 5.496	0.127 $\pm$ 0.069
	MSGD	98.971 $\pm$ 0.086	72.518 $\pm$ 0.403	77.628 $\pm$ 0.329	0.267 $\pm$ 0.004
	MSGD Reset	98.939 $\pm$ 0.100	72.823 $\pm$ 0.374	77.876 $\pm$ 0.305	0.264 $\pm$ 0.004

Appendix

Table 5: Oracle reset results across benchmarks. Main quantitative metrics for all optimizers across benchmarks under joint training with momentum resets. Bold values indicate the best result per column. For Adam and MSGD, color highlights denote the metrics in which the momentum-reset variant outperforms its non-reset counterpart ( Green for Adam Reset and Blue for MSGD Reset).

Figure 10: Task 1 accuracy across replay experiments with a 1K buffer. Each panel shows per-iteration accuracy at task switches for all optimizers under a limited replay memory. On simple benchmarks (Split MNIST and Rotated MNIST), the stability gap is visible, whereas on more complex datasets (Split CIFAR-10, Split mini-ImageNet, and Domain CIFAR-100), the small buffer induces substantial steady-state forgetting that dominates over the transient drop. This illustrates how poor approximations of the joint loss can obscure the stability gap we aim to study.

Figure 11: Effect of backbone freezing on the stability gap in Split CIFAR-10. a) Task 1 test accuracy across five sequential tasks for NGM-SGD (Algorithm 1 ; orange), momentum-SGD (MSGD; blue), and MSGD with the convolutional backbone frozen after Task 1 (MSGD-frozen; cyan). Dashed horizontal segments and circular markers indicate the minimum Task 1 accuracy within each task. b) Training accuracy across tasks. Curves and shaded regions show the mean

\pm

standard deviation across seeds.

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 304 papers

Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning
Feb 24, 2026JinLi He, Liang Bai, Xian YangContinual Learning Catastrophic Forgetting
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning
Aug 5, 2026Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung +2Class-Incremental Learning Dynamic Neural Networks
Department of Computer Science Simon Fraser University · Department of Computer Science McGill Univeristy, Mila - Quebec AI Institute
Preserving Plasticity in Continual Learning via Dynamical Isometry
Jun 8, 2026Andries Rosseau, Robert Müller, Ann NowéLoss of Plasticity Continual Learning
AI Lab, Vrije Universiteit Brussel · Aganthos

Feb 24, 2026J/K move · Enter open · S save

Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning

JinLi He, Liang Bai, Xian Yang

Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK

Continual Learning Catastrophic Forgetting

Open paper AskPDF arXiv

The magnitude of parameter updates are considered a key factor in continual learning. However, most existing studies focus on designing diverse update strategies, while a theoretical understanding of the underlying mechanisms remains limited. Therefore, we characterize model's forgetting from the perspective of parameter update magnitude and formalize it as knowledge degradation induced by task-specific drift in the parameter space, which has not been fully captured in previous studies due to their assumption of a unified parameter space. By deriving the optimal parameter update magnitude that minimizes forgetting, we unify two representative update paradigms, frozen training and initialized training, within an optimization framework for constrained parameter updates. Our theoretical results further reveals that sequence tasks with small parameter distances exhibit better generalization and less forgetting under frozen training rather than initialized training. These theoretical insights inspire a novel hybrid parameter update strategy that adaptively adjusts update magnitude based on gradient directions. Experiments on deep neural networks demonstrate that this hybrid approach outperforms standard training strategies, providing new theoretical perspectives and practical inspiration for designing efficient and scalable continual learning algorithms.

Abstract

Continual Learning

Explore similar work

Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning

NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning

Preserving Plasticity in Continual Learning via Dynamical Isometry