InTAct: Interval-based Task Activation Consolidation for Continual Learning

Patryk Krukowski; Jan Miksa; Piotr Helm; Jacek Tabor; Paweł Wawrzyński; Przemysław Spurek

cs.LGNov 21, 2025

InTAct: Interval-based Task Activation Consolidation for Continual Learning

Authors: Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek

Organizations: Jagiellonian University · AKCES NCBR · IDEAS Research Institute.

Abstract

Continual learning seeks to acquire new knowledge while preserving previously learned representations. Despite the success of prompt-based methods, they remain fragile in domain-incremental learning scenarios, where shifts in the input distribution cause representation drift in shared layers and lead to forgetting. We introduce InTAct, a method that preserves functional behavior in shared representations without freezing parameters or retaining past data. InTAct constrains updates within activation regions associated with prior domains, while allowing flexible adaptation elsewhere, thereby stabilizing neuron functionality rather than directly restricting parameter values. The approach is architecture-agnostic, integrates seamlessly with existing prompt-based frameworks, and matches or outperforms state-of-the-art methods on domain-incremental benchmarks.

Figures & tables

Figure 1: Method Overview. During training on task $(N{+}1)$ , InTAct preserves the functional region established by task $N$ (pink) while allowing the model to learn new knowledge in a distinct region (orange). After training, these regions merge into an expanded protected region (yellow), defining where the layer’s transformation should remain stable in future updates. The dashed curves illustrate transformations from earlier tasks, highlighting that their behaviors remain preserved within their respective regions, while adaptation is freely allowed outside them.

Panel of Figure 2 — Figure 2: Illustration of localized functional preservation in InTAct. A model is trained sequentially on three tasks to approximate different regions of a Gaussian-shaped function. The dashed lines denote the protected intervals (1D hypercubes) estimated after each task. During subsequent training stages, these intervals constrain functional drift within previously occupied regions while allowing the function to adapt freely in unexplored regions of the input space. As a result, InTAct preserves prior knowledge locally without restricting plasticity globally.

Figure 3: Overview of the proposed regularization framework. $\mathcal{L}_{\mathrm{IntDrift}}$ and $\mathcal{L}_{\mathrm{Feat}}$ preserve functional behavior within protected hypercubes corresponding to previously learned tasks (pink). $\mathcal{L}_{\mathrm{Var}}$ encourages compact activation regions for newly acquired tasks (orange), while $\mathcal{L}_{\mathrm{Align}}$ limits excessive expansion of the cumulative feature space by aligning task centers across tasks. Together, these constraints produce compact and stable feature regions, illustrated by the purple contours.

Method	Ref.	DomainNet	ImageNet-R	ImageNet-C	ImageNet-Mix	Average
EWC	PNAS 2017	$47.10{\pm}\scriptstyle 0.58$	$51.83{\pm}\scriptstyle 0.33$	$65.20{\pm}\scriptstyle 1.23$	$57.82{\pm}\scriptstyle 0.90$	$55.49{\pm}\scriptstyle 0.42$
LwF	T-PAMI 2017	$54.85{\pm}\scriptstyle 0.06$	$57.11{\pm}\scriptstyle 0.95$	$69.01{\pm}\scriptstyle 1.21$	$65.27{\pm}\scriptstyle 0.93$	$61.56{\pm}\scriptstyle 0.45$
L2P	CVPR 2022	$53.74{\pm}\scriptstyle 0.04$	$56.55{\pm}\scriptstyle 0.33$	$77.86{\pm}\scriptstyle 0.44$	$64.20{\pm}\scriptstyle 0.34$	$63.09{\pm}\scriptstyle 0.16$
S-Prompts	NeurIPS 2022	$44.08{\pm}\scriptstyle 0.05$	$27.23{\pm}\scriptstyle 0.16$	$60.25{\pm}\scriptstyle 0.28$	$24.25{\pm}\scriptstyle 0.19$	$38.95{\pm}\scriptstyle 0.09$
DualPrompt	ECCV 2022	$55.18{\pm}\scriptstyle 0.02$	$59.47{\pm}\scriptstyle 1.00$	$78.52{\pm}\scriptstyle 0.30$	$63.57{\pm}\scriptstyle 0.17$	$64.19{\pm}\scriptstyle 0.26$
ESN	AAAI 2023	$45.74{\pm}\scriptstyle 0.25$	$16.39{\pm}\scriptstyle 0.23$	$68.34{\pm}\scriptstyle 0.39$	$14.99{\pm}\scriptstyle 0.53$	$36.37{\pm}\scriptstyle 0.19$

Table 1: Benchmarking against the SOTA. We report AA across four diverse DIL benchmarks as the mean over ten runs for InTAct and over three runs for the baselines. InTAct achieves the highest average performance, with the largest gains on ImageNet-R and ImageNet-Mix. On DomainNet and ImageNet-C, InTAct remains highly competitive, matching or slightly exceeding the best baseline results. Presented results are achieved by integrating InTAct into KA-Prompt. Baseline results are sourced from Xu et al. (2025) .

Figure 4: AA Evolution on ImageNet-R. Above plot reports the AA across 15 incremental tasks. InTAct (black line with stars) outperforms SOTA prompt-based baselines (e.g., KA-Prompt, DualPrompt) and regularization methods (e.g., EWC, LwF) from $t=7$ onward. Notably, our method maintains high accuracy even during challenging task transitions (e.g., $t=9$ ), where most other methods exhibit significant forgetting. Baseline results are sourced from Xu et al. (2025) .

Panel of Figure 5 — Figure 5: Hyperparameter sensitivity of CODA-Prompt + InTAct on ImageNet-R: final accuracy on each task, averaged over three independent runs. (a) Lower drift penalties preserve classifier plasticity, allowing adaptation to new tasks. (b) Stronger variance regularization promotes more compact representations and improves stability. (c) A sufficiently strong feature consistency penalty is crucial for preventing drift in the backbone’s representations.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	KA-Prompt + InTAct			CODA-Prompt + InTAct
Dataset	$\lambda_{\mathrm{Feat}}$	$\lambda_{\mathrm{IntDrift}}$	$\lambda_{\mathrm{Var}}$	$\lambda_{\mathrm{Feat}}$	$\lambda_{\mathrm{IntDrift}}$	$\lambda_{\mathrm{Var}}$
DomainNet	$10^{-2}$	$10^{-4}$	$10^{-1}$	$10^{-4}$	$10^{-3}$	$1.0$
ImageNet-R	$10^{-1}$	$10^{-3}$	$1.0$	$10^{-1}$	$10^{-3}$	$10^{-1}$
ImageNet-C	$10^{-2}$	$10^{-4}$	$1.0$	$10^{-1}$	$10^{-4}$	$10^{-1}$
ImageNet-Mix	$10^{-2}$	$10^{-4}$	$1.0$	$10^{-1}$	$10^{-4}$	$10^{-1}$

Appendix

Table 3: Optimal Hyperparameters. Selected hyperparameters for InTAct integrated into KA-Prompt and into CODA-Prompt across different benchmarks.

Method	DomainNet	ImageNet-R	Average
L2P	$48.44{\pm}\scriptstyle 0.09$	$50.34{\pm}\scriptstyle 0.26$	$49.39{\pm}\scriptstyle 0.18$
+ InTAct	$\mathbf{53.44{\pm}\scriptstyle 0.02}$	$\mathbf{58.34{\pm}\scriptstyle 0.24}$	$\mathbf{55.89{\pm}\scriptstyle 0.13}$
DualPrompt	$50.87{\pm}\scriptstyle 0.31$	$53.31{\pm}\scriptstyle 0.36$	$52.09{\pm}\scriptstyle 0.34$
+ InTAct	$\mathbf{56.83{\pm}\scriptstyle 0.28}$	$\mathbf{61.85{\pm}\scriptstyle 0.45}$	$\mathbf{59.34{\pm}\scriptstyle 0.37}$
CODA-Prompt	$52.52{\pm}\scriptstyle 0.22$	$61.21{\pm}\scriptstyle 0.49$	$56.87{\pm}\scriptstyle 0.36$
+ InTAct	$\mathbf{56.66{\pm}\scriptstyle 0.39}$	$\mathbf{74.23{\pm}\scriptstyle 0.16}$	$\mathbf{65.45{\pm}\scriptstyle 0.28}$

Appendix

Table 4: Performance results of integrating InTAct with L2P, DualPrompt, and CODA-Prompt. We report AA for ImageNet-R and DomainNet in the DIL setting. Incorporating InTAct consistently yields performance gains over prompt-based baselines. Results are averaged over 5 independent runs.

$\mathcal{L}_{\mathrm{IntDrift}}$	$\mathcal{L}_{\mathrm{Var}}$	$\mathcal{L}_{\mathrm{Align}}$	$\mathcal{L}_{\mathrm{Feat}}$	AA (%)	$\Delta$
✓	✓	✓	✓	74.23	—
$\times$	✓	✓	✓	70.75	-3.48
✓	$\times$	✓	✓	73.28	-0.95
✓	✓	$\times$	✓	73.76	-0.47
✓	✓	✓	$\times$	72.44	-1.79
✓	$\times$	$\times$	✓	73.08	-1.15

Appendix

Table 5: Pairwise ablation on ImageNet-R. Extension of Tab. with the joint removal of every pair of loss components. We report the performance drop (

\Delta

) relative to the full InTAct.

Method	ImageNet-R	DomainNet
HINT [ 7 ]	$41.42{\pm}\scriptstyle 1.26$	$52.02{\pm}\scriptstyle 0.08$
InTAct	$\mathbf{74.23{\pm}\scriptstyle 0.16}$	$\mathbf{56.66{\pm}\scriptstyle 0.39}$

Appendix

Table 6: Comparison with HINT. AA of HINT and InTAct, both implemented in the CODA-Prompt framework.

Method	Split MNIST	Split FMNIST	Split CIFAR-10
EWC	20.57	19.92	30.43
LwF	28.63	25.92	24.26
MAS	17.91	18.74	21.75
SI	18.40	17.30	20.18
InTAct (Ours)	35.27	34.25	31.28

Appendix

Table 7: AA in CIL settings. All results are averaged over 5 seeds.

$\alpha$	0.75	0.80	0.85	0.90	0.95	0.99	1.00
AA (%)	74.66	74.51	74.49	74.63	74.23	73.12	71.37

Appendix

Table 8: Sensitivity analysis of the hyperparameter

\alpha

for CODA-Prompt + InTAct on ImageNet-R.

Task Interval	$\frac{1}{d}\sum\mathrm{diam}_{j}(\mathcal{H})$
Tasks 1–5	$3.280\rightarrow 4.095$
Tasks 6–10	$4.096\rightarrow 4.196$
Tasks 11–15	$4.197\rightarrow 4.202$

Appendix

Table 9: Growth of the cumulative hypercube’s average coordinate diameter across ImageNet-R.

Task Interval	$\frac{1}{d}\sum\mathrm{diam}_{j}(\mathcal{H})$
Tasks 1–10	$2.991\pm 0.005\rightarrow 3.644\pm 0.037$
Tasks 11–20	$3.644\pm 0.037\rightarrow 3.662\pm 0.035$
Tasks 21–30	$3.664\pm 0.035\rightarrow 3.684\pm 0.035$

Appendix

Table 10: Growth of the cumulative hypercube’s average coordinate diameter across ImageNet-Mix (30 tasks), reported as mean

\pm

standard deviation over runs.

Figure 6: Evolution of Internal Drift. Activation stability on Split-MNIST (DIL). The plot tracks the cumulative $L_{1}$ shift in hidden activations for reference images from early tasks as the model trains on later tasks. InTAct successfully suppresses drift compared to LwF, validating the effectiveness of the proposed regularization.

Methods	Total Params	Trainable Params	GPU Memory (GB)	Batch time (s)	AA
L2P	86,263,843	311,387	19.33	0.66	$56.55{\pm}\scriptstyle 0.33$
S-Prompts	92,964,094	1,637,910	14.90	0.39	$27.23{\pm}\scriptstyle 0.16$
DualPrompt	86,514,211	561,755	19.45	0.66	$59.47{\pm}\scriptstyle 1.00$
CODA-Prompt	88,437,580	2,638,926	19.18	0.80	$55.21{\pm}\scriptstyle 0.33$
CPrompt	92,485,224	2,689,168	41.64	1.18	$59.48{\pm}\scriptstyle 1.09$
C-Prompt	89,180,505	3,381,851	11.99	0.42	$62.43{\pm}\scriptstyle 0.49$

Appendix

Table 12: Efficiency comparison on ImageNet-R. We report total parameters, trainable parameters, GPU memory usage, batch time, and AA. Baseline measurements are taken from [ 21 ] . ‡ Measured by us under identical conditions on a single NVIDIA H100 (80 GB) with batch size 128. All other GPU memory and batch time measurements are from [ 21 ] and were obtained on different hardware; therefore, the two groups are not directly comparable.

Prompt-Based Continual Learning

See all 8 papers

Continual Learning without Continual Training
Oct 7, 2026Nikita Narayanan, Ritham Majumdarr, Sonali ParbhooContinual Learning Class-Incremental Learning
Imperial College London
CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds
Mar 16, 2026Vaishnavi Nagabhushana, Kartikay Agrawal, Ayon BorthakurClass-Incremental Learning Spiking Neural Networks
SustainAI Lab, MFSDS&AI, IIT Guwahati
Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning
Mar 13, 2026Gyutae Oh, Jungwoo Bae, Jitae ShinPrompt-Based Continual Learning Continual Learning
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea

Oct 7, 2026J/K move · Enter open · S save

Continual Learning without Continual Training

Nikita Narayanan, Ritham Majumdarr, Sonali Parbhoo

Imperial College London

Continual Learning Class-Incremental Learning Domain-Incremental Learning Prior-Data Fitted Networks In-Context Learning

Open paper AskPDF arXiv

Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge. Many existing methods rely on continued optimization, using regularization, replay, or parameter expansion to prevent new updates from overwriting previously learned knowledge. Instead, we propose replacing continual training with continual inference: a PFN-based model that is meta-trained, and then frozen, adapting to new classes only by extending an in-context evidence set. Our model, Latent Concept PFN, performs in-context Bayesian inference over a latent concept space that captures semantic structure shared across domains and classes. As each new domain or class arrives, exemplars are added to the memory; adaptation reflects updated posterior beliefs over latent concepts rather than gradient updates. No parameters are changed, reducing forgetting. The same method handles both domain and class incremental continual learning without task identity. Concept annotations are only used during meta-training, acting as a soft anchor on the latent space rather than a fixed bottleneck. Unlike fixed-vocabulary concept methods, the model also handles noisy, ambiguous, or incomplete annotations by combining concept labels with raw input evidence to discover distinctions beyond the predefined concept set. Experiments on class and domain incremental learning datasets demonstrate competitive continual learning performance while learning interpretable latent concepts.

Abstract

Prompt-Based Continual Learning

Explore similar work

Continual Learning without Continual Training

CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds

Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning