Functional compatibility as a determinant of persistent neural learning

Hossein Javidnia

cs.LGAug 23, 2026

Functional compatibility as a determinant of persistent neural learning

Authors: Hossein Javidnia

Organizations: School of Computing Dublin City University Dublin, Ireland

Abstract

Neural networks can acquire new capabilities while damaging existing ones, but what determines whether new learning persists remains unclear. We identify functional compatibility, the extent to which incoming learning can coexist with behaviour that must be preserved, as an experimentally manipulable causal determinant of persistence. From identical neural states, we vary compatibility while matching unrestricted learning opportunity and imposing a common retention requirement. Persistent learning increases with compatibility across independent directions, convolutional and transformer architectures, vision and text, and a ten-seed replication. Learning rules and retention constraints determine how much compatible opportunity is retained, whereas nonlinear geometry limits the matched intervention at larger update norms. Functional compatibility therefore reframes stability-plasticity from preventing forgetting to determining which new learning can coexist with existing function and persist.

Figures & tables

Figure 1: Functional compatibility causally controls persistent learning. a , Experimental logic. All interventions at a causal state start from the identical parent neural state, including the same model, optimizer, stream position, replay state and random-number-generator state. Functional compatibility is manipulated while unrestricted learning progress

\Delta_{0}

is matched, after which the same retention requirement is applied and persistent learning is measured. b , Persistent-progress ratio versus realized functional compatibility at

\beta=0.5

in three fully unfrozen systems. c , Compatibility slope as a function of retention budget. d , Matched compatibility slopes for distinct learning rules at

\beta=0.5

. For b–d ,

n=5

independent seed trajectories per system and 50 reconstructed causal states per seed; state-level slopes are aggregated within seed, and shaded bands and error bars show two-sided 95% Student-

t

intervals across the five seed effects.

System	Mean $\kappa$ at request 0	Nonzero MAE	Maximum error	Mean $\Delta_{0}$ spread	Maximum spread
CNN	0.010681	$1.1\times 10^{-5}$	$1.16\times 10^{-4}$	0.283%	0.391%
ViT	0.041266	$4.0\times 10^{-6}$	$8.9\times 10^{-5}$	0.282%	0.396%
Text transformer	0.000083	$5.0\times 10^{-6}$	$1.30\times 10^{-3}$	0.268%	0.390%

Table 1: Fidelity of the controlled compatibility intervention

$\beta$	CNN	ViT	Text transformer
0	0.000 [0.000,0.000]	0.000 [0.000,0.000]	0.000 [0.000,0.000]
0.01	0.072 [0.008,0.136]	0.682 [0.592,0.772]	0.012 [0.002,0.021]
0.05	0.999 [0.981,1.016]	0.996 [0.987,1.005]	0.131 [0.103,0.158]
0.10	1.014 [1.010,1.017]	1.001 [1.000,1.002]	0.287 [0.233,0.341]
0.25	1.014 [1.010,1.017]	1.001 [1.000,1.002]	0.728 [0.660,0.795]
0.50	1.014 [1.010,1.017]	1.001 [1.000,1.002]	0.965 [0.945,0.985]

Table 2: Full compatibility slopes across retention budgets

System	Modality	Seeds	Causal states/seed	Natural states/seed
CIFAR-10 CNN	vision	11, 29, 47, 71, 101	50	50
CIFAR-10 ViT	vision	11, 29, 47, 71, 101	50	50
Text transformer	text	11, 29, 47, 71, 101	50	50

Table 3: Experimental matrix. All neural representations remain unfrozen during the reported causal and natural-state experiments.

System	Mean $\kappa$ at request 0	Nonzero MAE	Nonzero max error	Mean $\Delta_{0}$ spread	Max spread
CIFAR-10 CNN	0.010681	$1.1\times 10^{-5}$	$1.16\times 10^{-4}$	0.283%	0.391%
CIFAR-10 ViT	0.041266	$4.0\times 10^{-6}$	$8.9\times 10^{-5}$	0.282%	0.396%
Text transformer	0.000083	$5.0\times 10^{-6}$	$1.30\times 10^{-3}$	0.268%	0.390%

Table 4: Audit of causal intervention fidelity. Compatibility errors exclude the requested-zero condition. Relative

\Delta_{0}

spread is computed within each matched causal state.

Method	CIFAR-10 CNN	CIFAR-10 ViT	Text transformer	Cross-system
Projection / AFM-base	1.014 [1.010, 1.017]	1.001 [1.000, 1.002]	0.965 [0.945, 0.985]	0.993 [0.980, 1.006]
Unrestricted	0.606 [0.591, 0.622]	0.586 [0.581, 0.592]	0.509 [0.488, 0.529]	0.567 [0.542, 0.592]
Linearized distillation	0.790 [0.772, 0.809]	0.606 [0.567, 0.645]	0.558 [0.542, 0.573]	0.651 [0.593, 0.710]
EWC-prox	0.570 [0.564, 0.577]	0.584 [0.580, 0.589]	0.506 [0.485, 0.528]	0.554 [0.533, 0.574]
Replay	0.013 [-0.013, 0.039]	0.005 [-0.026, 0.036]	0.002 [-0.003, 0.006]	0.007 [-0.004, 0.017]
DER++	0.019 [-0.015, 0.053]	0.010 [-0.019, 0.038]	0.002 [-0.004, 0.008]	0.010 [-0.001, 0.021]

Table 5: Matched causal slopes of

\rho_{\mathrm{persistent}}

on realized

\kappa

\beta=0.5

. Each system interval uses five seed-level matched slopes. The final column is a descriptive mean over 15 system-seed slopes and is not a claim about an unobserved population of architectures.

$\beta$	CIFAR-10 CNN	CIFAR-10 ViT	Text transformer
0	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]
0.01	0.072 [0.008, 0.136]	0.682 [0.592, 0.772]	0.012 [0.002, 0.021]
0.05	0.999 [0.981, 1.016]	0.996 [0.987, 1.005]	0.131 [0.103, 0.158]
0.10	1.014 [1.010, 1.017]	1.001 [1.000, 1.002]	0.287 [0.233, 0.341]
0.25	1.014 [1.010, 1.017]	1.001 [1.000, 1.002]	0.728 [0.660, 0.795]
0.50	1.014 [1.010, 1.017]	1.001 [1.000, 1.002]	0.965 [0.945, 0.985]

Table 6: Retention-budget sensitivity for the projected AFM-compatible proposal. Entries are mean matched seed slopes with 95% Student-

t

intervals.

System	Req. $\kappa$	Realized $\kappa$	Mean $\rho_{\mathrm{persistent}}$	Mean $\widehat{\lambda}$	Min. margin	Accept	Margin pass	Finite
CIFAR-10 CNN	0.00	0.010681	0.00524	0.9519	-0.003857	97.2%	78.6%	97.2%
CIFAR-10 CNN	0.10	0.100000	0.08937	1.0000	0.008058	100%	100%	99.6%
CIFAR-10 CNN	0.25	0.250000	0.24836	1.0000	0.083615	100%	100%	99.6%
CIFAR-10 CNN	0.50	0.499998	0.51097	1.0000	0.241686	100%	100%	99.6%
CIFAR-10 CNN	0.75	0.749999	0.76107	1.0000	0.407596	100%	100%	99.6%
CIFAR-10 CNN	1.00	0.999986	0.99974	1.0000	0.593211	100%	100%	99.6%

Table 7: Native AFM controlled-intervention results. “Margin pass” is the fraction of accepted rows with

\rho_{\mathrm{persistent}}-\widehat{\lambda}\kappa/3\geq 0

. “Finite” is successful finite completion divided by all rows at that condition.

System	$n$	Mean $\kappa$	$\kappa$ range	Mean $\widehat{\lambda}$	Median $\rho_{\mathrm{persistent}}$	Min. margin
CIFAR-10 CNN	250	0.228	[0.067, 0.443]	0.552	0.348	0.0995
CIFAR-10 ViT	250	0.422	[0.278, 0.618]	0.670	0.277	0.0848
Text transformer	250	0.950	[0.903, 0.976]	0.107	0.119	0.0199

Table 8: Native AFM on unmanipulated natural states. Median

\rho_{\mathrm{persistent}}

is emphasized because the ratio can be heavy-tailed when

\Delta_{0}

is small.

Method	CIFAR-10 CNN	CIFAR-10 ViT	Text transformer
AFM	100.0%	100.0%	100.0%
Projection	12.0%	34.8%	0.0%
Linearized distillation	11.6%	32.0%	0.0%
EWC-prox	0.0%	0.0%	0.0%
Replay	0.0%	0.0%	0.0%
DER++	0.0%	0.0%	0.0%

Table 9: Retention-pass rate on the 250 natural states per system under the stored common tolerance

D\leq 0.005

. This table measures retention feasibility under this specific criterion, not general method quality.

System	Causal mean $\Delta_{0}$	Causal median	Natural mean $\Delta_{0}$	Natural median
CIFAR-10 CNN	$1.64\times 10^{-6}$	$1.48\times 10^{-6}$	0.1199	0.1204
CIFAR-10 ViT	$8.85\times 10^{-6}$	$7.01\times 10^{-6}$	0.1582	0.1535
Text transformer	$2.32\times 10^{-4}$	$1.67\times 10^{-4}$	0.0270	0.0266

Table 10: Finite unrestricted decrease in the controlled causal experiment and in the ordinary natural-state validation.

Method	$\beta$	CNN	ViT	Text	Pooled
Projection / AFM-base	0.00	0.000	0.000	-0.000	-0.000
Projection / AFM-base	0.01	0.072	0.682	0.012	0.255
Projection / AFM-base	0.05	0.999	0.996	0.131	0.708
Projection / AFM-base	0.10	1.014	1.001	0.287	0.767
Projection / AFM-base	0.25	1.014	1.001	0.728	0.914
Projection / AFM-base	0.50	1.014	1.001	0.965	0.993

Table 11: Aggregate cross-system slopes.

System	Method	Mean $\rho_{\mathrm{persistent}}$	Median $\rho_{\mathrm{persistent}}$	Retention pass
CIFAR-10 CNN	AFM	0.937	0.348	100.0%
CIFAR-10 CNN	Projection	1.787	0.631	12.0%
CIFAR-10 CNN	Linearized distillation	1.789	0.632	11.6%
CIFAR-10 CNN	EWC-prox	1.075	1.008	0.0%
CIFAR-10 CNN	Replay	-13.393	-3.674	0.0%
CIFAR-10 CNN	DER++	-2.477	-0.223	0.0%

Table 12: Natural-state persistent-progress ratios and retention pass rate.

Figure 2: The compatibility effect generalizes across directions, models and local scales. a , Four independently constructed directions at each nonzero compatibility level; light traces and points show individual directions and the dark line shows the mean. b , Ratio of within-compatibility directional variation to between-compatibility variation at

\beta=0.5

. For a,b ,

n=5

seed trajectories per system, 50 states per seed, and four directions at each of four nonzero compatibility levels. c , Ten-seed compatibility slopes for the convolutional network, baseline vision transformer, stronger vision transformer and text transformer;

n=10

seed trajectories per system and intervals use 100,000 deterministic seed-bootstrap resamples. d , Multiscale compatibility slopes in the designed local expansion;

n=5

seed trajectories per system and 50 states per seed. The scale coordinate is an enlargement of the local intervention and is not a percentage of a natural training update.

$\beta$	CNN $R$ /slope	ViT $R$ /slope	Text $R$ /slope
0.01	0.3718 / 0.0725	0.2467 / 0.2740	0.4353 / $-0.0066$
0.05	0.0922 / 0.6876	0.0409 / 0.8535	0.5321 / 0.0927
0.10	0.0577 / 0.8917	0.0220 / 0.9566	0.3445 / 0.3367
0.25	0.0400 / 1.0130	0.0254 / 1.0133	0.1151 / 0.8285
0.50	0.0379 / 1.0465	0.0206 / 1.0294	0.0302 / 0.9861
1.00	0.0347 / 1.0556	0.0205 / 1.0295	0.0178 / 1.0033

Table 13: Independent-direction robustness across budgets

$\beta$	Equal-system pooled slope	95% CI
0.01	0.36331	[0.35075,0.37667]
0.05	0.77171	[0.76572,0.77759]
0.10	0.82114	[0.81433,0.82756]
0.25	0.93121	[0.92425,0.93841]
0.50	0.99360	[0.99123,0.99600]
1.00	1.00379	[1.00326,1.00442]

Table 14: Ten-seed pooled compatibility response

Figure 3: Functional compatibility arises during ordinary learning and AFM converts it into a protected learning frontier. a , Natural compatibility distributions across 750 unmanipulated states, comprising $n=5$ seed trajectories and 50 states per seed in each of three systems. b , Natural compatibility, accepted AFM path fraction and persistent-progress ratio. c , Retention-pass rates under the common recorded $D\leq 0.005$ criterion; rates describe feasibility under this criterion rather than universal method quality. d , Schematic of the AFM transaction separating persistent learning in ordinary network parameters from finite output restoration.

Dataset	AFM	CCL-DC	MKD	FGH	aL-SAR	LPR	OCAR
CORe50	0.2478	0.2016	0.1799	0.2546	0.1801	0.1640	0.1436
CLEAR-10	0.3807	0.3731	0.3513	0.3773	0.3572	0.3485	0.3164
CLAD-C	0.5129	0.4921	0.4931	0.3058	0.3296	0.4694	0.3747

Table 15: AFM chronological retention-plasticity benchmarks

Figure 4: Nonlinear geometry sets the finite-scale boundary of compatibility-controlled learning. a , Largest designed causal update norms compared with median natural unrestricted update norms; natural calibration uses 250 ordinary states per original system. b , Fraction of 500 preselected states per architecture, 10 seeds by 50 states, supporting all four positive same-norm compatibility endpoints at 1% to 100% of median natural update norm. Zero feasibility denotes absence of the matched four-level intervention, not a zero causal effect. c , In 231 feasible vision-transformer states at 1% natural norm, distributed across all 10 seeds, compatibility increases absolute persistent progress whereas the normalized ratio does not show a positive slope. d , Stratification of the same 231 states by unrestricted-progress heterogeneity leaves the absolute persistent slope essentially invariant while the ratio slope reverses sign. Bridge intervals use

n=10

seed-level effects and 100,000 deterministic seed-bootstrap resamples.

System	Negative requested-zero empirical margins
CNN	52
ViT	1
Text transformer	12
Total	65

Table 16: Near-zero theorem-aligned boundary assessment

System	Natural-norm fraction	Feasible / attempted	Seeds with feasibility	Target norm
CNN	0.01	1/500 (0.2%)	1	0.00060570
CNN	0.10-1.00	0/1500	0	0.00605702-0.06057018
ViT	0.01	231/500 (46.2%)	10	0.00039144
ViT	0.10-1.00	0/1500	0	0.00391441-0.03914410

Table 17: Fixed-norm bridge feasibility and decomposition

System	Matched slope	95% CI
CIFAR-10 CNN	1.01377	[1.01027, 1.01727]
CIFAR-10 ViT	1.00082	[0.99999, 1.00165]
Text transformer	0.96497	[0.94527, 0.98466]
Cross-system descriptive mean	0.99319	[0.98043, 1.00594]

Table 18: Reference primary causal slopes at retention budget

\beta=0.5

. Intervals are two-sided 95% Student-

t

intervals over seed-level effects; the cross-system row is a descriptive interval over the 15 system–seed effects.

System	$s=0.05$	$s=0.2$	$s=0.5$	$s=0.9$
CIFAR-10 CNN	1.01290	1.01032	1.00359	0.99390
CIFAR-10 ViT	1.00135	0.99886	0.99567	0.98347

Table 19: AFM/projection matched

\kappa

slopes across the designed multiscale range at

\beta=0.5

System	Mean causal norm	Median natural norm	Causal/natural ratio
CIFAR-10 CNN	$4.9766\times 10^{-6}$	0.0605702	$8.2163\times 10^{-5}$
CIFAR-10 ViT	$3.5026\times 10^{-5}$	0.0391441	$8.9479\times 10^{-4}$
Text transformer	$1.0863\times 10^{-3}$	0.0146534	0.0741306

Table 20: Largest designed multiscale intervention compared with the median natural unrestricted update norm.

Figure 5: Native AFM multiscale dose responses. a–c , Native-AFM responses across the designed local multiscale suite for the a , CIFAR-10 convolutional network, b , CIFAR-10 vision transformer, and c , text transformer. Internal scale indices are logarithmic expansion coordinates and must not be interpreted as percentages of a natural update.

$\beta$	CIFAR-10 CNN	CIFAR-10 ViT	Text transformer
0.01	0.3718 / 0.0725	0.2467 / 0.2740	0.4353 / $-0.0066$
0.05	0.0922 / 0.6876	0.0409 / 0.8535	0.5321 / 0.0927
0.10	0.0577 / 0.8917	0.0220 / 0.9566	0.3445 / 0.3367
0.25	0.0400 / 1.0130	0.0254 / 1.0133	0.1151 / 0.8285
0.50	0.0379 / 1.0465	0.0206 / 1.0294	0.0302 / 0.9861
1.00	0.0347 / 1.0556	0.0205 / 1.0295	0.0178 / 1.0033

Table 21: Independent-direction robustness for AFM/projection. Each cell gives

R_{\mathrm{dir}/\kappa}

/ matched

\kappa

slope.

System	Mean matched slope	95% CI
CIFAR-10 CNN	1.01544	[1.01324, 1.01778]
CIFAR-10 ViT	1.00064	[1.00010, 1.00114]
Stronger CIFAR-10 ViT	0.99883	[0.99824, 0.99942]
Text transformer	0.95950	[0.95049, 0.96863]

Table 22: Ten-seed AFM/projection matched slopes at

\beta=0.5

$\beta$	Pooled slope	95% CI
0.01	0.36331	[0.35075, 0.37667]
0.05	0.77171	[0.76572, 0.77759]
0.10	0.82114	[0.81433, 0.82756]
0.25	0.93121	[0.92425, 0.93841]
0.50	0.99360	[0.99123, 0.99600]
1.00	1.00379	[1.00326, 1.00442]

Table 23: Equal-system pooled AFM/projection slope across ten common seeds.

System	Count
CIFAR-10 CNN	52
CIFAR-10 ViT	1
Text transformer	12
Total	65

Table 24: Classification of all negative theorem-aligned-margin cases near requested

\kappa=0

System	Natural-norm fraction	Target norm	Feasible/attempted	Seeds with any feasible
CNN	0.01	0.00060570	1/500 (0.2%)	1
CNN	0.10	0.00605702	0/500	0
CNN	0.50	0.03028509	0/500	0
CNN	1.00	0.06057018	0/500	0
ViT	0.01	0.00039144	231/500 (46.2%)	10
ViT	0.10	0.00391441	0/500	0

Table 25: Fixed-norm bridge feasibility.

Method	$\kappa\mapsto\Delta_{\mathrm{persistent}}$ slope [95% CI]	$\kappa\mapsto\rho$ slope [95% CI]
EWC-prox	$4.3693\times 10^{-5}$ [3.5585, 5.1115] $\times 10^{-5}$	$-7.895$ [ $-10.701$ , $-5.197$ ]
Linearized distillation	$2.59767\times 10^{-4}$ [2.54328, 2.65080] $\times 10^{-4}$	$-3.494$ [ $-5.588$ , $-1.502$ ]
Projection / AFM base	$3.06428\times 10^{-4}$ [3.03053, 3.09322] $\times 10^{-4}$	$-0.572$ [ $-1.591$ , 0.455]
Unrestricted branch	$4.38287\times 10^{-5}$ [3.55718, 5.12805] $\times 10^{-5}$	$-7.891$ [ $-10.717$ , $-5.190$ ]

Table 26: ViT at 1% natural update norm,

\beta=0.5

: absolute persistent-progress and normalized-ratio slopes.

$\Delta_{0}$ -CV band	States	$\kappa\mapsto\Delta_{\mathrm{persistent}}$ slope	$\kappa\mapsto\rho$ slope
Low	78	$3.06394\times 10^{-4}$	2.2463
Medium	77	$3.06677\times 10^{-4}$	3.1477
High	76	$3.06452\times 10^{-4}$	$-7.5366$

Table 27: Projection branch at ViT 1% natural update norm, stratified by

\Delta_{0}

heterogeneity. Values are identical across the tested projection retention budgets because the projection endpoint saturates to the same branch in this setting.

Figure 6: Full AFM protected-update transaction. Detailed original AFM transaction showing same-state comparator construction, compatibility projection, normalized persistent assimilation, finite functional constraints, compact-cardinal shield, endpoint checks and accept/reject logic.

	CORe50	CLEAR-10	CLAD-C
Dataset structure	Official $128\times 128$ CORe50 stream Lomonaco and Maltoni (2017) ; $10$ classes and $12$ predeclared episodes covering new contexts, recurrence, long-dormancy return, an explicit $0\leftrightarrow 1$ target conflict, gradual visual drift, and capacity pressure.	Official CLEAR-10 imagery Lin et al. (2021) ; $11$ labels including background, $10$ chronological supervised buckets, and natural temporal drift.	CLAD-C chronological object classification Verwimp et al. (2023) on labeled SODA10M Han et al. (2021) ; $6$ classes and $6$ official chronological training segments. Only labeled object crops are used: no SODA10M unlabeled pretraining, no CLAD-D, and no external pretrained backbone.
Learner stream	Batch size $32$ ; task/session/episode metadata withheld; evaluator-only semantic regimes and validity intervals.	$32960$ learner examples, $3296$ per bucket, batch size $32$ , $1030$ updates; bucket and period metadata withheld.	$22249$ official object crops in six preserved segments with item counts 5157, 1154, 6742, 2560, 4517 and 2119; maximum batch size $10$ with partial boundary batches retained. Segment metadata and original labels remain evaluator-only.
Representation prefix	$20$ batches / $640$ images, then backbone freeze.	Complete first supervised episode: $103$ batches / $3296$ images, then backbone freeze. Optional unlabeled bucket- $0$ pretraining is not used.	Complete first official training segment: $516$ batches / $5157$ crops, then backbone freeze.
Candidate fitting	$256$ routed examples, $20$ fixed epochs, validation horizon $2048$ , risk threshold $0.7$ .	$256$ routed examples, $100$ fixed epochs, validation horizon $4096$ , risk threshold $0.7$ .	$256$ routed examples, $100$ fixed epochs, validation horizon $4096$ , risk threshold $0.7$ .
Evaluation	Checkpoint matrix over active semantic regimes; held-out test and complete original-semantics evaluator.	Checkpoint at every bucket boundary; $1137$ validation and $4363$ held-out test examples from supervised buckets $1$ - $10$ ; next-bucket evaluation for near-future accuracy.	Checkpoint after every official training segment; the official data loader yields $32967$ validation and $46915$ held-out test object crops.
Matrix	$17$ variants per seed and $5$ seeds: $85$ jobs per dataset, $255$ jobs total across the three frozen matrices.

Table 28: Experimental protocols. All learners receive labels for supervised updates but no task, session, episode, bucket, segment, intervention, or boundary identifiers.

Dataset	Comparator	AFM HV	Comparator HV	Mean $\Delta$	95% CI	Wins	Relative
CORe50	No protection	0.2478	0.2388	+0.0090	[0.0059, 0.0126]	5/5	3.76%
CLEAR-10	No protection	0.3807	0.3740	+0.0068	[0.0053, 0.0082]	5/5	1.81%
CLAD-C	A-GEM	0.5129	0.5019	+0.0110	[0.0054, 0.0166]	5/5	2.20%

Table 29: Primary paired retention-plasticity hypervolume results across five seeds. Confidence intervals are paired across seeds; relative improvement is normalized by the comparator mean.

Family	CORe50	CLEAR-10	CLAD-C
AFM	0.2478	0.3807	0.5129
No protection	0.2388	0.3740	0.4872
Matched SGD	0.2031	0.3596	0.4714
Replay	0.1839	0.3503	0.4797
A-GEM	0.1845	0.3521	0.5019
Online EWC	0.1829	0.3511	0.4834

Table 30: Five-seed mean family hypervolume for AFM and the classical comparison families. Oracle EWC receives task information and is reported only as a task-aware reference.

Dataset	Point	$R$	$P$	Fgt. $\downarrow$	BWT $\uparrow$	Test $\uparrow$	Online $\uparrow$	NF $\uparrow$
CORe50	AFM 0.10	0.9648	0.2088	0.0352	-0.0186	0.2291	0.2125	-
CORe50	AFM 0.50	0.9185	0.2458	0.0815	-0.0658	0.2362	0.2594	-
CORe50	AFM 1.00	0.9020	0.2595	0.0980	-0.0842	0.2398	0.2764	-
CORe50	No protection	0.8866	0.2693	0.1134	-0.0991	0.2341	0.2866	-
CLEAR-10	AFM 0.10	0.9933	0.3750	0.0067	0.0056	0.3412	0.3753	0.3263
CLEAR-10	AFM 0.50	0.9828	0.3793	0.0172	-0.0013	0.3417	0.3799	0.3309

Table 31: Five-seed mean operating points. Fgt. denotes mean forgetting, BWT backward transfer, Test held-out final-test accuracy, Online chronological stream accuracy, and NF CLEAR-10 next-bucket accuracy.

Dataset	AFM HV	CCL-DC HV	MKD HV	FGH HV	aL-SAR HV	LPR HV	OCAR HV
CORe50	0.2478	0.2016	0.1799	0.2546	0.1801	0.1640	0.1436
CLEAR-10	0.3807	0.3731	0.3513	0.3773	0.3572	0.3485	0.3164
CLAD-C	0.5129	0.4921	0.4931	0.3058	0.3296	0.4694	0.3747

Table 32: Five-seed mean retention-plasticity hypervolume for AFM and six modern challengers. AFM contributes its predeclared three-point frontier; each challenger contributes the

RP

area of its fixed operating configuration.

Method	$R\uparrow$	$P\uparrow$	Fgt. $\downarrow$	BWT $\uparrow$	Online $\uparrow$	Test $\uparrow$	Primary $\uparrow$
AFM	0.9127	0.4031	0.0873	-0.0312	0.4092	0.3849	0.3805
CCL-DC	0.9424	0.3850	0.0576	0.0081	0.3955	0.3950	0.3556
MKD	0.9634	0.3604	0.0366	0.0175	0.3714	0.3916	0.3414
LPR	0.9538	0.3512	0.0462	0.0202	0.3634	0.3799	0.3273
FGH	0.8716	0.3592	0.1284	-0.0215	0.3816	0.3591	0.3126
aL-SAR	0.9625	0.3018	0.0375	0.0343	0.3277	0.2934	0.2890

Table 33: Equal-weighted descriptive summary of conventional metrics across the three datasets. AFM uses the coordinate with highest AFM five-seed mean final-test accuracy on each dataset; the Primary column remains the mean of the declared benchmark-level primary scores.

Comparison	$\Delta R$	$\Delta P$	$\Delta$ Fgt.	$\Delta$ BWT	$\Delta$ Online	$\Delta$ Test	$\Delta$ Primary
AFM vs CCL-DC	-3.14%	+4.72%	+2.96 pp	-3.93 pp	+3.46%	-2.55%	+7.01%
AFM vs MKD	-5.26%	+11.86%	+5.07 pp	-4.87 pp	+10.19%	-1.70%	+11.44%
AFM vs LPR	-4.31%	+14.80%	+4.11 pp	-5.14 pp	+12.62%	+1.32%	+16.26%
AFM vs FGH	+4.72%	+12.24%	-4.12 pp	-0.97 pp	+7.22%	+7.19%	+21.73%
AFM vs aL-SAR	-5.17%	+33.58%	+4.98 pp	-6.55 pp	+24.88%	+31.20%	+31.67%
AFM vs OCAR	-8.29%	+44.00%	+8.25 pp	-3.62 pp	+34.54%	+37.93%	+36.76%

Table 34: AFM gain or loss relative to each modern challenger in the equal-weighted descriptive summary. R, P, Online, Test, and Primary are relative changes; Fgt. and BWT are AFM-minus-challenger percentage-point differences.

Ablation	Full AFM	Ablation	Mean $\Delta$	95% CI	Wins	Relative gain
Base-only, finite normalized budget	0.2257	0.1903	+0.0354	[+0.0276, +0.0445]	5/5	+18.61%
Base-only, fixed-absolute budget	0.2257	0.1903	+0.0354	[+0.0266, +0.0446]	5/5	+18.59%
Single-timescale AFM	0.2257	0.2257	+0.0000	[-0.0008, +0.0005]	4/5	+0.01%

Table 35: Paired five-seed CORe50

R\times P

ablation results. Differences are full AFM at

\eta=0.50

minus the indicated ablation.

Dataset	Runs	Certs	Commits	Protected nonzero	Exact restore accepted/attempts	Max endpoint error	Min deployed ratio	Min analytic margin
CORe50	15	90	90	8621	8636/8829	$4.768\times 10^{-7}$	0.99999494	$+2.107\times 10^{-3}$
CLEAR-10	15	99	99	10416	10431/10494	$9.537\times 10^{-7}$	0.99999718	$+2.940\times 10^{-2}$
CLAD-C	15	488	488	22917	22932/24980	$9.537\times 10^{-7}$	0.99993644	$+1.378\times 10^{-4}$

Table 36: Execution-level mechanism audit over the three protected AFM coordinates and five seeds on each real-world benchmark. The analytic margin is the minimum realized persistent-base progress ratio minus the theorem lower bound.

Component	CORe50	CLEAR-10	CLAD-C
Ordinary learning	0.36%	0.44%	0.24%
Same-state no-protection comparator	0.52%	0.27%	0.25%
Protection geometry	37.33%	16.12%	42.06%
Candidate machinery	1.94%	2.17%	2.00%
Finite counterfactual completion	25.95%	22.90%	28.55%
Endpoint verification	31.54%	52.68%	25.17%

Table 37: Component-level host-timing decomposition of AFM at the independently selected operating points. Entries are median percentages across the five frozen seeds. Profiling uses low-perturbation host timers without per-region CUDA synchronization; CUDA-event timing is recorded separately. Because each component is summarized independently, columns need not sum exactly to

100\%

Seed	CORe50 AFM	CORe50 no-prot.	$\Delta$	CLEAR AFM	CLEAR no-prot.	$\Delta$	CLAD AFM	CLAD A-GEM	$\Delta$
11	0.2481	0.2403	+0.0078	0.3852	0.3763	+0.0089	0.5065	0.4961	+0.0104
29	0.2281	0.2205	+0.0076	0.3693	0.3652	+0.0041	0.5001	0.4951	+0.0051
47	0.2520	0.2371	+0.0149	0.3836	0.3778	+0.0058	0.5231	0.5198	+0.0033
71	0.2621	0.2586	+0.0036	0.3752	0.3684	+0.0069	0.5216	0.5010	+0.0207
101	0.2489	0.2377	+0.0111	0.3903	0.3822	+0.0081	0.5133	0.4977	+0.0156

Table 38: Per-seed hypervolume for AFM and the strongest non-oracle comparator within each protocol. Hypervolume magnitudes are not compared across datasets.

Dataset	Coordinate	Forgetting reduction	Relative reduction	Test difference
CORe50	0.10	0.0783	69.00%	-0.499
CORe50	0.50	0.0319	28.12%	+0.205
CORe50	1.00	0.0154	13.59%	+0.565
CLEAR-10	0.10	0.0177	72.66%	+0.358
CLEAR-10	0.50	0.0072	29.42%	+0.408
CLEAR-10	1.00	0.0012	5.06%	+0.101

Table 39: AFM operating-point trade-offs relative to no protection. Test differences are percentage points; positive values favor AFM.

Method	Persistent auxiliary state	Optimization	Principal frozen settings
OCAR	Replay memory, 128 examples	Shared learning rate $10^{-3}$	Curvature/Fisher, damping, and replay rules retained
LPR	Replay memory, 128 examples	Learning rate $0.01$	$\omega_{0}=1$ , $\beta=1$ , preconditioner refresh every 100 iterations
aL-SAR	Replay memory, 4000 examples	Adam, $3\times 10^{-4}$	Update batch 16, online factor $0.015625$ , unfreeze rate $0.5$ , temperature $0.125$ , $k=4$ , warm-up 50
FGH	Prototype state	Adam, $5\times 10^{-3}$	Hypergradient rate 1, gradient-weight clamp 1000, one online epoch, prototype coefficient 1
CCL-DC	Second learner, replay memory 1000	AdamW, $5\times 10^{-4}$ , weight decay $10^{-4}$	Replay batch 64, one memory iteration, distillation weight 2, temperature 4
MKD	EMA teacher, replay memory 1000	Adam, $5\times 10^{-4}$	Replay batch 64, distillation weight 5.5, temperature 4, EMA $0.01$ , correction interval 10

Table 40: Frozen configurations used for the six modern challengers. Settings are method-native unless a shared benchmark value is stated.

Method	Equal-weighted three-benchmark primary score
AFM	0.3805
CCL-DC	0.3556
MKD	0.3414
LPR	0.3273
FGH	0.3126
aL-SAR	0.2890

Table 42: Equal-weighted mean of the three benchmark-level primary scores. This descriptive summary complements, but does not replace, the dataset-specific primary analysis.

Dataset	Method	$R\uparrow$	$P\uparrow$	Fgt. $\downarrow$	BWT $\uparrow$	Test $\uparrow$	Online $\uparrow$
CORe50	AFM 0.10	0.9648	0.2088	0.0352	-0.0186	0.2291	0.2125
	AFM 0.50	0.9185	0.2458	0.0815	-0.0658	0.2362	0.2594
	AFM 1.00	0.9020	0.2595	0.0980	-0.0842	0.2398	0.2764
	CCL-DC	0.9756	0.2066	0.0244	0.0339	0.2648	0.2240
	MKD	0.9952	0.1808	0.0048	0.0374	0.2415	0.1929
	FGH	0.8755	0.2910	0.1245	-0.1088	0.2493	0.3087

Table 43: Five-seed mean conventional continual-learning metrics for AFM and the six modern challengers. Fgt. denotes forgetting, BWT backward transfer, Test held-out final-test accuracy, and Online chronological stream accuracy.

Quantity	AFM $\eta=1.00$	No protection
Pedestrian peak accuracy at boundary 2	77.24%	60.34%
Pedestrian accuracy at boundary 3	0.00%	0.00%
Final Pedestrian accuracy	1.58%	1.76%
Pedestrian peak-to-final forgetting	75.66%	58.59%
Mean Pedestrian margin at boundary 2	+0.482	+0.053
Mean Pedestrian margin at boundary 3	-4.202	-5.587

Table 44: Exploratory CLAD-C Pedestrian diagnostic from saved checkpoints, reported as five-seed means unless stated otherwise. This diagnostic is not part of the primary benchmark analysis, and no retraining is involved.

Method	$R\uparrow$	$P\uparrow$	Fgt. $\downarrow$	BWT $\uparrow$	Online $\uparrow$	Test $\uparrow$	$R\times P\uparrow$
Full AFM, $\eta=0.50$	0.9185	0.2458	0.0815	-0.0658	0.2594	0.2362	0.2257
Base-only, finite normalized budget	0.9984	0.1906	0.0016	-0.0005	0.1885	0.2007	0.1903
Base-only, fixed-absolute budget	0.9977	0.1908	0.0023	-0.0004	0.1886	0.2016	0.1903
Single-timescale AFM	0.9205	0.2452	0.0795	-0.0638	0.2589	0.2354	0.2257

Table 45: Five-seed mean CORe50 metrics for full AFM at

\eta=0.50

and the targeted ablations. The final column is the single-operating-point

R\times P

product, not the AFM family hypervolume.

Operation	Median calls	Median host-observed time/call	Host-timing share
Protected-projector construction	1,711	2861.552 ms	37.89%
Compact-cardinal shield construction	1,526	1611.751 ms	20.58%
Safe-base backtracking	1,665	613.933 ms	8.60%
Finite-address preparation	1,665	601.503 ms	8.08%
Protected prestate evaluation	1,711	599.454 ms	8.06%
Protected poststate evaluation	1,664	599.695 ms	8.05%

Table 46: Invocation frequency and host-observed per-call cost of the principal AFM operations on CLAD-C at

\eta=0.1

. Values are medians across the five frozen seeds. Host timing uses the low-perturbation profiling mode without per-region CUDA synchronization; CUDA-event timing is recorded separately. The final column reports the median share of profiled host-observed elapsed time.

Scenario	Seeds passed	Structural events	Observed behavior
Favourable recurrence	3/3	commits 6/6/6; protected steps 20/13/10	recurrent protected learning completed
Semantic conflict	3/3	reopenings 2/3/1; splits 0/0/0	conflict handled without an unsupported route split
Observable shift	3/3	commits 3/3/3; protected steps 41/27/27	observable-shift control completed
Within-route observable shift	3/3	splits 1/1/1	positive route splitting exercised on every seed
Capacity pressure	3/3	commits 11/11/11; protected steps 30/18/21	bounded-capacity control completed
Identical-observation impossibility	3/3	commits 3/3/3; protected steps 12/8/11	indistinguishable observations did not induce a false semantic resolution

Table 47: Controlled AFM mechanism tests on three seeds. These experiments test recurrence, conflict, route refinement, capacity pressure, and observational indistinguishability rather than external predictive performance.

Method	$R$	$P$	$R\times P$
Full AFM, $\eta=0.50$	0.9185	0.2458	0.2257
Base-only, normalized budget	0.9984	0.1906	0.1903
Base-only, fixed absolute budget	0.9977	0.1908	0.1903
Single-timescale AFM	0.9205	0.2452	0.2257

Table 48: AFM transaction ablation and execution validation

Explore similar work

Jun 7, 2026cs.LG

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

One of the critical limitations of artificial neural networks is their lack of ability to continually learn: training on new tasks often leads to interference and forgetting of the previous ones. While several algorithms have been proposed to protect old memories from interference, they are typically applied during or immediately after each new episode of training. In contrast, humans and animals can learn continuously, acquiring multiple new memories during active learning before consolidating all of them into long-term storage. Here we show that multiple new tasks can be trained sequentially before an unsupervised sleep-like replay phase is applied to partially restore performance across all previously learned tasks. Our study further suggests that task-specific information remains resilient to new training but decays gradually as network is trained on new tasks. These findings point to novel principles for developing a broad range of continual learning AI solutions.

Anthony Bazhenov, Jean Erik Delanois, Giri P. Krishnan

Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA

Catastrophic Forgetting Continual Learning Forgetting New Tasks

Ask PDF

May 11, 2026q-bio.NC

Joint sparse coding and temporal dynamics support context reconfiguration

Adaptive behavior requires the brain to transition between distinct contexts while maintaining representations of prior experience. The ability to reconfigure neural representations without erasing previously acquired knowledge is central to learning in dynamic environments, yet the neural mechanisms that support this balance remain unclear. Understanding these mechanisms is also critical for addressing catastrophic forgetting in artificial systems designed for lifelong learning. Here, we identify joint sparse coding and temporal dynamics in both the mouse medial prefrontal cortex (mPFC) and computational networks as mechanisms that help preserve prior representations during context transitions. Specifically, sparsity in context-dependent representations reduces cross-context interference, whereas temporal dynamics within the network activity further enhance context separability across time. Strikingly, networks endowed with both properties, such as spiking neural networks, exhibit improved retention during lifelong learning without auxiliary heuristics. These findings establish joint sparse coding and temporal dynamics as a core mechanism supporting flexible context reconfiguration in lifelong learning and, through their activity constraining nature, as an energy-efficient architectural principle for stable adaptation. Together, they provide a mechanistic framework for understanding how the brain preserves prior knowledge while flexibly adapting to new contexts.

Qianqian Shi, Yue Che, Faqiang Liu +6

Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing, China · Guangdong Institute of Intelligence Science and Technology, Hengqin, China · Optical Memory National Engineering Research Center, Tsinghua University, Beijing, China +3

Synaptic Plasticity Catastrophic Forgetting Human Visual Cortex Temporal Dynamics

Ask PDF

Jul 10, 2026cs.LG

Interference and Retention in Continual Learning

Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.

Julius Störk

VARTA Microbattery GmbH, Ellwangen, Germany

Catastrophic Forgetting Continual Learning Retention Interference

Ask PDF