Neural Succession: A Mesoscopic Theory of Invasion, Coexistence, and Stabilization in Continual Learning

Shoaib Ahmed Dipu; Md Salman Shamil; Sayeed Shafayet Chowdhury

cs.LGSep 28, 2026

Neural Succession: A Mesoscopic Theory of Invasion, Coexistence, and Stabilization in Continual Learning

Authors: Shoaib Ahmed Dipu, Md Salman Shamil, Sayeed Shafayet Chowdhury

Organizations: Indiana University Indianapolis · North South University

Abstract

Continual learning is usually studied through mechanisms that preserve old knowledge. We develop Successional Learning Theory (SLT), a mesoscopic account in which the current representation is a resident community, the incoming task is an invader, forgetting is resident displacement, joint retention is coexistence, replay is resident reinforcement, and training moves from establishment toward stabilization. Its empirical coordinate is directional pre-invasion compatibility, measured on the resident model before the incoming task is learned. Across eight experiments, compatibility orders later forgetting on the 20 directed Split-CIFAR-10 transitions (three-repeat r=-0.789, incoming-task cluster 95% CI [-0.90,-0.72], every repeat alone r<=-0.67), forecasts held-out forgetting with 24% lower error than a no-information baseline, and reproduces under controlled MNIST permutations and CIFAR-10 rotations (r=-0.804, -0.718). On an 84-transition suite, compatibility separates coexistence from exclusion at every retention threshold (AUC 0.93-0.97). Replay repairs every transition with at most 325 stored examples and is most efficient where displacement is largest. Compatibility reaches |r|=0.720, while activation, representation, Jacobian, and fixed-coefficient Lotka-Volterra specializations do not. Plasticity and feature turnover fall reliably from early to late training (15/15 and 14/15 runs). We formalize a minimum habitat-modification bound, a displacement floor, a sufficient coexistence condition, an identifiability law with a range-restriction corollary, successional stabilization, and local reinforcement. The identifiability law also predicts where the coordinate loses leverage, and the prediction matches three CIFAR-100 partitions and five optimizer regimes. SLT is a pre-adaptation diagnostic that complements replay, regularization, and projection methods.

Figures & tables

Panel of Figure 1 — Figure 1: The SLT pipeline. A resident representation is probed before the incoming task is learned. The resulting coordinate diagnoses the transition, which then unfolds as establishment followed by stabilization and ends in displacement or coexistence. Grey labels map each stage to the result that formalizes it and the experiment that tests it.

Family	Main object	Pre	Coexistence	Stages
Transferability / SPOT / NTK	task relation, kernel	$\checkmark$
EWC / SI / MAS	parameter importance
GEM / OGD / GPM	gradient subspaces		implicit
Replay / DER++	stored examples, logits
Task ordering	sequence relation	$\checkmark$
Plasticity studies	training dynamics			$\checkmark$

Table 1: Positioning by the question each family answers. “Pre”: computable before task-

B

adaptation.

Result	Observable prediction	Tests	Evidence	Status
Thm. 1 , Cor. 1	forgetting rises with $\Omega$ ; $\Omega$ tracks demand	E1, E2, E8	$r=-0.789$ ; $-0.804$ / $-0.718$ ; $\Omega$ –probe loss $0.767$	supported
Thm. 2	$\Omega$ separates coexistence from exclusion	E3	AUC $1.00$ (natural); $0.93$ – $0.97$ (84-suite)	supported
Thm. 3 , Cor. 2	$\|r\|$ falls with coordinate and outcome spread	scope	CIFAR-100: $0.38/0.18/{-0.05}$ vs. predicted $0.32/0.30/0.13$	supported
Prop. 1	early $>$ late plasticity and turnover	E7	$4.78$ $[4.26,5.53]$ ; $0.37$ $[0.11,0.48]$	supported
Prop. 2	replay repairs displacement; efficiency grows with displacement	E4	20/20 repaired ( $\leq 325$ ex.); $r(\eta,F_{0})=0.70$	supported
Scale claim	transition-level $\gg$ feature-level	E6	$\|r\|$ : $0.72$ vs. $\leq 0.45$	supported

Table 2: From theory to evidence. Each result is stated as an observable prediction and tested by a named experiment.

ID	Question	Setting	Units	Primary statistic
E1	Does $\rho$ order displacement?	Split-CIFAR-10, ResNet-18	$20\times 3$ repeats	$r(\rho,F)$ on pair means
E2	Is the order prospective?	same; $\rho$ from resident only	20, LOPO	held-out $r$ , MAE
E3	Is coexistence a phase?	natural; pure-domain; MNIST suite	20; 60; 84	rates; ROC AUC
E4	Does replay reinforce?	buffers $0$ – $1000$	20	efficiency, threshold vs. $\rho$
E5	Does $\rho$ help an intervention?	CIFAR-10/100, 6 methods	3 seeds	ACC, BWT
E6	At which scale?	5 predictors; probe/LV tests	20	$\|r\|$ , pre-registered gates

Table 3: The eight SLT experiments. Units are directed transitions unless noted.

Figure 2: Compatibility is prospective. (a) Pair means over three independent repeats (bars: s.d.; band: OLS 95%). (b) Leave-one-pair-out forecasts (dashed: identity; band:

\pm

MAE). (c) The resident-only score agrees with a score that also uses the reverse direction.

Figure 3: Left: controlled manipulations of task relation (E8); black line and band: linear fit with 95% interval; red dashed: quadratic fit on MNIST. Right: identifiability (Theorem 3 ). (a) Predictive strength falls with the between-transition spread of

\rho

along the range-restriction curve of Corollary 2 , anchored at E1. (b) Across optimizer regimes, the one regime with collapsed outcome variance is the one where ordering vanishes.

Figure 4: Coexistence is a phase of the transition. (a) Natural transitions: the three that coexist (green,

F<0.20

) are the three most compatible; dotted line: compatibility split. (b) Pure-domain construction: only same-domain transitions reach near-zero forgetting. (c) 84-transition suite: coexistence AUC of

\rho

and of a representation-compatibility score

S

across mutual-retention thresholds (band: 95% cluster bootstrap over 28 class-pair

\times

transform clusters).

Figure 5: Scale selection. (a) On the same 20 transitions, the transition-level coordinate predicts displacement far better than feature- or step-level quantities. (b) Increasing Jacobian probes 16-fold at two layers does not produce a predictive microscopic coefficient.

Figure 6: Establishment precedes stabilization. (a, b) Median and interquartile range over 15 runs of per-epoch plasticity and turnover of the top-quartile resident features, while

A

is established and then while

B

invades. (c) Pre-specified contrasts with 95% intervals; grey: interval includes zero.

Claim	Result	Uncertainty / comparison
E1 natural ordering	$r=-0.789$ , $R^{2}=0.622$	CI $[-0.90,-0.72]$ ; each repeat $\leq-0.67$ ; single run $-0.710$
E2 held-out forecast	MAE $7.62$ pp, $r=0.648$	no-information MAE $10.03$ pp; LOTO $r=0.669$
E8 controlled relation	$r=-0.804$ / $-0.718$	group-CV $R^{2}$ $0.626$ / $0.484$
E3 coexistence phase	AUC $1.00$ / $0.927$	natural / 84-suite (CI $[0.83,0.98]$ ); 8/24 vs. 0/36
E4 reinforcement	20/20 repaired	$\leq 325$ examples (median 100); $r(\eta,F_{0})=0.70$
E5 intervention	ACC $0.815$ / $0.531$	above fine-tune, EWC, A-GEM on both benchmarks

Table 4: Main results with uncertainty. CI: 95% interval (cluster bootstrap unless stated).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Threshold $\tau$	Coexistence rate	$\rho_{\mathrm{pre}}$ (SLT)	Representation compatibility $S$	Difference
0.80	0.667	0.936 $[0.85,0.99]$	0.866 $[0.74,0.96]$	$+0.070$ $[0.001,0.169]$
0.90	0.571	0.927 $[0.83,0.98]$	0.887 $[0.76,0.97]$	$+0.040$ $[-0.014,0.113]$
0.95	0.476	0.966 $[0.89,1.00]$	0.936 $[0.81,1.00]$	$+0.030$ $[-0.005,0.106]$
0.99	0.310	0.969 $[0.91,1.00]$	0.940 $[0.85,0.99]$	$+0.029$ $[-0.026,0.112]$

Appendix

Table 5: E3c: coexistence AUC on the 84-transition suite (95% cluster-bootstrap intervals).

Figure 7: Establishment bridge. Within incoming task,

\Omega

tracks the incoming probe loss on Split-CIFAR-10 (

r=0.767

, cluster 95% CI

[0.28,0.95]

; clipped deficit

H

0.692

[-0.14,0.94]

with five clusters) but not on semantic CIFAR-100 (

r=0.131

), where the coordinate has little spread.

Figure 8: Replay (E4). (a) The global minimum buffer is not ordered by

\rho

. (b) Initial per-sample efficiency decreases with

\rho

. (c) Efficiency rises with forgetting at zero replay, as Proposition 2 (ii) predicts; part of this relation is expected from headroom.

Figure 9: Interventions (E5). (a, b) Task-incremental accuracy (mean

\pm

s.d., three seeds). (c) Paired differences for the gated sign tests.

	Fine-tune	EWC	A-GEM	$\rho$ -EWC	$\rho$ -EWC+replay	DER++
Split-CIFAR-10	0.536 / $-$ 0.480	0.526 / $-$ 0.492	0.773 / $-$ 0.184	0.569 / $-$ 0.436	0.815 / $-$ 0.111	0.851 / $-$ 0.068
Split-CIFAR-100	0.229 / $-$ 0.522	0.349 / $-$ 0.330	0.460 / $-$ 0.259	0.243 / $-$ 0.501	0.531 / $-$ 0.117	0.596 / $-$ 0.071

Appendix

Table 6: E5 benchmark (task-incremental, three seeds): ACC / BWT.

Figure 10: Robustness. (a) All 24 estimator settings. (b) Label budget: agreement with the full score and correlation with forgetting; red dotted: label-free k-means variant. (c) Optimizer regimes with task-cluster intervals (grey: interval includes zero). (d) Leave-one-task-out slopes on a pretrained ResNet-50.

Test	Criterion	Result
Jacobian competition, task level	$r\geq 0.5$ , perm. $p\leq 0.05$	$r=0.177$ , $p=0.54$
Jacobian competition, feature level	median partial $r>0.10$	$\leq 0.105$ at 1024 probes; $\leq 6.7\%$ significant
Fixed-coefficient LV	beats baselines (Wilson $>0.5$ )	0/60; median held-out $R^{2}=-29.2$
Locally stationary LV	stable windows exist	0/330 stable; unstable $R^{2}=-1.08$
Neural $R^{\ast}$	resources identifiable	1.9%; precision 0.0 vs. base 0.58
Niche/fitness (ND–FD) decomposition	explains forgetting	$R^{2}=0.21$ (held-out $0.11$ ), 45 pairs

Appendix

Table 7: Pre-registered microscopic tests (E6). All run in a regime with forgetting s.d.

0.064

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 304 papers

Functional compatibility as a determinant of persistent neural learning
Aug 23, 2026Hossein JavidniaStability-Plasticity Dilemma Continual Learning
School of Computing Dublin City University Dublin, Ireland
Interference and Retention in Continual Learning
Jul 10, 2026Julius StörkContinual Learning Online Continual Learning
VARTA Microbattery GmbH, Ellwangen, Germany
Forgetting, plasticity, and co-observation: a third facet of continual learning
Aug 19, 2026Timm Hess, Abhishek Jha, Gido M. van de Ven +1Continual Learning Catastrophic Forgetting
VIB.AI † KU Leuven, Belgium · Bernoulli Institute ‡ University of Groningen, Netherlands

Aug 23, 2026J/K move · Enter open · S save

Functional compatibility as a determinant of persistent neural learning

Hossein Javidnia

School of Computing Dublin City University Dublin, Ireland

Stability-Plasticity Dilemma Continual Learning Neural Network Training Dynamics Catastrophic Forgetting

Open paper AskPDF arXiv

Neural networks can acquire new capabilities while damaging existing ones, but what determines whether new learning persists remains unclear. We identify functional compatibility, the extent to which incoming learning can coexist with behaviour that must be preserved, as an experimentally manipulable causal determinant of persistence. From identical neural states, we vary compatibility while matching unrestricted learning opportunity and imposing a common retention requirement. Persistent learning increases with compatibility across independent directions, convolutional and transformer architectures, vision and text, and a ten-seed replication. Learning rules and retention constraints determine how much compatible opportunity is retained, whereas nonlinear geometry limits the matched intervention at larger update norms. Functional compatibility therefore reframes stability-plasticity from preventing forgetting to determining which new learning can coexist with existing function and persist.

Abstract

Continual Learning

Explore similar work

Functional compatibility as a determinant of persistent neural learning

Interference and Retention in Continual Learning

Forgetting, plasticity, and co-observation: a third facet of continual learning