Information as Structural Alignment: A Dynamical Theory of Continual Learning

Radu Negulescu

cs.LGApr 8, 2026

Information as Structural Alignment: A Dynamical Theory of Continual Learning

Authors: Radu Negulescu

Organizations: The Informational Buildup Foundation

Abstract

Catastrophic forgetting is not an engineering failure. It is a mathematical consequence of storing knowledge as global parameter superposition. Existing methods, such as regularization, replay, and frozen subnetworks, add external mechanisms to a shared-parameter substrate. None derives retention from the learning dynamics themselves. This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning, based on the premise that information is the achievement of structural alignment rather than stored content. In IBF, two equations govern the dynamics: a Law of Motion that drives configuration toward higher coherence, and Modification Dynamics that persistently deform the coherence landscape in response to localized discrepancies. Memory, agency, and self-correction arise from these dynamics rather than being added as separate modules. We first demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains: a controlled non-stationary world, chess evaluated independently by Stockfish, and Split-CIFAR-100 with a frozen ViT encoder. Across all three, IBF achieves replay-superior retention without storing raw data. We observe near-zero forgetting on CIFAR-100 (BT = -0.004), positive backward transfer in chess (+38.5 cp), and 43% less forgetting than replay in the controlled domain. In chess, the framework achieves a mean behavioral advantage of +88.9 +/- 2.8 cp under independent evaluation, exceeding MLP and replay baselines.

Figures & tables

Condition	$\mathrm{Acc}_{A}$	$\mathrm{BT}_{A}$	$\mathrm{Acc}_{C}$
Full IBF	$.684\pm.051$	$-.234\pm.052$	$.913\pm.008$
No-Agency	$.721\pm.047$	$-.198\pm.052$	$.918\pm.006$
No-Crystallization	$.767\pm.103$	$-.152\pm.108$	$.920\pm.008$
No-Crucible	$.911\pm.007$	$-.005\pm.003$	$.921\pm.003$
MLP	$.442\pm.059$	$-.522\pm.066$	$.489\pm.098$
Replay MLP	$.536\pm.121$	$-.410\pm.113$	$.816\pm.029$

Table 1: RRW results (5 seeds, mean

\pm

std). Full IBF achieves substantially stronger backward transfer and forward learning than the compared MLP baselines. The ablations do not function here as weaker versions of the full system, but as controlled probes of distinct mechanisms inside an explicitly contradictory domain.

Agent	Mean cp	vs. Passive
Full IBF ( $\sigma^{*}=1.2693$ )	$-243.0$	$+90.2$
No-Agency ( $\sigma^{*}$ )	$-251.2$	$+81.9$
MLP	$-250.9$	$+82.3$
Replay	$-256.8$	$+76.4$
Passive	$-333.2$	$0.0$

Table 2: Chess reference run at the geometrically prescribed bandwidth under independent evaluation. Mean centipawn values are reported after deterministic greedy evaluation on

1{,}002

disjoint positions at

\sigma_{\mathrm{eval}}=\sigma^{*}=1.2693

. The No-Agency row quantifies the developmental contribution of agency at the prescribed geometric scale.

Method	Task-IL Avg	BT	Class-IL
Full IBF	$.892\pm.009$	$-.004\pm.000$	$.528$
Replay MLP	$.723$	$-.234$	$.392$
EWC	$.311$	$-.681$	$.052$
MLP	$.276$	$-.719$	$.049$
Coherence head	$.901$	$+.000$	—

Table 3: CIFAR-100 results. Task-IL values use the log-space readout described in Section 5.4 . IBF achieves replay-superior retention with zero stored raw data and retains a structural Class-IL advantage without a task oracle.

Primitive	Symbol	Definition
Field	$\mathcal{F}$	A nonempty universal set whose elements are informational patterns; the total space of possible informational configurations.
Pattern	$\psi\in\mathcal{F}$	An element of the Field. Patterns possess no intrinsic meaning or stability independent of their relations to systems and scales.
Closed Informational System (CIS)	$S\subset\mathcal{F}$	A connected region of configuration space where coherence exceeds a viability threshold and boundary gradients point inward, actively resisting dissolution.
Observational Scale	$\lambda\in\Lambda$	A resolution parameter specifying the level of abstraction at which patterns and systems are distinguished. Determines which distinctions are operational and which coherence function applies.
Configuration Space	$\mathcal{C}(S,\lambda)$	For each system $S$ at scale $\lambda$ , the differentiable manifold of all admissible internal states. Every point represents a possible internal configuration.
Coherence	$\widehat{\mathcal{R}}(x_{S},S,\lambda)$	A scalar function quantifying the degree of structural alignment between a system’s internal configuration and its environment. Relational, fragile, non-reducible.

Table 4: Formal Primitives of the Informational Buildup Framework. Nine primitives define the ontological substrate. The complete formal treatment, including structural properties and inter-primitive relations, is provided in a dedicated foundational work.

Ref.	Name	Statement / Role	Scope
Axioms (Existence and Motion)
I	The Field	$\mathcal{F}\neq\emptyset$ ; for each $(S,\lambda)$ , the Field admits an associated differentiable configuration space $\mathcal{C}(S,\lambda)$ .	Here
II	Coherence	$\widehat{\mathcal{R}}:\mathcal{C}(S,\lambda)\to\mathbb{R}^{+}$ ; relational, fragile, and non-decomposable. Defines viability and CIS boundaries.	Here
III	Informational Gravity	$G=\nabla_{x_{S}}\widehat{\mathcal{R}}$ ; systems experience a pull toward higher structural alignment.	Here
IV	Law of Motion	$\dot{x}_{S}=k_{S}\cdot\nabla_{x_{S}}\widehat{\mathcal{R}}^{\mathrm{eff}}$ ; motion follows the effective coherence gradient, modulated by responsiveness.	Here
Closure Postulates

Table 5: Formal Structure and Placement in the Broader Arc. The four axioms and the local modification postulate stated in this paper define the minimal structure needed to ground the continual learning experimental program. The table also situates that local arc within the broader foundational framework, whose remaining closure postulates and claims are shown here in condensed form. Rows marked Here are stated or empirically instantiated in this paper; rows marked Foundational belong to the larger dedicated treatment. Claims 9 and 10 are operationalized jointly in this paper as the Self-Correction capacity: the system reflexively tracks its cross-context coherence state (Thm. 9) to selectively trigger the thermodynamic dissolution of false alignment (Thm. 10).

Theoretical Construct	Engine Realization	Local Consequence
Axioms (Existence and Motion)
Field $\mathcal{F}$ (Axiom I)	Domain observation space together with the frozen encoder $\pi$ and induced latent configuration space $z\in\mathbb{R}^{d}$	The engine always operates on admissible latent configurations
Coherence $\widehat{\mathcal{R}}$ (Axiom II)	Frozen baseline evaluator $\hat{\widehat{\mathcal{R}}}_{\mathrm{base}}(z)$ plus effective correction field $\delta\hat{R}(z)$	Effective coherence landscape $\widehat{\mathcal{R}}^{\mathrm{eff}}(z)=\hat{\widehat{\mathcal{R}}}_{\mathrm{base}}(z)+\delta\hat{R}(z)$
Informational Gravity $G=\nabla_{x_{S}}\widehat{\mathcal{R}}$ (Axiom III)	Discrete coherence increments over candidate next states, $\Delta R^{\mathrm{eff}}_{j}$	Local directional preference toward higher alignment
Law of Motion (Axiom IV)	Boltzmann action selection with responsiveness $k_{\mathrm{eff}}(z)$ over candidate next states	Probabilistic ascent on the effective coherence landscape
Modification Postulate

Table 6: Claim-to-Engine Mapping. Domain-agnostic correspondence between the formal theory (Section 3 ) and the universal discrete engine (Section 4 ). The table records how each theoretical construct is realized at the engine level. Domain-specific architectures, losses, and evaluation protocols enter only later, in Section 5 .

Symbol	Definition	Engine Realization	Notes
Latent Configuration and Evaluation
$z$	Current latent configuration	Frozen encoder output $\pi(\omega)$	The engine operates only in latent space
$z_{j}$	Candidate next latent state	Latent image of action $a_{j}$	Used for discrete motion / action selection
$\hat{\widehat{\mathcal{R}}}_{\mathrm{base}}(z)$	Baseline coherence evaluator	Frozen domain-specific baseline	No backpropagation through engine state
$\delta\hat{R}(z)$	Coherence-correction field	Kernel readout from coherence particles	Additive modification of the baseline evaluator
$\widehat{\mathcal{R}}^{\mathrm{eff}}(z)$	Effective coherence	$\hat{\widehat{\mathcal{R}}}_{\mathrm{base}}(z)+\delta\hat{R}(z)$	Read path output for motion

Table 7: Engine Symbol Dictionary I: Latent Space and Particle State. Core symbols defining the latent configuration space, effective evaluation, and particle-level state used by the universal IBF engine.

Symbol	Definition	Engine Realization	Notes
Read Path
$\mathcal{K}(z,z_{i})$	Localization kernel	Gaussian RBF $\exp(-\\|z-z_{i}\\|^{2}/2\sigma_{i}^{2})$	Shared by read and write paths
$\gamma_{i}$	Read-gating variable	Same-context or verified cross-context read access	$\gamma_{i}\in\{0,1\}$
$\delta k(z)$	Responsiveness correction field	Intensive readout from crystallized responsiveness particles	Bounded local modulation
$k_{\mathrm{eff}}(z)$	Effective responsiveness	$\max(k_{\min},\,k_{0}+\delta k(z))$	Controls Boltzmann sharpness
$P(a_{j}\mid z_{\mathrm{current}})$	Action-selection probability	Boltzmann distribution over candidate next states	Discrete instantiation of the Law of Motion

Table 8: Engine Symbol Dictionary II: Read Path, Write Path, and Lifecycle. Core operational symbols governing kernel readout, localized writing, contradiction testing, and thermodynamic state transitions in the universal IBF engine.

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 304 papers

Interference and Retention in Continual Learning
Jul 10, 2026Julius StörkContinual Learning Online Continual Learning
VARTA Microbattery GmbH, Ellwangen, Germany
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Information-Theoretic Generalization Bounds Replay-Based Continual Learning
Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China
Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting
Mar 8, 2026Ran ChengCatastrophic Forgetting Online Continual Learning

Jul 10, 2026J/K move · Enter open · S save

Interference and Retention in Continual Learning

Julius Störk

VARTA Microbattery GmbH, Ellwangen, Germany

Continual Learning Online Continual Learning Task Interference Catastrophic Forgetting

Open paper AskPDF arXiv

Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.

Abstract

Continual Learning

Explore similar work

Interference and Retention in Continual Learning

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting