Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

Michael Chertkov

cs.LGMar 31, 2026

Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

Authors: Michael Chertkov

Organizations: Graduate Inter-Disciplinary Program in Applied Mathematics, & Department of Mathematics, University Arizona, Tucson, AZ 85721

Abstract

An agent that operates sequentially must incorporate new experience without forgetting old experience, under a fixed memory budget. We propose a framework in which memory is not a parameter vector but a stochastic process: a Bridge Diffusion on a replay interval $[0,1]$ , whose terminal marginal encodes the present and whose intermediate marginals encode the past. New experience is incorporated via a three-step Compress--Add--Smooth (CAS) recursion. We test the framework on the class of models with marginal probability densities modeled via Gaussian mixtures of fixed number of components~ $K$ in $d$ dimensions; temporal complexity is controlled by a fixed number~ $L$ of piecewise-linear protocol segments whose nodes store Gaussian-mixture states. The entire recursion costs $O(LKd^2)$ flops per day -- no backpropagation, no stored data, no neural networks -- making it viable for controller-light hardware. Forgetting in this framework arises not from parameter interference but from lossy temporal compression: the re-approximation of a finer protocol by a coarser one under a fixed segment budget. We find that the retention half-life scales linearly as $a_{1/2}\approx c\,L$ with a constant $c>1$ that depends on the dynamics but not on the mixture complexity~ $K$ , the dimension~ $d$ , or the geometry of the target family. The constant~ $c$ admits an information-theoretic interpretation analogous to the Shannon channel capacity. The stochastic process underlying the bridge provides temporally coherent movie'' replay -- compressed narratives of the agent's history, demonstrated visually on an MNIST latent-space illustration. The framework provides a fully analytical Ising model'' of continual learning in which the mechanism, rate, and form of forgetting can be studied with mathematical precision.

Figures & tables

Figure 1: One iteration of the compress–add–smooth recursion, illustrated for

L\!=\!4

segments. Top: the protocol at day

n

consists of

L

uniform segments on

[0,1]

. Middle: compression rescales the protocol to

[0,L/(L+1)]

; the new day is appended on

[L/(L+1)]

, producing

L\!+\!1

uniform segments. Bottom: rebinning averages the

L\!+\!1

segments back onto the

L

-segment grid. The right-hand labels indicate the information-theoretic role of each step: only smoothing is lossy. Dashed lines track a past-day readout time

t_{m|n}

, which contracts by factor

L/(L+1)

every day.

Figure 2: Single-Gaussian experiment ( $K=1$ , $d=2$ , $L=10$ ) under the default circular-drift setting. (a) Age-averaged normalized forgetting $\bar{F}(a)$ , showing retention half-life $a_{1/2}=30$ . The curve exhibits a low-error plateau for ages $0$ – $15$ , followed by a steep sigmoid transition crossing $\bar{F}=0.5$ at age 30, a slight overshoot to $\bar{F}\approx 1.08$ around age 50 (the confusion regime), and eventual saturation near $\bar{F}=1.0$ . The curve is weakly non-monotone due to the periodic geometry of the circular drift, which causes geometric recurrence at multiples of the half-period. (b) Full forgetting matrix $\bar{F}(m,n)$ as a function of recalled day $m$ and current day $n$ . The dominant trend is age-controlled forgetting (iso-forgetting contours run parallel to the diagonal), modulated by the periodic geometry of the underlying drift.

Figure 3: Default single-Gaussian circular-drift experiment ( $L=10$ ). (a) Original daily means (dots, coloured by day) and replayed means at the final day (crosses). The black star marks the prior mean (the origin), which serves as the protocol’s long-time attractor. Confusion is visible as the systematic inward displacement of crosses from the circle toward the star: recent memories (warm colours) are replayed near their true locations on the circle, while older memories (cool colours) are pulled progressively toward the prior mean. This convergence of replay means toward the star — rather than remaining on the circle — is the geometric signature of confusion. (b) Readout times $t_{m|n}$ versus current day $n$ , showing the geometric decay ( 9 ).

Figure 4: Selected replay ellipses in the default single-Gaussian circular-drift experiment (

L=10

). For each displayed day, the original target mean is shown by a dot, while the replayed mean is shown by a cross with a dashed ellipse representing the replay covariance. Recent memories (e.g. day 95) are replayed with small displacement and compact ellipses. Intermediate-age memories (e.g. day 35) show large displacement toward the origin and inflated covariances. Very old memories (e.g. day 25) collapse nearly to the origin with very large ellipses. Geometric aliasing is visible for day 5, whose true location on the circle lies close to day 95 (they differ by two full periods), producing an apparently accurate replay that is coincidental rather than genuine recall.

Figure 5: Segment-budget sweep for the single-Gaussian circular-drift experiment. (a) Age–forgetting curves for

L\in\{5,8,10,15,20,30\}

. Increasing

L

shifts the sigmoid transition to higher ages without changing the curve shape qualitatively. (b) Retention half-life

a_{1/2}

versus

L

, confirming approximate linear scaling.

Figure 6: (a) Drift-speed sweep (circle period

P

). Faster drift shortens the half-life, but the effect saturates for slow drift. (b) Circle vs. linear drift. The circular curve is non-monotone due to periodic spatial recurrence; the linear curve is a clean sigmoid with longer half-life. (c) Half-life versus period

P

Figure 7: (a) Age–forgetting curves for

K=1

(blue) and

K=3

(red), both with

L=10

and circular drift. The curves nearly coincide, with both yielding

a_{1/2}=30

. (b) Decomposed raw forgetting for

K=3

: mean misalignment (blue) dominates, covariance error (orange) is secondary, and weight error (green) is negligible.

Figure 8: Component-level trajectories for

K=3

. (a) Original daily component means, coloured by day. The three interleaved helical paths trace the rotating-triangle geometry. (b) Replayed component means (

\times

) at the final day, after Hungarian matching to the originals. Recent components are well-recalled; older ones collapse toward the origin.

Figure 9: (a)

L

sweep at

K=3

: age–forgetting curves for

L\in\{5,10,15,20,30\}

. The pattern mirrors the

K=1

case. (b)

K

sweep at

L=10

: age curves for

K\in\{1,2,3,5,8\}

nearly coincide. (c) Half-life summary for both sweeps.

Figure 10: Crowding sweep for Gaussian-mixture memories (

L=10

). (a) Retention half-life

a_{1/2}

versus crowding ratio

\chi

for

K=2,3,5,8

. The half-life is flat at 30 for

\chi\lesssim 1.5

and declines only for well-separated components. (b) Age–forgetting curves at

K=3

for six crowding values. (c) Mean-error share of total forgetting, stable at

{\sim}90\%

across all

\chi

Figure 11: Representative age-averaged forgetting curves across three crowding regimes (

K=3

L=10

). Strong (

\chi=0.3

) and medium (

\chi=0.9

) crowding yield indistinguishable curves (

a_{1/2}=30

). Weak crowding (

\chi=2.2

) produces a slightly earlier transition (

a_{1/2}=28

) and a more pronounced confusion overshoot.

Figure 12: Scaling with ambient dimension for a fixed 2D signal (

K=3

L=10

). (a) Age-forgetting curves for

d=2,4,8,16

without nuisance drift. Higher

d

slightly improves normalised retention due to the larger amnesia baseline. (b) Retention half-life versus ambient dimension under two nuisance settings. Both show gentle increase with

d

. (c) Mean-error share decreases with

d

as covariance mismatch grows in the extra dimensions.

Panel of Figure 13 — Figure 13: Schematic of the four-phase split-and-merge curriculum for $K=3$ . Coloured dots represent the three mixture component means; the dashed circle indicates the inter-component radius $r$ . Phase 1: normal rotating triangle ( $r=0.8$ ). Phase 2: two components merge ( $r_{01}\to 0.05$ ). Phase 3: split back to triangle ( $r\to 0.8$ ). Phase 4: all three collapse toward the centre ( $r\to 0.1$ ). Transitions are smoothed over 5-day ramps.

Figure 14: Split-and-merge curriculum experiment ( $K=3$ , $L=10$ ). (a) Daily component means, coloured by day, with phase-boundary markers (red diamonds) at days 1, 31, 51, and 81. The four phases — normal triangle, merge, split, collapse — are clearly visible. (b) Age-averaged forgetting curve. Despite the nontrivial curriculum, the retention half-life remains $a_{1/2}=30$ , identical to the stationary baseline.

Figure 15: PCA centroids at $d=12$ : the global mean (left) and per-class centroids for digits 0, 3, and 8. Despite capturing only $57\%$ of total variance, the centroids are visually recognisable.

Figure 16: MNIST age–forgetting curve (

L=10

d=12

K=3

). The half-life

a_{1/2}=37

and the curve shape is a clean sigmoid without the confusion overshoot seen in the synthetic mean-drift experiments.

Figure 17: Comparison of synthetic

K=3

(blue,

d=2

, circle drift,

P=30

) and MNIST (red,

d=12

, rotating-dominance weights,

P=30

) age–forgetting curves at

L=10

. The MNIST curve is shifted rightward due to the higher dimension and milder daily perturbation.

Figure 18: Decomposed raw forgetting for MNIST. Unlike the synthetic experiments where $F_{\rm mean}$ dominates, the MNIST construction is covariance-dominated because component means are fixed and only the weights rotate. The periodic oscillations at large age have period $P=30$ , matching the curriculum: dips occur at ages that are multiples of $P$ (where the recalled and current days share the same dominant digit class), while peaks occur at half-period offsets.

Figure 19: Per-component replayed means decoded to pixel space, for selected past days. Each row corresponds to one digit class (0, 3, 8 from top to bottom). Recent days show distinct digit identities; older days progressively converge toward a common blurred average.

Figure 20: Component movie: per-class decoded means from $t=0$ (oldest memories, left) to $t=1$ (current day, right). Each row is one digit class. Digit identities are preserved across the full protocol, with only gradual loss of sharpness at the oldest end.

Figure 21: Component weights $\pi_{k}(t)$ across the protocol grid. The weight trajectory encodes a compressed history of the 100-day rotating-dominance curriculum: recent dominance of digit 8 (green at $t=1$ ), earlier dominance of digit 0 (blue peak at $t\approx 0.4$ ), and convergence toward uniform weights at the oldest end.

Figure 22: MNIST PCA dimension sweep. (a) Age–forgetting curves for

d\in\{4,8,12,20,30\}

nearly coincide. (b) Half-life is flat at

a_{1/2}\approx 37

across all dimensions.

Memory Compression

See all 24 papers

Aug 12, 2026

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting. Yet its generalization behavior is shaped by two coupled effects that existing analyses fold into a single hypothesis-level quantity: finite memory replaces each past distribution with an empirical proxy, and repeated reuse couples the buffer, the current data, and the final hypothesis through a shared optimization trajectory. We develop a layer-wise information-theoretic framework that separates these effects at every depth. Our main result decomposes the expected generalization gap into a replay-induced representation drift and an optimization-dependence term, the latter further resolved into stability, plasticity, interaction, and residual-coupling components. Two refinements make the framework operational. A Wasserstein relaxation of the drift term, valid under support mismatch, yields a depth-dependent drift--sensitivity trade-off whose minimizer identifies which interior layer to stabilize. An SGLD instantiation of the optimization term reduces it to a trajectory-level log-determinant budget, exposing a curvature-aware gradient-alignment statistic that serves as an online diagnostic of task-wise forgetting. Controlled and benchmark experiments confirm the predicted memory scaling, the interior funnel, and the alignment signal's link to forgetting.

Tieliang Gong, Zhongbo Zhang, Wen Wen +1

Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China

Information-Theoretic Generalization Bounds Replay-Based Continual Learning Representation Drift Catastrophic Forgetting Generalization Bounds

Ask PDF

Mar 2, 2026

Position: Modular Memory is the Key to Continual Learning Agents

Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these models remain fundamentally limited in continuous operation, experience accumulation, and personalization, capabilities that are central to adaptive intelligence. While continual learning research has long targeted these goals, its historical focus on in-weight learning (IWL), i.e., updating a single model's parameters to absorb new knowledge, has rendered catastrophic forgetting a persistent challenge. Our position is that combining the strengths of In-Weight Learning (IWL) and the newly emerged capabilities of In-Context Learning (ICL) through the design of modular memory is the missing piece for continual adaptation at scale. We outline a conceptual framework for modular memory-centric architectures that leverage ICL for rapid adaptation and knowledge accumulation, and IWL for stable updates to model capabilities, charting a practical roadmap toward continually learning agents.

Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21

Toyota Motor Europe · University of Bremen · University of Florence +17

Continual Learning for LLM Agents Memory-Augmented Agents

Ask PDF

Apr 2, 2026

Adaptive Memory Crystallization for Autonomous AI Agent Learning in Dynamic Environments

Autonomous AI agents operating in dynamic environments face a persistent challenge: acquiring new capabilities without erasing prior knowledge. We present Adaptive Memory Crystallization (AMC), a memory architecture for progressive experience consolidation in continual reinforcement learning. AMC is conceptually inspired by the qualitative structure of synaptic tagging and capture (STC) theory, the idea that memories transition through discrete stability phases, but makes no claim to model the underlying molecular or synaptic mechanisms. AMC models memory as a continuous crystallization process in which experiences migrate from plastic to stable states according to a multi-objective utility signal. The framework introduces a three-phase memory hierarchy (Liquid--Glass--Crystal) governed by an Itô stochastic differential equation (SDE) whose population-level behavior is captured by an explicit Fokker--Planck equation admitting a closed-form Beta stationary distribution. We provide proofs of: (i) well-posedness and global convergence of the crystallization SDE to a unique Beta stationary distribution; (ii) exponential convergence of individual crystallization states to their fixed points, with explicit rates and variance bounds; and (iii) end-to-end Q-learning error bounds and matching memory-capacity lower bounds that link SDE parameters directly to agent performance. Empirical evaluation on Meta-World MT50, Atari 20-game sequential learning, and MuJoCo continual locomotion consistently shows improvements in forward transfer (+34--43% over the strongest baseline), reductions in catastrophic forgetting (67--80%), and a 62% decrease in memory footprint.

Rajat Khanda, Mohammad Baqar, Sambuddha Chakrabarti +1

Supermicro San Jose, USA · Cisco Systems San Jose, USA · Princeton University New Jersey, USA +1

Continual Learning Memory-Augmented Agents Continual RL Catastrophic Forgetting

Ask PDF

Abstract

Memory Compression

Explore similar work

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

Position: Modular Memory is the Key to Continual Learning Agents

Adaptive Memory Crystallization for Autonomous AI Agent Learning in Dynamic Environments