cs.LGMar 31, 2026
SaveTemporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth
Organizations: Graduate Inter-Disciplinary Program in Applied Mathematics, & Department of Mathematics, University Arizona, Tucson, AZ 85721
Abstract
An agent that operates sequentially must incorporate new experience without forgetting old experience, under a fixed memory budget. We propose a framework in which memory is not a parameter vector but a stochastic process: a Bridge Diffusion on a replay interval , whose terminal marginal encodes the present and whose intermediate marginals encode the past. New experience is incorporated via a three-step Compress--Add--Smooth (CAS) recursion. We test the framework on the class of models with marginal probability densities modeled via Gaussian mixtures of fixed number of components~ in dimensions; temporal complexity is controlled by a fixed number~ of piecewise-linear protocol segments whose nodes store Gaussian-mixture states. The entire recursion costs flops per day -- no backpropagation, no stored data, no neural networks -- making it viable for controller-light hardware. Forgetting in this framework arises not from parameter interference but from lossy temporal compression: the re-approximation of a finer protocol by a coarser one under a fixed segment budget. We find that the retention half-life scales linearly as with a constant that depends on the dynamics but not on the mixture complexity~, the dimension~, or the geometry of the target family. The constant~ admits an information-theoretic interpretation analogous to the Shannon channel capacity. The stochastic process underlying the bridge provides temporally coherent movie'' replay -- compressed narratives of the agent's history, demonstrated visually on an MNIST latent-space illustration. The framework provides a fully analytical Ising model'' of continual learning in which the mechanism, rate, and form of forgetting can be studied with mathematical precision.
Figures & tables
Figure 1: One iteration of the compress–add–smooth recursion, illustrated for segments. Top: the protocol at day consists of uniform segments on . Middle: compression rescales the protocol to ; the new day is appended on , producing uniform segments. Bottom: rebinning averages the segments back onto the -segment grid. The right-hand labels indicate the information-theoretic role of each step: only smoothing is lossy. Dashed lines track a past-day readout time , which contracts by factor every day.
Figure 2: Single-Gaussian experiment ( , , ) under the default circular-drift setting. (a) Age-averaged normalized forgetting , showing retention half-life . The curve exhibits a low-error plateau for ages – , followed by a steep sigmoid transition crossing at age 30, a slight overshoot to around age 50 (the confusion regime), and eventual saturation near . The curve is weakly non-monotone due to the periodic geometry of the circular drift, which causes geometric recurrence at multiples of the half-period. (b) Full forgetting matrix as a function of recalled day and current day . The dominant trend is age-controlled forgetting (iso-forgetting contours run parallel to the diagonal), modulated by the periodic geometry of the underlying drift.
Figure 3: Default single-Gaussian circular-drift experiment ( ). (a) Original daily means (dots, coloured by day) and replayed means at the final day (crosses). The black star marks the prior mean (the origin), which serves as the protocol’s long-time attractor. Confusion is visible as the systematic inward displacement of crosses from the circle toward the star: recent memories (warm colours) are replayed near their true locations on the circle, while older memories (cool colours) are pulled progressively toward the prior mean. This convergence of replay means toward the star — rather than remaining on the circle — is the geometric signature of confusion. (b) Readout times versus current day , showing the geometric decay ( 9 ).
Figure 4: Selected replay ellipses in the default single-Gaussian circular-drift experiment ( ). For each displayed day, the original target mean is shown by a dot, while the replayed mean is shown by a cross with a dashed ellipse representing the replay covariance. Recent memories (e.g. day 95) are replayed with small displacement and compact ellipses. Intermediate-age memories (e.g. day 35) show large displacement toward the origin and inflated covariances. Very old memories (e.g. day 25) collapse nearly to the origin with very large ellipses. Geometric aliasing is visible for day 5, whose true location on the circle lies close to day 95 (they differ by two full periods), producing an apparently accurate replay that is coincidental rather than genuine recall.
Figure 5: Segment-budget sweep for the single-Gaussian circular-drift experiment. (a) Age–forgetting curves for . Increasing shifts the sigmoid transition to higher ages without changing the curve shape qualitatively. (b) Retention half-life versus , confirming approximate linear scaling.
Figure 6: (a) Drift-speed sweep (circle period ). Faster drift shortens the half-life, but the effect saturates for slow drift. (b) Circle vs. linear drift. The circular curve is non-monotone due to periodic spatial recurrence; the linear curve is a clean sigmoid with longer half-life. (c) Half-life versus period .
Figure 7: (a) Age–forgetting curves for (blue) and (red), both with and circular drift. The curves nearly coincide, with both yielding . (b) Decomposed raw forgetting for : mean misalignment (blue) dominates, covariance error (orange) is secondary, and weight error (green) is negligible.
Figure 8: Component-level trajectories for . (a) Original daily component means, coloured by day. The three interleaved helical paths trace the rotating-triangle geometry. (b) Replayed component means ( ) at the final day, after Hungarian matching to the originals. Recent components are well-recalled; older ones collapse toward the origin.
Figure 9: (a) sweep at : age–forgetting curves for . The pattern mirrors the case. (b) sweep at : age curves for nearly coincide. (c) Half-life summary for both sweeps.
Figure 10: Crowding sweep for Gaussian-mixture memories ( ). (a) Retention half-life versus crowding ratio for . The half-life is flat at 30 for and declines only for well-separated components. (b) Age–forgetting curves at for six crowding values. (c) Mean-error share of total forgetting, stable at across all .
Figure 11: Representative age-averaged forgetting curves across three crowding regimes ( , ). Strong ( ) and medium ( ) crowding yield indistinguishable curves ( ). Weak crowding ( ) produces a slightly earlier transition ( ) and a more pronounced confusion overshoot.
Figure 12: Scaling with ambient dimension for a fixed 2D signal ( , ). (a) Age-forgetting curves for without nuisance drift. Higher slightly improves normalised retention due to the larger amnesia baseline. (b) Retention half-life versus ambient dimension under two nuisance settings. Both show gentle increase with . (c) Mean-error share decreases with as covariance mismatch grows in the extra dimensions.
Figure 13: Schematic of the four-phase split-and-merge curriculum for . Coloured dots represent the three mixture component means; the dashed circle indicates the inter-component radius . Phase 1: normal rotating triangle ( ). Phase 2: two components merge ( ). Phase 3: split back to triangle ( ). Phase 4: all three collapse toward the centre ( ). Transitions are smoothed over 5-day ramps.
Figure 14: Split-and-merge curriculum experiment ( , ). (a) Daily component means, coloured by day, with phase-boundary markers (red diamonds) at days 1, 31, 51, and 81. The four phases — normal triangle, merge, split, collapse — are clearly visible. (b) Age-averaged forgetting curve. Despite the nontrivial curriculum, the retention half-life remains , identical to the stationary baseline.
Figure 15: PCA centroids at : the global mean (left) and per-class centroids for digits 0, 3, and 8. Despite capturing only of total variance, the centroids are visually recognisable.
Figure 16: MNIST age–forgetting curve ( , , ). The half-life and the curve shape is a clean sigmoid without the confusion overshoot seen in the synthetic mean-drift experiments.
Figure 17: Comparison of synthetic (blue, , circle drift, ) and MNIST (red, , rotating-dominance weights, ) age–forgetting curves at . The MNIST curve is shifted rightward due to the higher dimension and milder daily perturbation.
Figure 18: Decomposed raw forgetting for MNIST. Unlike the synthetic experiments where dominates, the MNIST construction is covariance-dominated because component means are fixed and only the weights rotate. The periodic oscillations at large age have period , matching the curriculum: dips occur at ages that are multiples of (where the recalled and current days share the same dominant digit class), while peaks occur at half-period offsets.
Figure 19: Per-component replayed means decoded to pixel space, for selected past days. Each row corresponds to one digit class (0, 3, 8 from top to bottom). Recent days show distinct digit identities; older days progressively converge toward a common blurred average.
Figure 20: Component movie: per-class decoded means from (oldest memories, left) to (current day, right). Each row is one digit class. Digit identities are preserved across the full protocol, with only gradual loss of sharpness at the oldest end.
Figure 21: Component weights across the protocol grid. The weight trajectory encodes a compressed history of the 100-day rotating-dominance curriculum: recent dominance of digit 8 (green at ), earlier dominance of digit 0 (blue peak at ), and convergence toward uniform weights at the oldest end.
Figure 22: MNIST PCA dimension sweep. (a) Age–forgetting curves for nearly coincide. (b) Half-life is flat at across all dimensions.