The Geometry of Forgetting

Sambartha Ray Barman; Andrey Starenky; Sophia Bodnar; Nikhil Narasimhan; Ashwin Gopinath

q-bio.NCMar 27, 2026

The Geometry of Forgetting

Authors: Sambartha Ray Barman, Andrey Starenky, Sophia Bodnar, Nikhil Narasimhan, Ashwin Gopinath

Organizations: Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA

Abstract

Why do we forget? Why do we remember things that never happened? The conventional answer points to biological hardware. We propose a different one: geometry. Here we show that high-dimensional embedding spaces, subjected to noise, interference, and temporal degradation, reproduce quantitative signatures of human memory with no phenomenon-specific engineering. Power-law forgetting ( $b = 0.460 \pm 0.183$ , human $b \approx 0.5$ ) arises from interference among competing memories, not from decay. The identical decay function without competitors yields $b \approx 0.009$ , fifty times smaller. Time alone does not produce forgetting in this system. Competition does. Production embedding models (nominally 384--1{,}024 dimensions) concentrate their variance in only ${\sim}16$ effective dimensions, placing them deep in the interference-vulnerable regime. False memories require no engineering at all: cosine similarity on unmodified pre-trained embeddings reproduces the Deese--Roediger--McDermott false alarm rate ( $0.583$ versus human ${\sim}0.55$ ) with zero parameter tuning and no boundary conditions. We did not build a false memory system. We found one already present in the raw geometry of semantic space. These results suggest that core memory phenomena are not bugs of biological implementation but features of any system that organizes information by meaning and retrieves it by proximity.

Figures & tables

Figure 1: Interference from competing memories produces human-like forgetting in low-dimensional embedding spaces. a , Forgetting curves steepen progressively as competing memories are added (light to dark blue), converging toward the classical human forgetting curve (red dashed; Ebbinghaus, 1885). All curves at

d=64

with identical decay function. b , The fitted forgetting exponent

b

increases monotonically with competitor count at

d=64

(blue) but remains near zero at

d=128

(grey). Human

b\approx 0.5

(red dashed). Shaded regions: bootstrap 95% CI. c , Maximum forgetting exponent as a function of effective dimensionality. Interference is substantial only below

d\approx 100

, placing biological neural codes (estimated

d=100

–

500

22 , 23 ; shaded) near the transition zone. Orange diamond: MiniLM (

d_{\text{nom}}=384

d_{\text{eff}}\approx 16

) shows strong interference consistent with its low effective dimensionality.

n=5

seeds throughout.

Figure 2: False memories arise from semantic clustering in the embedding space. a , UMAP projection of the SLEEP word list shows the critical lure ( sleep , red star) falling within the cluster of studied words (blue dots), while unrelated words (grey) remain separated. b , At

\theta=0.82

, the critical-lure false alarm rate (

0.583

) closely matches the human value (

{\sim}0.55

, red diamonds). c , Threshold operating curves show the lure false alarm rate remains elevated above unrelated across a wide range of thresholds. d , Per-list lure cosine similarity is consistently high across all 24 DRM word lists.

n=5

seeds for panels b – d .

Figure 3: Spaced repetition survives age-dependent degradation through recency of the most recent trace. a , Timeline showing repetition schedules: long-spaced items retain a recent trace at test (14 days old), while massed items have only old traces (30 days). Retention values shown at right. b , Retention increases monotonically with spacing interval (blue), matching the human ordering (red diamonds): long

>

medium

>

short

>

massed. c , The spacing gradient emerges as noise increases: at

\sigma=0

, all conditions achieve ceiling; as noise grows, conditions separate, with massed dropping fastest. Selected

\sigma=0.25

(dashed line). Error bars: bootstrap 95% CI,

n=5

seeds.

Figure 4: Exploratory: topological structure of the memory manifold shows a connectivity transition with transient loop structure. a , Connected components ( $H_{0}$ , blue) exhibit a sharp $1{,}000\to 1$ transition while loops ( $H_{1}$ , red) peak at $534\pm 24$ at $\epsilon=1.0$ (shaded: phase transition zone). b , Persistence diagram showing long-lived topological features (far from diagonal) versus noise (near diagonal). c , UMAP visualizations at three edge lengths: disconnected clusters ( $\epsilon=0.7$ ), critical connectivity with loops ( $\epsilon=1.0$ ), and fully connected ( $\epsilon=1.5$ ). Points colored by topic. Error bands: 95% CI, $n=5$ seeds.

Figure 5: Lightweight projections align independent encoders for cross-modal retrieval. a , Cross-modal similarity matrix shows strong diagonal structure: image queries retrieve their matched text descriptions with high cosine similarity (dark diagonal), while unrelated pairs show low similarity. b , Recall at $k\in\{1,5,10\}$ for image-to-text and text-to-image retrieval, exceeding random baseline by two orders of magnitude. c , Cross-dataset transfer (COCO $\to$ Flickr30k) shows minimal degradation. Error bars: 95% CI, $n=5$ seeds.

Figure 6: Qualitative comparison to canonical human benchmarks. For each phenomenon, the observed value (blue dot, with 95% CI) is compared to the corresponding human reference (red diamond). Short connecting lines indicate close matches; longer lines indicate discrepancies. The forgetting exponent and DRM false alarm rate show the closest correspondence; other phenomena show qualitative but not tight quantitative agreement.

Figure 7: Extended Data Fig. 1: System architecture. Conceptual overview of the embedding-based memory framework. Input modalities are encoded by frozen pre-trained models, projected into a shared space, and retrieved via cosine similarity with temporal decay modulation.

Figure 8: Extended Data Fig. 2: Memory-dependent reasoning on bAbI. a , Accuracy by task for contextual retrieval versus baselines. b , Memory scaling: precision declines with store size. c , Retrieval precision heatmap. Mean $\pm$ 95% CI, $n=5$ seeds.

Figure 9: Extended Data Fig. 3: Ebbinghaus parameter sweep. a , Fitted forgetting exponent $b$ across the $\sigma\times\beta$ parameter grid. Contour line at $b=0.5$ (human value). b , Goodness-of-fit $R^{2}$ for the same configurations.

Figure 10: Extended Data Fig. 4: Interference experiment full results. a , All forgetting curves at

d=64

(near condition). b , Curves at

d=128

remain flat. c , Near versus far distractor comparison. d , Raw retention by age bin.

n=5

seeds.

Figure 11: Extended Data Fig. 5: Consolidation as an informative negative result. a , Backward transfer worsens with consolidation despite b , substantial memory compression. c , The compression–accuracy trade-off: geometric merging achieves compression at the cost of retrieval accuracy.

Figure 12: Extended Data Fig. 6: Cross-modal retrieval detailed results. a , Recall curves from

k=1

k=10

for both retrieval directions. b , Transfer comparison. c , Comparison with random baseline.

n=5

seeds.

Figure 13: Extended Data Fig. 7: DRM per-list analysis. a , Lure cosine similarity across all 24 DRM word lists. b , Similarity distributions for studied words, critical lures, and unrelated words. c , Threshold operating curves for hit rate and false alarm rates.

n=5

seeds.

Figure 14: Extended Data Fig. 8: Spacing sweep full results. a , Retention by noise level at 25K distractors. b , Retention by distractor count at $\sigma=0.25$ . c , Effect size (Cohen’s $d$ , long versus massed) across the full $\sigma\times$ distractor grid. $n=5$ seeds.

Figure 15: Extended Data Fig. 9: Tip-of-tongue analysis. TOT rate (

3.66\pm 0.13\%

) compared to the human baseline (

{\sim}1.5\%

). The qualitative phenomenon (correct items known but not immediately retrieved) emerges from retrieval competition, though the rate exceeds the human value by a factor of

{\sim}2.4

, suggesting the geometric operationalisation (rank 2–20) is looser than the human phenomenological criterion.

n=5

seeds.

Figure 16: Extended Data Fig. 10: Reproducibility across seeds. a , Key metrics across all five random seeds, showing consistency of results. b , Bootstrap distribution of the forgetting exponent

b

with 95% confidence interval.

Figure 17: Extended Data Fig. 11: Effective dimensionality resolves the dimensionality paradox. a , Eigenvalue spectra of three embedding models show rapid drop-off; vertical dashed lines mark the participation ratio (

d_{\text{eff}}

). b , Cumulative explained variance: only 17–18 components account for 95% of variance regardless of nominal dimensionality (384, 768, or 1,024). c , Maximum interference exponent

b

as a function of effective dimensionality. MiniLM at native

d=384

(

d_{\text{eff}}\approx 16

, orange diamond) shows stronger interference than BGE-large at PCA

d=64

, consistent with its low effective dimensionality. d , Direct comparison: MiniLM

d=384

without PCA shows catastrophic retrieval failure with near distractors, exceeding BGE-large PCA

d=64

interference.

n=5

seeds; error bars: bootstrap 95% CI.

Phase	Parameter	Value	Notes
1	Embedding model	MiniLM-L6-v2	384-dim
	Positional encoding dim	384	Sinusoidal
	ContextProjector	$768\to 384$	Linear + LayerNorm
	InfoNCE temperature	0.07	Fixed
	Learning rate	$1\times 10^{-3}$	AdamW
	Weight decay	$1\times 10^{-4}$	—

Table 1: Hyperparameters across all phases. All values stored in YAML configuration files.

Dataset	Phase	Split	Size	Licence
bAbI QA (en-10k)	1	Train / Test	10K / 1K per task	BSD
TempLAMA	2	Full	Variable	MIT
CIFAR-100	3	Train / Test	50K / 10K	BSD
COCO Captions 2017	4	Train / Val	118K / 5K	CC BY 4.0
Flickr30k	4	Test	1K	Research
DRM word lists	5	—	24 lists $\times$ 16 words	Public domain

Table 2: Datasets used across all phases. All publicly available with permissive licences.

Method	Task 1	Task 2	Task 3	Task 4	Task 5
Contextual retrieval	$0.597\pm 0.009$	$0.341\pm 0.014$	$0.281\pm 0.012$	$0.365\pm 0.006$	$0.793\pm 0.008$
No memory	$0.000$	$0.000$	$0.006$	$0.011$	$0.000$
Full context	$0.773$	$0.487$	$0.267$	$0.305$	$0.840$
Random retrieval	$0.531\pm 0.008$	$0.317\pm 0.011$	$0.229\pm 0.013$	$0.360\pm 0.006$	$0.620\pm 0.018$
Overall	Contextual: $0.475\pm 0.007$ ; No memory: $0.003$ ; Random: $0.411\pm 0.004$

Table 3: bAbI accuracy (mean

\pm

std, 5 seeds) across tasks and methods.

Dim	Cond.	Distractors	$b$	95% CI	Retention
64	near	0	0.009	[0.006, 0.012]	0.990
	near	1,000	0.038	[0.023, 0.062]	0.961
	near	4,000	0.073	[0.057, 0.095]	0.919
	near	10,000	0.105	[0.087, 0.124]	0.878
	near	40,000	0.161	[0.126, 0.200]	0.814
	far	10,000	0.078	[0.054, 0.116]	0.917

Table 4: Fitted power-law exponent

b

and retention by distractor count, condition, and dimensionality (mean, 95% CI, 5 seeds). MiniLM rows show native 384-dim without PCA projection, confirming that effective dimensionality, not nominal dimensionality, governs interference.

Model	$d_{\text{nom}}$	$d_{\text{eff}}$	$d_{95}$	$d_{99}$	$d_{\text{eff}}/d_{\text{nom}}$
MiniLM-L6-v2	384	$15.7\pm 0.0$	$17.0\pm 0.0$	$19.0\pm 0.0$	0.041
BGE-base-en-v1.5	768	$16.6\pm 0.1$	$18.0\pm 0.0$	$19.0\pm 0.0$	0.022
BGE-large-en-v1.5	1,024	$16.3\pm 0.1$	$17.6\pm 0.5$	$19.0\pm 0.0$	0.016

Table 5: Effective dimensionality analysis. Despite 25

\times

–65

\times

differences in nominal dimensionality, all models concentrate variance in

{\sim}16

effective dimensions.

N=10{,}000

Wikipedia sentences per seed, 5 seeds.

Condition	Backward Transfer	Compression	Notes
No consolidation	$-0.100\pm 0.003$	$1.000$	Baseline
Gentle consolidation	$-0.394\pm 0.034$	$0.375\pm 0.036$	Age-based
Full (gentle + replay)	$-0.398\pm 0.030$	$0.432\pm 0.056$	+ replay

Table 6: Consolidation conditions on CIFAR-100 (mean

\pm

std, 5 seeds).

	Image $\to$ Text			Text $\to$ Image
Method	R@1	R@5	R@10	R@1	R@5	R@10
Contextual	$0.203$	$0.470$	$0.602$	$0.231$	$0.492$	$0.620$
Random	$0.001$	$0.007$	$0.018$	$0.002$	$0.012$	$0.021$

Table 7: Cross-modal retrieval on Flickr30k (mean

\pm

std, 5 seeds).

Phenomenon	Observed	Human	Notes
DRM FA (critical lure)	$0.583$	${\sim}0.55$	Unbaked
Spacing: long	$0.994\pm 0.008$	$0.65$	$\sigma=0.25$ , 25K
Spacing: massed	$0.230\pm 0.073$	$0.30$	$\sigma=0.25$
TOT rate	$3.66\pm 0.13\%$	$1.5\%$	PCA 128-dim
$H_{1}$ peak	$534\pm 24$	—	At $\epsilon=1.0$

Table 8: Emergent phenomena summary (mean

\pm

std, 5 seeds).

Phase	GPUs	Operations	Time
1	0, 1	Embedding + generation	${\sim}2$ h
2	0, 1	Temporal + Ebbinghaus	${\sim}1$ h
2 (interference)	1	Distractor sweep	${\sim}3$ h
3	1, 2	CLIP + HDBSCAN	${\sim}3$ h
4	1, 2	Projection + retrieval	${\sim}2$ h
5	1–3	DRM + spacing + topology	${\sim}2$ h

Table 9: Computational resources. All experiments on 4

\times

NVIDIA A100 GPUs.

List	Studied sim	Lure sim	Unrelated sim	FA rate
SLEEP	1.000	0.821	0.672	0.583
NEEDLE	1.000	0.830	0.699	0.583
ROUGH	1.000	0.819	0.693	0.583
SWEET	1.000	0.835	0.700	0.583
(Full table with all 24 lists in Extended Data Fig. 13 )

Table 10: Per-list DRM cosine similarities and false alarm rates at

\theta=0.82

(mean across 5 seeds).

Associative Memory

Associative memory studies how systems store patterns or associations and retrieve them from partial, noisy, or context-dependent cues.

See all 120 papers

The Price of Meaning: Why Every Semantic Memory System Forgets
Mar 28, 2026Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar +2Semantic Search Associative Memory
Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA
When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting
Oct 6, 2026Vedant Palit, Florent Draye, Nicolas Zucchet +2Memorization in Language Models LLM Fine-Tuning
MPI for Intelligent Systems, Tübingen · Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI +3
The Geometry of Agent Memory: Forecasting Retrieval-Bound Memory Failures, and Characterizing Its Limits
Oct 19, 2026Iliya Zhechev, Radoslav Ralev, Chris Coleman +4LLM Memory Conversational Memory
Redis

Mar 28, 2026J/K move · Enter open · S save

The Price of Meaning: Why Every Semantic Memory System Forgets

Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan+1

Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA

Semantic Search Associative Memory Catastrophic Forgetting

Open paper AskPDF arXiv

Every major AI memory system in production today organises information by meaning. That organisation enables generalisation, analogy, and conceptual retrieval -- but it comes at a price. We prove that the same geometric structure enabling semantic generalisation makes interference, forgetting, and false recall inescapable. We formalise this tradeoff for semantically continuous kernel-threshold memories: systems whose retrieval score is a monotone function of an inner product in a semantic feature space with finite local intrinsic dimension. Within this class we derive four results: (1) semantically useful representations have finite effective rank; (2) finite local dimension implies positive competitor mass in retrieval neighbourhoods; (3) under growing memory, retention decays to zero, yielding power-law forgetting curves under power-law arrival statistics; (4) for associative lures satisfying a

δ

-convexity condition, false recall cannot be eliminated by threshold tuning. We test these predictions across five architectures: vector retrieval, graph memory, attention-based context, BM25 filesystem retrieval, and parametric memory. Pure semantic systems express the vulnerability directly as forgetting and false recall. Reasoning-augmented systems partially override these symptoms but convert graceful degradation into catastrophic failure. Systems that escape interference entirely do so by sacrificing semantic generalisation. The price of meaning is interference, and no architecture we tested avoids paying it.

Abstract

Associative Memory

Explore similar work

The Price of Meaning: Why Every Semantic Memory System Forgets

When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting

The Geometry of Agent Memory: Forecasting Retrieval-Bound Memory Failures, and Characterizing Its Limits