The Price of Meaning: Why Every Semantic Memory System Forgets
Authors: Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan, Ashwin Gopinath
Organizations: Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA
Every major AI memory system in production today organises information by meaning. That organisation enables generalisation, analogy, and conceptual retrieval -- but it comes at a price. We prove that the same geometric structure enabling semantic generalisation makes interference, forgetting, and false recall inescapable. We formalise this tradeoff for semantically continuous kernel-threshold memories: systems whose retrieval score is a monotone function of an inner product in a semantic feature space with finite local intrinsic dimension. Within this class we derive four results: (1) semantically useful representations have finite effective rank; (2) finite local dimension implies positive competitor mass in retrieval neighbourhoods; (3) under growing memory, retention decays to zero, yielding power-law forgetting curves under power-law arrival statistics; (4) for associative lures satisfying a δ-convexity condition, false recall cannot be eliminated by threshold tuning. We test these predictions across five architectures: vector retrieval, graph memory, attention-based context, BM25 filesystem retrieval, and parametric memory. Pure semantic systems express the vulnerability directly as forgetting and false recall. Reasoning-augmented systems partially override these symptoms but convert graceful degradation into catastrophic failure. Systems that escape interference entirely do so by sacrificing semantic generalisation. The price of meaning is interference, and no architecture we tested avoids paying it.
Figures & tables
Figure 1: The No-Escape Theorem: logical structure (paper roadmap). This figure maps the paper’s argument. Under the kernel-threshold theorem class (Axioms A1–A5): the semantic kernel and rate-distortion optimality yield finite semantic effective rank (Theorem 1); local regularity yields positive cap mass (Theorem 2); growing memory yields inevitable forgetting (Theorem 3), with power-law arrival and population heterogeneity producing power-law forgetting curves. Independently, associative δ -convexity yields lure inseparability (Theorem 4). No architecture within this class avoids these consequences without abandoning semantic continuity or adding an external symbolic verifier. Each arrow represents a step derived under stated assumptions and supported by empirical tests across the architectures studied here.
Figure 2: Interference produces forgetting across architecturally distinct memory systems. a , Vector DB and b , Graph show smooth power-law forgetting curves converging toward the human range ( b≈0.3 – 0.7 , red dashed). c , Attention shows a phase transition (logistic fit: n0≈120 , k≈0.03 ; power-law fitting is inappropriate for this sigmoid failure mode). d , Filesystem (BM25) shows b=0 (no semantic interference). e , Parametric (PopQA) shows monotonic accuracy decline with neighbour density. Category 1 systems degrade continuously; Category 2 systems fail discontinuously. n=5 seeds throughout.
Figure 3: The forgetting exponent depends on competitor count, not architecture. Forgetting exponent b vs. number of near competitors for embedding architectures (Vector DB, Graph) with human reference ( b≈0.5 , dashed). Both converge toward the human range at high competitor counts. Shaded: bootstrap 95% CI, n=5 seeds.
Figure 4: False recall is geometrically inevitable. a , Hit rate, lure false alarm rate, and unrelated FA for all five architectures and human data. Embedding architectures show elevated lure FA; LLM architectures show FA =0 at behavioural level (explicit list-checking). b , Lure FA rates compared directly. The geometric prediction ( 24/24 lures within spherical caps) holds for all architectures regardless of behavioural output. n=5 seeds, 24 DRM lists.
Figure 5: Effective dimensionality converges far below nominal regardless of architecture. deff (participation ratio) vs. dnom for all five architectures. Grey: biological range ( deff=100 – 500 19 , 7 ). Qwen hidden states ( dnom=3,584 ) compress to deff=17.9 , a 200× reduction. All architectures cluster below the interference threshold.
Figure 6: No proposed solution achieves both immunity and usefulness. Every solution that reduces interference moves along a tradeoff frontier toward reduced usefulness; no solution escapes the frontier itself. This is the empirical corollary to Theorem 1. a , Zero-padding does not reduce b ( deff unchanged). b , BM25 eliminates false recall but semantic agreement drops to 15.5% . c , Gram–Schmidt eliminates interference; semantic accuracy =0% . d , Compression reduces b but degrades retrieval.
Figure 7: Architecture comparison across four memory phenomena. Heatmap of forgetting exponent b , DRM lure FA, spacing Cohen’s d , and TOT rate for all five architectures and human reference. The three prototypical behavioural categories are visible: pure geometric (top two rows), reasoning overlay (middle), SPP-violating (bottom). Dashes indicate metrics not measurable for that architecture (attention b : sigmoid, not power-law; parametric spacing: no controlled paradigm). † Parametric TOT ( 69% ) uses a different operational definition than human/embedding TOT and is not directly comparable (see Discussion). Metrics are architecture-specific and not all directly numerically comparable (see Methods for protocol differences).
Parameter
Value
Description
Seeds
[42,123,456,789,1024]
Random seeds
Bootstrap
10,000
Resamples for 95% CI
Decay β
0.20
Temporal decay rate (calibrated)
Decay ψ
0.50
Temporal decay exponent
Noise σ (Ebb.)
0.50
Ebbinghaus query noise
Noise σ (Sp.)
0.25
Spacing noise
Table 1: Hyperparameters for all architectures and experiments.
Table 4: Effective dimensionality per architecture.
Solution
Configuration
b
Accuracy
1: High dim
PCA d=64
0.370
reduced
Original d=1,024
0.831
baseline
Zero-pad d=2,048
0.332
baseline
Zero-pad d=4,096
0.308
baseline
2: BM25
Full BM25
0.000
15.5%
3: Gram–Schmidt
500 vectors
0.000
0.0%
Table 5: Solution analysis data points.
Figure 8: Extended Data Fig. 1: The five memory architectures. Each architecture implements a fundamentally different storage and retrieval mechanism: cosine similarity (Vector DB), attention over context (Attention), BM25 + LLM re-ranking (Filesystem), personalised PageRank (Graph), and parametric knowledge in weights (Parametric). Despite architectural diversity, all except Filesystem strongly satisfy SPP ( p<0.001 ). n=143 sentence pairs per architecture.
Figure 9: Extended Data Fig. 2: Vector Database full results. a , Forgetting exponent b at each competitor count, showing monotonic increase. b , DRM hit rate, lure FA, and unrelated FA. c , Spacing retention: long =0.902 , massed =0.360 . d , Eigenvalue spectrum ( deff=158 ). Error bars: bootstrap 95% CI, n=5 seeds.
Figure 10: Extended Data Fig. 3: Graph Memory full results. a , b=0.478 at 10,000 competitors. b , DRM lure FA =0.208 at θ=0.82 . c , Spacing: long =0.996 , massed =0.920 . d , Eigenvalue spectrum ( deff=127 ). n=5 seeds.
Figure 11: Extended Data Fig. 4: Attention Memory full results. a , Phase transition: near-perfect accuracy at nnear<100 , then catastrophic collapse (logistic fit: n0≈120 , k≈0.03 ; power-law fitting is inappropriate for this sigmoid failure mode; y -axis values reflect interference severity, not power-law exponents). b , DRM FA =0 at behavioural level (geometric prediction holds: 24/24 lures within caps). c , Spacing: architectural capacity artefact: massed =1.0 , spaced =0.0 (context-window limit relocates interference to capacity domain). d , deff=17.9 from dnom=3,584 , a 200× compression. n=5 seeds.
Figure 12: Extended Data Fig. 5: Parametric Memory full results. PopQA interference: accuracy drops from 1.000 ( <50 neighbours) to 0.113 ( >1,000 neighbours). Power-law fit b=0.215 . DRM FA =0 behaviourally. TOT =69% , a very high partial retrieval rate. deff=17.9 . n=3 seeds for PopQA.
Figure 13: Extended Data Fig. 6: Filesystem Memory full results. BM25 keyword retrieval: b=0.000 (no forgetting), FA =0 (no false recall), all spacing conditions =1.0 . SPP correlation r=0.210 ; BM25 weakly satisfies semantic proximity. This architecture demonstrates Solution 2: immunity at the cost of usefulness.
Figure 14: Extended Data Fig. 7: SPP verification. Mean similarity for related pairs (same article) vs. unrelated pairs (different articles) across all five architectures. All satisfy SPP ( p<0.001 ), with embedding architectures showing stronger separation. n=143 pairs.
Figure 15: Extended Data Fig. 8: Spherical cap verification. a , Analytical cap volume (fraction of sphere) vs. dimension for five cap half-angles θ∈{10∘,20∘,30∘,45∘,60∘} , showing exponential collapse with increasing d . Shaded region marks the interference regime ( deff≈10 – 50 ) where all tested architectures operate. b , Monte Carlo verification: simulated vs. analytical cap volume on log–log axes for the 7 ( d , θ ) combinations where the Monte Carlo sample detected non-zero signal. Six of seven points fall within ±20% of the y=x line; the single outlier ( d=8 , θ=20∘ , ratio =0.48 ) reflects finite-sample resolution at analytical volume ≈8×10−5 , not analytical error. Marker shape encodes dimension; colour encodes θ (matching a ). Confirms Theorem 2 .
Figure 16: Extended Data Fig. 9: Reproducibility across seeds. Per-seed values of b , DRM lure FA, and deff for Vector DB and Graph architectures. Low variance confirms reproducibility. n=5 seeds.
Figure 17: Extended Data Fig. 10: Full solution analysis. a , b vs. nominal dimensionality. b , BM25 immunity vs. usefulness. c , Orthogonalisation methods. d , Compression: b (blue) and accuracy (red) vs. cluster count. Every solution traces a tradeoff; none achieves both immunity and usefulness.
Why do we forget? Why do we remember things that never happened? The conventional answer points to biological hardware. We propose a different one: geometry. Here we show that high-dimensional embedding spaces, subjected to noise, interference, and temporal degradation, reproduce quantitative signatures of human memory with no phenomenon-specific engineering. Power-law forgetting (b=0.460±0.183, human b≈0.5) arises from interference among competing memories, not from decay. The identical decay function without competitors yields b≈0.009, fifty times smaller. Time alone does not produce forgetting in this system. Competition does. Production embedding models (nominally 384--1{,}024 dimensions) concentrate their variance in only ∼16 effective dimensions, placing them deep in the interference-vulnerable regime. False memories require no engineering at all: cosine similarity on unmodified pre-trained embeddings reproduces the Deese--Roediger--McDermott false alarm rate (0.583 versus human ∼0.55) with zero parameter tuning and no boundary conditions. We did not build a false memory system. We found one already present in the raw geometry of semantic space. These results suggest that core memory phenomena are not bugs of biological implementation but features of any system that organizes information by meaning and retrieves it by proximity.
Sambartha Ray Barman, Andrey Starenky, Sophia Bodnar +2
Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA
Current agentic memory systems (vector stores, retrieval-augmented generation, scratchpads, and context-window management) do not implement memory: they implement lookup. We argue that treating lookup as memory is a category error with provable consequences for agent capability, long-term learning, and security. Retrieval generalizes by similarity to stored cases; weight-based memory generalizes by applying abstract rules to inputs never seen before. Conflating the two produces agents that accumulate notes indefinitely without developing expertise, face a provable generalization ceiling on compositionally novel tasks that no increase in context size or retrieval quality can overcome, and are structurally vulnerable to persistent memory poisoning as injected content propagates across all future sessions. Drawing on Complementary Learning Systems theory from neuroscience, we show that biological intelligence solved this problem by pairing fast hippocampal exemplar storage with slow neocortical weight consolidation, and that current AI agents implement only the first half. We formalize these limitations, address four alternative views, and close with a co-existence proposal and a call to action for system builders, benchmark designers, and the memory community.
Binyan Xu, Xilin Dai, Kehuan Zhang
The Chinese University of Hong Kong, Hong Kong, China · Zhejiang University, Hangzhou, China
We present SCM (Sleep-Consolidated Memory), a research preview of a memory architecture for large language models that draws on neuroscientific principles to address a fundamental limitation in current systems: the absence of persistent, structured, and biologically plausible memory. Existing approaches rely on truncating context windows, growing vector databases without bound, or tiered storage systems that lack consolidation and forgetting mechanisms. SCM implements five core components inspired by human memory: a limited-capacity working memory, multi-dimensional importance tagging, offline sleep-stage consolidation with distinct NREM and REM phases, intentional value-based forgetting, and a computational self-model enabling introspection. Across a standardized benchmark suite of eight tests, the prototype achieves perfect recall accuracy over ten-turn conversations while reducing memory noise by 90.9% through adaptive forgetting. Memory search latency remains below one millisecond even with hundreds of stored concepts. This work establishes the architectural foundations for memory systems that consolidate, prioritize, and forget, offering a testable platform for advancing LLM memory research.