Atoms, Not Knowledge: What Production-Scale Addressed-Memory Writes Deposit

Wei Gao; Cheng Tian; Duan Yang; Jianqing Yin

cs.CLOct 18, 2026

Atoms, Not Knowledge: What Production-Scale Addressed-Memory Writes Deposit

Authors: Wei Gao, Cheng Tian, Duan Yang, Jianqing Yin

Organizations: Shibei Qiyuan AI Technology · Xi’an Jiaotong University

Abstract

Deterministic hash-addressed conditional memory permits local post-hoc knowledge injection. Extending arXiv:2610.21553's bounded-feasibility audit rejecting under-specified generality claims with repaired probes and training-dose/capacity axes, we give, to our knowledge, the first production-scale write-content verdict. We test 124M-7B self-trained/publicly pretrained beds, Qwen3.8-Flash-Next-FP8 (176.95B; 320,001,536 FP8 rows), and DeepSeek-V4.1-Flash. Write coverage requires a sufficiently trained backbone prior, full-width readout, trainable gates; undertrained-bed intervention confirms a threshold, not a slope. At fixed budget, coverage falls from 98% (50 facts) to 12.7% (1,000); table size is not binding. Two-hop composition does not exceed the independent single-hop baseline (26% observed vs 25% expected; 15% lexical-distortion floor). Synergy remains absent across dose/load in two independent bed families. A 28-percentage-point in-context gap (given vs recalled intermediates) localizes failure to retrieval-for-reasoning, not gate access. Writes are independently retrievable, precisely addressable, reversible atoms, not composable into reasoning. A layered protocol, the first, to our knowledge, to cover compositionality in behavioral memory evaluation, separates baseline, synergy and distortion floor. Any addressed/key-value memory must report coverage, synergy ratio and distortion rate before deployment claims. Composition must be system-orchestrated; multi-constraint queries should explicitly chain written atoms. In-context controls show given intermediate entities are exploited at twice the implicit rate. Production load fits exponential interference ( $R^2=0.98$ ); its mechanism, between budget dilution and write interference, remains open. All numbers are recomputed from archived raw logs/JSON; a fresh 52-target production rerun gives 52/52, exactly matching archived baselines.

Figures & tables

Condition	Bed pair	Evidence (greedy or bias-4 of 150)	Status
Pathway width	283M full vs. narrow	$146$ vs. $68$	recomputed
Pathway width	1.7B full-dim bed	$70$	recomputed
Prior strength	7B undertrained pair	$8$ vs. $1$	recomputed
Gate trainability	frozen gate, 3 seeds	$0/150$ (seed 42)	recomputed

Table 1: Controlled demonstrations of the three conditions. All counts are re-read from archived run logs on the original beds and checkpoints.

Figure 1: Interventional dose experiment: identical 2,500-step chain writing on six checkpoints of one 779M-parameter trajectory (Table 2 ). Write coverage is gated by prior training state.

Pretraining step	9,999	19,999	40,000	60,000	80,000	99,999
Backbone perplexity	100.2	59.6	41.2	37.0	35.6	34.2
Write coverage (of 150)	4	8	48	52	56	63

Table 2: Dose axis raw values.

Figure 2: Write coverage as a function of memory load at production scale. The decay is attributable to address collisions within salted sub-tables, not to table size.

	Archived record	Fresh re-run
Targets	26	52
Writes succeeded	26 (record match)	52
New promotions (non-top $\to$ rank 1)	5	24
Failures / clipping	0 / 0	0 / 0
Per-target baseline ranks (first 26)	all match

Table 3: Fresh-inference validation of the production pool prefix on the same checkpoint. Left: the audit’s archived 26-target prefix; right: the expanded 52-target re-run. The first 26 baselines match the archived record exactly.

Seed	L1 hop1	L2 hop2	L3 two-hop	L3 expected (indep.)	L0 floor
42	27/50	26/50	12/50	14.0	6/50
777	22/50	28/50	13/50	12.3	10/50
1234	22/50	27/50	14/50	11.9	6/50
Mean	$23.7$ ( $47\%$ )	$27.0$ ( $54\%$ )	$13.0$ ( $26\%$ )	$12.8$ ( $26\%$ )	$7.3$ ( $15\%$ )

Table 4: Layered two-hop evaluation over 50 written chains, three seeds. L3 observed composition matches the independent single-hop product

L1\times L2

within noise; no synergy is present. L0 is the mismatched-subject distortion floor.

Figure 3: Composed two-hop targets (observed

L3

) against the independent-retrieval baseline across the dose axis (six checkpoints) and the capacity axis (four loads). On the dose axis, composition is trivially absent below the writing threshold (dose 1–2) and tracks the baseline above it; on the capacity axis, observed composition tracks the baseline and deviations are non-systematic in sign.

Chains	L1	L2	L3 observed	L3 indep. $L1{\times}L2/50$	L0
12	8	12	6	1.9	10
25	10	12	3	2.4	7
50	16	17	15	5.4	6
100	26	26	12	13.5	8

Table 5: Composition under memory load (one seed). Probes are normalized to the written-chain count of each condition; counts are over the 50-chain probe set, so per-condition hit rates are the comparable quantity. L3 tracks the independent baseline at every load level; no load level produces synergy above it.

Memory-Augmented Language Models

Memory-augmented language models combine persistent or temporary memory with neural computation to retain, retrieve, and use information beyond the active context.

See all 291 papers

Oct 15, 2026

Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory

Addressed conditional memory provides a local surface for post-hoc knowledge injection, but a successful write need not transfer to a new question or preserve untargeted behavior. This study combines a controlled GPT-2 comparison with production-memory experiments on Qwen3.8-Flash-Next-FP8 and DeepSeek-V4.1-Flash. Newly initialized modules attached to frozen GPT-2 are tested in a 2 x 2 design: row-only versus row-plus-projection updates, each with or without fixed module-residual anchoring. All arms reproduce 50 trained completions in each of three sampling seeds. Natural-question target-string hits average 5.3% for both row-only arms, 28.7% for unanchored projection updates, and 22.0% with anchoring, while strict answer scores remain zero. Anchoring lowers measured drift; lexical hits do not establish entity-specific retrieval. In the archived Qwen experiments, gradient writes make the designated first answer token top-ranked for 132 independently reset targets, comprising 93 new promotions and 39 already top-ranked targets. A separate matched 24-target comparison gives 13 new promotions with Adam versus two with SPSA at equal iteration counts but unequal computational procedures. An 80-row case includes saved continuations and a replay record, while reformulations bound its scope. Six DeepSeek zero-order cases yield one new target-token promotion and one regression. These results support bounded writing feasibility and separate trained expression, question-form behavior, and preservation; first-token rankings do not establish complete-answer success or universal conditions for writable memory.

Wei Gao, Cheng Tian, Duan Yang +1

Shibei Qiyuan AI Technology · Xi'an Jiaotong University

Knowledge Augmentation for Language Models Knowledge Editing Memory-Augmented Language Models

Ask PDF

May 23, 2026

WhenLoss: Diagnosing Write and Retrieval Bottlenecks in Long-Context Memory Systems

Long-context memory systems often fail under fixed budgets, but end-to-end evaluation does not reveal whether evidence was discarded during compression or preserved but never retrieved. We introduce a four-condition diagnostic protocol that evaluates a fixed reader under truncated full context (TFC), oracle evidence (OE), complete stored memory (CSM), and retrieved memory (RM). Under this fixed-budget LongMemEval setup, write-side gaps exceed retrieval-side gaps for most tested baselines, with four of six baselines robustly write-dominant under our default diagnosis margin. Motivated by this diagnosis, we propose Expected Predictive Compression (EPC), which moves the key decision--what information to retain--to write time by using an LLM to anticipate likely future questions and preserve the minimal supporting evidence under the token budget, while leaving retrieval unchanged at question time. Across all 500 LongMemEval questions with three readers (GPT-5.2, Claude Sonnet 4, Gemini 2.5 Pro), EPC achieves the highest CSM scores among all systems (0.49 vs. 0.44 for Summary (LLM), the strongest baseline), reducing Delta_write to 0.04 while leaving Delta_retr comparable to other LLM-based systems. These results suggest that, on this benchmark and evaluation setup, improving what the write stage preserves is a key avenue for performance gains in the tested systems.

Jiangnan Yu, Kisson Songqi Lin, Jilong Wu

Long-Context Retrieval Memory-Augmented Language Models Memory Compression Long-Term Conversational Memory

Ask PDF

May 4, 2026

MEMAUDIT: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing

Long-term LLM agents must compress streams of past interactions into persistent memory before future queries are known. Existing evaluations usually measure final question-answering accuracy, which entangles memory writing with retrieval, prompting, and reader reasoning. We introduce MEMAUDIT, an exact packageoracle evaluation protocol for budgeted long-term memory writing. A MEMAUDIT package fixes an experience stream, candidate memory representations, storage costs, semantic evidence units, future-query requirements, and a budget, turning write-time memory selection into a finite auditable optimization problem with a certified denominator. We instantiate this protocol with a concave-over-modular semantic coverage objective under storage and one-representation-per-experience constraints, and compute exact package optima using branch-and-bound with MILP certification. Across controlled exact packages, validity-heavy stress tests, human-audited natural support slices, and exported Mem0, A-Mem, and Letta stores, MEMAUDIT separates representation quality, validity-state preservation, and budget-aware selection effects that end-to-end QA cannot localize. The resulting artifact provides reusable package generators, certified solvers, natural package exports, external-system scorers, and cached reproducibility metadata for evaluating what memory writers actually preserve under fixed storage budgets.

Nishant Bhargava, Rodrigo Sobral Barrento

Purdue University

Persistent Memory for Language Models LLM Agent Evaluation Long-Term Conversational Memory Agent Memory Management

Ask PDF

Abstract

Memory-Augmented Language Models

Explore similar work

Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory

WhenLoss: Diagnosing Write and Retrieval Bottlenecks in Long-Context Memory Systems

MEMAUDIT: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing