MemCode: Discrete Semantic Representations for Long-Term Agent Memory

Xin Li; Liang Hu; Duoqian Miao; Ming Peng; Qi Zhang

cs.AIOct 14, 2026

MemCode: Discrete Semantic Representations for Long-Term Agent Memory

Authors: Xin Li, Liang Hu, Duoqian Miao, Ming Peng, Qi Zhang

Organizations: Tongji University · WorldMind

Abstract

Long-term memory remains a fundamental bottleneck for large language model (LLM) agents across sessions. Existing systems typically adopt a write-then-retrieve paradigm, storing context-dependent dialogue fragments and retrieving them via nearest-neighbor search in a continuous embedding space. This design yields unstable access units and systematically misses logically related but lexically distant evidence. We propose MemCode, a long-term agent memory framework that couples retrieval-oriented writing with discrete associative retrieval. MemCode rewrites dialogue into self-contained atomic memory units (AMUs), maps AMUs into a learnable codebook with isomorphic semantic quantization (ISQ), and uses codebook co-occurrence topology as a retrieval path complementary to vector search. For long-term memory stores, MemCode separates online atomic writes from asynchronous offline consolidation. On LongMemEval-S and LoCoMo with GPT-4o-mini and Qwen3-30B-A3B-Instruct-2507 backbones, MemCode achieves the best overall accuracy while keeping per-dialogue memory construction within roughly 40~~130 seconds, making it 3~~21 times faster than LightMem and over an order of magnitude faster than A-MEM and MemoryOS.

Figures & tables

GPT-4o-mini
Method	Runtime $\downarrow$	Total Tokens $\downarrow$	Calls $\downarrow$	Overall $\uparrow$	Temporal	Multi-Session	Know.-Upd.	Single-User	Single-Asst.	Single-Pref.
Method	(s)	(k)		ACC (%)	( $n=133$ )	( $n=133$ )	( $n=78$ )	( $n=70$ )	( $n=56$ )	( $n=30$ )
Full Text	-	-	-	56.80	31.58	45.45	76.92	87.14	89.29	36.67
Naive RAG	-	-	-	61.00	39.85	48.48	67.95	90.00	98.21	53.33
LangMem	2293.70	1102.16	520.62	37.20	15.79	20.30	66.67	60.00	46.43	60.00
A-MEM	5132.06	1605.81	986.55	62.60	47.36	48.87	64.11	92.86	96.43	46.67

Table 1: Accuracy and memory-construction cost on LongMemEval-S. We report Overall/category ACC (

\uparrow

), runtime per dialogue (

\downarrow

), Total Tokens, and Calls. Asyn. denotes asynchronous consolidation. Bold and underline denote the best and 2 nd -best results within each backbone.

GPT-4o-mini
Method	Runtime $\downarrow$	Total Tokens $\downarrow$	Calls $\downarrow$	Overall $\uparrow$	Multi	Open	Single	Temp
Method	(s)	(k)		(ACC %)	( $n=282$ )	( $n=96$ )	( $n=841$ )	( $n=321$ )
Full Text	-	-	-	73.83	68.79	56.25	86.56	50.16
Naive RAG	-	-	-	63.64	55.32	47.92	70.99	56.39
A-MEM	6060.73	1149.43	1175.47	64.16	56.03	31.25	72.06	60.44
MemoryOS (eval)	2422.05	287.00	553.45	58.25	56.74	45.83	67.06	40.19

Table 2: Category-level accuracy and memory-construction cost on LoCoMo. Asyn. denotes asynchronous consolidation. Bold and underline denote the best and 2 nd -best results on each backbone.

Retrieval path	Without consolidation	With consolidation
Dense only	59.57 (168)	65.25 (184)
Dense + codebook	64.89 (183)	69.15 (195)

Table 3: Component attribution on LoCoMo Multi-Hop (GPT-4o-mini,

n=282

). All settings use the same base AMU store and a total evidence budget of 40. Counts of correct answers are shown in parentheses.

Method	Evidence budget	Overall accuracy (%)
Dense-20	20	73.18
Dense-40	40	74.61
Dense-20 + BM25-20	20+20	76.75
Dense–sparse hybrid + reranker	40	76.36
HyDE + reranker [ 40 ]	40	76.69
Dense-20 + codebook-20 (MemCode)	20+20	78.12

Table 4: Fixed-budget retrieval controls on LoCoMo (GPT-4o-mini). Panel (a) uses the same frozen 5,911-AMU store and at most 40 evidence items. Panel (b) isolates the second path under a 20+20 budget;

\Delta

is codebook minus BM25, C/B-only counts paired wins, and BM25 inj. reports additional base facts/time buckets beyond the vector stage.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Use	Hardware
Memory runtime	Ingestion, retrieval, consolidation, and benchmark evaluation	12th Gen Intel Core i5-12500H @ 3.10GHz; NVIDIA GeForce RTX 3050 Ti Laptop GPU, 4 GB; 16 GB RAM
Offline ISQ training	VQ-VAE codebook learning before memory evaluation	12 vCPU Intel Xeon Platinum 8352V @ 2.10GHz; one 32 GB vGPU; 90 GB RAM

Appendix

Table 5: Compute environments used by MemCode. Offline ISQ codebook training is performed once and is not included in the per-dialogue memory-runtime measurements in Tables 1 and 2 .

Block	Item	Setting
Data	Training pairs	DailyDialog context–response pairs; the context contains the two previous utterances
Data	Sequence length	Source and target sequences are truncated or padded to 128 tokens
Model	Base encoder	T5-base encoder; code dimension equals the T5 hidden size
Model	Projection and codebook	Stride-1 Conv1d projection, kernel size 3, padding 1; 1024 Euclidean code vectors
Optimization	Optimizer and schedule	AdamW, weight decay 0.01; batch size 16; gradient accumulation 2; 10 epochs; cosine schedule with 10% warmup; gradient clipping at 1.0
Optimization	Learning rates	$10^{-4}$ for T5 parameters; $10^{-3}$ for VQ, projection, and decoder-side parameters

Appendix

Table 6: Offline VQ-VAE training and ISQ codebook settings. These settings are fixed for all memory-runtime experiments.

Dataset	Ingestion unit	Extraction background	Dense path	Codebook path	Final evidence budget
LoCoMo	A–B dialogue turn	10 prior sentences	20 AMUs	20 AMUs	40 AMUs + 25 raw dialogue turns
LongMemEval-S	User–Assistant turn	5 prior sentences	20 AMUs	20 AMUs	40 AMUs + 10 raw dialogue turns

Appendix

Table 7: Dataset-specific ingestion and evidence-reconstruction budgets.

Hyperparameter	Value	Accuracy (%)	$\Delta$ vs. default
$K$	1	77.40	-0.72
	3 (default)	78.12	+0.00
	5	77.47	-0.65
	7	77.47	-0.65
	9	76.49	-1.63
$\alpha$	0.1	77.79	-0.33

Appendix

Table 8: Additional hyperparameter ablations for the codebook retrieval path on LoCoMo (GPT-4o-mini).

\Delta

denotes the signed percentage-point difference relative to the deployed default configuration

(K{=}3,\alpha{=}0.3,\tau_{\mathrm{IDF}}{=}0.5)

with 78.12% overall accuracy.

Run	Correct / Total	Overall	Multi-Hop	Temporal	Open	Single-Hop
1	1200 / 1540	77.92	64.54	77.88	48.96	85.73
2	1199 / 1540	77.86	64.89	76.95	50.00	85.73
3	1210 / 1540	78.57	62.77	78.19	48.96	87.40
4	1207 / 1540	78.38	65.25	76.32	47.92	87.04
5	1212 / 1540	78.70	65.60	77.26	47.92	87.16
6 (main table)	1203 / 1540	78.12	64.89	76.64	43.75	87.04

Appendix

Table 9: Repeated runs for the main results of LoCoMo on GPT-4o-mini setting under temperature-0 decoding. Category columns report accuracy. The main-table result corresponds to Run 6, which is close to the center of the repeated-run distribution and is not the best run.

Benchmark	Backbone	Overall	Temporal	Multi	Open	Single	Know.-Upd.	Single-Asst.	Avg. Ans.
LoCoMo	GPT-4o-mini	78.12	76.64	64.89	43.75	87.04	–	–	73.12
LoCoMo	Qwen3	76.36	67.91	59.22	52.08	88.11	–	–	128.88
LongMemEval-S	GPT-4o-mini	77.20	69.92	65.41	–	92.86	82.05	92.86	75.52
LongMemEval-S	Qwen3	73.80	57.89	64.66	–	94.29	78.21	92.86	146.37

Appendix

Table 10: Backbone-level performance and generation behavior for MemCode without consolidation. Category columns report accuracy. Avg. Ans. reports average answer length in words.

Benchmark	Backbone	AMUs / dialog	Active codes	Code assignments	Max code freq.	Avg. AMU words
LoCoMo	GPT-4o-mini	591.1	216.9	3678.6	294.0	16.48
LoCoMo	Qwen3	940.4	266.1	6150.2	461.4	15.92
LongMemEval-S	GPT-4o-mini	1553.4	420.4	9941.1	798.0	21.41
LongMemEval-S	Qwen3	2130.3	450.2	14532.2	1356.1	21.80

Appendix

Table 11: Backbone-induced memory-store characteristics. Values are averaged over dialogue-level memory stores. Qwen3 writes more AMUs and activates more codebook assignments, which increases coverage but also raises the density of potential distractors.

Benchmark	Shared $n$	Both correct	Both wrong	GPT-only	Qwen-only	Largest GPT-only source
LoCoMo	1540	1055	216	148	121	Temporal: 55 vs. 27
LongMemEval-S	500	322	67	64	47	Temporal: 28 vs. 12

Appendix

Table 12: Paired question-level disagreements between GPT-4o-mini and Qwen3 under MemCode. GPT-only means GPT-4o-mini answers correctly while Qwen3 does not; Qwen-only is the reverse.

Category	$n$	Accuracy			Paired Wins		BM25 Injection
Category	$n$	Codebook	BM25	$\Delta$	C-only	B-only	New Base	New Time
Overall	1540	78.12	76.75	+1.36	105	84	18.58	10.96
Multi-Hop	282	64.89	60.64	+4.26	33	21	18.50	10.65
Temporal Reasoning	321	76.64	70.41	+6.23	35	15	18.66	11.34
Open Domain	96	43.75	50.00	-6.25	5	11	18.59	10.78
Single-Hop / Basic	841	87.04	87.63	-0.59	32	37	18.58	10.93

Appendix

Table 13: Full details for the second-path replacement ablation on LoCoMo (GPT-4o-mini).

\Delta

denotes codebook minus BM25. “C-only” counts cases where the codebook path answers correctly while BM25 does not, and “B-only” denotes the reverse. “New Base” and “New Time” report the average number of additional base facts and time buckets injected by the BM25 stage beyond the vector stage.

Metric	MemCode	Full HippoRAG2@40
Additional model calls	0	12,779
Additional tokens	0	8.47M
Explicit graph construction	None	34.49 min
Graph size	None	17,275 nodes / 109,779 edges
Query latency	0.060 s	2.864 s

Appendix

Table 14: Additional retrieval-system cost beyond the shared AMU construction stage. Query latency is averaged over LoCoMo questions.

Category	$n$	bge Acc.	MiniLM Acc.	$\Delta$	bge-only	MiniLM-only	Shared correct
Overall	1540	78.12	76.23	-1.88	90	61	1113
Multi-Hop	282	64.89	62.77	-2.13	23	17	160
Temporal Reasoning	321	76.64	73.83	-2.80	20	11	226
Open Domain	96	43.75	43.75	+0.00	4	4	38
Single-Hop / Basic	841	87.04	85.37	-1.66	43	29	689

Appendix

Table 15: Embedding-model sensitivity on LoCoMo (GPT-4o-mini). The bge-base-en-v1.5 row corresponds to the main MemCode setting.

\Delta

is the signed percentage-point difference of all-MiniLM-L6-v2 minus bge-base-en-v1.5. “bge-only” and “MiniLM-only” count paired question-level wins.

Asset	Role in this paper	Citation/source	License or terms
LoCoMo	Long-term conversational-memory benchmark for QA evaluation	Maharana et al. [32]	CC BY-NC 4.0 release license
LongMemEval-S	Dynamic multi-session memory benchmark for QA evaluation	Wu et al. [33]	MIT-licensed public release
DailyDialog	Conversation data used to train the discrete codebook component	Li et al. [25]	CC BY-NC-SA 4.0 release license
T5-base	Encoder initialization for isomorphic semantic quantization	Raffel et al. [36]	Apache-2.0 release license
bge-base-en-v1.5	Dense embedding backbone for vector retrieval	Xiao et al. [37]	MIT public model-release terms
GPT-4o-mini API	Memory writing, answer generation, and LLM-as-a-judge evaluation	OpenAI [34]	OpenAI service terms

Appendix

Table 16: Existing datasets, models, APIs, and software assets used by MemCode. License and terms information follows the public release pages associated with the cited assets.

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 358 papers

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory
Jul 17, 2026Jing Yu, Yibo Zhao, Jiaming Zhang +1Persistent Memory for Language Models Long-Term Conversational Memory
School of Data Science and Engineering, East China Normal University
DimMem: Dimensional Structuring for Efficient Long-Term Agent Memory
May 15, 2026Wentao Qiu, Haotian Hu, Fanyi Wang +2LLM Agent Memory Persistent Memory for Language Models
1StepOS · 2Xiamen University · 3ShanghaiTech University
LeanMem: Simple and Efficient Long-Term Memory for LLM Agents
Aug 4, 2026Yuxin Liao, Le Wu, Min Hou +3LLM Agent Memory Persistent Memory for Language Models
Hefei University of Technology

Jul 17, 2026J/K move · Enter open · S save

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

Jing Yu, Yibo Zhao, Jiaming Zhang, Xiang Li

School of Data Science and Engineering, East China Normal University

Persistent Memory for Language Models Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Open paper AskPDF arXiv

Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.

Abstract

LLM Agent Memory

Explore similar work

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

DimMem: Dimensional Structuring for Efficient Long-Term Agent Memory

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents