LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound

Baixi Sun; Le Chen; Anjir Ahmed Chowdhury; Xiaolong Ma; Chih-Hsuan Yang; Mingze Xia; Syed Zawad; Sheng Di

cs.AISep 26, 2026

LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound

Authors: Baixi Sun, Le Chen, Anjir Ahmed Chowdhury, Xiaolong Ma, Chih-Hsuan Yang, Mingze Xia, Syed Zawad, Sheng Di,

Organizations: Argonne National Laboratory · University of Houston · Oregon State University · IBM Research

Abstract

Agent memory grows as agents read inputs, reason, and call tools. Longer histories increase inference cost and eventually exceed the context window. LLM-based summarization reduces this history but adds latency and provides no explicit bound on information loss. We propose LAM, a Lossy Agent Memory system with three components: a deterministic deduplication rule with a substitution bound on retrieval scores - a bound on score perturbation, not a certificate of unchanged ranking; a memory manager that preserves the cached prefix and overlaps compaction with inference; and a performance model that estimates compaction costs before deployment. On 600 agent trajectories, LAM removes 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence. At a fixed deletion set, the performance model predicts a 71.4x-91.6x end-to-end speedup from removing records before prefill instead of deleting them from a prefilled context. That benefit comes from the schedule rather than the rule and applies to any prefix-preserving test.

Figures & tables

Figure 1: Compaction needs the right schedule. Modeled end-to-end time under the two schedules, where the grey bars delete a record already in the context and so pay to prefill it and then prefill the invalidated suffix again, while ours (green) tests the arriving record first and pays neither.

Figure 2: System design, where (a) traces one arriving record through the host memory manager to either a retained record or a reference stub, and (b) contrasts the two schedules, the upper one voiding the shaded cache suffix that must then be prefilled again and the lower one leaving the prefilled prefix intact.

	symbol	quantity	value / source
offline	$R_{p},t_{d}$	prefill rate, decode step time	GB10 anchors, Table 22
	$R_{e}$	encoder throughput	71,708 tok/s
	$R_{h}$	xxHash3	38.2 GB/s
	$R_{v}$	exact $\ell_{2}$ verify	13.0 M pairs/s
	$R_{z}$	cuSZp	53.3 GB/s
	$\beta$	bytes per token	3.68 B/tok

Table 1: Inputs to equation 2 , split into offline terms measured once per device by kernel microbenchmark and online terms read from the run itself.

Figure 3: Token removal against evidence retention, one marker per measured setting and up-and-left better, where the three accented series are LAM’s cumulative layers — L0, L0

+

L1, and the full pipeline — and the error bars give SemDeDup’s spread across seeds.

method	schedule	$r_{\mathit{eff}}$	batch 1	batch 32
observation masking ( $k=5$ )	wall	0.75	107.7 (9.4 $\times$ )	105.1 (11.8 $\times$ )
LLM summarization ( $\rho=10$ )	wall	0.90	498.0 (43.6 $\times$ )	495.5 (55.8 $\times$ )
SemDeDup (global $k$ -means, $0.95$ )	wall	0.32	647.9 (56.7 $\times$ )	645.4 (72.7 $\times$ )
LLMLingua-2 (rate $0.80$ )	wall	0.24	790.5 (69.1 $\times$ )	787.9 (88.7 $\times$ )
LAM, L0 only	admit	0.17	12.0 (1.1 $\times$ )	9.5 (1.1 $\times$ )
LAM, full	admit	0.22	11.4	8.9

Table 2: Modeled end-to-end latency per turn, with the ratio to LAM in parentheses, where “wall” defers compaction to the context budget and “admit” removes before prefill, the shaded row is the shipped configuration, and accented cells are the best in their column.

$\cos t$	$\delta$	mean $\hat{\delta}/\delta$	keep	R@10	tokens removed	evidence	resolve
$0.99$	0.1414	0.350	0.916	0.9978	18.89%	100.000%	75.83%
$0.98$	0.2000	0.409	0.896	0.9947	19.64%	100.000%	75.83%
$0.96$	0.2828	0.532	0.856	0.9881	21.53%	100.000%	75.83%
$0.95$	0.3162	0.559	0.837	0.9846	22.50%	99.984%	75.67%
$0.90$	0.4472	0.706	0.696	0.9518	32.02%	98.709%	70.33%

Table 3: Merge-threshold sweep, in which

\delta=\sqrt{2-2t}

is what the theorem permits against the

\hat{\delta}

the run attained, “evidence” is gold-patch retention and “resolve” the resolve-rate ceiling, and the shaded row is the shipped default.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

stage	setting
chunking	fixed 512-token windows over tool observations, no overlap
encoder	bge-base-en-v1.5 , unit-normalized, $D=768$
threshold	$\cos t=0.95$ , i.e. $\delta=0.316{}$ by equation 1
candidates	banded LSH over $Q_{\varepsilon}$ , verified by exact $\ell_{2}$
byte codec	cuSZp, relative error bound $1\%$ of $\max\|x\|$
stubs	one line per drop, charged as input (0.69%)

Appendix

Table 4: Default configuration. Parameter sweeps are in Appendix F.10 (band width, quantizer step), Appendix F (threshold) and Table 22 (device).

granularity	unit	removable
record	one whole tool observation	1.61%
paragraph	blank-line separated block	1.61%
chunk-512	fixed 512-token window	2.61%
line	a single line	18.14%

Appendix

Table 5: Byte-exact duplicate tokens by granularity, within-trajectory. The line-level figure, not the record- or chunk-level one, is the floor any near-duplicate contribution must be measured above.

Figure 4: Layer composition over 600 trajectories, weighted by tokens. Running L0 before L1 reduces joint removal by 2.45 percentage points at

\cos\geq 0.95

. Assuming independence overstates removal by 4.0–11.7 points. Net joint removal is 22.50%, or 4.47 points above L0 alone.

		chunks removed (cardinality)			tokens removed
$t$	$\delta$	attained	ceiling ( $t^{\prime}$ )	% of ceiling	at $t$	at $t^{\prime}$
$0.99$	$0.141$	8.42%	14.16%	59.48%	6.16%	11.47%
$0.95$	$0.316$	15.86%	61.19%	25.92%	13.13%	62.55%
$0.90$	$0.447$	28.35%	91.86%	30.87%	26.12%	91.63%

Appendix

Table 6: The removal ceiling, measured. For each operating point

t

we run greedy first-occurrence a second time at

t^{\prime}=4t-3

(radius

2\delta

); by Theorem 2 its removal upper-bounds what any

\delta

-bounded compactor could achieve here. Cardinality over 27,986 chunks is the bounded quantity. Token columns report observed removal in the two passes and are not upper bounds on token removal.

Figure 5: Representative selection and the substitution bound. Left: observation-token removal for connected components (DSU) and greedy first-occurrence. Right: maximum substitution distance relative to

\delta=\sqrt{2-2t}

. Greedy remains within the bound. The diamond marks

\hat{\delta}=0.268

\cos\geq 0.95

. Results use 600 trajectories and 9.07M observation tokens.

	$C=3$ K	$C=8$ K	$C=32$ K	$C=128$ K
batch 1	2.5% (1.03 $\times$ )	6.4% (1.07 $\times$ )	21.6% (1.28 $\times$ )	52.4% (2.10 $\times$ )
batch 32	23.5% (1.31 $\times$ )	45.0% (1.82 $\times$ )	76.6% (4.28 $\times$ )	92.9% (14.1 $\times$ )

Appendix

Table 7:

\varphi

, the context-scaling share of one turn, and the ceiling it puts on any compactor. Llama-3.1-8B on GB10,

\Delta=637{}

arriving tokens and

G_{a}=39{}

decoded tokens per turn (measured). Weight reads amortize across a decode batch; KV reads do not.

policy	batch 1	batch 32	% of total
delete in place (prefill, then remove)	$1.186\times 10^{6}$ s	$1.183\times 10^{6}$ s	98.6%
$\varepsilon$ error-bounded (remove, then prefill)	$1.66\times 10^{4}$ s	$1.29\times 10^{4}$ s	0.1%
policy gain	71.4 $\times$	91.6 $\times$

Appendix

Table 8: Modeled latency with the same compressor, corpus, and deletion set. Only the schedule differs. Delete-in-place prefills the arriving tokens and then removes duplicates; the

\varepsilon

error-bounded rule removes before prefill. Llama-3.1-8B, growth regime, 600 trajectories.

fixed arrival rate, 5 req/s — server unsaturated
compactor	GPU server			CPU compactor
threads	out tok/s	TTFT p50	TPOT p50	ctx tok/s	vs. solo
none	990.8	157.0 ms	11.81 ms	—	—
8	990.8	164.7 ms	11.84 ms	2,937	0.966
16	990.7	166.5 ms	11.84 ms	4,647	0.929
32	989.0	192.5 ms	13.68 ms	3,809	0.620

Appendix

Table 9: CPU compaction overlapped with GPU serving on two sirius nodes. At 32 threads the compactor contends with the server’s own CPU work and both sides lose.

quantity	value	note
gross removal	23.19%	L1 followed by L0
net of stubs	22.50%	stubs cost 0.69% of input
chunks encoded	27,986	one encoding per chunk
per-turn latency	18.1 ms mean / 115 ms p99	a turn itself costs 306 ms–3.3 s
measured bound	$\hat{\delta}=0.268\leq 0.316$	attained, not configured
online $\equiv$ batch	byte-identical residuals	verified deletion-set equality, 600/600 pass

Appendix

Table 10: The

\varepsilon

error-bounded test over 600 SWE-smith trajectories, 15,169 observation records, 9.07M tokens,

\cos\geq 0.95

	substitution distance $d_{i}$ , token-weighted					retrieval
$\cos t$	bound $\delta$	median	p99	max	mean $/\delta$	keep	R@10
$0.99$	0.1414	0.0376	0.1401	0.1412	0.350	0.916	0.9978
$0.98$	0.2000	0.0820	0.1979	0.1999	0.409	0.896	0.9947
$0.95$	0.3162	0.2103	0.3148	0.3162	0.559	0.837	0.9846
$0.9$	0.4472	0.3598	0.4457	0.4472	0.706	0.696	0.9518

Appendix

Table 11: Substitution distances and retrieval quality. Distance summaries are weighted by deleted tokens;

\hat{\delta}

is the maximum within a run. Keep is the retained-chunk fraction. R@10 compares with the full index’s top-10 after applying the substitution map.

configuration	removal	vs. default
L0 alone (exact hashing)	18.04%	$-\,4.93$ pt
L1 alone (near-duplicate)	13.81%	$-\,9.16$ pt
A: L0 $\to$ L1	20.52%	$-\,2.45$ pt
B: L1 $\to$ L0 (default)	22.97%	—
independence estimate (rejected)	29.35%	$+\,6.38$ pt

Appendix

Table 12: Layer composition and order,

\cos\geq 0.95

, same denominator. The independence estimate is compared with measured joint removal.

stage	seconds	% of total
bge-base encode	13.945508	99.989%
$Q_{\varepsilon}$ quantize	0.000138	0.001%
band digests	0.000078	0.001%
exact $\ell_{2}$ verify	0.001201	0.009%
cuSZp byte layer	0.000069	0.000%
total	13.946994

Appendix

Table 13: Per-stage cost of one full compaction pass at trajectory scale.

method	removal	evidence kept	trajectories intact	lines lost
no compaction	0.00%	100.000%	100.0%	0
L0 alone (exact hashing)	18.02%	100.000%	100.0%	0
ours, $\cos\geq 0.99$	18.86%	100.000%	100.0%	0
ours, $\cos\geq 0.96$	21.49%	100.000%	100.0%	0
ours, $\cos\geq 0.95$	22.47%	99.984%	99.8%	1
ours, $\cos\geq 0.90$	31.99%	98.709%	93.0%	81

Appendix

Table 14: Gold-patch safety, 598 trajectories, 6,276 witnessed evidence lines. “Kept” pools over lines (micro retention); “intact” is the fraction of trajectories losing none (macro). The no-compaction and L0 rows are harness checks and must read

100.000\%

; composing L1 with L0 must score what L1 scores alone.

method	removal	context gain	cost / event	share of turn
L0 alone (xxHash3)	18.04%	1.220 $\times$	3.08 $\mu$ s	$3.1\times 10^{-6}$
L1 alone (near-duplicate)	13.81%	1.160 $\times$	8.88 ms	0.0089
L1 $\to$ L0, gross	23.19%	1.302 $\times$	8.88 ms	0.0089
L1 $\to$ L0, net	22.50%	1.290 $\times$	8.88 ms	0.0089
ours $-$ hashing	$+4.47$ pt	1.058 $\times$	2,883 $\times$	—

Appendix

Table 15: Ratio ledger, 600 trajectories, 9.07M observation tokens,

\cos\geq 0.95

. Context gain is

1/(1-r)

; “share of turn” is per-event cost against a turn costing 1.003 s. “Net” charges us for reference stubs.

floor $\tau$	admissible	evidence loss	binding point	RSC drop	binding point
$1.000$	7	0.0%	(all tie)	$-$ 0.00 pp	(all tie)
$0.999$	10	0.7%	SemDeDup $\cos 0.9999$	$-$ 0.17 pp	ours $t{=}0.95$
$0.990$	16	5.5%	SemDeDup $\cos 0.995$	$-$ 1.83 pp	ours $t{=}0.92$
$0.950$	22	12.0%	ours $t{=}0.88$	$-$ 8.50 pp	ours $t{=}0.88$

Appendix

Table 16: Maximum evidence loss allowed by each floor across 35 settings from four families. “Tasks losing evidence” is

1-\mathrm{macro\_all\_intact}

. “RSC decrease” is the largest percentage-point decrease from 75.83% among the two families with measured RSC.

	best	$\mathrm{ECR}@\tau$
method	(unconstrained)	$\tau=1.000$	$0.999$	$0.990$	$0.950$
L0 alone	18.02%	18.02%	18.02%	18.02%	18.02%
near-duplicate layer alone	26.09%	11.42%	13.08%	13.08%	26.09%
observation masking	81.44%	—	—	4.42%	4.42%
SemDeDup, 5 seeds	47.19%	—	19.60%	22.84%	33.74%
ours: near $\to$ line	31.99%	21.49%	22.47%	22.47%	31.99%

Appendix

Table 17: Removal and evidence in one place, same corpus and denominator.

\mathrm{ECR}@\tau

is the most a family can remove while retaining

\geq\tau

of the gold-patch evidence lines, maximized over its own operating points; “best” drops the constraint. Shading marks the column winner. SemDeDup is swept to

\cos 0.9999

, tighter than our own family.

	$\rho$	$\eta(1)$	$\eta(3)$	$\eta(8)$	break-even $\kappa$ vs. ours
ours, $\cos\geq 0.95$	0.2247	0.2246	0.2245	0.2241	—
ours, $\cos\geq 0.90$	0.3199	0.3141	0.3027	0.2741	16.9
masking, $k{=}20$	0.3406	0.3097	0.2478	0.0932	3.8
masking, $k{=}10$	0.6738	0.5884	0.4178	$-0.0087$	5.3
masking, $k{=}5$	0.7794	0.6603	0.4222	$-0.1732$	4.7

Appendix

Table 18: Net reduction

\eta(\kappa)

from equation 9 , assuming

\kappa

repair turns per lost evidence line. Break-even values show when repair costs offset additional removal.

	$B{=}8$ K	$B{=}16$ K	$B{=}32$ K	$B{=}64$ K
uncompacted	22.0%	67.2%	93.2%	99.3%
ours, $\cos\geq 0.99$	23.0%	71.5%	94.2%	99.8%
ours, $\cos\geq 0.95$	25.2%	75.0%	96.7%	100.0%
ours, $\cos\geq 0.90$	33.0%	83.7%	99.3%	100.0%

Appendix

Table 19: Budget-feasibility rate

\mathrm{BFR}@B

: the fraction of 600 trajectories whose observation stream fits under

B

tokens. Necessary, not sufficient.

Figure 6: (a) Metric-entropy bounds from Corollary 3 and observation-token removal. The upper and lower edges use greedy passes at radii

\delta

and

2\delta

. Index rate depends logarithmically on retained-set size, while context cost depends on token count. (b) Conditional resolve-rate ceiling (RSC, equation 10 ) versus token removal for LAM and observation masking on 600 trajectories, of which 455 were originally resolved. RSC remains near baseline through 22.50% removal and decreases at more aggressive settings.

$\cos t$	byte codec	dedup	codec	composite	recall@10
$0.9$	binary (sign)	$1.396\times$	$30.72\times$	$42.88\times$	0.6204
$0.9$	cuSZp/fixed, $\varepsilon_{b}\!=\!0.03$	$1.396\times$	$9.46\times$	$13.20\times$	0.6354
$0.9$	cuSZp/fixed, $\varepsilon_{b}\!=\!0.01$	$1.396\times$	$6.53\times$	$9.11\times$	0.6491
$0.9$	cuSZp/fixed, $\varepsilon_{b}\!=\!0.003$	$1.396\times$	$4.94\times$	$6.90\times$	0.6549
$0.9$	int8 (per-vector)	$1.396\times$	$3.96\times$	$5.53\times$	0.6556
operating point we ship

Appendix

Table 20: Composite storage reduction and recall@10, including the selected setting. Reduction multiplies deduplication and codec ratios on the embedding index. The full-precision, full-size index has recall 0.6536; small differences around it should not be interpreted as quality gains.

$\cos t$	all context	observations	observations	ratio
	LHTB		SWE-bench
L0 (byte-exact)	2.08%	3.35%	—	—
$0.999$	6.28%	8.07%	—	—
$0.99$	14.44%	16.78%	6.52%	$2.6\times$
$0.98$	18.71%	20.54%	8.39%	$2.4\times$
$0.95$	33.07%	33.39%	13.81%	$2.4\times$

Appendix

Table 21: Near-duplicate rate on LHTB, token-weighted, within trajectory, against the SWE-bench observation rate from Table 15 . Same encoder, window and rule.

Figure 7: The removal axis on a second scaffold. LHTB provides no gold patches, so only removal is plotted. L0 is record-level there and line-level on SWE-smith.

quantity	measured	benchmark
streaming bandwidth	237.4 GB/s	triad $c=a+b$ , $3\times 512$ MB fp32; $\mu_{m}=0.869$ of the 273 GB/s spec
fp16 tensor matmul	90.3 TFLOP/s	$8192^{3}$ GEMM
machine balance	380 FLOP/byte	derived $F/B$ ; everything below is bandwidth-bound
bge-base encode	71,708 tok/s	512 synthetic docs $\times$ 512 tok = 262K tok, batch 64
$Q_{\varepsilon}$ quantize	65.4 GB/s	$2^{26}$ fp32 $\to$ int16, 6 B/element moved; unfused PyTorch
xxHash3 (CPU)	38.2 GB/s	64 MB buffer, single thread

Appendix

Table 22: Hardware anchors and the benchmark used to obtain each.

Figure 8: Compaction latency on a second device (NVIDIA A100-SXM4-40GB), the same comparison as Figure 9 (a). The ordering and the crossovers are unchanged, so the gap is structural rather than an artifact of one bandwidth-starved machine. Dotted verticals mark where the KV cache no longer fits at batch 1.

		bits per chunk
$t$	$\delta$	$\log_{2}\|R(2\delta)\|$	$H(\pi(X))$	$\log_{2}\|R(\delta)\|$	tokens rm.	traj. intact	RSC
$0.995$	$0.1000$	14.61	14.59	14.67	18.47%	100.0%	75.83%
$0.990$	$0.1414$	14.55	14.56	14.65	18.89%	100.0%	75.83%
$0.980$	$0.2000$	14.40	14.50	14.61	19.64%	100.0%	75.83%
$0.970$	$0.2449$	14.14	14.45	14.58	20.45%	100.0%	75.83%
$0.960$	$0.2828$	13.80	14.40	14.55	21.53%	100.0%	75.83%

Appendix

Table 23: Rate, removal and the outcome ceiling across the sweep. Bits per chunk;

\log_{2}|S|=14.77

lossless.

			queries with bounded-invariant top- $k$
relation	parameter	bound on $\|\Delta\cos\|$	$k{=}1$	$k{=}3$	$k{=}5$	$k{=}10$	$k{=}20$
$\ell_{\infty}$ relaxed, $2\varepsilon\sqrt{D}$	$\varepsilon=0.001$	0.0554	15.62%	0.59%	0.13%	0.00%	0.00%
	$\varepsilon=0.003$	0.1663	0.26%	0.00%	0.00%	0.00%	0.00%
	$\varepsilon=0.01$	0.5543	0.00%	0.00%	0.00%	0.00%	0.00%
	$\varepsilon=0.03$	1.6628	0.00%	0.00%	0.00%	0.00%	0.00%
$\ell_{2}$ tight, $\sqrt{2-2t}$	$t=0.995$	0.1000	3.71%	0.00%	0.00%	0.00%	0.00%

Appendix

Table 24: Share of queries whose top-

k

is provably invariant, by bound and by

k

. 1,536 queries against 5,882 LoCoMo records. The default operating point is

t=0.95

$k$	median	p90	p99	max
$1$	0.0185	0.0702	0.1296	0.2358
$3$	0.0056	0.0201	0.0483	0.0886
$5$	0.0036	0.0133	0.0299	0.0698
$10$	0.0018	0.0067	0.0148	0.0247
$20$	0.0010	0.0034	0.0075	0.0147

Appendix

Table 25: Rank-

k

/rank-

(k{+}1)

cosine margin over 1,536 queries. Compare against the bound column of Table 24 .

recall of true $\cos\geq 0.99$ pairs
	$\varepsilon/\sigma$
$r$	$0.01$	$0.02$	$0.03$	$0.05$	$0.08$	$0.12$	$0.2$	$0.3$
$2$	0.974	1.000	1.000	1.000	1.000	1.000	1.000	1.000
$4$	0.581	0.774	0.885	0.990	1.000	1.000	1.000	1.000
$8$	0.400	0.506	0.590	0.743	0.879	0.986	1.000	1.000
$16$	0.332	0.398	0.467	0.537	0.668	0.779	0.934	0.992

Appendix

Table 26: Band-width sweep over the exhaustive within-trajectory pair set. Candidate rate is the fraction of pairs sent to distance verification; its reciprocal gives the reduction in comparisons. We select settings with full recall and low candidate rate.

Figure 9: Compaction cost, removal, scheduling, and evidence retention. (a) Modeled compaction latency on

T

tokens, excluding subsequent agent inference. (b) Observation-token removal versus input size over 600 trajectories. Removal rises from 0.80% at

T=1K{}

to 25.58% at

T=33K{}

, with whole-trajectory mean 22.50%. Curves stop when too few trajectories reach the input size; summarization removal is assumed at

1-1/\rho

. (c) Scheduling at fixed compressor, corpus, and removal. Methods that rewrite or delete existing context pay re-prefill costs; observation masking costs 9.4

\times

times LAM’s modeled total time. For rules supporting both schedules, testing first gives 71.4

\times

–91.6

\times

speedup. (d) Gold-patch evidence loss on an inverted logarithmic axis. Zero-loss methods appear in the shaded band. The plot is clipped at 10% loss. The default setting loses 1 of 6,276 lines (0.016%), compared with masking losses of 6.98%, 19.2%, and 26.9% at

k=20,10,5

. The

\tau

floors and 20% band mark retention requirements. Summarization evidence retention is unmeasured.

	snapshot assumption	per-arrival (correct)
update rule	$C\leftarrow C(1-r)$ every event	$C\leftarrow C+\Delta(1-r)$ per arrival
steady state	$C^{*}=\Delta/r_{\mathit{eff}}$	none; only eviction bounds $C$
mean context, 32K budget	2.8K	28K
speedup of L1 over L0 alone	1.22 $\times$	1.07 $\times$ growth, 1.00 $\times$ hard budget

Appendix

Table 27: Snapshot and per-arrival accounting for the same measured removal rate.

Agent Memory Management

Agent memory management studies how AI agents store, update, retrieve, compress, and forget information across long-running tasks.

See all 449 papers

Jul 17, 2026

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.

Jing Yu, Yibo Zhao, Jiaming Zhang +1

School of Data Science and Engineering, East China Normal University

Persistent Memory for Language Models Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Ask PDF

Aug 4, 2026

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents

Long-term memory is essential for LLM-based agents to sustain interactions and reliably leverage distant history. However, existing memory systems typically process heterogeneous dialogue content through a uniform summarization and retrieval pipeline, leading to either excessive token consumption or irreversible loss of fine-grained evidence. We argue that historical dialogue content should be handled differently according to its compressibility, temporal dynamics, and fidelity requirements. Based on this insight, we propose LeanMem, a lightweight long-term memory framework. LeanMem first filters out low-value content, then stores informative segments as compact profile memory, temporally structured event memory, or source-grounded record memory, depending on the nature of the information. During maintenance, only dynamically evolving event memories are selectively updated, avoiding redundant consolidation of stable profiles and immutable records. During inference, LeanMem dynamically selects memory types and allocates retrieval budgets according to query-specific evidence demands, assembling relevant evidence on demand. On LoCoMo and LongMemEval-S with GPT-4.1-mini and Qwen3-8B, LeanMem improves accuracy over the strongest memory-based baseline in every setting, by up to 15.1 points, at the lowest or near-lowest construction cost, inference tokens, and latency. The code and datasets are included in the supplementary materials.

Yuxin Liao, Le Wu, Min Hou +3

Hefei University of Technology

LLM Agent Memory Persistent Memory for Language Models Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Ask PDF

Jun 23, 2026

Are We Ready For An Agent-Native Memory System?

Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.

Wei Zhou, Xuanhe Zhou, Shaokun Han +5

Shanghai Jiao Tong University · Tsinghua University · MemTensor (Shanghai) Technology Co., Ltd

Agent Memory LLM Agent Memory Agent Memory Management

Ask PDF

Abstract

Agent Memory Management

Explore similar work

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents

Are We Ready For An Agent-Native Memory System?