Large language model (LLM) agents that serve users across many sessions must choose, for every request, the few stored memories that enter a limited context window. This choice largely decides whether the agent answers correctly, and it runs on every request at Web scale. This selection faces three challenges: a large and growing store, a per-request cost budget, and multi-session questions that need all of their evidence. Many memory systems build notes or graphs with hundreds to thousands of LLM calls per history, and each is designed on its own, so it is unclear which component yields the accuracy. These challenges mirror the properties of recommendation, so we formalize agent memory as budgeted slate recommendation, with the request as the user, stored memories as items and the context window as a slate. The formalization places memory systems in one design space of recommender cascades and factorizes accuracy into the completeness that each stage controls. Guided by it, we propose AgentMemRec, which adapts candidate generation, learning to rank and slate construction from recommender systems and selects memories without any LLM call. On multi-session questions of LongMemEval, AgentMemRec outperforms memory and context-selection methods, including A-Mem, HippoRAG 2, DynamicRAG and T-Mem, by 10 to 31 points of accuracy. It matches an LLM reranker that makes 64 calls per query at 3.2 times lower serving cost, and on LoCoMo it outperforms Mem0 and Zep without any LLM call at write time. The factorization attributes the gains to slates that more often hold all of the evidence, so recommender-system design offers an effective and efficient foundation for the long-term memory of LLM agents.
Figures & tables
Figure 1. Agent memory as recommendation. Evidence for the request lies in two of 50 sessions, and a selector with no LLM call fills a slate of stored turns under a token budget B , from which the agent’s LLM answers. Orange labels give the recommender role of each element. A long-term memory of 50 dated sessions is drawn as a row of chat cards with user and assistant turns. Sessions 3 and 17 are highlighted because they hold the evidence, pottery on Tuesdays and moved to Thursdays. A user asks on June 20 which day the pottery class is now. AgentMemRec, which recalls, ranks and fills with no LLM call, selects a slate of at most B tokens that contains both evidence turns. An answering LLM reads the slate and answers Thursdays, which is judged correct. Orange labels mark the history as items, the request as the user, the context as the slate and the judged answer as feedback.
Figure 2. Long-term agent memory as budgeted slate recommendation. (a) Memory construction, recall, ranking and context assembly narrow the stored turns of a long-term memory to the agent’s context under a token budget B , and each is a recommender stage (orange). (b) The compared systems by their choice at each stage, where marks an LLM call and the last column summarizes the stages that call an LLM. AgentMemRec learns its ranker over cheap features and makes no LLM call at any stage. Two panels share four stage columns: write-time map, candidate generation, ranker and slate builder. Panel a shows a funnel that narrows from left to right. A request and a history enter it. The first segment holds the dated sessions of turns, among them two gold evidence units and some optional records. The second holds a smaller pool that still contains both gold units. The third holds a ranking with the gold units on top, and the fourth a slate under the token budget B. An answering LLM reads the slate, and the judged answer is the reward. Braces under the funnel state that the first two stages decide whether the pool keeps all evidence and that the ranker decides whether the slate keeps the pooled evidence. The slate builder fills the budget with the ranked turns that fit, and the judged answer gives the accuracy. Panel b lists the compared systems in three groups: LLM-built memories, context selectors and an LLM reranker. Each row has one chip per stage and a last column, set apart by a dashed line, that summarizes the stages at which the system calls an LLM, none for Adaptive-k and AgentMemRec. LongMemEval design, HippoRAG 2, A-Mem and T-Mem spend LLM calls on the write-time map, and HippoRAG 2 and A-Mem also at candidate generation. Adaptive-k and DynamicRAG use per-question cutoffs, and DynamicRAG and Ref call an LLM in the ranker. AgentMemRec uses turns with optional records, a fused lexical and dense pool of 128 units, a ranker learned over cheap features and a greedy fill, with no LLM call.
LongMemEval-S (50 sessions, 100k tokens per history)
LLM calls ↓
Accuracy (%) ↑
System
Write
Sel.
Tokens
Qwen
DeepSeek
Adaptive-k ( Taguchi et al., 2025 )
0
0
1,470
69.1
76.6
DynamicRAG-8B ( Sun et al., 2025 )
0
1
2,140
67.0
74.2
HippoRAG 2 ( Gutiérrez et al., 2025 )
442
1
2,520
59.9
63.2
A-Mem ( Xu et al., 2025 )
979
1
2,540
54.4
58.1
Table 1. Main results on the test questions of LongMemEval-S and LongMemEval-M with evidence in at least two sessions ( κ≥2 , n=209 each). Gains are paired margins. LLM calls are per question, at write time (Write), amortized over the histories each build covers, and at selection time (Sel.). Tokens are mean slate lengths. Best in bold, second underlined.
Figure 3. Accuracy on the LongMemEval-S test questions with evidence in at least two sessions, for Qwen (filled) and DeepSeek (hollow). (a) Every compared system by its LLM calls per question at write and selection time ( n=209 ). (b) The questions whose evidence spans three or more sessions. AgentMemRec reaches the accuracy of the 64-call reranker with no LLM call, and it is the most accurate system on the questions with the most scattered evidence. Two panels of accuracy on LongMemEval-S. Panel a plots accuracy against LLM calls per question: AgentMemRec at zero calls is as accurate as the reranker with 64 calls, and the LLM-built memories with hundreds to thousands of calls are lower. Panel b shows accuracy on questions whose evidence spans three or more sessions, where AgentMemRec has the highest accuracy for both answering models.
LongMemEval-S
System
Acc. ↑
Zep ∗ ( Rasmussen et al., 2025 )
63.8
RecMem ∗ ( Dai et al., 2026 )
69.2
TiMem ∗ ( Li et al., 2026a )
76.9
MemOS ∗ ( Li et al., 2025 )
77.8
AgentMemRec
83.2
Table 2. Accuracy (%) under the shared protocols of memory papers, with GPT-4o-mini answering. AgentMemRec scores above every reported system on LongMemEval-S and is at parity with our T-Mem reproduction on LoCoMo. ∗ Copied (Appendix D ), reprod. our reproduction, and † with the optional write-time records. Best in bold, second underlined.
Qwen
Acc. ↑
System
rslate↑
ac
ai
Qwen
DeepSeek
Adaptive-k
0.684
87.1
30.3
69.1
76.6
DynamicRAG
0.630
86.3
34.0
67.0
74.2
HippoRAG 2
0.502
89.5
30.0
59.9
63.2
A-Mem
0.502
86.2
22.4
54.4
58.1
Ref (64 calls)
0.915
83.8
36.6
79.8
89.1
Table 3. Stage factorization on LongMemEval-S test, κ≥2 ( n=209 ), accuracies in %, so Acc=ai+(ac−ai)rslate . Slate completeness separates the systems, and AgentMemRec has the most complete slates with no LLM call. Lower rows rank its 128-turn pool ( rpool=0.981 , against 0.938 for the 64-turn pool of Ref). Best in bold, second underlined.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Features
rslate , κ≥2
rslate , κ≥3
All ten
0.949
0.852
without turn context
0.815
0.541
retrieval signals only
0.813
0.541
cross-encoder only
0.676
0.352
key overlap only
0.644
0.219
turn context only
0.537
0.582
Appendix
Table 4. Feature-group ablation of the ranker (label-only, LongMemEval-S test, each variant retrained). The learned combination exceeds every single feature group by 13 to 41 points of slate completeness at κ≥2 .
κ=2 ( n=160 )
κ≥3 ( n=49 )
System
Qwen ↑
DeepSeek ↑
Qwen ↑
DeepSeek ↑
Adaptive-k
78.4
85.0
38.8
49.0
DynamicRAG-8B
73.6
81.2
45.4
51.0
HippoRAG 2
67.8
72.3
34.2
33.2
A-Mem
59.8
65.9
36.7
32.7
T-Mem
74.4
78.1
57.1
57.1
Appendix
Table 5. Accuracy (%) on the LongMemEval-S test questions by the number of evidence sessions. AgentMemRec is the most accurate system at κ≥3 , and at κ=2 it exceeds every memory and context-selection method by 6.1 to 27.0 points, all gains. Best in bold, second underlined.
Concerts
Festivals
Investment
System
Ev. ↑
OK ↑
Ev. ↑
OK ↑
Ev. ↑
OK ↑
Adaptive-k
2/5
0/4
3/6
0/4
0/1
0/4
DynamicRAG-8B
2/5
0/4
5/6
0/4
0/1
0/4
HippoRAG 2
0/5
0/4
3/6
0/4
0/1
0/4
A-Mem
1/5
0/4
4/6
2/4
0/1
0/4
T-Mem
own
0/1
own
1/1
own
0/1
Appendix
Table 6. Three LongMemEval-S test questions. Concerts : What is the order of the concerts and musical events I attended in the past two months, starting from the earliest? Festivals : How many movie festivals that I attended? Investment : I mentioned an investment for a competition four weeks ago? What did I buy? Ev. counts evidence turns in the 2,700-token slate, and OK counts correct slates per answering model (T-Mem answers once from its own memory). AgentMemRec keeps all evidence and answers correctly in all three, and bold marks every evidence turn kept and every slate correct.
Write
Sel.
Accuracy ↑
Δ
System
Pool / ranker
LLM calls
LLM calls
Tokens
Qwen
DeepSeek
Qwen
DeepSeek
Adaptive-k ( Taguchi et al., 2025 )
history / largest gap
0
0
1,180
57.8
60.8
+12.2 G
+20.6 G
DynamicRAG-8B ( Sun et al., 2025 )
fused 40 / listwise 8B
0
1
2,150
59.3
62.1
+10.6 G
+19.3 G
LongMemEval design ( Wu et al., 2025 )
rounds / dense, fact keys
251,622
0
2,530
56.3
60.3
+13.6 G
+21.1 G
native top-10, uncapped
rounds / dense, fact keys
251,622
0
4,945
64.1
65.6
+5.9 ref.
+15.8 ref.
HippoRAG 2 ( Gutiérrez et al., 2025 )
rounds / fact graph, PPR
≈ 512k
1
2,520
46.1
48.1
+23.9 G
+33.3 G
Appendix
Table 7. LongMemEval-M test, κ≥2 ( n=209 ), with the GPT-4o judge and four perturbed slates per question. Accuracies are in %. LLM calls are generations per question at selection time, or in total at write time, and tokens are approximate mean slate lengths. Δ is AgentMemRec minus the row in points, with its label: G gain, P parity, ref. a reference arm without a label. Best accuracy in bold, second underlined.
AgentMemRec
Ref
Ratio
LLM calls per request
0
64
n/a
LLM tokens per request
0
20,861
n/a
GPU time per request (ms)
280
891
3.2 ×
Dollars per million requests
69
220
3.2 ×
Median latency, S (ms)
277
924
3.3 ×
90th-percentile latency, S (ms)
321
1,156
3.6 ×
Appendix
Table 8. Selection-stage serving cost on one A100 80 GB GPU for the 209 LongMemEval-S test questions with κ≥2 . AgentMemRec makes no LLM call and needs about a third of the GPU time and latency of the 64-call reranker.
Long-term memory systems allow LLM agents to preserve information beyond a single context window, but most systems focus on storing and retrieving facts after extraction, leaving the write decision under-specified. What deserves memory can depend on the user's current task, topic, activity, or interaction partner, while uniform extraction applies one notion of importance across these different situations. We formulate this challenge as preference-conditioned write control and introduce AdaMem, which uses adaptive natural-language Memory Policies to personalize what an agent writes to memory. Each policy represents the user's memory preference for a particular interaction context, is updated from periodic feedback, and controls subsequent memory writing. We evaluate this loop in AdaMem-Bench, which assigns different memory preferences to six concurrent interaction personas across five ten-week stories. Across two extraction models and two feedback modes, AdaMem improves average QA accuracy over Mem0 from 80.0% to 84.35% while reducing persistent memory by 9.27%. Our analyses show that explicit feedback helps models learn better memory policies, but current models still struggle to translate those policies into reliably selective writing behavior. AdaMem thus demonstrates the promise of adaptive write control while exposing policy execution as a central limitation of current memory agents. Our code is publicly available: https://github.com/galaxyChen/AdaMem
Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.
Jing Yu, Yibo Zhao, Jiaming Zhang +1
School of Data Science and Engineering, East China Normal University
Large language model (LLM) agents work in scenarios with very different memory needs. A question about past conversations needs the exact facts it names, while a multi-step task needs a trace of the steps already taken together with details that the next step cannot name. Existing memories are each designed and tested for one kind of scenario. Retrieval memories such as A-Mem miss steps that the current query does not express, summaries drop exact strings, and pairing the two fixes the split of the budget in advance. We propose FlexA-Mem, a general memory that writes each unit of history once into an immutable store and renders a fresh memory for every call. The rendered memory is a compact timeline of records plus detail that a fixed rule fits into the remaining room without an LLM call. The detail is either relevant units as raw text or excerpts or the verbatim pieces that best match the query. With one store, FlexA-Mem serves both kinds of scenario. On AppWorld challenge tasks at 4k, it outperforms the second-best method by 11.5%, and on 147 unused tasks with predictions fixed in advance it outperforms A-Mem with a summary by 12.2%. On LongMemEval and LoCoMo, it matches a dedicated keyword retriever that writes no memory, while memories built on text written by an LLM fall far behind at tight budgets. Ablations show that the timeline carries AppWorld completion, whereas recovery mainly saves steps there and decides accuracy on exact facts. Code is available at https://github.com/WujiangXu/Flex-Amem
Wujiang Xu, Jiaojiao Han, Xi Zhu +5
Rutgers University · Independent Researcher · Stanford University +2