A conversational agent that remembers has to decide what to keep. Retrieval-augmented generation keeps the transcript and searches it; the alternative is to compact each turn into atomic facts and search those. We compare the two on LoCoMo, with both arms answering all 1,985 questions in one window, against the same stores, with the same generator and judge, and both predictions on one row, which licenses McNemar's exact test on the 512 questions where they disagree. On the four categories comparable work reports, compaction scores 71.4 per cent against 64.6, a lead of 6.8 points (p = 5.5e-08) at 34 per cent fewer prompt tokens. On all five, including the adversarial set comparable work drops, the lead falls to 2.0 and stops being significant, because the baseline leads adversarial by 14.6. That loss has a structure. Our lead falls as the reference answer demands more source wording, 9.6 points ahead at one or two reference words and 3.1 behind past eight, with token overlap moving the same way, so the judge is not the cause. Compaction keeps what a fact asserts and discards how it was said, so it wins where the answer is a value and loses where it is a phrasing. We also report four negative results with their power calculations.
Figures & tables
Figure 1: The two paths through the system. Write path across the top, read path across the bottom, the two storage tiers in the middle. The dotted return arrow is reinforcement: reading a fact rewrites it.
Category
n
NMAFC
RAG
Lead
Won/lost
McNemar p
Context
single-hop
281
50.9 [45.1, 56.7]
45.6 [39.8, 51.4]
+5.3
51/36
0.133
980 vs 1,486
temporal
321
71.7 [66.5, 76.3]
46.1 [40.7, 51.6]
+25.5
101/19
1.1e-14
943 vs 1,422
multi-hop
96
57.3 [47.3, 66.7]
44.8 [35.2, 54.7]
+12.5
15/3
0.0075
961 vs 1,410
open-domain
841
79.8 [76.9, 82.4]
80.3 [77.4, 82.8]
-0.5
71/75
0.804
967 vs 1,461
adversarial
446
35.9 [31.6, 40.4]
50.4 [45.8, 55.1]
-14.6
38/103
4.1e-08
960 vs 1,486
scored (4)
1,539
71.4 [69.1, 73.6]
64.6 [62.2, 66.9]
+6.8
238/133
5.5e-08
964 vs 1,454
Table 1: Paired LoCoMo result, 1,985 questions, both arms in one window against the same stores. Intervals are Wilson 95 per cent. Discordant counts are questions won and lost. Context is mean retrieved tokens.
Reference length
n (all 5)
Lead, all 5
p
n (scored 4)
Lead, scored 4
p
1 to 2 words
683
+6.0
0.0027
532
+9.6
1.2e-05
3 to 4 words
498
+0.8
0.781
375
+7.5
0.0023
5 to 8 words
510
+2.5
0.305
404
+8.2
0.0017
9 or more words
294
-6.1
0.057
228
-3.1
0.410
Table 2: The paired lead as a function of reference-answer length. The split is on the reference, so it is fixed before either arm runs.
Figure 2: Left, the paired lead by reference-answer length, on all five categories and on the scored four. Right, mean token-overlap F1 against the reference by category, which has no model in the loop.
Parameter
Value
Role
lambda_core_anchor
0.0
core anchors never decay
lambda_active_context
0.05
base decay rate per turn
lambda_ephemeral
0.69
base decay rate per turn
eta
0.15
consolidation constant
beta
0.0
clustering protection, ablation control
w_prune
0.1
demotion threshold
Table 3: Shipped configuration. Every parameter that differs from zero or from a library default is listed. Values marked as ablation controls are inactive at their defaults and exist so that the corresponding mechanism can be switched off and attributed.
Figure 3: Decay under the shipped constants. Left, the three memory types at k=0 . Centre, the effect of retrieval count on an active-context fact. Right, one fact through three retrievals; each resets the weight to 1 and lowers the rate for the next interval.
Paired questions
Expected discordant
Smallest detectable lead
108
27
9.4 points
214
54
6.7 points
322
80
5.5 points
830
208
3.4 points
1,539
385
2.5 points
1,985
496
2.2 points
Table 4: Smallest lead detectable by a two-sided paired test at α=0.05 , at the discordance rate of 0.25 observed here.
Figure 4: Discordant questions per category. Bars to the right are questions only NMAFC answered correctly, bars to the left only the baseline. Agreement counts and exact p-values are on the right. The information in the agreements is zero, which is the argument for pairing.
Conversation
n
NMAFC
RAG
Lead
Won/lost
p
conv-26
199
68.3
58.8
+9.5
42/23
0.025
conv-30
105
57.1
55.2
+1.9
16/14
0.856
conv-41
193
66.3
60.6
+5.7
33/22
0.177
conv-42
259
56.4
59.5
-3.1
30/38
0.396
conv-43
242
60.3
61.6
-1.2
27/30
0.791
conv-44
158
64.6
58.9
+5.7
25/16
0.211
Table 5: Paired accuracy by conversation, all five categories. Won and lost are the discordant counts, and p is McNemar exact. No single conversation is significant after any correction for ten comparisons, which is the expected result at these sample sizes given the minimum detectable effects above.
Figure 5: Accuracy by category with Wilson 95 per cent intervals, plotting Table 1 . Significance markers are McNemar exact on the paired discordant counts, so they do not follow from whether the intervals overlap.
Figure 6: Left, the open-domain accuracy and context of three retrieval widths against the baseline. Right, all five categories; each arrow runs from the baseline’s position to ours, so an arrow pointing up and to the left is a win on both axes.
Figure 7: Latency percentiles on a log scale, and the distribution clipped at ten seconds. The arms separate only above the ninetieth percentile. Both maxima near 93.8 seconds are the provider’s retry ladder rather than either system’s work.
Question
Reference
NMAFC
RAG
What is Melanie excited about in her adoption process?
creating a family for kids who need one
inspired by her friend’s positive experience
making a family for kids who need one
What did Caroline and her family do while camping?
explored nature, roasted marshmallows, and went on a hike
Caroline did not go camping with her family
explored nature, roasted marshmallows, hiked
What motivated Melanie to pursue counseling?
her own journey and the support she received, and how counseling improved her life
Melanie is not pursuing counseling
her own journey and the support she got
What inspired Melanie’s painting for the art show?
visiting an LGBTQ center and wanting to capture unity and strength
nature and sunsets
the LGBTQ center visit
How did Caroline’s son handle the accident?
He was scared but reassured by his family
Caroline does not have a son mentioned in the facts
He was scared but reassured and explained he would be OK.
What does Jon’s tattoo symbolize?
Freedom and expressing himself through dance
No information available
freedom, dancing without worrying what people think
Table 6: Adversarial questions the baseline answered and we did not. Rows one to four show wording drift, where our answer is defensible and shares almost nothing with the reference. Rows five and six show the other failure, where retrieval missed and our arm declined.
Question
Reference
NMAFC
RAG
When did Melanie paint a sunrise?
2022
2022
last year
When is Melanie planning on going camping?
June 2023
June 2023
next month
When did Caroline give a speech at a school?
The week before 9 June 2023
early June 2023
the week before 25 May 2023
How long ago was Caroline’s 18th birthday?
10 years ago
ten years ago
Not mentioned in the excerpts.
When did Caroline have a picnic?
The week before 6 July 2023
late June or early July 2023
yesterday
When did Caroline go to the LGBTQ conference?
10 July 2023
10 July 2023
two days ago on 4 July 2023
Table 7: Temporal questions we answered and the baseline did not. The baseline’s answers are frequently the relative phrase from the retrieved chunk, unresolved.
Mechanism
n
Variant
Shipped
Won/lost
p
list-shape gate, wide retrieval
214
36.4
36.9
9/10
1.0
short-phrase clause, all
1,984
63.4
63.4
72/72
1.0
short-phrase clause, adversarial
445
41.1
35.7
41/17
0.0022
short-phrase clause, temporal
321
67.6
71.7
2/15
0.0024
exact-wording clause, both
767
49.5
50.8
30/40
0.282
exact-wording clause, adversarial
446
36.1
35.9
28/27
1.0
Table 8: Three of the four mechanisms of Section 5.6, with the paired counts behind each. Won and lost are discordant questions counted for the variant against the shipped setting, and p is McNemar exact. The weight signal is excluded because it was measured as answer reachability rather than as a paired accuracy test.
File
Contents
data/locomo_all.json
1,985 rows, one per question, both arms’ predictions, correctness, retrieved-context size and latency
data/od_ship.json
841 open-domain rows at the shipped retrieval width
data/od_ground.json
841 open-domain rows with the grounding term active
data/od_wide.json
841 open-domain rows at ten hydrated turns
data/lg_ship.json
214 rows where the list-shape detector fires, shipped width
data/lg_wide.json
214 rows where the list-shape detector fires, wide
Table 9: The files released with the paper for recomputing every number and figure in it. The scripts share no state and none of them contacts a model.
Language-model agents compact their context for the question at hand, but the compacted context must also serve questions that come later. We evaluate compaction on these later questions, using four corpora that annotate the evidence of several questions per text and recorded agent trajectories, and compare every method with random deletion at the same budget. On MultiRC, keeping the sentences that gpt-6-luna (through the OpenAI Decisions API) judges necessary for the current question raises its accuracy on that question by 11.4 points at 25% of the tokens. On the later questions, however, no method that selects sentences for the current question does detectably better than deleting sentences at random. Deleting words instead of sentences does better: LLMLingua-2, which ignores the question, beats random sentence deletion at the same token count by 12.5 points on the later MultiRC questions. It keeps about twice random deletion's share of the evidence's content words, and at equal content-word retention its gain shrinks to 7.3 points (exploratory). On FEVER, label accuracy can overstate what the kept text supports: the gain of sentence selection is no longer detected when answers are scored against the kept evidence, and the 21.0-point gain of LLMLingua-2 is not shown to rest on the kept text. The ranking also depends on the setting: on Evidence Inference sentence selection beats LLMLingua-2 at a nominal 25% budget, and on τ-bench trajectories, whose later tool calls need exact identifiers, it beats random deletion while LLMLingua-2 shows no detected gain. Scores on the question a compaction was made for do not reliably show how it serves later ones.
Existing conversational memory systems rely on complex hierarchical summarization or reinforcement learning to manage long-term dialogue history, yet remain vulnerable to context dilution as conversations grow. In this work, we offer a different perspective: the primary bottleneck may lie not in memory architecture, but in the Signal Sparsity Effect within the latent knowledge manifold. Through controlled experiments, we identify two key phenomena: Decisive Evidence Sparsity, where relevant signals become increasingly isolated with longer sessions, leading to sharp degradation in aggregation-based methods; and Dual-Level Redundancy, where both inter-session interference and intra-session conversational filler introduce large amounts of non-informative content, hindering effective generation. Motivated by these insights, we propose \method, a minimalist framework that brings conversational memory back to basics, relying solely on retrieval and generation via Turn Isolation Retrieval (TIR) and Query-Driven Pruning (QDP). TIR replaces global aggregation with a max-activation strategy to capture turn-level signals, while QDP removes redundant sessions and conversational filler to construct a compact, high-density evidence set. Extensive experiments on multiple benchmarks demonstrate that \method achieves robust performance across diverse settings, consistently outperforming strong baselines while maintaining high efficiency in tokens and latency, establishing a new minimalist baseline for conversational memory.
Yuqian Wu, Wei Chen, Zhengjun Huang +5
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · National University of Singapore
Long conversations with an AI agent create a simple problem for one user: the history is useful, but carrying it verbatim is expensive. We study personalized agent memory: one user's conversation history with an agent, distilled into a compact retrieval layer for later search. Each exchange is compressed into a compound object with four fields (exchange_core, specific_context, thematic room_assignments, and regex-extracted files_touched). The searchable distilled text averages 38 tokens per exchange. Applied to 4,182 conversations (14,340 exchanges) from 6 software engineering projects, the method reduces average exchange length from 371 to 38 tokens, yielding 11x compression. We evaluate whether personalized recall survives that compression using 201 recall-oriented queries, 107 configurations spanning 5 pure and 5 cross-layer search modes, and 5 LLM graders (214,519 consensus-graded query-result pairs). The best pure distilled configuration reaches 96% of the best verbatim MRR (0.717 vs 0.745). Results are mechanism-dependent. All 20 vector search configurations remain non-significant after Bonferroni correction, while all 20 BM25 configurations degrade significantly (effect sizes |d|=0.031-0.756). The best cross-layer setup slightly exceeds the best pure verbatim baseline (MRR 0.759). Structured distillation compresses single-user agent memory without uniformly sacrificing retrieval quality. At 1/11 the context cost, thousands of exchanges fit within a single prompt while the verbatim source remains available for drill-down. We release the implementation and analysis pipeline as open-source software.