Do Personalized LLMs Need the Whole Conversation? Matching the Full History with Only 5% of Its Text via Relevance-Filtered Context

Erika Hayashi; Andrew Shin

cs.CLOct 10, 2026

Do Personalized LLMs Need the Whole Conversation? Matching the Full History with Only 5% of Its Text via Relevance-Filtered Context

Authors: Erika Hayashi, Andrew Shin

Organizations: Keio University

Abstract

Personalized assistants accumulate long conversation histories, and passing the whole history to an LLM with every request is costly. We show that it is also unnecessary. On PersonaMem-v2 (180 questions), with GPT-5.6-terra and Claude Opus 5.5 as judges, a relevance-filtered context, built by sentence-level retrieval that ranks the history by similarity to the current request and stops at a fixed budget, matches full-history accuracy with only 5% of the text, cutting the judge's input about 20-fold. An LLM summary of the same length performs on par, but it needs an LLM to read the entire history, whereas retrieval needs only a small embedding model. While both sentence-level retrieval and LLM summary perform strongly, retrieval does so while placing the relevant evidence in the context for only 61% of the questions, and an oracle given only that evidence beats the full history by 15.6 points, making better context selection a promising path to surpassing the full history.

Figures & tables

Figure 1: Overview of the compared contexts. The history is turned into a compact context of

B

characters by (a) sentence-level retrieval of parts relevant to the current request ( relevance-filtered context ) or (b) an LLM summary of the whole history.

				GPT-5.6-terra		Claude Opus 5.5
Context	Chars	% full	Built by	Acc.	Gain recovered	Acc.	Gain recovered
No context	30	0.0	—	20.0	0.00	21.7	0.00
Full history (target)	156,594	100	—	43.3	1.00	54.4	1.00
Head slice, 8k	7,473	4.8	—	30.0	0.43	31.1	0.29
Head slice, 50%	77,636	49.6	—	36.1	0.69	48.3	0.81
Retrieval, 2k	1,893	1.2	embedding model	37.2	0.74	46.7	0.76

Table 1: Accuracy (%) and gain recovered (

n=180

; chance 25%) with GPT-5.6-terra and Claude Opus 5.5 as judges (

n=180

; chance 25%). Summaries are written by GPT-5.6-terra. Chars: mean characters given to the judge. Gain recovered:

(\mathrm{Acc}-\mathrm{Acc}_{\text{none}})/(\mathrm{Acc}_{\text{full}}-\mathrm{Acc}_{\text{none}})

. Bold: best non-oracle context. Details: Appendix B .

Request	What’s the most effective way to tackle early signs of aging like fine lines and uneven skin tone?
Correct response	Since you prefer modern synthetic skincare over Ayurvedic remedies, a targeted routine with a gentle chemical exfoliant like glycolic acid, a peptide-infused moisturizer, and daily broad-spectrum SPF can be highly effective.
Retrieval, 8k	[…] USER: I reached for the cool, luminous serum from my bathroom shelf — the one in the minimalist frosted bottle […] and smoothed it across my face. […] USER: It made me think about how, despite the pull of heritage remedies and nostalgia, I’ve always been drawn to the polished precision of modern formulations. […]
Summary, 8k	## Preferences […] The user is highly engaged with visual art, especially mixed media, layered compositions, tactile surfaces, found imagery, textiles, reclaimed materials, gold leaf, and the relationship between physical texture and memory […]

Table 2: Excerpts of the two compact contexts for one anti-stereotypical question. Bold: sentences that also appear in the annotated evidence. The retrieved sentences contain the user’s preference for modern formulations over heritage remedies, whereas the summary, organized around the user’s art interests, does not mention the serum or this preference. Both judges answered correctly with the retrieval context and incorrectly with the summary context (excerpts rebuilt with the same procedure).

Figure 2: Accuracy against the characters given to the judge. Retrieval and summaries with about 5% of the history reach the full history (dashed), whereas head slices keep improving up to it. Error bars: 95% bootstrap intervals.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Context	GPT-5.6-terra	Claude Opus 5.5
No context	$-$ 23.3 [ $-$ 30.0, $-$ 16.7]	$-$ 32.8 [ $-$ 40.6, $-$ 25.0]
Evidence only	+15.6 [8.3, 23.3]	+15.6 [7.2, 23.9]
Head slice, 50%	$-$ 7.2 [ $-$ 13.3, $-$ 0.6]	$-$ 6.1 [ $-$ 12.8, 0.0]
Retrieval, 2k	$-$ 6.1 [ $-$ 13.3, 1.1]	$-$ 7.8 [ $-$ 16.7, 1.1]
Retrieval, 8k	$-$ 2.8 [ $-$ 10.0, 3.9]	0.0 [ $-$ 7.8, 7.8]
Summary, 2k	$-$ 6.7 [ $-$ 14.4, 1.1]	$-$ 12.8 [ $-$ 21.1, $-$ 4.4]

Appendix

Table 3: Paired accuracy difference (points) from the full history, with 95% bootstrap intervals over questions (

n=180

Context	Chars	% full	GPT-5.6- terra	Claude Opus 5.5
No context	30	0.0	20.0	21.7
Evidence only (oracle)	1,780	1.1	58.9	70.0
Full history	156,594	100	43.3	54.4
Head slice, 2k	1,350	0.9	24.4	30.0
Head slice, 8k	7,473	4.8	30.0	31.1
Head slice, 25%	38,490	24.6	33.3	42.2

Appendix

Table 4: Accuracy (%) of all conditions (

n=180

) with GPT-5.6-terra and Claude Opus 5.5 as judges. Bold: the highest value among non-oracle contexts.

		GPT-5.6-terra					Claude Opus 5.5
Type	$n$	None	Evid.	Full	Retr. 8k	Summ. 8k	None	Evid.	Full	Retr. 8k	Summ. 8k
Stereotypical	30	16.7	63.3	46.7	53.3	66.7	16.7	83.3	53.3	56.7	66.7
Anti-stereotypical	30	3.3	60.0	46.7	50.0	36.7	3.3	70.0	56.7	50.0	33.3
Neutral	30	6.7	66.7	26.7	33.3	40.0	10.0	80.0	40.0	46.7	60.0
Therapy background	30	33.3	46.7	46.7	30.0	56.7	33.3	60.0	50.0	56.7	56.7
Health and medical	30	13.3	36.7	20.0	20.0	20.0	10.0	50.0	36.7	43.3	26.7

Appendix

Table 5: Accuracy (%) by preference type (30 questions each) for GPT-5.6-terra and Claude Opus 5.5. None: no context; Evid.: evidence only ; Full: full history; Retr. 8k: retrieval with 8k characters; Summ. 8k: GPT-5.6-terra summary with 8k characters.

Domain	Units	Summary $-$ truncation
Entertainment	17	$-$ 29.4 [ $-$ 44.1, $-$ 14.7]
Food	17	$-$ 35.3 [ $-$ 50.0, $-$ 20.6]
Health	70	$-$ 38.1 [ $-$ 49.5, $-$ 26.7]
Sports	15	$-$ 6.7 [ $-$ 23.3, +10.0]
Technology	16	$-$ 31.2 [ $-$ 46.9, $-$ 15.6]
Travel	17	$-$ 35.3 [ $-$ 55.9, $-$ 14.7]

Appendix

Table 6: Earlier experiments (English conversations, questions written in Japanese): accuracy difference (points) between summaries and truncation at the same length of 2,000 characters, with the judges GPT-5.6-terra and Claude Haiku 4.5 pooled, and 95% bootstrap intervals over persona units.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Do LLMs Benefit From Their Own Words?
Feb 27, 2026Jenny Y. Huang, Leshem Choshen, Wei Sun +5LLM Context Management Context Compression
Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research
LANTERN: Layered Archival and Temporal Episodic Retrieval Network for Long-Context LLM Conversations
Apr 18, 2026Rahul SubramaniMemory-Augmented Language Models Persistent Memory for Language Models
Cisco Systems, Inc.
PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations
Sep 9, 2026Hyojeong Yu, Hyukhun Koh, Minsung Kim +2Conversational Memory Persistent Memory for Language Models
Dept. of ECE, Seoul National University · IPAI, Seoul National University

Feb 27, 2026J/K move · Enter open · S save

Do LLMs Benefit From Their Own Words?

Jenny Y. Huang, Leshem Choshen, Wei Sun, Omar Khattab+4

Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research

LLM Context Management Context Compression

Open paper AskPDF arXiv

In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses. We revisit this design choice by comparing full-context prompting to four alternative, substantially-reduced context configurations. Analyzing in-the-wild multi-turn conversations across three open reasoning and one state-of-the-art model, we find that response quality is largely preserved under aggressive context filtering: replacing all prior assistant turns with one-sentence summaries or keeping only the most recent user--assistant exchange often matches storing full context in performance while using roughly 8x less context. To understand this result, we observe that a substantial fraction of user turns (36.4%) in multi-turn conversations are self-contained and that many follow-up turns can be addressed by seeing only the immediately preceding user--assistant exchange. Furthermore, we find that when models condition on their own past responses, this can lead to context pollution, a phenomenon in which reasoning errors, hallucinations, or stylistic artifacts propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits the assistant-side history. Taken together, these findings suggest moving away from storing full dialogue transcripts and instead retaining only what is relevant.

Abstract

LLM Context Management

Explore similar work

Do LLMs Benefit From Their Own Words?

LANTERN: Layered Archival and Temporal Episodic Retrieval Network for Long-Context LLM Conversations

PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations