In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses. We revisit this design choice by comparing full-context prompting to four alternative, substantially-reduced context configurations. Analyzing in-the-wild multi-turn conversations across three open reasoning and one state-of-the-art model, we find that response quality is largely preserved under aggressive context filtering: replacing all prior assistant turns with one-sentence summaries or keeping only the most recent user--assistant exchange often matches storing full context in performance while using roughly 8x less context. To understand this result, we observe that a substantial fraction of user turns (36.4%) in multi-turn conversations are self-contained and that many follow-up turns can be addressed by seeing only the immediately preceding user--assistant exchange. Furthermore, we find that when models condition on their own past responses, this can lead to context pollution, a phenomenon in which reasoning errors, hallucinations, or stylistic artifacts propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits the assistant-side history. Taken together, these findings suggest moving away from storing full dialogue transcripts and instead retaining only what is relevant.
Figures & tables
Figure 1: Real example of overrelying on irrelevant context. In Turn 2, the user requests UMAP clustering code. In Turn 5, the user says, “use t-SNE instead.” Left: When the previous assistant response remains in context, the model incorrectly carries over the Jaccard metric from UMAP into the t-SNE implementation, introducing a bug. Right: Without the previous response in context, the model generates correct t-SNE code.
Figure 2: Pairwise win rates between Full Context (FC) prompting and three reduced-context configurations across four models (Qwen3-4B, DeepSeek-R1-Distill-Llama-8B, GPT-OSS-20B, and GPT-5.2), evaluated on the Quality and On-Topic dimensions. Top row: FC vs. summarized assistant history (SUM), where each prior assistant response is replaced with a one-sentence summary. Second row: FC vs. Last Turn Only (LT), which retains only the most recent user–assistant round. Third row: FC vs. Assistant Omitted (AO), which retains all user turns but replaces assistant responses with a placeholder. Bottom row: FC vs. Delete Everything (DE), which provides only the current user prompt. Error bars indicate binomial proportion 95% confidence intervals.
Figure 3: Three example conversations on WildChat illustrating different types of follow-ups. Left: the user provides feedback that is concrete enough that the model can respond from scratch using the previous user turn and the updated specifications alone. Middle: the follow-ups reference specific parts in past assistant turns, making it necessary to see the referenced assistant turn. Right: the follow-up references a prior user turn without concrete feedback; no assistant history is necessary.
Figure 4: Left: Distribution of conversation rounds referenced by user prompts from round 2 onward, across 150 WildChat conversations. The X-axis represents the number of rounds back from the current round that a given prompt references. The red stem at 0 indicates prompts that are self-contained (no prior rounds referenced). Middle: Mean cumulative context length across conversations on WildChat as a function of conversation round, comparing Full Context, Assistant Omitted, Last Turn Only , and Summarized context. Shaded band shows ± 2 standard errors. Right: The number of conversations that can be answered using a 5K-token budget as a function of conversation round.
Figure 5: Left: Accuracy (%) on multi-turn GSM8K under Full Context (FC) and Assistant Omitted (AO) context. Right: Confusion matrices showing the number of problems answered correctly across models and context configurations.
Assistant-response History
Filler History
Category
Count
Conf.
Count
Conf.
Intent Override
39
0.74
0
0.96
Incorrect Code Carryover
30
0.77
0
0.95
Factual Hallucination Carryover
27
0.75
0
0.95
Reasoning Trace Carryover
6
–
–
–
Formula Carryover
1
–
–
–
Table 1: Prevalence of context pollution instances across 729 responses generated by GPT-5.2 on WildChat conversations, under the original history and a history in which assistant-side responses are replaced with length-matched filler text. Confidence values indicate the LLM annotator’s average confidence across instances in that category.
Figure 6: Ratio of adaptive wins to full-context-only wins vs. mean token count (in thousands) for operating points obtained by sweeping the decision rule’s threshold τ and cost weight λ . The green triangle marks a heuristic that omits assistant responses on all turns categorized as “New Asks.”
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Config
Qwen3-4B
DeepSeek-R1
GPT-OSS-20B
GPT-5.2
Sum
≥
≥
≥
≥
LT
≥
≥
∼
∼
AO
∼
∼
DE
Appendix
Table 2: On-Topic: comparison of reduced-context configurations against Full Context (FC), per model. ≥ indicates that reduced context performs significantly better than or is statistically indistinguishable from FC (95% CIs overlap); ∼ indicates that the win-rate gap is within 10pp.
Config
Qwen3-4B
DeepSeek-R1
GPT-OSS-20B
GPT-5.2
Sum
≥
≥
≥
LT
≥
≥
∼
AO
∼
DE
Appendix
Table 3: Quality: comparison of reduced-context configurations against Full Context (FC), per model. ≥ indicates that the reduced context performs significantly better than or is statistically indistinguishable from FC (95% CIs overlap); ∼ indicates that the win-rate gap is within 10pp.
Figure 8: Pairwise win rates by prompt category (new ask, follow up with feedback, follow up without feedback) for Qwen3-4B (top) and GPT-5.2 (bottom), comparing Full-Context (FC) and Assistant-Omitted (AO) responses. Stars indicate statistically significant differences. Error bars indicate binomial proportion 95% confidence intervals.
Figure 9: Pairwise win rates between Full Context (FC) and three reduced-context configurations across all four models (Qwen3-4B, DeepSeek-R1-Distill-Llama-8B, GPT-OSS-20B, and GPT-5.2), evaluated on the Quality and On-Topic dimensions. Top row: FC vs. Last Turn Only (LT), which retains only the most recent user–assistant round. Middle row: FC vs. Assistant Omitted (AO), which retains all user turns but replaces assistant responses with a placeholder. Bottom row: FC vs. Delete Everything (DE), which provides only the current user prompt. Error bars indicate binomial proportion 95% confidence intervals.
Figure 10: Pairwise win rates between Full Context (FC) and three reduced-context configurations across all four models (Qwen3-4B, DeepSeek-R1-Distill-Llama-8B, GPT-OSS-20B, and GPT-5.2), evaluated on the Quality and On-Topic dimensions. Top row: FC vs. Last Turn Only (LT), which retains only the most recent user–assistant round. Middle row: FC vs. Assistant Omitted (AO), which retains all user turns but replaces assistant responses with a placeholder. Bottom row: FC vs. Delete Everything (DE), which provides only the current user prompt. Error bars indicate binomial proportion 95% confidence intervals.
Figure 11: Pairwise win rates between Full-Context (FC) and Assistant-Omitted (AO) responses when the judge evaluates only final answers. Compare with Fig. 9 , in which the judge receives both thinking traces and final answers.
Model
Fraction of Turns ≥ FC
Tokens Saved
GPT-5.2 (self-generated context)
0.916
37.7%
Qwen3-4B (self-generated context)
0.915
49.2%
DeepSeek-R1 (historical context)
0.875
49.4%
GPT-OSS-20B (historical context)
0.927
43.5%
Appendix
Table 4: Fraction of turns meeting or exceeding the full-context (FC) baseline, and the corresponding token savings, across four models. Parentheses labels indicate whether the conversation history was self-generated or drawn from historical logs.
Feature
Coef.
SE
95% CI
PromptPC3
−0.2003
0.1200
[−0.4355,0.0350]
HistoryPC17
−0.1606
0.1056
[−0.3677,0.0464]
HistoryPC2
−0.1412
0.1320
[−0.1176,0.4000]
category_new_ask
−0.1308
0.1098
[−0.3460,0.0843]
PromptPC24
−0.0963
0.0941
[−0.0881,0.2807]
HistoryPC26
−0.0920
0.0880
[−0.0805,0.2644]
Appendix
Table 5: Logistic regression coefficients with 95% confidence intervals. This table displays the 20 features with the largest absolute coefficient magnitudes.
Users interacting with Large Language Models (LLMs) in a multi-turn conversation routinely refine their requests or pivot to new topics. LLMs, however, often miss these topic shifts and carry over irrelevant context from previous turns, leading to inaccurate responses. In this paper, we stress-test the multi-turn understanding of LLMs and study the following two sub-tasks: (1) detecting whether the user pivots or refines in the current turn, and (2) shortlisting relevant context from previous turns. To this end, we construct synthetic benchmarks based on real-world datasets from varied domains, as to simulate context shifts of different levels of difficulty. We then evaluate the zero-shot performance of ten LLMs (open-weight, closed-source and reasoning), and demonstrate that only some reasoning and strongly instructed LLMs are accurate in detecting pivots; open-weight LLMs struggle with the task and frequently carry stale context even with explicit cues; and all models suffer from a position bias. Based on the results, we discuss key takeaways for improving long-term robustness in multi-turn capabilities for LLMs.
Large Language Model (LLM) interactions are typically underspecified, with users clarifying all necessary details across multiple conversational turns. Yet recent work shows that LLMs perform far worse in this multi-turn setting than in a single turn with same information being available at once, a phenomenon termed "Lost-in-Conversation." However, bridging this gap effectively remains an open problem. Here we introduce Found in Conversation (FiC), a training framework where a model teaches itself to find and recover its single-turn competence given underspecified multi-turn prompts. We develop View-Asymmetric Self-Distillation, which distills across two views of the same task information--single-turn view for the teacher, multi-turn view for the student--transferring strong single-turn behavior into weak multi-turn behavior. This requires no stronger external teacher, which is unavailable as even frontier LLMs exhibit this gap. Across model families (Llama, Qwen, Phi, and OLMo) and sizes (3B-14B), FiC recovers at least 92% of single-turn performance and reaches 100% on two Llama backbones, yielding more efficient and helpful multi-turn conversations with single-turn capabilities intact.
Personalized assistants accumulate long conversation histories, and passing the whole history to an LLM with every request is costly. We show that it is also unnecessary. On PersonaMem-v2 (180 questions), with GPT-5.6-terra and Claude Opus 5.5 as judges, a relevance-filtered context, built by sentence-level retrieval that ranks the history by similarity to the current request and stops at a fixed budget, matches full-history accuracy with only 5% of the text, cutting the judge's input about 20-fold. An LLM summary of the same length performs on par, but it needs an LLM to read the entire history, whereas retrieval needs only a small embedding model. While both sentence-level retrieval and LLM summary perform strongly, retrieval does so while placing the relevant evidence in the context for only 61% of the questions, and an oracle given only that evidence beats the full history by 15.6 points, making better context selection a promising path to surpassing the full history.