Organizations: School of Artificial Intelligence Jilin University Changchun, China · School of Cyber Science and Technology, University of Science and Technology of China Hefei, China · School of Information and Library Science, University of North Carolina at Chapel Hill Chapel Hill, North Carolina, USA
Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant-generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns, and inside a model. We study these gaps across six task families and five models. Degradation from fully specified single-turn input (FULL) to progressively revealed multi-turn interaction (SHARDED) is clearly task- and model-dependent, and stronger one-shot performance does not imply greater interaction robustness. We then retrospectively analyze completed SHARDED conversations by replaying the user messages already observed in each trajectory while editing only assistant-generated history. Replacing prior assistant responses with neutral content (termed neutralization) changes downstream min-max normalized performance by +.027 across 2,973 trajectories. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus +.068), showing that simple context shortening is insufficient to explain the effect of history editing. Turn Surgery further intervenes on one assistant turn at a time. Among 237 selected degraded trajectories, 63.7% contain at least one beneficial intervention, while most tested positions remain unchanged; for binary tasks, 48.4% admit a fail-to-success reversal. An open-weight case study links behaviorally consequential history changes to measurable downstream state differences, but finds task-dependent rather than universal internal signatures. Overall, assistant-generated history has active but selective effects on multi-turn performance, motivating selective rather than uniform history management.
Figures & tables
Figure 1. Motivation: self-generated assistant history as the future context. (a) Assistant responses become part of later context. (b) Performance often degrades from FULL to SHARDED , motivating whether this self-generated history contributes to multi-turn degradation.
Figure 2. Overview of our research work, which covers (1) degradation landscape, (2) assistant-generated history effects, (3) influential turns, and (4) internal model states.
Figure 3. Multi-turn degradation exhibits a structured task × model landscape. (A) Overall task-macro performance under FULL and SHARDED , showing the aggregate degradation from one-shot input to multi-turn interaction. (B) The corresponding FULL - SHARDED gaps across tasks and models, revealing heterogeneous degradation patterns. Error bars denote 95% paired-bootstrap confidence intervals; † indicates intervals including zero.
Figure 4. History effects persist under surface-length matching. (A) Effects of short and length-matched neutralization relative to SHARDED on the common subset of 264 trajectories. (B) Conditional binary outcome transitions under short neutralization among trajectories that initially succeed or fail under SHARDED . Error bars denote 95% paired-bootstrap confidence intervals.
Figure 5. Turn Surgery reveals broad but structured history sensitivity within the selected degraded trajectories. (A) Average surgery effects under two aggregation schemes. (B) Trajectory-level reach of positive interventions and fail-to-success reversals. (C) Turn-level positive and negative effects across normalized assistant-turn positions, where 0 and 1 correspond to the earliest and latest intermediate assistant turns within each trajectory, respectively. (D) Surgery effects across distance-to-final bins.
Figure 6. Internal changes accompany positive-effect history interventions, but their signatures are task dependent rather than universal. (A) Controlled state divergence between positive and no-change interventions. (B) State restoration profiles under held-out and prospective evaluations. (C) Held-out task-level outcomes across complementary probes for Actions, Code, and D2T. Symbols in panel C denote significant positive/negative effects or confidence intervals including zero.
In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses. We revisit this design choice by comparing full-context prompting to four alternative, substantially-reduced context configurations. Analyzing in-the-wild multi-turn conversations across three open reasoning and one state-of-the-art model, we find that response quality is largely preserved under aggressive context filtering: replacing all prior assistant turns with one-sentence summaries or keeping only the most recent user--assistant exchange often matches storing full context in performance while using roughly 8x less context. To understand this result, we observe that a substantial fraction of user turns (36.4%) in multi-turn conversations are self-contained and that many follow-up turns can be addressed by seeing only the immediately preceding user--assistant exchange. Furthermore, we find that when models condition on their own past responses, this can lead to context pollution, a phenomenon in which reasoning errors, hallucinations, or stylistic artifacts propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits the assistant-side history. Taken together, these findings suggest moving away from storing full dialogue transcripts and instead retaining only what is relevant.
Jenny Y. Huang, Leshem Choshen, Wei Sun +5
Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research
Users interacting with Large Language Models (LLMs) in a multi-turn conversation routinely refine their requests or pivot to new topics. LLMs, however, often miss these topic shifts and carry over irrelevant context from previous turns, leading to inaccurate responses. In this paper, we stress-test the multi-turn understanding of LLMs and study the following two sub-tasks: (1) detecting whether the user pivots or refines in the current turn, and (2) shortlisting relevant context from previous turns. To this end, we construct synthetic benchmarks based on real-world datasets from varied domains, as to simulate context shifts of different levels of difficulty. We then evaluate the zero-shot performance of ten LLMs (open-weight, closed-source and reasoning), and demonstrate that only some reasoning and strongly instructed LLMs are accurate in detecting pivots; open-weight LLMs struggle with the task and frequently carry stale context even with explicit cues; and all models suffer from a position bias. Based on the results, we discuss key takeaways for improving long-term robustness in multi-turn capabilities for LLMs.
Large Language Model (LLM) interactions are typically underspecified, with users clarifying all necessary details across multiple conversational turns. Yet recent work shows that LLMs perform far worse in this multi-turn setting than in a single turn with same information being available at once, a phenomenon termed "Lost-in-Conversation." However, bridging this gap effectively remains an open problem. Here we introduce Found in Conversation (FiC), a training framework where a model teaches itself to find and recover its single-turn competence given underspecified multi-turn prompts. We develop View-Asymmetric Self-Distillation, which distills across two views of the same task information--single-turn view for the teacher, multi-turn view for the student--transferring strong single-turn behavior into weak multi-turn behavior. This requires no stronger external teacher, which is unavailable as even frontier LLMs exhibit this gap. Across model families (Llama, Qwen, Phi, and OLMo) and sizes (3B-14B), FiC recovers at least 92% of single-turn performance and reaches 100% on two Llama backbones, yielding more efficient and helpful multi-turn conversations with single-turn capabilities intact.