Agentic Context Management for Multi-Turn Human-AI Conversations

Matthew Ho; Chenglong Wang; Baolin Peng; Nebojsa Jojic; Philippe Laban; Michel Galley; Jianfeng Gao; Lianhui Qin

cs.AIOct 31, 2026

Agentic Context Management for Multi-Turn Human-AI Conversations

Authors: Matthew Ho, Chenglong Wang, Baolin Peng, Nebojsa Jojic, Philippe Laban, Michel Galley, Jianfeng Gao, Lianhui Qin

Organizations: University of California, San Diego · Microsoft Research

Abstract

LLMs degrade in multi-turn conversation partly because of context pollution: the model's own earlier outputs stay in context and anchor later turns, so an early misreading of the user's intent can persist after the user corrects it. Assistant omission (AO), dropping every assistant message, removes this pollution but fails when interaction is referential: later turns refer back to drafts, decisions, and tool-call results that exist only in assistant turns. We propose Agentic Conversation Context Curation (AC3), a training-free method in which a separate analyzer (1) consolidates the task specification from the user's messages alone, (2) audits the assistant's prior outputs against it, and (3) edits the context, either appending the analysis or discarding the assistant side and rebuilding the context from the specification. A second finding: pollution is contagious. An analyzer that sees the assistant's reasoning while extracting the specification can do worse than no intervention, and instructing it to ignore them did not close the gap, so we exclude assistant turns structurally. On Lost in Conversation (replaying difficulty-selected instances), where AO is an oracle by construction, AC3 closes 55-84% of the gap between full context and AO on three of four tasks and exceeds AO on the fourth, and an LLM that faithfully summarizes the history does not reproduce the gain even with more compute. On real WildChat conversations, a GPT-5-mini judge prefers AC3's next turn over both full context and AO in every configuration of our main table, though this preference depends on the judge; on CollabLLM results are mixed. In stateful tool use (tau2-bench), AO collapses to 0% on every model; AC3 does not collapse, though it falls slightly behind full context. Detector studies show that Reset removes injected pollution but is not span-selective: the gain comes from rebuilding the specification.

Figures & tables

Figure 1: The value of omitting assistant turns falls as task referentiality grows. Results for one respondent (gpt-5.4) on the four benchmarks, ordered by how much the next user turn depends on the assistant’s prior turns. Values are taken from Table 2 (LiC, CollabLLM, tau2-bench) and Table 3 (WildChat); the AC3 bar is the best operator for this respondent on each benchmark (on WildChat, best against AO). On LiC, where the user’s messages together fully specify the task, AO is strong and AC3 is higher still. On CollabLLM, AC3-Reset (62.5) is one problem out of 40 ahead of AO (60.0) and three ahead of full context (55.0). On WildChat, a GPT-5-mini judge prefers AC3-Reset’s next turn over both AO and full context (dashed line: parity). On tau2-bench, where tool-call results live only in assistant turns, AO collapses to 0% and AC3 does not collapse, but full context remains best.

Figure 2: Illustrative example. (a) Context pollution: an early incorrect interpretation by the assistant (a zero prepaid balance) stays in the context and re-anchors a later response even after the user has corrected it. (b) Coarse omission: AO removes the invalidated reasoning but also the state the next action needs (the restart the agent already recommended), so the agent repeats itself. The deficit is behavioral rather than a lack of capability: on LiC, the same models solve the task when it is posed in a single turn ( Laban et al., 2025 ) . Our method (Section 3 , Figure 3 ) supplies the missing control at the harness level by rebuilding the conversation around a specification drawn from the user’s messages and dropping the invalidated reasoning.

Figure 3: Agentic Conversation Context Curation (AC3). Before the assistant generates, a separate analyzer runs two analysis subtasks and one intervention step. Subtask 1 (specification extraction) sees only user turns, so it cannot be biased by the assistant’s prior reasoning. Subtask 2 (approach evaluation) compares that specification against the assistant’s full history. An operator (Augment, Reset, or Rewrite; the figure shows a rewritten context) then edits the context; the “retained” content in the rebuilt context is restated by the analyzer, not copied from assistant turns. Structural exclusion in Subtask 1 is essential: when the extractor sees assistant turns, accuracy falls below the full-context baseline on math and code (Table 4 ).

Prior work & baselines
Strategy	Math $\uparrow$	Code $\uparrow$	Database $\uparrow$	Actions $\uparrow$
Baseline (full context)	60.0	15.8	4.0	34.8
+ Memory	55.0	21.1	4.0	34.8
AO ( Huang et al., 2026 ) ⋄	85.0	73.7	32.0	82.6
Concat User ( Laban et al., 2025 ) ⋄	80.0	68.4	32.0	87.0
ERGO ( Khalid et al., 2025 ) §	80.0	43.9	12.0	43.5–52.2

Table 1: Per-task LiC results (GPT-5-mini, accuracy %, replay mode). Instances are difficulty-selected (those the full-context baseline most often failed), so absolute accuracies are low by construction and are not estimates of GPT-5-mini’s multi-turn accuracy. ⋄ Design oracles: single-turn by construction. ‡ Mean of

N{=}3

runs; other rows are single trials. § Interval; see Appendix B.1.4 , which also gives pools, denominators, and paired tests.

Panel of Table 2 — Table 2: Results across respondents on LiC, CollabLLM, and tau2-bench. Benchmark-averaged accuracy (LiC: four tasks on a second difficulty-selected pool, Section 5.1 ; CollabLLM: MATH-Hard and BigCodeBench) or task reward (tau2-bench telecom_small , mean of $N{=}3$ seeds). Respondents: gpt-5.4, dsv4f (DeepSeek-V4-Flash), k2.6 (Kimi-K2.6). AC3-Reset has no turn floor or reset cap; ∗ Gated Reset, the only Reset variant run on tau2-bench.

Panel of Table 3 — Table 3: WildChat: pairwise win rates (%) by respondent. Each cell is the rate at which a GPT-5-mini judge prefers the AC3 response over assistant omission (AO) or full context (FC); 50% is parity. —: not reported. Turn sets and judge checks are in Appendix B.6 .

Configuration	Math	Code	Database
AC3-Augment	80%	53%	32%
$-$ structural exclusion	40%	11%	8%
$-$ subtask decomposition	55%	21%	4%
Full context (no analysis)	60%	16%	4%

Table 4: Progressive ablation of the analyzer (Augment, no memory; single trials on the Table 1 pools). Row 2 lets the specification extractor see assistant messages; row 3 also collapses the two subtasks into a single pass.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 5 — Table 5: CollabLLM per-task accuracy (MATH-Hard / BigCodeBench, $n{=}20$ per cell except k2.6 Reset MATH-Hard, $n{=}19$ ), with all BigCodeBench cells execution-scored in one sandbox. ‡ Served through OpenRouter (Appendix B.2 ); unmarked cells were served through Azure. Baseline is full context. The two-task averages are the CollabLLM block of Table 2 .

Model	Strategy	Math	Code	Database	Actions	Avg. $\Delta$
GPT-5-mini	Baseline	20%	10%	5%	60%
GPT-5-mini	Reset	55%	25%	40%	80%	+26pp
GPT-5	Baseline	40%	21%	27%	85%
GPT-5	Reset	70%	47%	65%	70%	+20pp
DeepSeek V3.2	Baseline	50%	7%	10%	35%
DeepSeek V3.2	Reset	80%	62%	45%	70%	+39pp

Appendix

Table 6: Multi-model LiC results on a hard subset selected via high baseline failure rate by GPT-5.2. Reset improves accuracy for 15 of 16 model-task pairs across four models from three families.

Category	Count	Description
Task too hard	4	Agent never tries the right fix
Step budget / Hard persona	4	Correct approach but runs out of turns
Context pollution	1	Agent stuck in repetitive loop
User sim issue	1	User stopped after first message
Premature transfer	1	Agent transfers instead of troubleshooting

Appendix

Table 7: Baseline failure-mode breakdown (11 failures; GPT-5-mini, one trial). One author’s reading without a rubric or a second annotator, so a hypothesis rather than a measured taxonomy.

Condition	Math	Code	Database
Augment
Soft attention (no memory)	40%	13%	31%
Soft attention + memory	56%	14%	46%
Hard attention	80%	40%	54%
Reset
Soft attention (no memory)	42%	23%	31%

Appendix

Table 8: Soft- vs. hard-attention results on held-out test splits (

n{\approx}7

–13 per cell). Some math and code cells use a per-run false-negative adjustment that we do not use elsewhere and that favors reset arms, so compare within a row family only. Memory closes up to 65% of the gap to hard attention on database.

DeepSeek V3.2 (eval split)
Condition	vs AO	vs FC	$\Delta$ vs AO
Reset	72.5%	72.5%	—
+ memory	86.3%	80.4%	+13.8pp
GPT-5-mini (online)
Reset	90.8%	86.8%	—
+ memory	86.7%	80.0%	$-$ 4.1pp

Appendix

Table 9: WildChat memory ablation. Indented + memory rows add the memory-based learner to the Reset row above. Memory improves DeepSeek’s win rate by +13.8pp when there is headroom; GPT-5-mini’s near-ceiling baseline leaves no room for improvement. For DeepSeek, memory was learned from one-third of WildChat turns and frozen for evaluation; GPT-5-mini uses the online regime. These rows use the evaluation split (DeepSeek) or the online regime (GPT-5-mini), so they are not directly comparable to Table 3 .

Turn	User message	FC vs AO	GR vs AO	GR vs FC
2	“Why do I keep getting error…”	AO	GR	GR
4	“I am using Codespaces”	FC	GR	GR
6	“Now I get: [new error]”	FC	GR	GR
8	“Still getting [same error]”	FC	GR	GR
10	“I do not have a bashrc file”	AO	GR	GR
12	[Shares GitHub Actions YAML]	AO	GR	GR

Appendix

Table 10: Per-turn winners for the WildChat Maven debugging conversation. GR wins all 14 pairwise comparisons. On context-dependent turns (4, 6, 8), AO fails because it cannot connect the user’s references to prior discussion.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 256 papers

Do LLMs Benefit From Their Own Words?
Feb 27, 2026Jenny Y. Huang, Leshem Choshen, Wei Sun +5LLM Context Management Context Compression
Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research
Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap
May 23, 2026Tianlang Chen, Shirley Wu, Jure LeskovecLanguage Model Distillation Teacher-Student Learning
Stanford
What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction
Sep 5, 2026Jinnan Li, Zheren Fu, Yue Wang +3LLM Context Management Causal Interventions in Language Models
School of Artificial Intelligence Jilin University Changchun, China · School of Cyber Science and Technology, University of Science and Technology of China Hefei, China · School of Information and Library Science, University of North Carolina at Chapel Hill Chapel Hill, North Carolina, USA

Feb 27, 2026J/K move · Enter open · S save

Do LLMs Benefit From Their Own Words?

Jenny Y. Huang, Leshem Choshen, Wei Sun, Omar Khattab+4

Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research

LLM Context Management Context Compression

Open paper AskPDF arXiv

In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses. We revisit this design choice by comparing full-context prompting to four alternative, substantially-reduced context configurations. Analyzing in-the-wild multi-turn conversations across three open reasoning and one state-of-the-art model, we find that response quality is largely preserved under aggressive context filtering: replacing all prior assistant turns with one-sentence summaries or keeping only the most recent user--assistant exchange often matches storing full context in performance while using roughly 8x less context. To understand this result, we observe that a substantial fraction of user turns (36.4%) in multi-turn conversations are self-contained and that many follow-up turns can be addressed by seeing only the immediately preceding user--assistant exchange. Furthermore, we find that when models condition on their own past responses, this can lead to context pollution, a phenomenon in which reasoning errors, hallucinations, or stylistic artifacts propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits the assistant-side history. Taken together, these findings suggest moving away from storing full dialogue transcripts and instead retaining only what is relevant.

Abstract

LLM Context Management

Explore similar work

Do LLMs Benefit From Their Own Words?

Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap

What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction