Context-Agent: Dynamic Discourse Trees for Non-Linear Dialogue

Junan Hu; Shudan Guo; Wenqi Liu; Jianhua Yin; Yinwei Wei

cs.CLApr 7, 2026

Context-Agent: Dynamic Discourse Trees for Non-Linear Dialogue

Authors: Junan Hu, Shudan Guo, Wenqi Liu, Jianhua Yin, Yinwei Wei

Organizations: Shandong University, China

Abstract

Large Language Models demonstrate outstanding performance in many language tasks but still face fundamental challenges in managing the non-linear flow of human conversation. The prevalent approach of treating dialogue history as a flat, linear sequence is misaligned with the intrinsically hierarchical and branching structure of natural discourse, leading to inefficient context utilization and a loss of coherence during extended interactions involving topic shifts or instruction refinements. To address this limitation, we introduce Context-Agent, a novel framework that models multi-turn dialogue history as a dynamic tree structure. This approach mirrors the inherent non-linearity of conversation, enabling the model to maintain and navigate multiple dialogue branches corresponding to different topics. Furthermore, to facilitate robust evaluation, we introduce the Non-linear Task Multi-turn Dialogue (NTM) benchmark, specifically designed to assess model performance in long-horizon, non-linear scenarios. Our experiments demonstrate that Context-Agent enhances task completion rates and improves token efficiency across various LLMs, underscoring the value of structured context management for complex, dynamic dialogues. The dataset and code is available at GitHub.

Figures & tables

Figure 1: A schematic diagram of linear (upper) vs. non-linear (lower) dialogue flow.

Method	Structure	Construction Basis	Retrieval Unit	Local Coherence	Update Efficiency
Linear & Compression Methods
Full Context	Linear Sequence	Token Concatenation	Entire History	High	Very Low ( $O(N^{2})$ )
MemGPT	OS-like Hierarchy	Event-Triggered/Function	Paginated Memory	High (Self-Edit)	Medium
Retrieval-Augmented Generation (RAG)
Standard RAG	Flat Index	Semantic Similarity	Indep. Chunks	Low (Disjointed)	High
DH-RAG	Chain	Semantic Clustering	Query Chains	High (Dynamic)	Medium (Incremental)

Table 1: Comparison of context management paradigms. We compare our method with linear methods, standard RAG, advanced RAG, and tree-based memory.

Figure 2: An overview of the Context-Agent framework. It illustrates the dynamic evolution of a multi-turn dialogue represented as a forest of topic trees, with branches indicating sub-dialogue paths. The number in each node represents the turn number in the conversation. Solid edges represent the active path, while dashed edges indicate inactive paths.

Figure 3: A 15-turn NTM dialogue example on trip planning, featuring topic shifts and instruction refinements. The right panel lists checkpoint questions for objective task completion evaluation. See Appendix A.6 for details.

Dataset	Avg. Turns	Max Turns	Total turns	Non-linear Evolution
Multichallenge	5	10	1365	No
MT-Eval	7	14	1170	No
MT-Bench-101	3	7	4208	No
NTM (Ours)	17	27	6931	Yes

Table 2: Comparison of NTM with existing multi-turn dialogue datasets.

Model	Open Source	Context Window
GPT-4.1	$\times$	1000k
DeepSeek-V3	$\checkmark$	64k
GLM-4-Plus	$\times$	128k
Llama 3.1-70B	$\checkmark$	128k

Table 3: Details of the LLMs used

Model	Method	TCR (%) $\Uparrow$	TCR Gain(%)	ACT $\Downarrow$				ACT Drop (%)
Model	Method	TCR (%) $\Uparrow$	TCR Gain(%)	10-turn	15-turn	20-turn	25-turn	ACT Drop (%)
GPT-4.1	Full-History	86.0	–	4070	6382	9535	12803	–
	Truncation	55.2	-35.8	1839	2378	2981	3142	–
	Context-Agent	88.9	+3.4	2108	2894	4137	6227	-52.3
DeepSeek-V3	Full-History	64.3	–	3540	5428	7805	10693	–
	Truncation	42.8	-33.4	1732	2088	2535	2883	–

Table 4: Main Results on Context Management Efficiency and Effectiveness. Performance on our proposed NTM Benchmark (Task-Oriented) across varying dialogue lengths. TCR : Task Completion Rate; ACT : Average Context Tokens. Context-Agent consistently outperforms baselines.

Method	EM(Exact Match)	F1 Score	ACT
Full-History	13.3	25.2	4261
Truncation	7.1	12.8	1703
Context-Agent	16.2	28.9	2435

Table 5: Result of Llama 3.1-70B on TopiOCQA.

Panel of Figure 4 — Figure 4: (a) TCR comparison across different methods and models. (b) A typical example of context tokens change trend in a 20-turn dialogue.

Figure 5: Trade-off between TCR and ACT, where the ideal point is the top-left corner (high TCR, low ACT).

Method	TCR (%)	TCR Drop (%)
Full-History	64.3	-
w/o Tree	41.5	-35.5
w/o RAG	45.3	-29.5
Context-Agent	69.3	+7.8

Table 6: Ablation study results on DeepSeek-V3.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Average Response Time(s)	Relative Increase(%)
Full-History	12.5	-
Context-Agent	13.5	+8.0%

Appendix

Table 7: Average response time for different context management methods on a 20-turn dialogue.

Figure 13

Figure 14

Figure 15

Figure 6: The topic tree structure corresponding to the dialogue example in Figure 3 . Each node represents a turn in the dialogue, with branches indicating topic shifts and refinements. The solid edges represent the active path, while the dashed edges represent inactive branches.

Figure 7: An example of a 15-turn dialogue from the NTM benchmark in the coding support domain. The dialogue features multiple topic shifts and instruction refinements, culminating in a clear task of generating a Python calculator function.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
Jun 10, 2026Yeongseo Jung, Jaehyeok Kim, Eunseo Jung +5Truncated Backpropagation Through Time Memory-Augmented Language Models
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan +1
Conversation Tree Architecture: A Structured Framework for Context-Aware Multi-Branch LLM Conversations
Mar 22, 2026Pranav Hemanth, Sampriti SahaLLM Context Management
Pandemonium Research
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
May 14, 2026Renning Pang, Tian Lan, Leyuan Liu +3Memory-Augmented Language Models Long-Term Conversational Memory
University of Electronic Science and Technology of China

Jun 10, 2026J/K move · Enter open · S save

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

Yeongseo Jung, Jaehyeok Kim, Eunseo Jung, Jiachuan Wang+4

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan+1

Truncated Backpropagation Through Time Memory-Augmented Language Models Long-Context Modeling LLM Context Management LLM Context Compression

Open paper AskPDF arXiv

Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues. We revisit the context compression under conversational dynamics and empirically present its fragility. To improve both efficiency and robustness, we introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve, revise, and write-back loop shares information across turns and updates stale memories, stabilizing long-horizon behavior. In addition, we adapt truncated backpropagation-through-time (TBPTT) to our multi-turn setting, learning cross-turn dependencies without full-history backpropagation. Extensive experiments on long-form dialogue benchmarks demonstrate superior performance and efficiency of C-DIC; notably, C-DIC shows stable inference latency and perplexity over hundreds of dialogue turns, supporting a scalable path to high-quality dialogue modeling.

Abstract

LLM Context Management

Explore similar work

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

Conversation Tree Architecture: A Structured Framework for Context-Aware Multi-Branch LLM Conversations

Improving Multi-turn Dialogue Consistency with Self-Recall Thinking