TPBench: A Turning-Point Benchmark for Dialogue Compression

Minji Park; Seunghyun Yoon; Hyuk Lim

cs.CLOct 2, 2026

TPBench: A Turning-Point Benchmark for Dialogue Compression

Authors: Minji Park, Seunghyun Yoon, Hyuk Lim

Organizations: Korea Institute of Energy Technology (KENTECH)

Abstract

A compressor can keep the facts of a dialogue and still drop the turn that changed them. A user corrects a price, reverses a choice, or adds a constraint. We call this failure turning-point eviction. One overall retention score hides it, because that score mixes what the user first wanted with what the user wants now. We introduce TPBench, which evaluates three complementary information targets at shared nominal retention budgets. P1 asks for the user's initial goal. P2 asks for the current value of a slot the user revised. P3 asks for both, in dialogues with a late annotated slot update. The current-value answers come from the human dialogue-state annotations of MultiWOZ and SGD. The initial-goal answer is the first sentence of the first user turn. Neither requires new crowdsourcing. The probe-specific evaluations rank compression methods differently. On the joint probe at a retained fraction of 0.30, every tested compressed method remains below full context with the main Llama reader. Deleting the turn that carries the update sharply lowers current-value accuracy, while deleting one matched irrelevant turn leaves it unchanged. A Mistral reader repeats the P2/P3 rankings and the joint-probe gap. Current-value recovery is tested on an additional corpus, LongMemEval-KU, and on Chinese RiSAWOZ: full context has the highest accuracy, and recency has the highest compressed-method mean in both evaluations.

Figures & tables

Panel of Figure 1 — Figure 1: An illustrative dialogue separates the three probes. P1 needs the initial goal, P2 needs the current value, and P3 needs both. On the left, the heavy box is the turning point, the user turn that revises the slot. On the right, each box is one compressed context; a check mark means the probe can be answered from it and a cross means it cannot. The heavy box on the right is the only context that answers P3.

Probe	Question	Reference answer
P1	What was the user’s initial goal?	first sentence of the first user turn
P2	What is the current value of slot $s$ ?	value after the final update to $s$
P3	What is the task, and what is current?	first-turn goal and value after the final update

Table 1: The three TPBench probes. The Question column paraphrases each prompt. Appendix A gives the prompts verbatim.

Comparison	Reader	Examples and budget	Scoring
Main turn selectors	Llama	Three sampled sets; P1/P3 $r=.30$ , P2 $r=.10,.30$	P1 lexical; P2 value; P3 segment-local joint
P1 semantic check	Llama answers, GPT-5.4-mini judge	Same three sampled sets; $r=.30$ ; six selectors	AI-assisted full equivalence
First user + recent	Llama	Same three sampled sets; $r=.30$	Matched generation run; same probe scorers
Deletion controls	Llama	Seed-42 P2; paired turn deletion	P2 strict/loose; superseded-value diagnostic
Cache methods	Llama	Seed 42; cache $r=.30$	Four methods on P2; ChunkKV on P3
Cross-reader	Mistral	Same three P2/P3 sampled sets	Same probe scorers

Table 2: Experiment scope. Main seeds sample probe sets. LongMemEval-KU and RiSAWOZ instead keep the eligible set fixed and vary random selection. Turn fractions and cache fractions have different physical units.

	SGD $r{=}0.30$				MultiWOZ $r{=}0.30$
Method	P1 s/l	P2	P3 l	P3 s	P1 s/l	P2	P3 l	P3 s
full context	$\mathit{0.487\,/\,0.850}$	$\mathit{0.691}$	$\mathit{0.728}$	$\mathit{0.652}$	$\mathit{0.430\,/\,0.867}$	$\mathit{0.781}$	$\mathit{0.505}$	$\mathit{0.475}$
recency	$0.010\,/\,0.135$	$0.254$	$0.067$	$0.065$	$0.002\,/\,0.092$	$0.125$	$0.040$	$0.040$
random selection	$0.145\,/\,0.530$	$0.278$	$0.238$	$0.162$	$0.147\,/\,0.543$	$\mathbf{0.314}$	$0.145$	$0.128$
earliest turns	$0.480\,/\,0.877$	$0.156$	$0.313$	$0.160$	$\mathbf{0.410}\,/\,\mathbf{0.867}$	$0.296$	$0.058$	$0.048$
uniform stride	$\mathbf{0.493}\,/\,0.875$	$0.283$	$\mathbf{0.432}$	$\mathbf{0.288}$	$0.407\,/\,0.852$	$0.286$	$\mathbf{0.220}$	$\mathbf{0.178}$

Table 3: Main Llama results at

r{=}0.30

, averaged over seeds

\{42,43,44\}

. P1/P3 use

200

examples per dataset and seed; P2 uses

\{128,125,132\}

SGD and

\{161,168,168\}

MultiWOZ examples. P1 columns show strict / loose scores, P2 is strict, and P3 l and P3 s are the loose and strict joint scores of Section 3 . Italic denotes full context; bold denotes the best compressed method in each column.

Panel of Figure 2 — Figure 2: Rank changes among the seven compressed turn selectors at $r{=}0.30$ . Rank 1 is best and ties take the average rank. Each line follows one method from P1-strict through P1-loose to P2-strict. Solid lines are the four selectors that use position alone and dashed lines are the three that use dialogue content, as defined in Section 5 . A line that falls steeply between P1-loose and P2-strict marks a method that keeps the opening but loses the revision. Table 3 reports the corresponding accuracies.

Method	SGD	MultiWOZ
full context (uncompressed)	$\mathit{0.482}$	$\mathit{0.180}$
recency	$0.018$	$0.007$
random selection	$0.232$	$0.082$
earliest turns	$0.472$	$\mathbf{0.172}$
uniform stride	$0.458$	$0.168$
attention/H2O proxy	$\mathbf{0.480}$	$0.162$

Table 4: AI-assisted full semantic equivalence of initial-goal answers at

r{=}0.30

, averaged over three seeds (

200

examples per dataset and seed). This analysis covers six selectors and full context. LLMLingua-2 is outside the judged set. Italic denotes full context; bold denotes the best compressed method in each column.

Method	Answer retained
full context (uncompressed)	$\mathit{0.988}$
LLMLingua-2	$\mathbf{0.509}$
embedding MMR	$0.422$
attention/H2O proxy	$0.429$
uniform stride	$0.472$
earliest turns	$0.422$

Table 5: Fraction of MultiWOZ P2 examples whose current value remains in the compressed dialogue (

r{=}0.30

n{=}161

, seed

42

). Italic denotes full context; bold denotes the best compressed method in each column.

	SGD			MultiWOZ
Method	P2-strict	Retained	Read $\|$ kept	P2-strict	Retained	Read $\|$ kept
full context (uncompressed)	$\mathit{0.691}$	$\mathit{1.000}$	$\mathit{0.691}$	$\mathit{0.781}$	$\mathit{0.988}$	$\mathit{0.790}$
recency	$0.254$	$0.309$	$0.781$	$0.125$	$0.217$	$0.573$
random selection	$0.278$	$0.372$	$0.744$	$\mathbf{0.314}$	$\mathbf{0.505}$	$0.613$
earliest turns	$0.156$	$0.211$	$0.667$	$0.296$	$0.411$	$0.696$
uniform stride	$0.283$	$\mathbf{0.402}$	$0.664$	$0.286$	$0.481$	$0.584$

Table 6: P2-strict, answer retention, and accuracy on the retained examples at

r{=}0.30

, averaged over seeds

\{42,43,44\}

. An answer is retained when the complete current value occurs at phrase boundaries in the compressed dialogue, and “Read

|

kept” scores only those examples. Italic denotes full context; bold denotes the highest compressed value in the first two columns of each corpus.

Method	Time	Date	Place and name	Other
recency	$0.255\,/\,0.347$	$0.216\,/\,0.299$	$0.252\,/\,0.243$	$0.316\,/\,0.331$
random selection	$\mathbf{0.269}\,/\,0.346$	$0.239\,/\,0.303$	$0.340\,/\,0.396$	$0.273\,/\,0.489$
earliest turns	$0.100\,/\,0.092$	$0.157\,/\,0.185$	$0.269\,/\,0.332$	$0.128\,/\,0.327$
uniform stride	$0.231\,/\,0.390$	$\mathbf{0.387}\,/\,0.453$	$0.279\,/\,0.358$	$0.228\,/\,0.398$
attention/H2O proxy	$0.072\,/\,0.063$	$0.141\,/\,0.157$	$0.152\,/\,0.234$	$0.116\,/\,0.255$
embedding MMR	$0.219\,/\,0.269$	$0.299\,/\,0.307$	$\mathbf{0.417}\,/\,0.482$	$\mathbf{0.372}\,/\,0.430$

Table 7: SGD P2-strict and answer retention by slot type at

r{=}0.30

, averaged over seeds

\{42,43,44\}

. Each entry reads P2-strict / retention, and the four groups together cover every example. Bold denotes the highest P2-strict in each column.

Method	SGD P3	MultiWOZ P3
full context	$\mathit{0.645}$	$\mathit{0.478}$
earliest turns	$0.162$	$0.048$
recency	$0.065$	$0.040$
first USER + recent	$0.247$	$0.137$
uniform stride	$\mathbf{0.287}$	$\mathbf{0.178}$

Table 8: Matched first-user-plus-recent comparison at

r{=}0.30

, three seeds, scored with the segment-local P3-strict rule. This experiment uses the same

200

examples per seed as Table 3 but a separate generation run. Its full-context values differ from that table by at most

0.007

. Italic denotes full context; bold denotes the best compressed method in each column.

	P3-loose		P3-strict
Reader / corpus	Full	Best	Full	Best
Llama SGD	$0.700$	$0.437$ (U)	$0.601$	$0.262$ (U)
Llama MultiWOZ	$0.518$	$0.222$ (U)	$0.502$	$0.189$ (U)
Mistral SGD	$0.310$	$0.256$ (U)	$0.260$	$0.143$ (U)
Mistral MultiWOZ	$0.165$	$0.086$ (L)	$0.151$	$0.083$ (L)

Table 9: Saved-output P3 sensitivity after excluding pairs accepted by the P2 alias heuristic. SGD denominators are

152/147/147

and MultiWOZ

181/183/180

for seeds

42/43/44

. U denotes uniform stride and L the LLMLingua-2. “Best” is the highest mean among seven turn selectors, selected separately for each threshold.

	SGD				MultiWOZ
Method	Segment	Whole	Parsed	Seg. $\|$ parsed	Segment	Whole	Parsed	Seg. $\|$ parsed
full context	$\mathit{0.652}$	$\mathit{0.658}$	$\mathit{0.977}$	$\mathit{0.667}$	$\mathit{0.475}$	$\mathit{0.497}$	$\mathit{0.945}$	$\mathit{0.503}$
recency	$0.065$	$0.075$	$0.870$	$0.075$	$0.040$	$0.058$	$0.885$	$0.045$
random selection	$0.162$	$0.182$	$0.967$	$0.167$	$0.128$	$0.148$	$0.907$	$0.142$
earliest turns	$0.160$	$0.172$	$0.953$	$0.168$	$0.048$	$0.085$	$0.835$	$0.058$
uniform stride	$\mathbf{0.288}$	$\mathbf{0.298}$	$0.973$	$\mathbf{0.296}$	$\mathbf{0.178}$	$\mathbf{0.222}$	$0.967$	$\mathbf{0.184}$

Table 10: P3-strict on Llama under the two scorers, averaged over seeds

\{42,43,44\}

(

200

examples per dataset and seed). “Segment” is the scorer used in the main tables and “Whole” applies both criteria to the complete answer. “Parsed” is the share of answers the segment parser reads, and “Seg.

|

parsed” scores only those. Italic denotes full context; bold denotes the best compressed method in each accuracy column.

Condition	SGD	MultiWOZ
full context	$\mathit{0.672}$	$\mathit{0.801}$
remove update turn	$0.508$	$0.391$
remove matched single turn	$0.672$	$0.801$
remove update turn and all answer-bearing turns	$0.070$	$0.019$
remove matched control turns	$0.648$	$0.811$

Table 11: Strict P2 accuracy in the paired deletion experiment, on the seed-

42

P2 sets. The single-turn comparison uses

128

SGD and

161

MultiWOZ examples. The comparison that removes the update turn together with all answer-bearing turns uses

128

and

159

, respectively. The displayed full-context row uses the single-turn sample; all-evidence comparisons pair their last two rows.

Panel of Figure 3 — Figure 3: Deleting the update turn moves the reader from the current value to the value it replaced, and the matched control does not. Each column is one deletion condition, red marks the turn removed, and the heavy boxes mark the condition whose answers change. Rates are MultiWOZ P2 at seed $42$ ( $n{=}161$ ) from Tables 12 and 13 .

Condition	SGD	MultiWOZ
full context	$\mathit{0.672~/~0.695}$	$\mathit{0.801~/~0.820}$
remove update turn	$0.508~/~0.531$	$0.391~/~0.422$
remove matched single turn	$0.672~/~0.695$	$0.801~/~0.814$
remove update turn and all answer-bearing turns	$0.070~/~0.078$	$0.019~/~0.019$
remove matched control turns	$0.648~/~0.672$	$0.811~/~0.830$

Table 12: P2 deletion experiment on seed

42

, reported as strict / loose accuracy. SGD uses

128

examples throughout. MultiWOZ uses

161

for the first three rows and

159

for the last two, because two items have no valid matched control for the all-evidence comparison. The displayed full-context row uses the single-turn sample.

	SGD		MultiWOZ
Condition	Abstain	Old value	Abstain	Old value
full context	$\mathit{0.000}$	$\mathit{0.094}$	$\mathit{0.000}$	$\mathit{0.062}$
$-$ update turn	$0.008$	$0.234$	$0.000$	$0.286$
matched control	$0.000$	$0.094$	$0.000$	$0.062$
$-$ update and answer-bearing turns	$0.008$	$0.445$	$0.013$	$0.484$
matched control	$0.000$	$0.102$	$0.000$	$0.044$

Table 13: What the reader answers when evidence is deleted, on the seed-

42

P2 sets. Abstention is the rate of explicit no-answer replies and old value uses normalized exact matching against the replaced value. SGD uses

n=128

; MultiWOZ uses

n=161

, and

n=159

for the last two rows.

MultiWOZ
Method	$r{=}0.50$	$r{=}0.70$
full context (uncompressed)	$\mathit{0.779\pm 0.041}$ / $\mathit{0.805\pm 0.027}$	$\mathit{0.779\pm 0.041}$ / $\mathit{0.805\pm 0.027}$
recency	$0.348\pm 0.052$ / $0.390\pm 0.051$	$0.583\pm 0.021$ / $0.608\pm 0.021$
random selection	$0.493\pm 0.011$ / $0.553\pm 0.006$	$0.607\pm 0.042$ / $0.650\pm 0.041$
earliest turns	$0.527\pm 0.005$ / $0.575\pm 0.020$	$\mathbf{0.741}\pm 0.035$ / $0.759\pm 0.029$
uniform stride	$\mathbf{0.575}\pm 0.046$ / $0.702\pm 0.057$	$0.640\pm 0.029$ / $0.670\pm 0.027$

Table 14: P2 strict / loose accuracy at

r\in\{0.50,0.70\}

, mean

\pm

standard deviation over three seeds. The three samples contain

\{128,125,132\}

SGD and

\{161,168,168\}

MultiWOZ examples. Full context does not use the budget, so its row repeats across the two columns. Italic denotes full context; bold denotes the best compressed method in each column within each corpus.

Setting	ChunkKV	SnapKV	PyramidKV	StreamingLLM	Best turn selector
SGD P2 $r{=}0.30$	$0.133$	$0.469$	$\mathbf{0.492}$	$0.297$	$0.297$ (MMR/random)
SGD P3-strict $r{=}0.30$	$0.140$	—	—	—	$0.280$ (uniform stride)
MultiWOZ P2 $r{=}0.30$	$0.093$	$\mathbf{0.689}$	$0.683$	$0.584$	$0.329$ (LLMLingua-2)
MultiWOZ P3-strict $r{=}0.30$	$0.085$	—	—	—	$0.195$ (uniform stride)

Table 15: Strict accuracy for KV-cache methods at

r{=}0.30

, seed

42

. The P3 rows use the segment-local scorer. Higher is better. Bold denotes the best KV-cache method in each row with more than one entry. The last column repeats the best turn selector on the same seed; MMR/random denotes a tie between embedding MMR and random selection. A dash marks a condition we did not evaluate.

Chunk length	P2-strict	P2-loose	Abstain	Malformed
$10$	$0.087$	$0.099$	$0.323$	$0.124$
$20$	$0.087$	$0.093$	$0.342$	$0.199$
$40$	$0.050$	$0.056$	$0.311$	$0.478$

Table 16: ChunkKV chunk length on the MultiWOZ P2 set at

r{=}0.30

(

n{=}161

, seed

42

Method	SGD $r{=}0.10$	SGD $r{=}0.30$	MultiWOZ $r{=}0.10$	MultiWOZ $r{=}0.30$
full context (uncompressed)	$\mathit{0.626\pm 0.055}$	$\mathit{0.626\pm 0.055}$	$\mathit{0.765\pm 0.025}$	$\mathit{0.765\pm 0.025}$
recency	$0.000\pm 0.000$	$0.254\pm 0.013$	$0.006\pm 0.006$	$0.123\pm 0.019$
random selection	$0.120\pm 0.019$	$0.273\pm 0.031$	$\mathbf{0.135\pm 0.009}$	$\mathbf{0.311\pm 0.040}$
earliest turns	$0.015\pm 0.007$	$0.148\pm 0.012$	$0.028\pm 0.007$	$0.265\pm 0.010$
uniform stride	$0.094\pm 0.016$	$0.278\pm 0.019$	$0.058\pm 0.030$	$0.267\pm 0.046$
attention/H2O proxy	$0.016\pm 0.008$	$0.115\pm 0.021$	$0.024\pm 0.010$	$0.217\pm 0.013$

Table 17: Mistral P2-strict accuracy, mean

\pm

standard deviation over three seeds. Italic denotes full context; bold denotes the best compressed method in each column.

	SGD $r{=}0.30$		MultiWOZ $r{=}0.30$
Method	P3-loose	P3-strict	P3-loose	P3-strict
full context (uncompressed)	$\mathit{0.342\pm 0.055}$	$\mathit{0.303\pm 0.045}$	$\mathit{0.158\pm 0.020}$	$\mathit{0.142\pm 0.023}$
recency	$0.033\pm 0.012$	$0.033\pm 0.012$	$0.025\pm 0.010$	$0.020\pm 0.010$
random selection	$0.150\pm 0.013$	$0.103\pm 0.010$	$0.070\pm 0.009$	$0.060\pm 0.005$
earliest turns	$0.180\pm 0.026$	$0.073\pm 0.003$	$0.018\pm 0.003$	$0.012\pm 0.003$
uniform stride	$\mathbf{0.258\pm 0.023}$	$\mathbf{0.173\pm 0.013}$	$0.082\pm 0.003$	$0.058\pm 0.008$

Table 18: Mistral P3 accuracy at

r{=}0.30

, mean

\pm

standard deviation over three seeds (

n{=}200

per seed). Italic denotes full context; bold denotes the best compressed method in each column.

Method	Accuracy	Bootstrap $95\%$ CI
full context (uncompressed)	$\mathit{0.417}$	$\mathit{[0.306,\ 0.528]}$
recency	$\mathbf{0.319}$	$[0.222,\ 0.431]$
random selection	$0.228$	$[0.164,\ 0.294]$
uniform stride	$0.250$	$[0.153,\ 0.347]$
earliest turns	$0.097$	$[0.041,\ 0.167]$

Table 19: LongMemEval-KU P2-strict accuracy at

r{=}0.30

(

n{=}72

). Random selection averages five selector seeds. The reader uses a 32K input window and removes tokens from the beginning if a prompt exceeds it. Italic denotes full context; bold denotes the best compressed method in each column.

Method	P2-strict	Bootstrap $95\%$ CI
full context	$\mathit{0.232}$	$\mathit{[0.174,0.290]}$
recency	$\mathbf{0.155}$	$[0.106,0.208]$
random selection	$0.150$	$[0.120,0.183]$
earliest turns	$0.029$	$[0.010,0.053]$
uniform stride	$0.121$	$[0.077,0.164]$

Table 20: RiSAWOZ P2-strict at

r{=}0.30

(

n{=}207

). Random selection averages five selector seeds. Italic denotes full context; bold denotes the best compressed method in each column.

	Synthetic		MultiWOZ-derived		SGD-derived
Method	$r{=}0.10$	$r{=}0.30$	$r{=}0.10$	$r{=}0.30$	$r{=}0.10$	$r{=}0.30$
full context (uncompressed)	$\mathit{0.87}$	$\mathit{0.88}$	$\mathit{0.67}$	$\mathit{0.65}$	$\mathit{0.61}$	$\mathit{0.63}$
recency	$0.00$	$0.00$	$0.00$	$0.05$	$0.00$	$0.17$
random selection	$\mathbf{0.09}$	$0.32$	$0.03$	$0.20$	$0.02$	$0.20$
earliest turns	$0.00$	$0.43$	$0.04$	$0.08$	$0.00$	$0.06$
uniform stride	$0.03$	$0.21$	$0.07$	$\mathbf{0.23}$	$0.06$	$0.34$

Table 21: Free-form answer accuracy with the GPT-5.4-mini reader (seed

42

). Italic denotes full context; bold denotes the best compressed method among the six evaluated selectors in each column. LLMLingua-2 appears only in Table 22 . Full-context answers are generated and scored separately for each budget, producing differences of at most

0.02

Method	Synthetic	MultiWOZ-derived	SGD-derived
full context (uncompressed)	$\mathit{1.000\,/\,1.000}$	$\mathit{0.990\,/\,1.000}$	$\mathit{1.000\,/\,1.000}$
recency	$0.000\,/\,0.235$	$0.190\,/\,0.030$	$0.245\,/\,0.075$
random selection	$0.410\,/\,0.290$	$0.310\,/\,0.180$	$0.290\,/\,0.135$
earliest turns	$0.550\,/\,0.275$	$0.170\,/\,0.030$	$0.105\,/\,0.000$
uniform stride	$0.260\,/\,0.310$	$0.370\,/\,0.285$	$0.490\,/\,0.245$
attention/H2O proxy	$0.690\,/\,0.350$	$0.225\,/\,0.080$	$0.120\,/\,0.045$

Table 22: FMTS diagnostics on the compressed contexts at

r{=}0.30

: lexical answer-retention heuristic / boundary-turn recall. Italic denotes full context; bold denotes the highest compressed lexical-retention value. These diagnostics do not use generated reader answers.

	strict		loose		token- $F_{1}$
Method	SGD	MultiWOZ	SGD	MultiWOZ	SGD	MultiWOZ
full context (uncompressed)	$\mathit{0.487}$	$\mathit{0.430}$	$\mathit{0.850}$	$\mathit{0.867}$	$\mathit{0.577}$	$\mathit{0.414}$
recency	$0.010$	$0.002$	$0.135$	$0.092$	$0.049$	$0.026$
random selection	$0.145$	$0.147$	$0.530$	$0.543$	$0.286$	$0.214$
earliest turns	$0.480$	$\mathbf{0.410}$	$0.877$	$\mathbf{0.867}$	$0.582$	$\mathbf{0.413}$
uniform stride	$\mathbf{0.493}$	$0.407$	$0.875$	$0.852$	$0.587$	$0.400$

Table 23: P1 at

r{=}0.30

under three scoring rules, mean

\pm

sample standard deviation over seeds

\{42,43,44\}

(

200

examples per dataset and seed). Italic denotes full context; bold denotes the best compressed method in each column.

Method	SGD $r{=}0.10$	SGD $r{=}0.30$	MultiWOZ $r{=}0.10$	MultiWOZ $r{=}0.30$
full context (uncompressed)	$\mathit{0.691\pm 0.017}$	$\mathit{0.691\pm 0.017}$	$\mathit{0.781\pm 0.042}$	$\mathit{0.781\pm 0.042}$
recency	$0.003\pm 0.005$	$0.254\pm 0.028$	$0.020\pm 0.019$	$0.125\pm 0.011$
random selection	$0.130\pm 0.010$	$0.278\pm 0.031$	$\mathbf{0.177}\pm 0.007$	$\mathbf{0.314}\pm 0.024$
earliest turns	$0.008\pm 0.008$	$0.156\pm 0.024$	$0.024\pm 0.001$	$0.296\pm 0.009$
uniform stride	$0.088\pm 0.005$	$0.283\pm 0.017$	$0.080\pm 0.029$	$0.286\pm 0.037$
attention/H2O proxy	$0.008\pm 0.008$	$0.114\pm 0.005$	$0.034\pm 0.004$	$0.244\pm 0.017$

Table 24: P2-strict accuracy, mean

\pm

standard deviation over seeds

\{42,43,44\}

. The three samples contain

\{128,125,132\}

SGD and

\{161,168,168\}

MultiWOZ examples. Italic denotes full context; bold denotes the best compressed method in each column.

	SGD $r{=}0.30$		MultiWOZ $r{=}0.30$
Method	P3-loose	P3-strict	P3-loose	P3-strict
full context (uncompressed)	$\mathit{0.728\pm 0.032}$	$\mathit{0.652\pm 0.025}$	$\mathit{0.505\pm 0.013}$	$\mathit{0.475\pm 0.017}$
recency	$0.067\pm 0.003$	$0.065\pm 0.005$	$0.040\pm 0.010$	$0.040\pm 0.010$
random selection	$0.238\pm 0.010$	$0.162\pm 0.008$	$0.145\pm 0.005$	$0.128\pm 0.003$
earliest turns	$0.313\pm 0.006$	$0.160\pm 0.028$	$0.058\pm 0.003$	$0.048\pm 0.006$
uniform stride	$\mathbf{0.432\pm 0.006}$	$\mathbf{0.288\pm 0.008}$	$\mathbf{0.220\pm 0.010}$	$\mathbf{0.178\pm 0.021}$

Table 25: P3 accuracy at

r{=}0.30

under the segment-local scorer, mean

\pm

standard deviation over seeds

\{42,43,44\}

(

n{=}200

per seed). Higher is better. ChunkKV uses seed

42

. Italic denotes full context; bold denotes the best turn selector.

Multi-Turn Dialogue Evaluation

Multi-turn dialogue evaluation measures whether AI systems remain accurate, coherent, safe, and useful as conversations evolve.

See all 202 papers

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
Jun 10, 2026Yeongseo Jung, Jaehyeok Kim, Eunseo Jung +5Truncated Backpropagation Through Time Memory-Augmented Language Models
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan +1
KEEP-Bench: Evaluating Evidence Preservation in Context Compression
Oct 19, 2026Ziqing Man, Di Wu, Jia-Chen Gu +3LLM Context Compression LLM Evaluation
University of California, Los Angeles
Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
May 17, 2026Natalia Trukhina, Vadim VashkelisLLM Compression LLM Context Compression
Embedded Intelligence Lab (EMILAB)

Jun 10, 2026J/K move · Enter open · S save

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

Yeongseo Jung, Jaehyeok Kim, Eunseo Jung, Jiachuan Wang+4

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan+1

Truncated Backpropagation Through Time Memory-Augmented Language Models Long-Context Modeling LLM Context Management LLM Context Compression

Open paper AskPDF arXiv

Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues. We revisit the context compression under conversational dynamics and empirically present its fragility. To improve both efficiency and robustness, we introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve, revise, and write-back loop shares information across turns and updates stale memories, stabilizing long-horizon behavior. In addition, we adapt truncated backpropagation-through-time (TBPTT) to our multi-turn setting, learning cross-turn dependencies without full-history backpropagation. Extensive experiments on long-form dialogue benchmarks demonstrate superior performance and efficiency of C-DIC; notably, C-DIC shows stable inference latency and perplexity over hundreds of dialogue turns, supporting a scalable path to high-quality dialogue modeling.

Abstract

Multi-Turn Dialogue Evaluation

Explore similar work

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

KEEP-Bench: Evaluating Evidence Preservation in Context Compression

Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression