Language-model agents compact their context for the question at hand, but the compacted context must also serve questions that come later. We evaluate compaction on these later questions, using four corpora that annotate the evidence of several questions per text and recorded agent trajectories, and compare every method with random deletion at the same budget. On MultiRC, keeping the sentences that gpt-6-luna (through the OpenAI Decisions API) judges necessary for the current question raises its accuracy on that question by 11.4 points at 25% of the tokens. On the later questions, however, no method that selects sentences for the current question does detectably better than deleting sentences at random. Deleting words instead of sentences does better: LLMLingua-2, which ignores the question, beats random sentence deletion at the same token count by 12.5 points on the later MultiRC questions. It keeps about twice random deletion's share of the evidence's content words, and at equal content-word retention its gain shrinks to 7.3 points (exploratory). On FEVER, label accuracy can overstate what the kept text supports: the gain of sentence selection is no longer detected when answers are scored against the kept evidence, and the 21.0-point gain of LLMLingua-2 is not shown to rest on the kept text. The ranking also depends on the setting: on Evidence Inference sentence selection beats LLMLingua-2 at a nominal 25% budget, and on τ-bench trajectories, whose later tool calls need exact identifiers, it beats random deletion while LLMLingua-2 shows no detected gain. Scores on the question a compaction was made for do not reliably show how it serves later ones.
Figures & tables
accuracy (%)
model
policy
budget
known
hidden
hidden − random
open-decider
typed
25%
70
62
− 0.6 [ − 1.9 , +0.7 ]
open-decider
agnostic
25%
63
63
+0.3 [ − 0.9 , +1.4 ]
open-decider
random
25%
61
62
—
open-decider
typed
50%
72
66
− 0.6 [ − 1.7 , +0.6 ]
open-decider
agnostic
50%
67
67
+0.4 [ − 0.7 , +1.6 ]
Table 1: Typed selection keeps known-question accuracy above random deletion, while hidden-question accuracy stays near random. Readers: open-decider and gpt-6-luna , the typed readers whose full-text accuracy exceeds the all-“no” rate; agnostic: question-agnostic selection. Paragraph-macro candidate-level accuracy (%) on the known and hidden questions after compaction, and policy − random hidden accuracy (points, paragraph-bootstrap 95% interval). Table 7 gives the sufficiency check for the same rows.
Figure 2: Hidden questions that share evidence with the known question gain from typed selection; on Evidence Inference, disjoint claims gain too, and more of their evidence is kept. 25% budget. Left: typed − random hidden accuracy with bootstrap 95% intervals, for questions whose evidence overlaps the known question’s and for disjoint ones. Right: share of disjoint questions’ evidence kept by typed selection and by random deletion (for MultiRC and FEVER the exact expectation for a random subset of the same size). Values and overlap shares in Table 8 .
MultiRC, gpt-6-luna
MultiRC, open-decider
FEVER, gpt-6-luna
text
acc.
words
content
acc.
words
content
acc.
words
content
random sentences
68.3
26.3
26.1
62.1
26.2
26.0
61.2
25.7
25.6
typed sentences
67.9
24.4
24.3
61.0
26.3
26.3
79.0
59.1
59.2
random words
70.8
25.0
25.2
62.6
25.0
25.1
67.5
25.2
25.2
LLMLingua-2
80.8
26.3
48.0
68.1
26.3
48.0
82.2
26.4
40.7
hybrid
76.6
25.7
37.3
65.7
26.1
37.6
84.7
55.3
59.2
Table 2: Word-level compression leads on hidden MultiRC and FEVER questions and keeps more of their evidence’s content words than random sentence deletion; on MultiRC, about the same share of its words. 25% of the tokens; FEVER accuracy is label accuracy. Accuracy (%, document-macro) on hidden questions with annotated evidence, and share of the hidden questions’ annotated evidence kept (%), counted over all words and over content words, answer-weighted (content words: words outside a standard English stop list). Random words: a uniform sample of words. Hybrid: typed whole sentences up to half the budget, the rest LLMLingua-2 words (Appendix H ).
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
analysis
appendix
predictions
met
partly met
not met
unresolved
Evidence Inference overlap test
E
2
1
—
1
—
KEEP–DROP contrast
K
4
4
—
—
—
LoCoMo test
M
6
3
—
—
3
Follow-ups A–D
G , J , M
9
6
1
1
1
LLMLingua-2 granularity test
H
2
1
1
—
—
Hybrid compactor
H
3
1
2
—
—
Appendix
Table 3: Outcomes of the prespecified predictions. Partly met: met in some of the cells the prediction covers.
hidden questions, 25%
hidden
corpus
known, 25%
all
overlapping
disjoint
50%
MultiRC
+10.4 [ +7.4 , +13.3 ]
− 0.6 [ − 1.8 , +0.6 ]
+3.5 [ +1.5 , +5.7 ]
− 1.7 [ − 3.2 , − 0.2 ]
+0.5 [ − 1.0 , +1.9 ]
FEVER
+36.0 [ +28.5 , +43.5 ]
+18.5 [ +14.3 , +22.5 ]
+27.3 [ +22.7 , +32.0 ]
+1.3 [ − 3.4 , +6.1 ]
+10.1 [ +6.6 , +13.9 ]
Ev. Inference
+17.4 [ +14.5 , +20.3 ]
+11.0 [ +8.9 , +13.1 ]
+18.8 [ +14.6 , +23.0 ]
+8.5 [ +6.1 , +10.8 ]
+4.3 [ +2.7 , +5.9 ]
Appendix
Table 5: qwen3.8-flash as reader: typed − random accuracy (points, document-bootstrap 95% interval) on the known and hidden questions, with the hidden questions split by evidence overlap, at 25% and (last column) 50%.
corpus, reader
contrast
b
all hidden
overlapping
disjoint
MultiRC, open-decider
typed − random
25
− 0.1 [ − 1.1 , +0.8 ]
+6.0 [ +4.3 , +7.7 ]
− 2.9 [ − 4.0 , − 1.9 ]
50
− 0.6 [ − 1.6 , +0.3 ]
+3.9 [ +2.4 , +5.4 ]
− 3.2 [ − 4.4 , − 1.9 ]
agnostic − random
25
+0.8 [ − 0.1 , +1.7 ]
50
+0.4 [ − 0.5 , +1.3 ]
MultiRC, gpt-6-luna
typed − random
25
− 0.2 [ − 1.1 , +0.7 ]
+4.4 [ +2.6 , +6.1 ]
− 2.2 [ − 3.4 , − 1.0 ]
50
− 0.2 [ − 1.3 , +0.8 ]
+2.8 [ +1.3 , +4.4 ]
− 1.5 [ − 2.8 , − 0.3 ]
Appendix
Table 6: Hidden-question contrasts against random deletion averaged over seeded masks (points, document bootstrap 95% interval), at budget b (%), over all hidden questions and split by evidence overlap with the known question. Compare Table 8 , which uses one mask.
sufficiency AUROC vs.
hidden acc. after 25% re-read
model
policy
budget
evidence lost
answer flipped
lowest sufficiency
random
open-decider
typed
25%
0.81
0.69
67
66
open-decider
agnostic
25%
0.79
0.69
68
66
open-decider
random
25%
0.81
0.68
67
66
open-decider
typed
50%
0.78
0.70
71
69
open-decider
agnostic
50%
0.71
0.66
71
70
Appendix
Table 7: How well the sufficiency check finds damaged hidden questions, and what re-reading them recovers (MultiRC, the rows of Table 1 ). The AUROC (area under the receiver operating characteristic curve) columns rank hidden questions by low sufficiency and score that ranking against two question-level outcomes: at least one annotated evidence sentence deleted (evidence lost), and at least one candidate answer turned from right to wrong (answer flipped). The last two columns give hidden candidate-level accuracy averaged over questions (not exact match) after re-reading the full text for 25% of hidden questions: those with the lowest sufficiency, or a random 25%.
typed − random
overlap
disjoint kept
corpus, reader
b
all hidden
overlapping
disjoint
share
typed/random
MultiRC, open-decider
25
− 0.6 [ − 1.9 , +0.7 ]
+5.3 [ +2.9 , +7.7 ]
− 3.0 [ − 4.5 , − 1.4 ]
35
14/25
50
− 0.6 [ − 1.7 , +0.6 ]
+4.2 [ +1.9 , +6.3 ]
− 3.4 [ − 5.0 , − 1.8 ]
—
MultiRC, gpt-6-luna
25
− 0.8 [ − 1.9 , +0.3 ]
+4.3 [ +2.2 , +6.4 ]
− 2.9 [ − 4.3 , − 1.4 ]
35
18/25
50
− 0.4 [ − 1.7 , +0.9 ]
+2.6 [ +0.5 , +4.7 ]
− 1.6 [ − 3.3 , +0.1 ]
—
FEVER, gpt-6-luna
25
+17.2 [ +13.4 , +21.2 ]
+25.8 [ +21.3 , +30.3 ]
+0.2 [ − 4.8 , +5.2 ]
51
22/24
Appendix
Table 8: Hidden questions whose evidence overlaps the known question’s gain more from typed selection. Typed − random hidden accuracy (points, paragraph or document bootstrap 95% interval) at budget b (%), over all hidden questions and split by evidence overlap with the known question. Overlap share: evidence-classified hidden questions sharing an evidence sentence with the known question (%). Disjoint kept: share of disjoint questions’ evidence kept by typed selection and by random deletion (%; for MultiRC and FEVER the exact expectation for a random subset of the same size, for Evidence Inference the kept set of the random-deletion read). Each accuracy cell averages within documents, over the documents that have such questions, so the two cells need not combine to the all-question contrast by the overlap share. The overlap split is post hoc on MultiRC and FEVER; † on Evidence Inference the overlapping and disjoint contrasts are prespecified, and the all-hidden contrast and the kept column exploratory.
Figure 3: For typed sentence selection, the hidden-question gain follows the extra evidence kept. Typed − random hidden accuracy against typed minus random evidence kept (points), at 25%, on overlapping (circles) and disjoint (squares) hidden questions. Open markers: the paper’s known question (Figure 2 ); filled: a different known question per document. Dashed: the line specified before the second experiment, 0.366× extra evidence +0.343 .
hidden questions
overlapping
corpus, reader
known
all
overlapping
disjoint
− disjoint
MultiRC, open-decider
+6.2 [ +3.0 , +9.7 ]
− 0.7 [ − 2.0 , +0.6 ]
+4.5 [ +2.3 , +6.9 ]
− 3.3 [ − 4.9 , − 1.8 ]
+7.6 [ +4.9 , +10.4 ]
MultiRC, gpt-6-luna
+8.0 [ +5.0 , +11.1 ]
− 0.4 [ − 1.5 , +0.6 ]
+1.3 [ − 0.7 , +3.5 ]
− 1.2 [ − 2.7 , +0.4 ]
+3.0 [ +0.3 , +5.8 ]
FEVER, gpt-6-luna
+39.5 [ +33.0 , +46.5 ]
+16.0 [ +11.9 , +20.3 ]
+29.6 [ +24.7 , +34.0 ]
− 6.6 [ − 12.0 , − 1.6 ]
+35.7 [ +29.0 , +42.7 ]
Ev. Inference, gpt-6-luna
+22.6 [ +19.6 , +25.8 ]
+11.6 [ +9.5 , +13.8 ]
+20.6 [ +16.9 , +24.5 ]
+8.1 [ +5.6 , +10.5 ]
+12.7 [ +7.6 , +18.1 ]
Appendix
Table 9: With a different known question, typed selection still helps the known question, hidden MultiRC questions stay near random deletion, and overlapping hidden questions gain more than disjoint ones. Typed − random accuracy at 25% (points, document-bootstrap 95% interval). Overlapping − disjoint is the paired contrast within the documents that have both kinds of hidden question. Bottom: extra evidence kept (typed − random, points) and the gain predicted by the line specified before this experiment, 0.366× extra evidence +0.343 ; the observed gains are the overlapping and disjoint columns above.
hidden questions
known question
corpus, reader
b
L2 − random
typed − L2
L2 − random
typed − L2
MultiRC, open-decider
25
+6.5 [ +5.4 , +7.6 ]
− 6.6 [ − 8.0 , − 5.2 ]
+6.9 [ +4.6 , +9.3 ]
+1.7 [ − 1.3 , +4.8 ]
50
+5.3 [ +4.4 , +6.3 ]
− 6.0 [ − 7.3 , − 4.6 ]
+3.4 [ +1.3 , +5.5 ]
+2.8 [ +0.3 , +5.2 ]
MultiRC, gpt-6-luna
25
+12.9 [ +11.8 , +14.1 ]
− 13.1 [ − 14.5 , − 11.8 ]
+12.8 [ +10.3 , +15.1 ]
− 3.0 [ − 5.9 , − 0.1 ]
50
+10.2 [ +9.3 , +11.1 ]
− 10.4 [ − 11.7 , − 9.2 ]
+9.9 [ +8.0 , +12.0 ]
− 2.3 [ − 4.3 , − 0.2 ]
FEVER, gpt-6-luna
25
+20.8 [ +18.8 , +23.0 ]
− 3.7 [ − 6.7 , − 0.6 ]
+22.0 [ +16.2 , +27.9 ]
+12.5 [ +7.0 , +18.5 ]
Appendix
Table 10: LLMLingua-2 (L2), at a rate targeting random deletion’s share, compared with random deletion averaged over the seeded masks and with typed selection, on hidden and known questions (points, paragraph or document bootstrap 95% interval) at budget b (%; exploratory).
MultiRC, gpt-6-luna
MultiRC, open-decider
FEVER, gpt-6-luna
budget
25%
50%
25%
50%
25%
50%
policy
known
hidden
known
hidden
known
hidden
known
hidden
known
hidden
known
hidden
typed
78.7
67.9
83.2
76.1
69.0
61.0
73.6
66.1
92.5
79.0
93.0
85.6
agnostic
68.5
67.5
76.0
75.5
63.3
62.2
66.4
66.4
62.3
61.6
76.0
76.7
TF-IDF
76.9
67.0
82.8
74.8
65.4
61.7
68.2
66.1
88.0
77.2
90.0
86.7
random (3 masks)
67.3
68.3
76.2
76.0
61.1
62.2
66.7
66.9
56.8
61.2
72.2
75.2
Appendix
Table 11: At token-matched budgets, typed selection beats random deletion on the known question, and LLMLingua-2 is highest on hidden questions. Accuracy (%) at 25% and 50% of each document’s o200k_base tokens, on the known and hidden questions. MultiRC: candidate-level, paragraph-macro; FEVER: claim accuracy, document-macro. Random deletion averages three masks. Each level uses every answer its policy gave; differences of these levels are within 1 point of the paired contrasts of Table 12 .
MultiRC, gpt-6-luna
MultiRC, open-decider
FEVER, gpt-6-luna
policy
policy/random
difference
policy/random
difference
policy/random
difference
hidden questions, 25% of tokens
typed
68.1/68.5
− 0.4 [ − 1.5 , +0.6 ]
61.0/62.2
− 1.1 [ − 2.1 , − 0.2 ]
79.0/61.2
+17.8 [ +15.0 , +20.8 ]
agnostic
67.6/68.6
− 1.0 [ − 1.8 , − 0.1 ]
62.2/62.2
+0.1 [ − 1.0 , +1.2 ]
61.6/61.2
+0.4 [ − 2.4 , +3.2 ]
TF-IDF
67.1/68.5
− 1.3 [ − 2.3 , − 0.4 ]
61.7/62.2
− 0.5 [ − 1.5 , +0.5 ]
77.2/61.2
+16.0 [ +13.0 , +19.0 ]
LLMLingua-2
80.7/68.6
+12.1 [ +11.0 , +13.3 ]
68.0/62.2
+5.9 [ +4.7 , +7.0 ]
82.2/61.2
+21.0 [ +18.6 , +23.2 ]
Appendix
Table 12: On hidden MultiRC questions, every policy except LLMLingua-2 is near or below random deletion; on FEVER, every policy except question-agnostic selection is above it. Hidden questions at token-matched budgets, paired: an answer refused in either read is dropped from both. Policy/random: the two accuracies (%) on the remaining answers, document-macro, with random deletion averaged over its three masks within each document. Difference: policy − random, which equals the difference of the two levels up to rounding (points, document-bootstrap 95% interval).
words − sentences
scores − random
corpus, reader
b
by scores (1)
at random (2)
sentences (3)
words (4)
MultiRC, dec
25
+4.6 [ +3.2 , +5.9 ]
+0.4 [ − 0.4 , +1.3 ]
+1.3 [ +0.2 , +2.3 ]
+5.4 [ +4.2 , +6.6 ]
50
+4.3 [ +3.0 , +5.5 ]
+0.5 [ − 0.3 , +1.3 ]
+0.5 [ − 0.4 , +1.5 ]
+4.4 [ +3.3 , +5.5 ]
MultiRC, luna
25
+11.7 [ +10.4 , +13.0 ]
+1.2 [ +0.3 , +2.1 ]
+0.4 [ − 0.7 , +1.5 ]
+10.0 [ +8.9 , +11.2 ]
50
+9.7 [ +8.6 , +10.9 ]
+2.4 [ +1.6 , +3.3 ]
+0.1 [ − 0.9 , +1.0 ]
+7.3 [ +6.5 , +8.3 ]
FEVER, luna
25
+19.1 [ +15.9 , +22.1 ]
+6.3 [ +4.4 , +8.2 ]
+1.9 [ − 1.3 , +5.4 ]
+14.7 [ +12.7 , +16.8 ]
Appendix
Table 13: The LLMLingua-2 scores help when they choose words, and in five of six cells not when they choose sentences. Top: contrasts on hidden questions at budget b (% of the tokens; points, document-bootstrap 95% interval), paired on the answers both policies gave. Bottom: hidden-question accuracy (%) of sentence scores and random words; Table 11 gives word scores ( LLMLingua-2 ) and random sentences. Random policies average three masks. luna: gpt-6-luna ; dec: open-decider .
MultiRC, gpt-6-luna
MultiRC, open-decider
FEVER, gpt-6-luna
policy
known
hidden
known
hidden
known
hidden
typed
78.7
67.9
69.0
61.0
92.5
79.0
LLMLingua-2
80.9
80.7
70.0
68.0
79.5
82.2
hybrid, half
82.2
76.5
70.7
65.7
89.0
84.7
hybrid, quarter
79.6
79.6
70.8
66.8
81.5
82.1
random sentences
67.3
68.3
61.1
62.2
56.8
61.2
Appendix
Table 14: Hybrid compaction at 25% of the tokens. Top: known and hidden accuracy (%); bottom: share of the known and hidden questions’ evidence words kept (%). The hybrid keeps typed whole sentences up to half (or a quarter) of the budget and fills the rest with LLMLingua-2 words. Hidden accuracy here covers all hidden questions; Table 2 covers those with annotated evidence, so the two differ by up to 0.1 points.
cell
gap
raw
all
content
+answer
breadth
linear
MultiRC, luna
LLMLingua-2 − random sent.
+12.5 [ +11.3 , +13.6 ]
+10.1
+7.3 [ +6.1 , +8.4 ]
+4.9
+2.6
+8.0
MultiRC, luna
hybrid − typed
+8.6 [ +7.3 , +9.9 ]
+7.2
+5.1 [ +3.8 , +6.4 ]
+3.2
− 0.7
+6.1
MultiRC, dec
LLMLingua-2 − random sent.
+6.0 [ +4.7 , +7.3 ]
+5.5
+2.0 [ +0.7 , +3.2 ]
+0.6
− 2.2
+2.3
MultiRC, dec
hybrid − typed
+4.7 [ +3.5 , +5.9 ]
+4.6
+2.5 [ +1.1 , +3.7 ]
+1.1
− 2.9
+2.8
FEVER, luna
LLMLingua-2 − random sent.
+21.0 [ +18.7 , +23.4 ]
+2.7
+1.3 [ − 1.1 , +3.5 ]
—
− 4.1
+14.9
FEVER, luna
hybrid − typed
+5.7 [ +3.6 , +7.9 ]
+0.4
− 0.5 [ − 1.8 , +0.8 ]
—
− 5.8
+5.7
Appendix
Table 15: Hidden-question gaps (points) at 25% before and after conditioning on a retention measure (exploratory; document-bootstrap 95% intervals). Columns after the raw gap: binned all-word (all), content-word, content-word plus answer and breadth retention, and content-word retention in the linear form (linear). Raw: the model without retention terms, on hidden questions with annotated evidence; the paired contrasts of Appendices H and G differ slightly. luna: gpt-6-luna ; dec: open-decider .
document label recovered
corpus
text
supported
refuted
balanced
“not enough info”
FEVER
full
93.8
83.5
88.6
–
typed 25%
67.2
57.3
62.2
74.6
random 25%
37.0
37.1
37.1
81.1
typed 50%
78.5
68.3
73.4
71.7
random 50%
60.0
57.0
58.5
75.3
Appendix
Table 16: Typed selection recovers the document’s label more often than random deletion. Three-way read of the hidden claims ( gpt-6-luna ; %, document-macro, over all claims each read answered). Document label recovered: per gold label and label-balanced. “Not enough info”: share of claims whose annotated evidence was all deleted that the reader answers so. Bottom: Label recovery and annotation-derived support accuracy on common claim sets. Hidden claims of the three-way read ( gpt-6-luna ) on the common set of each row: claims answered in both texts and, for support, with a target in both. Claims, documents: size of that set. Typed, random: label-balanced document-macro accuracy St (%, defined in the text). Typed − random: Styped−Srandom , equal to the difference of the two levels up to rounding (points, document-bootstrap 95% interval).
corpus
claims
n
docs
KEEP − DROP
KEEP
DROP
typed
random
full
FEVER
all
5,132
200
+50.0 [ +47.4 , +52.4 ]
95
45
76
59
95
disjoint
2,551
158
+41.1 [ +36.2 , +45.8 ]
95
53
62
62
93
overlapping
2,581
196
+50.1 [ +46.7 , +53.5 ]
95
45
88
61
95
true
3,658
199
—
94
21
68
41
94
false
1,474
199
—
97
98
98
99
97
Ev. Inf.
all
2,915
792
+17.8 [ +15.9 , +19.7 ]
90
72
82
70
90
Appendix
Table 17: KEEP beats DROP at 25% ( gpt-6-luna ). Split by gold label, the gap is on true claims; false claims are answered correctly at 97% or more in every arm. KEEP − DROP hidden-claim accuracy (points, document bootstrap 95% interval; all and disjoint prespecified, overlapping secondary, label split post hoc) and document-macro accuracy (%) of KEEP, DROP and, descriptively, the main typed-selection, random and full-text reads of the same claims, which asked several claims per request.
measured, per document
modeled, per later question
corpus
kˉ
scoring
read 25%
read full
25%
50%
full
check sep.
check joint
re-read
MultiRC
10.2
2,605
15,081
15,431
1,259
1,320
1,446
245
177
361
FEVER
27.2
2,795
10,460
10,810
311
426
671
281
169
168
Evidence Inf.
3.7
22,181
3,321
6,479
1,535
2,708
4,889
1,488
184
1,222
LoCoMo
149.6
180,510
64,220
96,742
6,270
11,783
22,721
6,218
146
5,680
thousand input tokens per document, typed 25%
k=1
k=3
k=10
k=kˉ
ratio at kˉ
break-even k
Appendix
Table 18: Input tokens for typed selection at 25% ( gpt-6-luna ). Top: per document, measured scoring and batched reads, and modeled one-question costs (read, sufficiency check as a separate call or joint with the read, and a full re-read charged at a quarter of the full read). Bottom: thousand input tokens per document to answer k later questions, one request each, the ratio to always reading the full text at k=kˉ , the corpus’s mean number of later questions, and the break-even k , the number of later questions at which the policy and the full text cost the same (never: at no k ). Prompt caching is not modeled.
accuracy (%)
evidence kept,
memory
luna 25%
luna 50%
qwen 25%
25% (%)
typed
87.9
91.9
83.8
27.2
agnostic
90.4
94.8
—
48.6
TF-IDF
88.7
91.4
85.7
25.7
random
88.3
89.7
83.2
25.4
recency
83.1
87.6
82.4
25.3
Appendix
Table 19: LoCoMo hidden-question accuracy at the last checkpoint (conversation-macro %, chance 25) by memory, budget and reader (luna: gpt-6-luna ; qwen: qwen3.8-flash on the same memories), and the share of hidden questions’ evidence turns kept at 25% (%, question mean). Full and empty memory do not depend on the budget. Tests H1–H4: under Test results below.
gpt-6-luna
qwen3.8-flash
memory (last checkpoint)
refused
accuracy
answered acc.
refused
accuracy
answered acc.
empty, original options
10.0
81.4
90.6
0.0
85.1
85.1
empty
28.4
60.2
83.9
0.0
71.5
71.5
full
0.5
90.9
91.4
—
—
—
typed 25%
1.1
78.0
78.9
0.0
75.7
75.7
agnostic 25%
2.3
81.0
82.8
0.0
81.7
81.7
Appendix
Table 20: With harder options, gpt-6-luna refuses more with empty memory, KEEP stays far above DROP, and typed versus random stays unresolved. LoCoMo hidden questions at the last checkpoint, conversation-macro (%). Refused: share of questions refused. Accuracy: over all questions, refusal counts as wrong (chance 25). Answered acc.: over answered questions. First row: empty memory with the original options, on the same questions. KEEP and DROP are the texts of Section 5 . qwen3.8-flash did not read the full memory.
text
hidden, luna
known, luna
hidden, qwen
id kept
G
full state
91.0
70.5
96.4
—
—
oracle
93.2
45.5
97.3
100.0
1.00
anticipatory
75.9
47.2
80.0
77.1
0.64
ECov †
74.8
49.3
77.3
73.6
0.64
typed
67.1
59.2
74.0
66.4
0.46
recency
53.9
28.9
56.4
42.9
0.19
Appendix
Table 21: On τ -bench retail trajectories at 25% of the tokens, typed selection beats random deletion on hidden questions, LLMLingua-2 does not, and anticipating later calls costs the known question. Accuracy (%, state means) with gpt-6-luna (luna) and qwen3.8-flash (qwen); ECov: expected-coverage selection; id kept: share of hidden questions whose gold identifier survives; G : share of the random–oracle gap closed on hidden questions (luna). † On a 120-state subsample, where typed selection scores 62.1 (hidden) and 59.9 (known). Further arms are on all states; their contrasts with ECov (M4, M5 in Appendix N ) use the ECov subsample.
stubs
gists
θ
acc.
re-read
tokens
acc.
re-read
tokens
0.3
72.3
65.1
96.6
67.7
64.6
92.9
0.5
72.6
72.8
104.5
69.1
73.2
102.0
0.7
72.3
79.5
111.7
69.7
79.8
108.4
Appendix
Table 22: Recoverable arm (retail, 25%, gpt-6-luna ): hidden accuracy (%), share of hidden questions re-read (%) and passage tokens per question as % of the full state’s, by threshold θ .
compaction
known
hidden
T used
cut
− random, hidden
− LLMLingua-2 , hidden
− random, known
random sentences
67.3
68.3
typed
78.7
67.9
LLMLingua-2
80.9
80.7
qwen3.8-flash
generic
75.4
76.1
88.4
23.7
+7.6 [ +6.6 , +8.7 ]
− 4.5 [ − 5.7 , − 3.2 ]
+8.2 [ +6.1 , +10.3 ]
focused
88.5
65.4
69.5
0.7
− 2.9 [ − 4.0 , − 1.8 ]
− 15.1 [ − 16.7 , − 13.6 ]
+21.4 [ +19.2 , +23.5 ]
Appendix
Table 23: Abstractive summaries on MultiRC at 25% of the tokens, read by gpt-6-luna . Left: known and hidden accuracy (%), share of T used and share of summaries cut at T (%). Right: summary − comparator on hidden questions, and − random deletion on the known question (points, 95% interval).
Large Language Models (LLMs) have a bounded context window. The context window is the maximum input size an LLM can consume for a single inference. AI agents rely on a process called context compaction to fit their state within the context window when calling an LLM. Despite its ubiquity, context compaction has received essentially no formal analysis. In this paper, we initiate a formal study of context compaction. We first introduce a framework consisting of two games that capture the two algorithmic strategies for context compaction used by contemporary AI agents in practice. The Context Selection Game models context compaction algorithms that select a subset of an agent's accumulated state to retain. The Context Generation Game models context compaction algorithms that summarize an agent's state by an arbitrary message of bounded length. We then prove an equivalence between the Context Generation Game and one-way communication complexity. The minimum context compaction budget for answering a set of queries within a target error is equal to the one-way communication complexity of the induced communication problem at the same error. Known bounds from communication complexity therefore transfer directly to context compaction. We also show that the Context Selection Game corresponds to a restricted class of one-way communication protocols. Any gap between selection and generation is therefore a gap between two classes of communication protocols. We prove that there exists a set of queries for which generation needs strictly less budget than selection. The equivalence between the Context Generation Game and one-way communication also lets us measure how well a deployed context compaction algorithm performs on a query relative to the optimal strategy. As an example, we present a case study that evaluates Anthropic's context compaction endpoint on set membership queries.
Hayder Tirmazi, Sam Markelon, Allison Bishop +1
AllSpice Inc. · Boston University · Proof Trading +2
Agents that work for many turns outgrow the context window of the model that serves them, and on small language models (SLMs) and constrained inference systems that window is small. Context compaction replaces older turns with a summary so a session can continue, but it is lossy, costs a model call, and can help or hurt depending on when it runs. We describe Bud ContextCompact 2.0, the compaction built into the Bud agent runtime, and evaluate it in two settings. On recall questions over recorded agent sessions, compaction to a 16K budget raised a 1.2B-parameter model's accuracy to 0.335, from 0.247 with truncation and 0.176 with its full 32K history (p < 0.001), and raised a 9B model from 0.515 (truncation) to 0.577 (p = 0.04). On 100 SWE-bench Verified tasks run end to end by a 27B reasoning model on a 49K-token window (556 graded sessions), compaction raised the resolve rate on long sessions from 41% to 53% (+12.3 points, 95% CI +1.6 to +23.0, McNemar p = 0.043), left short sessions unchanged, and used 13% fewer tokens per solved task at equal total cost. The main failure without compaction was not the history overflowing but a reasoning turn exhausting the room the history left. The gain has limits: it vanished under a 45-minute session budget, because each summary took a median of 161 s, and in a threshold study it appeared only when compaction triggered early enough to leave room. An 8B model submitted after about ten turns at 5-8K tokens and never needed compaction. We argue that the accuracy results transfer across hardware, including CPU inference, for an equivalent model and window, while the time cost scales with the hardware and must be budgeted for.
Efficient context compression is critical for retrieval-augmented question answering in resource-constrained settings, where long retrieved contexts increase latency, memory use, and LLM reader cost. We propose a lightweight encoder-only framework for query-driven sentence pruning that preserves answer-critical evidence while aggressively reducing irrelevant context. Our method learns marginal contribution scores for sentences using counterfactual training signals and optimizes a contrastive ranking objective that separates critical evidence from noncritical context. Our approach scores all sentences from a single full-context encoding, enabling fast inference with low computational overhead. Experiments show that it maintains accuracy comparable to the strongest baseline while using 3.7× less peak memory and achieving nearly 3× lower compression latency, demonstrating an effective quality--efficiency trade-off for practical resource-constrained deployment.
Thao Do, Dinh Phu Tran, An Vo +2
Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea