QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents

Yingjie Tan; Haiquan Qiu; Quanming Yao; Manyi Zhong; Xianzhi Yu; Yaqing Wang

cs.LGNov 5, 2026

QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents

Authors: Yingjie Tan, Haiquan Qiu, Quanming Yao, Manyi Zhong, Xianzhi Yu, Yaqing Wang

Organizations: Tsinghua University · Huawei Technologies Ltd.

Abstract

DeepSearch LLM agents repeatedly search and carry retrieved sources into subsequent reasoning steps. This expanding context increases token consumption and latency, although many results do not contribute to later reasoning. We propose Query-Relevant Attention Entropy (QRAE), an online context compression method that uses sparse evidence-assessment heads to decide which intact retrieved chunks should persist across reasoning steps. It scores bounded candidate batches through paired prefills: a query-free control suppresses shared attention responses, while normalized entropy measures the breadth of the positive response within each chunk. High-scoring chunks enter a persistent evidence whitelist and are included in the reasoning context. Uncertain chunks remain outside the reasoning context but are retained in a candidate cache, where they can be retrieved for later reevaluation. QRAE requires neither an additional generative compression pass nor modification of the attention operator. Across multiple DeepSearch benchmarks and reasoning models, QRAE improves task success by an average of 2.35 percentage points and reduces end-to-end latency and reported token usage by an average of 30% and 45%, respectively. These benefits generalize to a model with nearly 100B parameters.

Figures & tables

Figure 1: Overview of QRAE for context compression in DeepSearch agents. (a) Retrieved chunks pass through QRAE before the LLM updates its hypothesis and takes the next action. (b) The QRAE signal is derived from the attention distributions of the model’s query-relevant heads. (c) Chunks with the highest running-mean scores in the candidate cache enter the whitelist and the reasoning context; deferred chunks remain in the candidate cache.

Figure 2: Construction of the QRAE signal. (a) Offline localization selects the model heads with the highest attention mass on answer tokens in the input. (b) During online scoring, prefills with and without the query yield signed attention differences. (c) Positive attention differences are first normalized within each chunk. Normalized entropy is then computed and averaged across the selected heads.

Dataset	Metric	DeepSearch	Lingua-2	ACON	Rerank	QRHead	SideQuest	QRAE
Dataset	Metric	No comp.	Compression baselines					Ours
BrowseComp- Plus	Succ. (%) $\uparrow$	13.8	14.3	12.1	14.7	11.6	10.6	17.2
	Time (s) $\downarrow$	29.5	28.8	50.0	35.6	45.1	24.3	20.5
	Tokens $\downarrow$	177k	171k	98k	148k	149k	66k	54k
XBench- DeepSearch	Succ. (%) $\uparrow$	38.0	43.0	40.0	41.0	35.0	36.0	44.0
	Time (s) $\downarrow$	276.0	245.3	324.8	213.6	357.5	167.2	150.4

Table 1: QRAE effectiveness in DeepSearch agents with DeepDiver-7B across four benchmarks. Results report task success, end-to-end time, and total processed tokens from one representative run per benchmark. Bold and underlined values denote the best and second-best results, respectively.

Method	OpenResearcher-30B			OpenPangu-2.0-Flash-92B
Method	Success (%) $\uparrow$	Time (s) $\downarrow$	Tokens $\downarrow$	Success (%) $\uparrow$	Time (s) $\downarrow$	Tokens $\downarrow$
DeepSearch	53.2	151.0	1981k	54.9	308.6	1077k
LLMLingua-2	52.1	154.7	2037k	54.2	304.5	1057k
ACON	49.4	188.1	842k	50.1	429.8	879k
Rerank	50.7	162.0	2157k	54.3	344.7	970k
QRHead	51.1	189.3	1771k	53.8	393.8	982k

Table 2: Cross-model evaluation of QRAE on BrowseComp-Plus, reporting task success, end-to-end time, and total processed tokens. Token counts for OpenPangu include inputs only. Bold and underlined values denote the best and second-best results, respectively.

Panel of Figure 3 — Figure 3: Global layer–head localization scores for DeepDiver-7B, computed from attention to answer-bearing spans. Heads are selected on HotpotQA (a) and validated on BrowseComp-Plus (b) and XBench-DeepSearch (c), neither of which is used for head selection.

Model	5 chunks			20 chunks
Model	AUC	Recall@1	Recall@3	AUC	Recall@1	Recall@10
LiteResearcher-4B	76.6	48.3%	76.6%	81.8	35.2%	83.2%
DeepDiver-7B	83.2	54.6%	83.2%	84.4	44.6%	88.7%
WebSeer-14B	84.1	53.9%	84.5%	85.1	47.8%	89.2%
OpenResearcher-30B	90.3	74.1%	90.5%	89.6	57.4%	93.0%
OpenPangu-2.0-Flash-92B	92.5	83.8%	93.4%	92.2	66.2%	96.8%

Table 3: Cross-model QRAE scoring on BrowseComp-Plus with 5- and 20-chunk candidate sets. AUC denotes AUROC; all metrics are percentages and higher is better.

Variant	5 chunks			20 chunks
Variant	AUC	Recall@1	Recall@3	AUC	Recall@1	Recall@10
QRAE (full)	83.2	54.6%	83.2%	84.4	44.6%	88.7%
w/o query-free subtraction	79.6	48.6%	80.9%	78.8	34.2%	84.5%
Attention sum	75.0	44.7%	79.5%	81.3	32.6%	80.0%

Table 4: Scoring ablations on BrowseComp-Plus with DeepDiver-7B. Candidate sets and retained-chunk budgets are shared across variants. AUC denotes AUROC; all metrics are percentages and higher is better.

Variant	BrowseComp-Plus			XBench-DeepSearch
Variant	Succ. (%) $\uparrow$	Time (s) $\downarrow$	Tokens $\downarrow$	Succ. (%) $\uparrow$	Time (s) $\downarrow$	Tokens $\downarrow$
QRAE (full)	17.2	20.5	54	44.0	150.4	117
w/o query-free subtraction	15.8	24.1	73	42.0	192.8	186
Attention sum	14.6	27.3	92	42.0	181.6	177
Current-search filtering	14.9	30.2	129	41.0	232.3	225

Table 5: Agent-level component ablations on BrowseComp-Plus and XBench-DeepSearch. Success is in percent, time in seconds, and tokens in thousands. End-to-end costs include the evaluator calls required by each variant.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$K_{\mathrm{H}}$	Selected layer–head pairs
LiteResearcher-4B	2	L20H15, L18H15
DeepDiver-7B	2	L18H22, L17H0
WebSeer-14B	2	L36H21, L47H33
OpenResearcher-30B	3	L26H9, L26H25, L19H18
OpenPangu-2.0-Flash-92B	5	L45H36, L39H24, L39H30, L39H32, L36H41

Appendix

Table 6: Fixed evidence-assessment heads. Layer and head indices are zero-based.

Setting	Value or rule
Calibration	100 HotpotQA trajectories; one answer-containing input each
Head set	Fixed per model; Table 6
Batch size $K$	Agent’s configured search-result count per query
Chunking	Agent’s search splitting rules and length limits
Trigger	Candidate count at least $K$ ; at most once per query update
Batch selection	$K$ candidates with the fewest prior scores

Appendix

Table 7: Implementation settings shared with the agent or fixed by QRAE.

Benchmark	Role	Evaluation set
BrowseComp-Plus	End-to-end and evidence ranking	830 questions
XBench-DeepSearch	End-to-end and head transfer	2510 release; 100 questions
FRAMES	End-to-end multi-hop QA	Test split; 824 questions
SimpleQA	End-to-end factual QA	Original test set; 4,326 questions
HotpotQA	Offline localization and evidence analysis	100 sampled trajectories

Appendix

Table 8: Datasets and evaluation coverage. All questions in the four end-to-end benchmarks are evaluated.

Model	Official checkpoint
DeepDiver-7B	openpangu/openPangu-Embedded-7B-DeepDiver
OpenResearcher-30B	OpenResearcher/OpenResearcher-30B-A3B
OpenPangu-2.0-Flash-92B	openpangu/openPangu-2.0-Flash
LiteResearcher-4B	simplex-ai-inc/LiteResearcher-4B
WebSeer-14B	99hgz/WebSeer-14b

Appendix

Table 9: Official checkpoint identifiers, linked to their model cards.

Model	Max. output tokens per call	Max. interaction rounds
DeepDiver-7B	256	50
OpenResearcher-30B	8,192	50
OpenPangu-2.0-Flash-92B	4,096	50

Appendix

Table 10: Generation and interaction budgets for the end-to-end agents. Within each model–benchmark setting, all compared methods use the same budgets.

Setting	Retrieval source	Top- $k$	Search snippet
BrowseComp-Plus: DeepDiver	BM25, official index	5	First 512 tokens per document; Qwen3-0.6B tokenizer
BrowseComp-Plus: OpenResearcher	Official dense-retrieval service	10	Query-highlighted summary, at most 180 characters
BrowseComp-Plus: OpenPangu	BM25, official index	10	At most 256 tokens per document
XBench-DeepSearch	Serper web search	10	API snippet, truncated to 512 tokens if longer
FRAMES	Serper web search	10	API snippet, truncated to 512 tokens if longer
SimpleQA	Serper web search	10	API snippet, truncated to 512 tokens if longer

Appendix

Table 11: Retrieval settings. All result counts are per query. Token and character limits apply to snippet text before compression.

Method	Configuration
DeepSearch	Uncompressed retrieved context under the matched agent and retrieval budgets.
QRAE	$\rho=0.5$ by default; fixed heads in Table 6 ; Search-aligned $K$ and $M=\lceil K/2\rceil$ ; online paired prefills; no evaluator generation.
LLMLingua-2	microsoft/llmlingua-2-xlm-roberta-large-meetingbank ; token retention rate 0.5 on each query’s returned snippet text, before it enters the agent context; source metadata and snippet boundaries are protected.
ACON	Observation-compression variant with GPT-4.1-mini and the published observation-compression guideline, conditioned on the task and interaction history. Compress a query’s search observation when its snippet text exceeds 400 tokens, targeting 50% of its original text length; retain only the refined observation and leave the existing history unchanged.
Rerank	Qwen/Qwen3-Reranker-0.6B , using its query–document relevance prompt and yes/no score. Rerank the same $K$ retrieved snippets and keep the top $\lceil K/2\rceil$ ; use the snippet limits above without retrieving additional candidates.
QRHead	QRRetriever scoring on the corresponding frozen reasoning model: 16 heads for DeepDiver and 32 for OpenResearcher and OpenPangu. Average query-token attention mass over selected heads, subtract the score for the null query N/A , and retain the top $\lceil K/2\rceil$ snippets per query.

Appendix

Table 12: Compression configurations under matched agent and retrieval budgets. Retention targets apply to search-derived content.

Model	Hardware	Precision
DeepDiver-7B	$4\times$ NVIDIA RTX 4090	BF16
OpenResearcher-30B	$4\times$ NVIDIA RTX 4090	BF16
OpenPangu-2.0-Flash-92B	$4\times$ NVIDIA H200	BF16

Appendix

Table 13: Hardware and precision for the end-to-end comparisons. Execution settings are matched across methods within each model–benchmark setting.

Metric	BrowseComp-Plus	XBench-DeepSearch
Delayed admissions per trajectory	4.36	11.17
Annotated evidence among delayed admissions per trajectory	1.31	2.69
Mean admission delay (steps)	2.00	2.10
Trajectories with delayed admission	745/830 (89.8%)	89/100 (89.0%)

Appendix

Table 14: Delayed admission and recovery of annotated evidence in the original compression runs. Evidence counts are a subset of all delayed admissions.

Example	Initially deferred	Reassessed and admitted	Final answer
Question 66: roast duck	Step 3; ranks $(6,6)$	Step 5; ranks $(2,5)$	Peking duck (correct)
Question 75: The Voice of China	Step 5; ranks $(14,12)$	Step 8; ranks $(2,1)$	Li Wei (correct)

Appendix

Table 15: Two XBench-DeepSearch examples of evidence deferred and later admitted. Rank pairs give the respective ranks assigned by the two selected heads; smaller ranks are better.

Panel of Figure 4 — Figure 4: Cross-model evidence-assessment head localization. Red boxes mark the annotated heads. The horizontal and vertical axes denote head and layer indices, respectively, both starting from zero.

Model	5 chunks			20 chunks
Model	AUC	Recall@1	Recall@3	AUC	Recall@1	Recall@10
DeepDiver	$83.2\pm 0.5$	$(54.6\pm 0.4)\%$	$(83.2\pm 0.1)\%$	$84.4\pm 0.2$	$(44.6\pm 0.6)\%$	$(88.7\pm 0.1)\%$

Appendix

Table 16: Input-order sensitivity of QRAE scoring with DeepDiver-7B on BrowseComp-Plus. Entries show the reference score from Table 3

\pm

STD across input permutations. All scores use the percentage scale; STDs are in percentage points.

Dataset	Method	Success (%) $\uparrow$	Time (s) $\downarrow$	Tokens (k) $\downarrow$
BrowseComp- Plus	DeepSearch	$13.7\pm 0.5$	$29.6\pm 0.3$	$177\pm 19$
BrowseComp- Plus	QRAE	$\bm{17.4\pm 0.3}$	$\bm{20.6\pm 0.1}$	$\bm{55\pm 4}$
XBench- DeepSearch	DeepSearch	$38.4\pm 3.1$	$265.8\pm 9.4$	$311\pm 24$
XBench- DeepSearch	QRAE	$\bm{44.6\pm 1.8}$	$\bm{152.3\pm 7.7}$	$\bm{120\pm 15}$
FRAMES	DeepSearch	$40.9\pm 1.1$	$160.2\pm 5.6$	$138\pm 14$
FRAMES	QRAE	$\bm{42.7\pm 1.2}$	$\bm{91.8\pm 3.7}$	$\bm{92\pm 10}$

Appendix

Table 17: Comparison of DeepSearch and QRAE over five paired seeds (41–45) per benchmark, with every run covering all evaluation trajectories. Entries show mean

\pm

sample STD; success is in percent, time in seconds, and tokens in thousands.

Target	Scope	$\beta=0$	2	3	4	5	6
Chunk	Prefill	11	11	12	15	16	19
Chunk	All	11	11	12	14	16	19
Token	Prefill	11	11	12	12	14	15
Token	All	11	11	12	13	14	16

Appendix

Table 18: Success (%) under attention intervention. “Prefill” applies the patch only during prefill; “all” retains it through decoding.

Dataset	Method	Time (s)			Tokens (k)
Dataset	Method	Reasoning	Tool	Compression	Input	Output	Compression
BrowseComp- Plus	DeepSearch	28.2	1.3	–	175	2.4	–
BrowseComp- Plus	QRAE	17.3	0.9	2.3	48	1.3	4.9
XBench- DeepSearch	DeepSearch	144.3	131.7	–	314	4.6	–
XBench- DeepSearch	QRAE	41.1	98.6	10.7	77	2.8	37

Appendix

Table 19: Per-trajectory time and token breakdown for DeepSearch and QRAE. Time is measured in seconds and token counts in thousands (k). Input and output refer to the reasoning agent; compression costs are listed separately. Dashes indicate no compression component.

Head set	5 chunks			20 chunks
Head set	AUC	Recall@1	Recall@3	AUC	Recall@1	Recall@10
QRAE (full)	83.2	54.6%	83.2%	84.4	44.6%	88.7%
Random heads	45.7	9.9%	38.8%	53.0	5.3%	40.2%
Heads selected by QRHead	68.3	24.5%	70.0%	70.4	13.9%	63.3%

Appendix

Table 20: Head-selection comparison on BrowseComp-Plus with DeepDiver-7B. All head sets use the same QRAE scoring rule. AUC denotes AUROC; all metrics are percentages and higher is better.

LLM Context Compression

See all 62 papers

CORE: Coarse-to-Fine Search with Context Management for Deep Research
Nov 5, 2026Hoang Tran Vuong, Ngoc Quan Pham, Hung Bui +2Agentic Retrieval Information Retrieval
Qualcomm AI Research* · Qualcomm Vietnam Company Limited
EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
Mar 10, 2026Thao Do, Dinh Phu Tran, An Vo +2Efficient Language Model Inference Knowledge-Intensive QA
Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea
Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG
Sep 4, 2026Shuyu Guo, Shuo Zhang, Zhaochun RenRL for RAG LLM Inference Acceleration
Shandong University Qingdao, China · Bloomberg London, United Kingdom · Leiden University Leiden, The Netherlands

Nov 5, 2026J/K move · Enter open · S save

CORE: Coarse-to-Fine Search with Context Management for Deep Research

Hoang Tran Vuong, Ngoc Quan Pham, Hung Bui, Fatih Porikli+1

Qualcomm AI Research* · Qualcomm Vietnam Company Limited

Agentic Retrieval Information Retrieval LLM Context Management Multi-Hop Retrieval

Open paper AskPDF arXiv

Deep research agents answer complex questions through repeated search. They rely on a harness that runs their tool calls and returns evidence to their context. This creates a trade-off: Search works best over long passages, which give enough context for reliable semantic matching. But the returned evidence should be short, because every observation stays in the agent's context for the rest of the trajectory. We introduce CORE, a training-free harness that combines coarse-to-fine search with context management. CORE retrieves passages with a dense retriever. It then reuses the token-level similarities from a late-interaction reranker to return only the most relevant snippet from each passage. Context management lets the agent expand the context around sources it has already retrieved and suppresses repeated snippets. In controlled closed-search comparisons with the Tongyi DeepResearch and OpenResearcher harnesses, CORE achieves state-of-the-art answer accuracy and evidence recall across three agent models and three benchmarks (BrowseComp-Plus, BrowseComp-Wikipedia and BrowseComp-Plus-CM), on corpora containing up to 553M documents.

Abstract

LLM Context Compression

Explore similar work

CORE: Coarse-to-Fine Search with Context Management for Deep Research

EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering

Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG