Rethinking Deep Research from the Perspective of Web Content Distribution Matching
Authors: Zixuan Yu, Zhenheng Tang, Tongliang Liu, Chengqi Zhang, Xiaowen Chu, Bo Han
Organizations: School of Computer science and engineering, Sun Yat-sen University · CSE, The Hong Kong University of Science and Technology · Sydney AI Centre, The University of Sydney · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · DSA Thrust, The Hong Kong University of Science and Technology (GuangZhou) · TMLR Group, Department of Computer Science, Hong Kong Baptist University
Despite the integration of search tools, Deep Search Agents often suffer from a misalignment between reasoning-driven queries and the underlying web indexing structures. Existing frameworks treat the search engine as a static utility, leading to queries that are either too coarse or too granular to retrieve precise evidence. We propose WeDas, a Web Content Distribution Aware framework that incorporates search-space structural characteristics into the agent's observation space. Central to our method is the Query-Result Alignment Score, a metric quantifying the compatibility between agent intent and retrieval outcomes. To overcome the intractability of indexing the dynamic web, we introduce a few-shot probing mechanism that iteratively estimates this score via limited query accesses, allowing the agent to dynamically recalibrate sub-goals based on the local content landscape. As a plug-and-play module, WeDas consistently improves sub-goal completion and accuracy across four benchmarks, effectively bridging the gap between high-level reasoning and low-level retrieval.
Figures & tables
Figure 1: Distributions of query–observation alignment metrics (TF-IDF, Jaccard, and normalized Levenshtein similarity) for successful vs. failed trajectories, highlighting the structural misalignment between agent-generated queries and retrieved web content.
Figure 2: Framework of Web Content Distribution Aware Search (WeDAS)
Method
Outcome
TF-IDF ↑
Jaccard ↑
NLS ↑
Baseline
Success
0.401
0.331
0.148
Failure
0.269
0.238
0.104
WeDAS
Success
0.431
0.364
0.165
Failure
0.292
0.241
0.108
Table 3: Comparison of Different Metrics of Query-Result Alignment between Baseline and WeDAS.
T
pass@1
pass@3
0
49.51
72.82
1
57.28
74.76
2
54.37
74.76
Table 4: Ablation on GAIA with different max probe iterations T (GPT-5-mini as agent model).
Existing deep-research agents use a Search--Visit workflow that retrieves whole webpages without considering the structure they expose through titles, headings, sections, and metadata. This prevents agents from directly constraining retrieval to parts of a webpage and often carries irrelevant content into their context. We introduce Sieve, a search--inspect--fetch strategy driven by a Boolean Query Language (BQL): it searches webpage fields to filter candidates, uses an interchangeable ranker to order them, presents structure-rich result cards for inspection, and fetches only selected sections. Across three QA collections, Sieve is more accurate than the strongest conventional Search--Visit configuration on each collection while using 20.7--50.6% fewer tokens. Boolean filtering improves every tested ranker, and the accuracy--context advantage persists across retriever choices and agent backbones. Our implementation is included in the SkimSearchAgent library at https://github.com/ielab/skim-search-agent.
Shuai Wang, Haodong Chen, Yu Yin +3
The University of Queensland, Brisbane, Australia · CSIRO, Brisbane, Australia
Deep Research agents are rapidly emerging as primary consumers of modern retrieval systems. Unlike human users who issue and refine queries without documenting their intermediate thought processes, Deep Research agents generate explicit natural language reasoning before each search call, revealing rich intent and contextual information that existing retrievers entirely ignore. To exploit this overlooked signal, we introduce: (1) Reasoning-Aware Retrieval, a retrieval paradigm that jointly embeds the agent's reasoning trace alongside its query; and (2) DR-Synth, a data synthesis method that generates Deep Research retriever training data from standard QA datasets. We demonstrate that both components are independently effective, and their combination yields a trained embedding model, AgentIR-4B, with substantial gains. On the challenging BrowseComp-Plus benchmark, AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch, compared to 50% with conventional embedding models twice its size, and 37% with BM25. Code and data are available at: https://texttron.github.io/AgentIR/.
Zijian Chen, Xueguang Ma, Shengyao Zhuang +3
University of Waterloo · University of Queensland · Carnegie Mellon University
Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additional hops, and longer trajectories. These global properties, however, are only indirect proxies for the local retrieval capabilities required during search. To address this mismatch, we introduce latent anchor reasoning, which consists of resolving an unnamed retrieval anchor from descriptive specifications and transferring the recovered anchor into a subsequent information demand. This primitive retrieval unit decomposes deep search into chains of coupled operations and organizes question construction around anchor resolution and relation transfer, without prescribing a canonical search path. Based on this formulation, we propose PrimeSeeker, a capability-oriented framework that constructs web-grounded anchor structures and jointly derives a question and a reference evidence skeleton. The skeleton preserves supporting evidence from construction and guides expert generation through extractive highlights of current tool observations. These highlights are removed before supervised fine-tuning, while the skeleton is subsequently reused to audit reference-step coverage for reinforcement-learning rewards. We construct 9,221 expert trajectories, training a 30B search agent. Across five deep-search benchmarks, PrimeSeeker achieves strong performance, while reference-step optimization further improves the supervised policy. The resulting trajectories exhibit low retrieval redundancy, and fixed-budget evaluation shows strong solution coverage with substantially fewer tool calls than long-horizon systems.
Linzhi Peng, Hanting Chen, Heng Chang +3
Beihang University · Huawei Technologies Ltd. · Tsinghua University