Rethinking Deep Research from the Perspective of Web Content Distribution Matching
Organizations: School of Computer science and engineering, Sun Yat-sen University · CSE, The Hong Kong University of Science and Technology · Sydney AI Centre, The University of Sydney · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · DSA Thrust, The Hong Kong University of Science and Technology (GuangZhou) · TMLR Group, Department of Computer Science, Hong Kong Baptist University
Abstract
Despite the integration of search tools, Deep Search Agents often suffer from a misalignment between reasoning-driven queries and the underlying web indexing structures. Existing frameworks treat the search engine as a static utility, leading to queries that are either too coarse or too granular to retrieve precise evidence. We propose WeDas, a Web Content Distribution Aware framework that incorporates search-space structural characteristics into the agent's observation space. Central to our method is the Query-Result Alignment Score, a metric quantifying the compatibility between agent intent and retrieval outcomes. To overcome the intractability of indexing the dynamic web, we introduce a few-shot probing mechanism that iteratively estimates this score via limited query accesses, allowing the agent to dynamically recalibrate sub-goals based on the local content landscape. As a plug-and-play module, WeDas consistently improves sub-goal completion and accuracy across four benchmarks, effectively bridging the gap between high-level reasoning and low-level retrieval.
Figures & tables
| Method | Outcome | TF-IDF | Jaccard | NLS |
|---|---|---|---|---|
| Baseline | Success | 0.401 | 0.331 | 0.148 |
| Failure | 0.269 | 0.238 | 0.104 | |
| WeDAS | Success | 0.431 | 0.364 | 0.165 |
| Failure | 0.292 | 0.241 | 0.108 |
| pass@1 | pass@3 | |
|---|---|---|
| 0 | 49.51 | 72.82 |
| 1 | 57.28 | 74.76 |
| 2 | 54.37 | 74.76 |