cs.AIMar 9, 2026

UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking

Authors: Chang Liu, Chuqiao Kuang, Tianyi Zhuang, Yuxin Cheng, Huichi Zhou, Xiaoguang Li, Lifeng Shang

Organizations: Huawei Technologies Ltd. · The University of Hong Kong · University College London

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent advancements in LLM-based information-seeking agents have achieved record-breaking performance on established benchmarks. However, these agents remain heavily reliant on search-engine-indexed knowledge, leaving a critical blind spot: Unindexed Information Seeking (UIS). This paper identifies and explores the UIS problem, where vital information is not captured by search engine crawlers, such as overlooked content, dynamic webpages, and embedded files. Despite its significance, UIS remains an underexplored challenge. To address this gap, we introduce UIS-QA, the first dedicated UIS benchmark, comprising 110 expert-annotated QA pairs. Notably, even state-of-the-art agents experience a drastic performance drop on UIS-QA (e.g., from 70.90 on GAIA and 46.70 on BrowseComp-zh to 24.55 on UIS-QA), underscoring the severity of the problem. To mitigate this, we propose UIS-Digger, a novel multi-agent framework that incorporates dual-mode browsing and enables simultaneous webpage searching and file parsing. With a relatively small ∼\sim30B-parameter backbone LLM optimized using SFT and RFT training strategies, UIS-Digger sets a strong baseline at 27.27%, outperforming systems integrating sophisticated LLMs such as O3 and GPT-4.1. This demonstrates the importance of proactive interaction with unindexed sources for effective and comprehensive information-seeking. Our work not only uncovers a fundamental limitation in current agent evaluation paradigms but also provides the first toolkit for advancing UIS research, defining a new and promising direction for robust information-seeking systems. The dataset has been released at: https://huggingface.co/datasets/UIS-Digger/UIS-QA.

Figures & tables

Figure 1: UIS problem. Previous information-seeking agents (bottom) focus primarily on indexed information and thus often fail to gather the evidence needed to answer complex queries, either rejecting to answer or generate hallucinations. In contrast, UIS agents (top) are equipped with additional tools and fine-tuned to excavate unindexed information, thus capable of interacting with websites deeply and solve UIS tasks reliably.
 Task TypeReal-Wold Web EnvironmentUnknown StartpointUnindexed-Information DependenceFinal Answer- oriented Evaluation
WebArenaComputer Use✗✗-✗
Mind2WebComputer Use✗✗-✗
Mind2Web-LiveComputer Use✓✗✓✗
Online-Mind2WebComputer Use✓✗✓✗
Browsecomp-en/zhInfo Seeking✓✓✗✓
xbench-DeepSearchInfo Seeking✓✓✗✓
Table 1: Comparison of our UIS-QA and existing benchmarks.
Figure 2: UIS-Digger multi-agent system. Planner, web searcher, web surfer and file reader works together to solve UIS problems. The web surfer can switch between textual- and visual-mode to observe webpages and hence make next-step decisions. Zoom-in for better view.
Figure 3: QA Pairs Construction Pipeline. ( Left ): Procedure for constructing QA pairs using real-world information. First, homepages potentially containing deep navigation structures and informative content are collected. UIS-Digger then explores these homepages to extract information from pages requiring multiple navigation steps. The collected information subsequently serves as context for query generation. ( Right ): Procedure for constructing QA pairs based on simulated webpages. We identify browsing actions that UIS-Digger struggles to perform, and generate webpages (along with a JSON database containing relevant statistics) that incorporate these actions. QA pairs are then generated using the information from the JSON database of these simulated webpages.
NamesAction SpaceBackboneUIS-QAGAIABC-zh
crawl(s)visualfilebrowser
Direct Inference
DeepSeek-V3.1----DeepSeek-V3.11.8--
Claude-sonnet-4----Claude-S42.7--
GPT-5----GPT-50.9--
Commercial System
Table 2: Evaluation results on UIS-QA, GAIA, and BrowseComp-zh (BC-zh). † indicates reasoning-oriented LLMs. ‡ denotes results measured on GAIA-text-103 rather than the full GAIA benchmark. § indicates that the UIS-QA score for Memento ( Zhou et al., 2025a ) is reported without using its case bank, since UIS is a new task type and only limited cases have been previously allocated. Action spaces including crawl (read webpage content), visual (read images), download file and opearte browser are included.
Figure 4: Action analysis. ( left ): Search behaviors of UIS-Digger and three baseline methods. The pie charts show the proportions of cases where the agent successfully retrieves the root URL via search, and whether the root URL is subsequently accessed through crawling or browsing. ( Right ): Action frequency distributions of correct and incorrect cases for Pangu-38B UIS-Digger at different training stages. Zoom-in for best view.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

 UIS-QABC-zhGAIAFinSearchComp(T2/T3)
Pangu-38B9.112.125.248/3.4
Pangu-SFT22.730.842.769.0/5.7
Pangu-RFT27.332.550.573.0/11.4
Appendix
Table 3: Performance of each training stage across different benchmarks.
Figure 8

Explore similar work

CardsList
Apr 18, 2026

SeekerGym: A Benchmark for Reliable Information Seeking

Despite their substantial successes, AI agents continue to face fundamental challenges in terms of trustworthiness. Consider deep research agents, tasked with searching for information relevant to a given topic-while AI agents can perform effective information retrieval, there is little guarantee regarding the completeness of this information. Gaps in retrieved information can leave biases that mislead users even if the information they are given is correct and relevant. We introduce SeekerGym, a benchmark designed to evaluate the completeness of information retrieved by AI agents. In addition, SeekerGym also measures how well agents quantify their uncertainty in the completeness of their information; if an agent fails to retrieve all relevant information, it is useful for it to at least quantify how much might be missing. At a high level, each task in SeekerGym is a document (e.g., a Wikipedia article), and the AI agent must issue queries to retrieve passages from that document. Intuitively, the document comprehensively covers a topic, so the ability to retrieve its sections directly measures completeness of information retrieval. In addition to Wikipedia, we also consider machine learning survey papers, where the goal is to retrieve relevant sections of a survey paper. We benchmark several models and algorithms; the best approaches retrieve 42.5% of passages on Wikipedia and 29.2% on ML Surveys, leaving substantial room for improvement.

Remy Kim, Minseung Lee, Shuo Li +1

University of Pennsylvania, Curation Labs · Curation Labs · Google DeepMind +1

AI Agent ReliabilityAgentic SearchAI Agent BenchmarksLLM Uncertainty EstimationIR Evaluation
AskPDF
Apr 3, 2026

InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seeking

Recent agentic search systems have made substantial progress by emphasising deep, multi-step reasoning. However, this focus often overlooks the challenges of wide-scale information synthesis, where agents must aggregate large volumes of heterogeneous evidence across many sources. As a result, most existing large language model agent systems face severe limitations in data-intensive settings, including context saturation, cascading error propagation, and high end-to-end latency. To address these challenges, we present \framework, a hierarchical framework based on principle of near-decomposability, containing a strategic Host, multiple Managers and parallel Workers. By leveraging aggregation and reflection mechanisms at the Manager layer, our framework enforces strict context isolation to prevent saturation and error propagation. Simultaneously, the parallelism in worker layer accelerates the speed of overall task execution, mitigating the significant latency. Our evaluation on two complementary benchmarks demonstrates both efficiency (3−5× 3-5 \times speed-up) and effectiveness, achieving a 8.4%8.4\% success rate on WideSearch-en and 52.9%52.9\% accuracy on BrowseComp-zh. The code is released at https://github.com/agent-on-the-fly/InfoSeeker

Ka Yiu Lee, Yuxuan Huang, Zhiyuan He +5

Huawei Noah’s Ark Lab · University of Liverpool · University College London

Multi-Agent LLM SystemsWeb Search Agents
AskPDF
Mar 4, 2026

iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topics

With the emergence of search-enabled generative QA systems, users are increasingly turning to tools that browse, aggregate, and reconcile evidence across multiple sources on their behalf. Yet many widely used QA benchmarks remain answerable by retrieving a single relevant passage, making them poorly suited for measuring cross-source sensemaking, such as integrating evidence, tracking causal links, and resolving dependencies across facets of a topic. We present iAgentBench, a dynamic ODQA benchmark that targets these higher-level information needs while keeping questions natural and grounded in realistic information-seeking behavior. iAgentBench draws seed topics from real-world attention signals and uses common user intent patterns to construct user-like questions whose answers require combining evidence from multiple sources, not just extracting a single snippet. Each instance is released with traceable evidence and auditable intermediate artifacts that support contamination checks and enable fine-grained diagnosis of failures in retrieval versus synthesis. Experiments across multiple LLMs show that retrieval improves accuracy, but retrieval alone does not reliably resolve these questions, underscoring the need to evaluate evidence use, not just evidence access.

Preetam Prabhu Srikar Dammu, Arnav Palkhiwala, Tanya Roosta +1

University of Washington Seattle, Washington, USA · UC Berkeley Berkeley, California, USA

AI Agent BenchmarksLLM Agent EvaluationKnowledge-Intensive QAMulti-Hop QAIR Evaluation
AskPDF