Researchers often need web corpora that are not answers to a single query, but reusable collections of documents spanning many decentralized sources. We refer to this problem as thematic web data collection: assembling semantically relevant documents that share a common theme but are scattered across heterogeneous and structurally disconnected regions of the web. Existing approaches, including traversal-based methods and LLM-driven web agents, are limited by local exploration or short-horizon stopping, preventing comprehensive collection. We formulate this task as a long-horizon information foraging problem over query-induced semantic patches, where the central challenge is deciding when to continue local exploitation and when to switch to new regions. We propose PatchScout, a multi-agent framework that instantiates a class of patch-switching policies to balance local exploitation and adaptive patch switching under a fixed budget. Experiments on four live-web topics show that PatchScout substantially improves yield and domain coverage over traversal-based and search-augmented reasoning agent baselines. In an applied social science setting with an incomplete prior collection, PatchScout discovers 102 previously unidentified entities, illustrating its ability to expand existing thematic datasets.
Figures & tables
Name
Condition
Description
Threshold
ρm(τ)<η
Realized productivity below a fixed topic-dependent threshold.
Realized-Mean
ρm(τ)<ρˉ<m
Realized productivity below the mean productivity of prior patches.
Realized-EMA
ρm(τ)<θm
Realized productivity below the EMA of prior patches.
Scent-Mean
γt(h)<ρˉ<m
Information scent below the mean productivity of prior patches.
Scent-EMA
γt(h)<θm
Information scent below the EMA of prior patches.
Table 1: Re-seeding policies derived from different trigger-baseline combinations. Scent conditions use forward-looking estimates, while realized conditions use observed within-patch performance.
Universities
Fortune 500
Law Schools
ASPs
System
HR
Cov
HR
Cov
HR
Cov
HR
Cov
ACHE
18.3
3.6
4.0
2.6
6.0
6.6
0.0
0.0
DeepResearcher
–
0.3
–
0.8
–
1.0
–
17.3
Claude Deep Research
–
1.1
–
15.0
–
91.9
–
17.3
Gemini Deep Research
–
1.8
–
3.0
–
93.4
–
9.8
No-Reseed
18.7
1.7
0.8
3.0
0.1
1.0
0.1
1.5
Table 2: Harvest rate (HR%) and gold-standard domain coverage (Cov%) at a 10,000-page budget. Deep-research systems are evaluated at completion by coverage over final and intermediate URLs; HR is omitted for them because they have no comparable page-budget denominator. Best completed result per topic per metric in bold.
Figure 1: Gold-standard domain coverage (%) over pages visited. Deep-research baselines reported as completion endpoints. Traversal-only methods plateau early, while re-seeding sustains discovery, particularly on sparse topics.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Role
Default Value
k
Number of search results retained per query.
10
b
Maximum branching factor (number of links to enque in frontier from a single page).
10
η
Fixed productivity threshold for re-seeding.
Topic dependent.
ρ0
Initial patch productivity estimate.
Topic dependent.
h
Number of frontier links used to estimate information scent.
10
β
Update rate for patch productivity.
0.3
Appendix
Table 3: List of hyperparameters for PatchScout .
University
ASP
Balanced
h
β
Relevant
Domain
Relevant
Domain
Relevant
Domain
5
0.1
743
190
72
36
0.884
0.764
5
0.3
813
172
73
50
0.930
0.866
5
0.5
868
159
49
30
0.807
0.638
10
0.1
777
204
78
43
0.941
0.864
10
0.3
880
235
74
42
0.974
0.920
Appendix
Table 4: Topic-separated totals for relevant pages and relevant domains across scent-dynamic hyperparameter settings. Balanced columns are computed across both topics: for each topic, values are normalized by that topic’s best value across settings (relevant/max relevant, domains/max domains), then averaged equally across topics.
Component
Total Tokens
Tokens per Page
Total Calls
Latency per Call
Forager
198,415,172
707.71
49,015
6.44
Critic
44,390,930
158.33
38,834
4.68
Scout
3,798,127
13.54
1,666
4.02
Fetch
0
0.0000
344,588
0.75
Search
0
0.0000
7,571
2.01
Appendix
Table 5: Per component cost metrics. Token usage is aggregated across all 28 collection runs included in Table 2 . Latency per call is in seconds.
Approach
Forager Tokens
Critic Tokens
Scout Tokens
Total Tokens
DeepResearcher
-
-
-
2,258,943
No-Reseed
39,005,430
5,442,936
2,472
44,450,838
Fixed-Reseed
23,891,382
6,762,778
563,082
31,217,242
Threshold
21,800,783
6,173,678
822,458
28,796,919
Realized-Mean
29,978,153
6,195,291
341,875
36,515,319
Realized-EMA
30,342,823
6,054,647
259,603
36,657,073
Appendix
Table 6: Token usage per approach, aggregated across all 4 topics included in Table 2 .
Universities
Fortune 500
Law Schools
ASPs
System
Tok
Pg/1K
Cov/1M
Tok
Pg/1K
Cov/1M
Tok
Pg/1K
Cov/1M
Tok
Pg/1K
Cov/1M
DeepResearcher
477K
0.16
10.5
340K
0.01
11.7
680K
0.00
2.9
761K
0.08
30.2
Claude Deep Research
–
–
–
–
–
–
–
–
–
–
–
–
Gemini Deep Research
707K
0.16
46.7
1.1M
0.08
14.1
1.8M
0.14
102.7
1.2M
0.07
10.7
No-Reseed
9.9M
0.19
3.1
11.6M
0.01
1.3
11.6M
0.00
0.2
11.4M
0.00
0.2
Fixed-Reseed
6.7M
0.93
44.4
7.6M
0.20
12.0
9.1M
0.30
15.7
7.8M
0.05
5.3
Appendix
Table 7: Full-budget token efficiency per topic. Tok is total LLM tokens; Pg/1K is relevant pages discovered per 1,000 tokens; Cov/1M is the number of reference-set entities discovered per million tokens (not a percentage). Claude Deep Research exposes no usage data. Best value per column in bold.
Universities
Fortune 500
Law Schools
ASPs
System
Pg
Cov
Pg
Cov
Pg
Cov
Pg
Cov
DeepResearcher
78
0.3
56
0.8
28
1.0
64
17.3
Claude Deep Research
-
1.1
-
15.0
-
91.9
-
17.3
Gemini Deep Research
113
1.8
82
3.0
247
93.4
87
9.8
No-Reseed
716
1.4
893
2.6
1,556
0.5
1,063
1.5
Fixed-Reseed
1,072
5.7
1,405
6.0
1,995
34.0
1,568
8.3
Appendix
Table 8: Token-matched comparison: pages visited (Pg) and gold-standard domain coverage (Cov%) for each PatchScout system if stopped at the same total LLM token budget the DeepResearcher baseline used for that topic. Claude and Gemini Deep Research rows show final, unmatched results for reference: Gemini reports only total usage and Claude reports none, so neither can be truncated. ACHE has no LLM token cost and is omitted. Best completed Pg/Cov per topic in bold (baseline reference rows excluded from bolding).
Universities
Fortune 500
Law Schools
ASPs
System
HR
Cov
HR
Cov
HR
Cov
HR
Cov
Random-Reseed
72.1
4.6
7.6
2.2
27.8
25.4
4.9
9.0
Random-Forager
53.5
1.2
10.6
2.0
21.5
11.7
4.6
12.0
Upfront-Scout
64.3
8.3
16.9
6.6
28.6
28.9
3.7
10.5
Realized-EMA
86.2
4.4
23.2
19.8
25.3
70.6
3.7
26.3
Scent-EMA
68.3
15.5
20.1
24.6
37.2
86.8
5.6
35.3
Appendix
Table 9: Ablation results. Each topic reports harvest rate (HR%) and gold-standard domain coverage (Cov%), matching the main results table. Best value within each topic/metric is bolded.
Reference model
DeepSeek
Qwen 9B
Qwen 27B
Qwen 397B
Prediction
Agr
Prec
Rec
Agr
Prec
Rec
Agr
Prec
Rec
Agr
Prec
Rec
DeepSeek
–
–
–
86.1
81.1
86.9
87.4
79.9
90.8
86.9
77.6
91.9
Qwen 9B
86.1
86.9
81.1
–
–
–
88.7
83.6
88.6
88.0
80.9
89.4
Qwen 27B
87.4
90.8
79.9
88.7
88.6
83.6
–
–
–
92.0
87.8
91.5
Qwen 397B
86.9
91.9
77.6
88.0
89.4
80.9
92.0
91.5
87.8
–
–
–
Appendix
Table 10: Pairwise agreement between critic models on the binary relevance decision. For each pair, agreement (Agr%), precision (Prec%), and recall (Rec%) treat the row model as the prediction and the column model as the reference. All four models judged the same 10,000 pages, with the same binary Yes/No prompt; only the model differs.
Topic
Orig. label
Not rel.
Topical, no ev.
Partial ev.
Direct ev.
Universities
Relevant
5.8
37.9
9.3
47.1
Universities
Not relevant
72.9
19.5
2.0
5.6
Fortune 500
Relevant
5.8
9.8
36.3
48.1
Fortune 500
Not relevant
85.4
10.3
3.8
0.4
Law Schools
Relevant
2.8
22.4
41.8
32.9
Law Schools
Not relevant
66.6
28.0
5.2
0.2
Appendix
Table 11: Critic 4-way relabel of sampled pages, reclassified in their original critic batches (same pages, same order, same batch-mates as the first pass): full breakdown of where originally relevant and originally not-relevant pages land under the four graded labels (not_relevant / topically_related_no_evidence / partial_evidence / direct_evidence). Rows sum to 100% (rounding). Combined pools all four topics.
The dominant approach to building domain-specific pretraining corpora is to filter large web archives such as CommonCrawl. This works well for popular domains but breaks down for specialized ones, where relevant content is sparse and often beyond the reach of popularity-driven crawlers. We present Data Scout, which inverts this: instead of filtering an archive, it directs a targeted crawl. An LLM expands a root topic into a taxonomy and thousands of search queries; the returned URLs (seeds) are grouped by subdomain and screened with a user-supplied classifier (the probe), admitting each subdomain on the basis of a small sample. This works because relevance has a sharp boundary at the subdomain level: in mathematics, a page is 21x more likely to be relevant than one on a sibling subdomain. With the FineMath classifier as the probe, 21.9% of crawled pages are high-quality math content, 70x the 0.31% rate from filtering a comparable web sample, so the crawl wastes far less effort. But the payoff is not just efficiency: 63.2% of these pages are missing from CommonCrawl altogether, yet just as useful for training. Continued pretraining of Llama-3.2-3B on 1.9B Data Scout tokens matches FineMath corpus on GSM8k. Because the probe is the only domain-specific component, Data Scout can in principle apply to any domain with such a classifier.
Large language model (LLM)-based web agents automate web navigation and data collection. However, live web data collection demands capabilities beyond task completion: agents must discover site-internal pages and retrieve text, image, and video artifacts in an accessible form within a fixed interaction budget. We formulate this setting as budget-constrained, site-level multimodal web data collection and propose Budget-constrained and Reliable Agent (BaRA). BaRA performs breadth-first search (BFS)-based link discovery with liveness verification to filter hallucinated and dead links, then validates extracted multimodal artifacts using rule-based provenance and accessibility checks. A history-based self-reflection module recovers from execution failures and incomplete outputs. On controlled synthetic and real-world websites, BaRA consistently improves valid-link discovery and download-valid multimodal extraction over existing agents. Our code is available at https://github.com/MLAI-Yonsei/BaRA-Agent.
Soojeong Lee, Joseph Lee, Yongseong Cho +3
Department of Statistics and Data Science, Yonsei University, Seoul, Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · HUSTLERS Corp., Seoul, Republic of Korea
During research, domain experts often ask analytical questions whose answers require integrating data from a wide range of web sources. Thus, they must spend substantial effort searching, extracting, and organizing raw data before analysis can begin. We formalize this process as the SODIUM task, where we conceptualize open domains such as the web as latent databases that must be systematically instantiated to support downstream querying. Solving SODIUM requires (1) conducting in-depth and specialized exploration of the open web, which is further strengthened by (2) exploiting structural correlations for systematic information extraction and (3) integrating collected information into coherent, queryable database instances. To quantify the challenges in automating SODIUM, we construct SODIUM-Bench, a benchmark of 105 tasks derived from published academic papers across 6 domains, where systems are tasked with exploring the open web to collect and aggregate data from diverse sources into structured tables. Existing systems struggle with SODIUM tasks: we evaluate 6 advanced AI agents on SODIUM-Bench, with the strongest baseline achieving only 46.5% accuracy. To bridge this gap, we develop SODIUM-Agent, a multi-agent system composed of a web explorer and a cache manager. Powered by our proposed ATP-BFS algorithm and optimized through principled management of cached sources and navigation paths, SODIUM-Agent conducts deep and comprehensive web exploration and performs structurally coherent information extraction. SODIUM-Agent achieves 91.1% accuracy on SODIUM-Bench, outperforming the strongest baseline by approximately 2 times and the weakest by up to 73 times.