SODIUM: From Open Web Data to Queryable Databases

Chuxuan Hu; Philip Li; Maxwell Yang; Daniel Kang

cs.DBMar 19, 2026

SODIUM: From Open Web Data to Queryable Databases

Authors: Chuxuan Hu, Philip Li, Maxwell Yang, Daniel Kang

Organizations: UIUC

Abstract

During research, domain experts often ask analytical questions whose answers require integrating data from a wide range of web sources. Thus, they must spend substantial effort searching, extracting, and organizing raw data before analysis can begin. We formalize this process as the SODIUM task, where we conceptualize open domains such as the web as latent databases that must be systematically instantiated to support downstream querying. Solving SODIUM requires (1) conducting in-depth and specialized exploration of the open web, which is further strengthened by (2) exploiting structural correlations for systematic information extraction and (3) integrating collected information into coherent, queryable database instances. To quantify the challenges in automating SODIUM, we construct SODIUM-Bench, a benchmark of 105 tasks derived from published academic papers across 6 domains, where systems are tasked with exploring the open web to collect and aggregate data from diverse sources into structured tables. Existing systems struggle with SODIUM tasks: we evaluate 6 advanced AI agents on SODIUM-Bench, with the strongest baseline achieving only 46.5% accuracy. To bridge this gap, we develop SODIUM-Agent, a multi-agent system composed of a web explorer and a cache manager. Powered by our proposed ATP-BFS algorithm and optimized through principled management of cached sources and navigation paths, SODIUM-Agent conducts deep and comprehensive web exploration and performs structurally coherent information extraction. SODIUM-Agent achieves 91.1% accuracy on SODIUM-Bench, outperforming the strongest baseline by approximately 2 times and the weakest by up to 73 times.

Figures & tables

Figure 1. Solving Sodium is driven by ( $\mathcal{C}$ 0) in-depth exploration of specialized websites, and strengthened through ( $\mathcal{E}$ 1) exploiting structural correlations for systematic information extraction and ( $\mathcal{E}$ 2) integrating collected information into coherent, queryable database instances.

Domain	# Surveys	# Tables	# Base Domains	# Primary Keys	# Cols	# Cells	Avg. # Search Depth
Demographics	8	8	7	29	49	188	3.75
Sports	7	30	9	162	184	885	4.35
Finance	11	22	10	99	85	379	3.33
Economics	5	12	6	46	41	154	3.39
Food	9	19	9	75	90	354	3.12
Climate	8	14	8	47	57	189	3.08

Table 1. Statistics of SodiumBench .

Figure 2. Data collection workflow of SodiumBench .

System	Cell Acc. ( $\mathcal{C}0$ )		Row Acc. ( $\mathcal{E}1$ )		Column Acc. ( $\mathcal{E}1$ )		Table Acc. ( $\mathcal{E}2$ )		Task Acc. ( $\mathcal{C}0$ , $\mathcal{E}1$ , $\mathcal{E}2$ )
System	GPT-4o	Match	GPT-4o	Match	GPT-4o	Match	GPT-4o	Match	GPT-4o	Match
AG2	45.51 (2)	36.39 (2)	15.10 (2)	8.17 (2)	18.69 (2)	15.61 (2)	1.90 (3)	0.95 (2)	46.48 (2)	35.90 (2)
AutoGPT	35.74 (3)	29.32 (3)	10.89 (3)	5.22 (3)	10.26 (4)	7.44 (4)	1.90 (3)	0.95 (2)	34.73 (3)	28.42 (3)
AutoGen	31.32 (4)	26.57 (4)	7.84 (4)	3.33 (4)	13.71 (3)	11.21 (3)	2.86 (2)	0.95 (2)	29.54 (4)	24.70 (4)
OpenAI ResearchBot	16.71 (5)	5.21 (5)	1.56 (5)	0.43 (5)	3.84 (5)	0.61 (5)	0.00 (5)	0.00 (5)	16.61 (5)	4.85 (5)
WebVoyager	2.05 (6)	1.12 (6)	0.24 (6)	0.00 (6)	0.61 (6)	0.14 (6)	0.00 (5)	0.00 (5)	2.93 (6)	1.54 (6)

Table 2. Accuracy (

\uparrow

) of different agents on SodiumBench in percentages (%). GPT-4o columns refer to LLM-as-a-judge evaluations, and Match columns refer to exact match evaluations. The optimal values for each metric are highlighted with both bold and underline formatting. We report the rank of each agent under each metric in italicized parentheses.

Figure 4. LLM-as-a-judge accuracy of different agents across different search depths

D

		$\ell=0$	$\ell=1$	$\ell=2$	$\ell\geq 3$
# Cells	Embedded	306	185	69	44
# Cells	Standalone	276	1006	524	343
Accuracy (%)	Embedded	91.2	91.9	92.8	93.2
Accuracy (%)	Standalone	81.2	86.2	93.9	81.3
Prune rate (%)	Embedded	–	63.0	96.6	93.5
Prune rate (%)	Standalone	–	78.8	96.9	93.6

Table 3. The execution statistics of SodiumAgent ’s web explorer across ATP-BFS levels

\ell

with (as an embedded component of SodiumAgent ) and without the cache manager (as a standalone agent).

Component	Mode	# Cells	Acc. (%) $\uparrow$	Cost ( $)$ \downarrow$
Web Explorer	Start from base	104	87.50	1.08
Web Explorer	Start from cached paths	500	92.60	0.91
Cache Manager	Left	885	93.67	0.23
Cache Manager	Up	660	89.24	0.16

Table 4. Comparison of SodiumAgent components in the utility of cached results during cell filling. Accuracy is measured using LLM-as-a-judge evaluation, which provides finer-grained assessment, as analyzed in Section 5.2 .

Web Search Agents

Web search agents use language models to find, verify, and synthesize information from the web for multi-step tasks.

See all 41 papers

Benchmarking Hybrid Deep Research Across Database Querying and Web Search
Sep 8, 2026Ruofan Wu, Peiran Xu, Xiaolong Li +9Deep Research Agents Agentic Search
University of Houston · University of California, Los Angeles · The University of Hong Kong +2
Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction
Apr 29, 2026Yuxuan Huang, Yihang Chen, Zhiyuan He +6Multi-Agent LLM Systems Agentic Search
University of Liverpool · University College London · Huawei Noah’s Ark Lab
WANDR: A Benchmark for Wide and Deep Research
Aug 14, 2026Vitaliy Polshkov, Marcin Pitera, Jeremy Yang +7AI Agent Benchmarks Deep Research Agents
Perplexity

Sep 8, 2026J/K move · Enter open · S save

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

Ruofan Wu, Peiran Xu, Xiaolong Li, Fan Shu+8

University of Houston · University of California, Los Angeles · The University of Hong Kong+2

Deep Research Agents Agentic Search Question Answering Hybrid IR Agent Benchmarks

Open paper AskPDF arXiv

While autonomous agents have made significant strides in "deep research" by iteratively navigating the open web to synthesize information, real-world problem-solving is rarely confined to a single environment. Complex analytical tasks inherently require agents to weave together evidence from both ambiguous unstructured text (e.g., the open web) and highly precise structured data (e.g., relational databases). However, existing benchmarks evaluate these modalities in isolation, failing to capture the critical "handoff" - the ability to preserve constraints when moving evidence between systems. We introduce HybridDeepResearch, to our knowledge the first deep-research benchmark that requires both web search and SQL to form a complete, verifiable answer. The benchmark contains 380 tool-dependent tasks grounded in LiveSQLBench-Base-Lite databases and public web corpora, validated through automated checks and human review, and covering three reasoning patterns: SQL2S, S2SQL, and Parallel. Evaluations across proprietary and open-weight models under various agentic scaffolds reveal that even state-of-the-art models like GLM-5.2, Claude-Sonnet-4.6 and GPT-5 achieve only about 50-54% Pass@8 on the hard subset. Notably, results show that directional reasoning is substantially more difficult than parallel intersection, highlighting that bridging structured and unstructured information spaces without losing constraints remains a major open challenge for agentic systems. Code and datasets are publicly available at GitHub (https://github.com/Snowflake-AI-Research/HybridDeepResearch) and Hugging Face (https://huggingface.co/datasets/Snowflake/HybridDeepResearch).

Abstract

Web Search Agents

Explore similar work

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction

WANDR: A Benchmark for Wide and Deep Research