OGDA: Ontology-Grounded Data Agents for Reliable Analysis over Heterogeneous Data Lakes
Organizations: The Hong Kong University of Science and Technology · Hong Kong Baptist University
Abstract
Full-pipeline data analysis over real-world data lakes requires LLM agents to locate and integrate information across large collections of tables with different schemas. Existing data agents follow task-specific exploration, which exhausts the context window in massive lakes. As the context fills, the agent's reasoning degrades, leading it to locate and process incorrect data files. These errors propagate into the final answer. To overcome these limitations, we introduce Ontology-Grounded Data Agent (OGDA), which utilizes a compact ontology to reduce exploration overhead and constrain data processing. OGDA constructs the ontology through agentic hypothesis testing. At each step, a proposer agent induces schemas, concepts, and relations from raw sources, and a verifier checks each proposal against the underlying data. During task execution, the ontology serves as a representation of the data lake for the agent to explore. Every data-processing action is constrained by a task graph, which the agent can extend only after observing the semantic meaning of the data. Across KramaBench and DiscoveryBench with three LLM backbones, OGDA outperforms the strongest baselines by up to 10.82 points. These results suggest that explicitly modeling the data lake is a key step toward reliable end-to-end data agents.
Figures & tables
| Model | Method | KramaBench | DiscoveryBench | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Answer | Pipeline | HMS | ctxF1 | varF1 | |||||||
| Score | Score | Score | Score | Score | |||||||
| DeepSeek- V4.1-Flash | Smolagents | ||||||||||
| Reflexion | |||||||||||
| DS-STAR | |||||||||||
| Data Interpreter | |||||||||||
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Construction model | Lakes | Schemas | Fields | Tables | Concepts | Links |
|---|---|---|---|---|---|---|---|
| KramaBench | DeepSeek-V4.1-Flash | 6 | 100 | 1,577 | 1,723 | 294 | 1,791 |
| GPT-6-Luna | 6 | 78 | 1,487 | 602 | 33 | 136 | |
| MiMo-V2.6-Pro | 6 | 80 | 779 | 1,706 | 116 | 336 | |
| DiscoveryBench | DeepSeek-V4.1-Flash | 10 | 21 | 837 | 21 | 210 | 1,285 |
| GPT-6-Luna | 10 | 20 | 800 | 20 | 88 | 265 | |
| MiMo-V2.6-Pro | 10 | 21 | 838 | 21 | 183 | 554 |
| Model | Method | Saved answers | Full | Partial | Dynamic only | No evidence | Empty code | Missing output | Explicit units / 311 | Explicit % |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek- V4.1-Flash | smolagents | 90/104 | 49 | 22 | 7 | 12 | 14 | 0 | 106/311 | 34.08 |
| Reflexion | 104/104 | 59 | 19 | 6 | 20 | 0 | 0 | 134/311 | 43.09 | |
| DS-STAR | 91/104 | 55 | 20 | 7 | 10 | 12 | 0 | 115/311 | 36.98 | |
| Data Interpreter | 102/104 | 60 | 26 | 10 | 8 | 0 | 0 | 141/311 | 45.34 | |
| OGDA | 103/104 | 67 | 25 | 5 | 7 | 0 | 0 | 182/311 | 58.52 | |
| DatawiseAgent | 89/104 | 70 | 26 | 3 | 5 | 0 | 0 | 188/311 | 60.45 |
| Model | Method | Generation / task | Construction total | Construction / task | Total / task |
|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash | smolagents | 0.01467 | 0.0000 | 0.00000 | 0.01467 |
| Reflexion | 0.03307 | 0.0000 | 0.00000 | 0.03307 | |
| DS-STAR | 0.02527 | 0.0000 | 0.00000 | 0.02527 | |
| Data Interpreter | 0.00535 | 0.0000 | 0.00000 | 0.00535 | |
| OGDA | 0.01242 | 4.2127 | 0.01228 | 0.02470 | |
| DatawiseAgent | 0.00936 | 0.0000 | 0.00000 | 0.00936 |
Data Analysis Agents
Data analysis agents use language models and tools to explore datasets, run analyses, and produce answers or reports grounded in evidence.
See all 120 papers