Searching for What Matters: Open-Ended Discovery of Multiple Hypotheses with Agentic Data Exploration

Sanchit Kabra; Nurendra Choudhary; Jiangshu Du; Eddie Huang; Danai Koutra; Chandan K Reddy

cs.LGOct 13, 2026

Searching for What Matters: Open-Ended Discovery of Multiple Hypotheses with Agentic Data Exploration

Authors: Sanchit Kabra, Nurendra Choudhary, Jiangshu Du, Eddie Huang, Danai Koutra, Chandan K Reddy

Organizations: Virginia Tech · Amazon

Abstract

Discovering what drives an outcome from data is central to science and often requires deciding what is worth investigating before a specific hypothesis is known. Yet most existing hypothesis generation settings substantially narrow this search by providing a research question, target relationship, or curated context. We introduce open-ended multiple hypothesis discovery, where a system receives only a large tabular dataset and an outcome and must discover multiple data-supported hypotheses without knowing in advance the relevant variables, functional forms, conditions, or number of underlying mechanisms. To study this problem, we introduce a benchmark built from structural causal models grounded in real-world data schemas, containing 351 planted hypotheses across 15 tasks and five domains, with 53--65 variables and 1M--10M rows per task. The causal model is hidden from the discovery method and retained only by the evaluator, allowing proposed hypotheses to be tested through interventions and separating observational discovery from causal validation. We further propose HypAgent, which treats the dataset as an interactive environment and performs iterative search by proposing candidate directions, triaging and refining promising hypotheses, and removing explained signal so that each accepted discovery redirects subsequent search. HypAgent achieves 0.47 precision, 0.51 recall, and 0.49 F1, compared with 0.35 F1 for the strongest baseline. Our analysis shows that many failures occur after a relevant variable has already been surfaced, suggesting that correctly characterizing a mechanism's functional form and conditions, rather than merely identifying relevant variables, remains a key challenge in open-ended discovery.

Figures & tables

Figure 1: Overview of Open-Ended Hypothesis Discovery. Left: Given only a tabular dataset

D

and outcome

Y

, the goal is to discover multiple unknown hypotheses. Middle: HypAgent iteratively proposes, triages, refines, and residualizes hypotheses so that each discovery guides the next search. Right: OpenHypBench evaluates proposed hypotheses through interventions on the hidden structural causal model.

Benchmark	Scale	Discovery setting	Open-ended evaluation	New targets	Causal
Context-based generation ( Qi et al., 2024 )	3K	Curated context	✗	✗	✗
Data-driven analysis ( Gu et al., 2024 )	$\sim$ 100K	Prespecified question	✗	✗	✗
Hypothesis suites ( Liu et al., 2025a ; Majumder et al., 2025 )	$\sim$ 100K	Prespecified goal	✗	✗	✗
Causal suites ( Sachs et al., 2005 ; Scutari, 2010 )	10K	Prespecified goal	✗	✗	✓
OpenHypBench (ours)	10M	Open-ended	✓	✓	✓

Table 1: Comparison of existing hypothesis discovery benchmarks. Scale denotes the maximum number of rows in a benchmark dataset. New targets indicates that the evaluated data and target effects are newly generated rather than published targets.

Figure 3: Overall discovery performance on OpenHypBench. (a) Point estimates at each method’s reported operating point; time is mean end-to-end discovery time per task and excludes the shared evaluator. Confidence intervals are reported in Figure 6 in the appendix. (b) Recall after uniformly subsampling each method to a common budget of up to 25 proposals per task.

Figure 4: Recall across hypothesis archetypes with coral outlines marking the best-performing method for each archetype.

Figure 5: $F_{1}$ as the number of rows, features, and planted hypotheses varies.

Configuration	P	R	$\mathbf{F_{1}}$	Impact
HypAgent	0.47	0.51	0.49	–
$-$ Residualization	$-0.15$	$-0.12$	$-0.13$	68.8% increase in duplicates
$-$ Triage	$-0.08$	$-0.13$	$-0.11$	Suboptimal direction in 25.5% of cases
$-$ Refinement	$-0.31$	$-0.36$	$-0.34$	58.5% increase in unsupported hypotheses

Table 3: Ablation of HypAgent components across all 15 tasks. Ablated rows report changes relative to the full method.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Form	Definition and interventional signature	Count
Monotone threshold	$T$ changes $Y$ after crossing a threshold, e.g., $\phi(T)=w\mathbb{I}[T>q]$ .	30
Non-monotone	The effect is strongest at both tails or within an interior band; the resulting curve contains at least one direction change.	47
Derived feature	The treatment is a function of multiple variables, $U=g(T_{1},T_{2})$ , such as a ratio, product, or difference.	50
Interaction reversal	The direction of the effect of $T$ changes across values of a moderator variable $M$ .	71
Subpopulation-specific	The effect is active only under a condition $c(M)$ .	57
Simpson’s reversal	The marginal association between $T$ and $Y$ has the opposite direction from the association within strata of $M$ .	47

Appendix

Table 4: Functional forms in OpenHypBench. Counts are aggregated across all 15 tasks.

Task ID	Domain	Source dataset	URL	License	Rows	Outcome ( $Y=1$ )	Features	Hyp.
census_acs	Census	ACS PUMS ( U.S. Census Bureau, 2026a )	link	U.S. public domain	1M	Housing cost burden	63	23
census_adult	Census	UCI Adult ( Becker and Kohavi, 1996 )	link	CC BY 4.0	5M	High income	63	25
census_cps	Census	CPS ASEC ( U.S. Census Bureau, 2026b )	link	U.S. public domain	10M	Long commute	64	25
ecom_hm	E-commerce	H&M ( García Ling et al., 2022 )	link	Competition rules	5M	Customer churn	61	23
ecom_instacart	E-commerce	Instacart ( Risdal and Cukierski, 2017 )	link	Competition rules	1M	Product reordered	63	21
ecom_onlineretail	E-commerce	Online Retail II ( Chen, 2012 )	link	CC BY 4.0	10M	High lifetime value	62	24

Appendix

Table 5: Task list of OpenHypBench.

Figure 6: Precision, recall, and $F_{1}$ at each method’s reported operating point. Whiskers show 95% cluster-bootstrap confidence intervals over tasks.

Figure 7: Failure modes among the 172 planted hypotheses missed by HypAgent.

Scientific Hypothesis Generation

Scientific hypothesis generation develops methods for proposing testable explanations and research directions from literature, data, and experiments.

See all 90 papers

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses
Oct 16, 2026Jianyuan Zhong, Chenwei Wu, Zhaochen Yu +20Scientific Hypothesis Generation Autonomous Research Agents
PhAI Labs · The Chinese University of Hong Kong · University of Michigan +13
From Scientific Observations to Mechanisms: Benchmarking Hypothesis Generation by AI Scientists
Oct 4, 2026Xiaxun Xie, Qingqing Long, Meng Xiao +4AI Agents for Scientific Discovery Scientific Hypothesis Generation
Computer Network Information Center, Chinese Academy of Sciences · National University of Singapore · Sichuan University
Autonomous Scientific Discovery via Iterative Meta-Reflection
Jul 1, 2026Bingchen Zhao, Sara Beery, Oisin Mac AodhaScientific Hypothesis Generation Autonomous Scientific Discovery
University of Edinburgh · Massachusetts Institute of Technology

Oct 16, 2026J/K move · Enter open · S save

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses

Jianyuan Zhong, Chenwei Wu, Zhaochen Yu, Xixian Jiang+19

PhAI Labs · The Chinese University of Hong Kong · University of Michigan+13

Scientific Hypothesis Generation Autonomous Research Agents Scientific Idea Evaluation Recursive Self-Improvement

Open paper AskPDF arXiv

Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.

Abstract

Scientific Hypothesis Generation

Explore similar work

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses

From Scientific Observations to Mechanisms: Benchmarking Hypothesis Generation by AI Scientists

Autonomous Scientific Discovery via Iterative Meta-Reflection