ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe
Authors: Weihan Li, Tianshi Zheng, Yangqiu Song, Ginny Y. Wong, Simon See
Organizations: Department of Mechanical Engineering, The University of Tokyo, Tokyo, Japan · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China · NVIDIA AI Technology Center (NVAITC), NVIDIA, Santa Clara, USA
Scientific discovery often begins when scattered clues call for a new way of describing the world. Such abductive exploration can require constructing the representation in which an explanation is stated, when the world is epistemically open, and composing evidence scattered across contexts, when it is structurally interconnected. Existing benchmarks rarely separate these two demands or control them independently. We introduce ULTRADISCOVERY, an interactive world of five domains in which an agent revises an initially successful theory and predicts the outcome of an unseen cross-domain intervention. A 2×2 design leaves the representation open or discloses it, and leaves the evidence distributed or aligns it, with the latent dynamics fixed. With the representation open, agents across eleven models often retract the axiom they were taught, and none introduces the unobserved entity or rewrites the variables that a replacement requires. Disclosure triples intervention requests and adds about one of the eighteen findings the world affords, and alignment adds less. Two vendor-harness systems carry discovery into more domains, and one of them rewrites the variables in Open episodes. No system makes the exact prediction within 200 paid actions. At larger budgets one exact prediction appears with both aids, while every Open episode remains inexact. The results locate the difficulty in the step from accumulating evidence to composing it into a representation that transfers.
Figures & tables
Figure 1: How much representation is supplied before inquiry. From top to bottom: a supplied hypothesis space, supplied variables, and the epistemically open setting of UltraDiscovery , where the agent constructs entities, variables and laws from instrument readings, under the instruments’ own names (Table A.1 ).
Figure 2: The epistemic structure of UltraDiscovery . Eighteen blocks in five domains, joined by the edges along which the findings support one another (upper panel). Three hyperedges each join three blocks read in three domains to a representation change, and HC joins the five blocks behind the terminal prediction (lower panel).
Figure 3: The world of UltraDiscovery , what it teaches, what is true and what it asks. Top: the five domains with their function, the 24-tick pass that resets while the notes persist, and 200 paid actions. (A) The doctrine the prologue teaches and confirms: one object, one individual, one rate. (B) The hidden law in five lines (Section 3.2 ) and one example: three objects under one latent individual take turns, and a retool splits one off. (C) The eight verbs, then the terminal intervention, posted at the pennant, and the three readings committed before it happens.
Figure 4: The 2×2 experimental design. Arrows mark the paired contrasts Δrepr and Δalign , I their interaction, and the star the primary condition.
Figure 5: Discovery over the episode. (a) The four factorial conditions in the baseline loop. (b) The baseline loop and the two vendor-harness systems on the same four instances, pooling conditions. Solid lines give grounded blocks and dotted lines the blocks whose evidence the agent holds, each a mean with every model weighted equally, and the flag marks the posting of the terminal intervention.
Table 1: Discovery by model and condition: mean Blocks, AUC, XC and stated RC. Bold : best of a column among the eleven models of the baseline loop; underlined : next two; † and ‡ mark malformed-reply and withheld-reply exclusions (Appendix ). Margins: Δalign per row and Δrepr per column, paired within instance, with RC reading uptake under disclosure (Section 4.3 ); the corner gives their interaction on Blocks and XC.
Figure 6: Grounded blocks by scaffold, model and Open condition. Bars average instances, and darker segments show grounded cross-domain bridges.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
In the world
In the paper
What it is
Appendix
Table 2: The world’s vocabulary and the paper’s. The main text names objects, instruments and the events of the protocol by their function; the world, the agent-facing materials of Appendix and the appendices use the names in the first column. The five domains keep the world’s names everywhere, and the block identifiers take their initials.
Autonomous scientific discovery with LLMs requires generating and testing hypotheses adaptively as evidence accumulates while maintaining statistical validity. Existing anytime-valid methods can handle data-dependent hypotheses, but open-ended discovery poses a deeper challenge: the best discovered hypothesis may still be the best of a bad lot, with better explanations yet undiscovered, while even background knowledge such as physical laws may require revision in light of new findings. In response, we formalize the problem as Abductive Autonomous Scientific Discovery (AASD) using possibility theory. We introduce abductive utility, a computable measure of discovery progress, and possibility frontier search, the first algorithm for AASD, which maintains anytime validity and achieves ε-optimal abductive utility asymptotically under suitable conditions. Experiments on synthetic and real-world scientific-discovery tasks show strong performance.
Anita Yang, Siu Lun Chau, Tomoya Wakayama +2
Lattice Lab, Toyota Motor Corporation, Japan · Department of Computer Science, University of Tokyo, Japan · EPIC Lab, College of Computing & Data Science, Nanyang Technological University, Singapore +2
Open-ended scientific discovery asks agents to move beyond executing analyses for predefined questions. Across multiple rounds of exploration, a discovery agent must decide which phenomena warrant investigation while avoiding overinterpretation, where emerging claims exceed the evidential scope of the analyses. This creates an evidence-calibration problem: the exploration trajectory must be coupled with claim status so that evidence can guide both what to investigate next and what can be claimed. We introduce StatefulDiscovery, a discovery framework that externalizes investigation state and uses it to coordinate frontier selection, evidence acquisition, and claim adjudication. We evaluate StatefulDiscovery across 40 real-data discovery tasks. Compared with several baselines, StatefulDiscovery produces more claims overall judged to be both well-supported and high-value. Ablations indicate distinct roles for structured hypotheses, local adjudication, frontier control and persistent states. Together, these results suggest that explicit discovery state can couple exploration with evidence-calibrated claim formation. Our code is released at https://github.com/SUSTech-GenAI/StatefulDiscovery.git.
Automated scientific discovery using large language models relies on identifying genuinely novel solutions. Standard reinforcement learning penalizes high-variance mutations, which leads the policy to prioritize familiar patterns. As a result, the maximum reward plateaus even as the average reward increases. Overcoming this limitation requires a signal that distinguishes unexplored regions from intrinsically difficult problems. This necessitates measuring disagreement across independently adapted weight hypotheses rather than relying on a single network's confidence. UG-TTT addresses this challenge by maintaining a small ensemble of low-rank adapters over a frozen base model. The per-token disagreement, quantified as the mutual information between ensemble predictions and weight hypotheses, isolates epistemic uncertainty and identifies positions where insufficient coverage leads to adapter divergence rather than intrinsic problem difficulty. This measure is incorporated as an exploration bonus into the policy gradient, directing the policy toward positions where persistent adapter disagreement signals low training coverage, the same frontier where genuine discovery is possible. A nuclear norm regularizer ensures the adapters remain distinct from one another, thereby preserving the exploration signal throughout training. Across four scientific discovery benchmarks, UG-TTT increases the maximum reward on three tasks, maintains substantially higher solution diversity, and an ablation study confirms that the regularizer is essential for sustaining this behavior.
Kainat Riaz, Muhammad Ahmed Mohsin, Ahsan Bilal +5
Stanford University · University of Oklahoma · National University of Sciences and Technology