Asterism: Exploring and Synthesizing Scattered Observations into Literature-Grounded Hypotheses and Theories
Authors: Joseph Chee Chang, Michael D'Arcy, Amy X. Zhang, Pao Siangliulue, Sangho Suh, Aakanksha Naik, Jena D. Hwang, Javier Ramos Benitez, Stella Wroblewski, Matt Latzke, Michael Cuoco, Ruben Lozano-Aguilera, Kris Ganjam, Joel Chan, Doug Downey, Peter Jansen, Kyle J. Travaglini, Daniel S. Weld
Organizations: Allen Institute for AI Seattle, Washington, United States · University of Washington Seattle, Washington, United States · Allen Institute Seattle, Washington, United States · University of Maryland College Park, Maryland, United States
A theory draws many independent observations into one framework with novel hypotheses. A researcher building such a theory must synthesize observations scattered across many papers, each describing related concepts but often in different terms. Which concepts matter most also depends on their preferences and research questions. Recent approaches scale theory synthesis with LLMs, but automate away choices and intuitions from researchers. We present Asterism, which extracts observations from hundreds of papers as concept-relation triples, with concepts unified in a hierarchical ontology. Researchers curate an evidence graph using the ontology and aggregate observations at different levels of granularity to focus theory formation on specific phenomena of interest. In a field deployment (n=10), researchers worked from observations to theories, and kept concepts and hypotheses fitting their preferences. In two case studies, teams of immunology and agriculture researchers discovered mechanisms outside their standard analyses and constructed hypotheses worth follow-up experiments.
Figures & tables
Figure 1. Asterism on a graph extracted from around 100 papers about how AI explanations affect human-AI teams. Left: a chat agent that searches the extracted evidence on the user’s behalf, answers questions about it, and proposes extracted concepts and relations to be added to the canvas (Figure 4 ). Middle: the canvas, holding the concepts and relations the user has curated for their inquiry, tinted by concept category and colored by relation, with a badge on every node and edge counting the unique papers behind it (Figures 6 and 5 ). Right: the hypothesis drawer, where hypotheses generated based on evidence from the canvas are reviewed by the user, marked interesting or dismissed, and selected for synthesis into a theory (Figure 9 ). Each hypothesis lists the concept-relation triples it was drawn from, hovering highlights them on the canvas (Figure 9 ). Screenshot of the \sysname{} web interface, divided into three vertical panels. The left panel is a chat conversation in which the user asks the system to help build up a graph for a research question and the agent replies with a ranked list of concepts to explore, each with a paper count and an Add or Expand button. The middle panel is a node-link canvas holding about a dozen labeled concept boxes, such as Feature-Based Explanation, Example-Based Explanation, Domain Expertise, Overreliance on AI, Underreliance on AI, and Decision-Making Accuracy, joined by curved colored edges. The right panel lists generated hypotheses; the open one is titled Asymmetric Reliance Calibration Law and shows its statement, its scope, and its supporting evidence with concept-relation triples.
Figure 2. Asterism extracts experiments and observations as concept-relation triples from a large set of papers, and infers a shared ontology that normalizes and generalizes concepts across papers (top). The researcher can steer theory development by selecting concepts across abstraction levels and relations between them to curate an evidence graph (left). Based on this shared representation, Asterism generates hypotheses for her to review and select (center), and draws the hypotheses she keeps into a unified theory that stays grounded in her evidence graph (right). A diagram of the \sysname{} process in four panels. The top panel shows a stack of paper icons with an arrow into a list of four extracted triples, each drawn as a flask icon, a lettered source concept, a green up arrow or a red down arrow, and a lettered target concept. Dashed lines run from each triple to the leaves of a tree on the right headed Shared Ontology, Concept Normalization, whose nodes are colored circles arranged over four levels. The lower left panel, outlined in green, is headed Zoom Nodes, Navigate abstraction levels in the ontology, and Select Interesting Triples, Curate an evidence graph of interest. It holds two controls reading Zoom out, more abstract, and Zoom in, more specific, a checklist of the same four triples with three of them ticked, and a laptop screen showing a node-link evidence graph over the eight lettered concepts. A researcher in a lab coat sits at a laptop beside it. An arrow leads down to a panel outlined in orange, headed Triage Hypotheses, Review generated hypotheses and choose which to keep, holding six cards; each card groups two or three triples under a lightbulb icon and is marked below with either a green tick or a red cross, four ticks and two crosses in total. An arrow leads right to a panel outlined in blue, headed Provisional Theory, Stitch together kept hypotheses, showing five lightbulbs on a ring around a central circle labelled Provisional theory. Below it a dashed arrow annotated outside scope.
Figure 3. Part of the ontology Asterism inferred for Dora’s corpus, shown in the outline panel. Each row carries the number of papers behind it: Overreliance on AI mentioned in 74 papers while Deception of Reliance only in 1. The ontology also powers the concept-zoom interaction, where zooming in on a concept offers the ones nested under it and zooming out merges it into its parent (Figures 5 and 6 ). A screenshot of a collapsible outline panel listing ontology concepts as an indented tree. Each row has a coloured dot, a concept name, and a badge on the right giving a paper count. Top-level rows with pink dots read AI Explanation Technique with 57 papers, Task Performance Outcome, AI Reliance Behavior, User Trust, User Experience Outcome, and User Characteristics. Indented under AI Explanation Technique, rows with green dots read Feature-Based Explanation with 41 papers, Example-Based Explanation with 16, Counterfactual Explanation, Contrastive Explanation, Natural Language Explanation, Rule-Based Explanation, Confidence Score Display, Attention-Based Explanation, and Concept-Based Explanation. A third level under Example-Based Explanation lists Nearest Neighbor Explanation, Gradient Similarity Explanation, and Representer Point Selection. Under AI Reliance Behavior the rows include Overreliance on AI with 74 papers, Under-Reliance on AI with 19, and Appropriate Reliance with 26.
Figure 4. The triage widget the agent returns instead of writing to the canvas itself, here for the 26 relations it found around Overreliance on AI . Rows are split by what they would cost: the first group connects concepts already on the canvas, the rest bring in new ones. Each row carries the relation, the number of papers behind it, and how much those papers agree, and the best-supported rows arrive pre-checked. A screenshot of a modal dialog inside the chat panel, headed SUGGESTED RELATIONS with a count of 26 and the line: Relations around Overreliance on AI. Rows are grouped under two orange headers, CONNECTS CONCEPTS YOU ALREADY HAVE with one row, and BRINGS IN NEW CONCEPTS with twenty-four. Each row has a checkbox, a source concept, an italic relation such as increases, decreases, or no effect, the target Overreliance on AI, and a small line reading a paper count and an agreement percentage. The first six rows are checked and the rest are not. A link at the bottom reads Show 5 more, 15 hidden, above an orange button reading: Add 5 concepts plus 6 relations, 6 to 11.
Figure 5. The concept panel for Example-Based Explanation . The two controls below move the concept along the ontology. Zooming in replaces it with the six specific techniques grouped under it; zooming out replaces it with its parent, AI Explanation , and the checkbox keeps the current concept nested inside that parent instead of folding it away, which is the state Figure 6 shows on the canvas. A vertical side panel. At the top, the concept name Example-Based Explanation, a category chip reading intervention, a one-sentence definition, and a long list of aliases. Below are two boxed sections. The first, Zoom-in to Specific Concepts, lists six concepts including Correspondence-Based Explanation, ExMatchina Explanation, and kNN Explanation. The second, Zoom out to Broader Concepts, shows the parent AI Explanation with a badge reading 55 nodes, a checkbox for keeping the current concept nested inside the new group, and a sample of the concepts in that group. A footer button reads: See 92 observations across 19 sources.
Figure 6. Nesting keeps a concept’s aggregated evidence without hiding the techniques beneath it. Feature-Based Explanation is drawn as a group with LIME and SHAP nested inside it, so the group’s merged edge to Overreliance on AI and the two techniques’ own edges are on the canvas at the same time. Figure 5 shows the controls that produce this state. On a node, the dark pill counts the unique papers behind the concept and the orange pill the number of sub-concepts folded into it. Hovering over the edge out of Example-Based Explanation shows the detailed distribution of reported relations from different papers. Clicking on the edge leads to Figure 7 . A screenshot of a small region of the canvas. On the left, a group node drawn with a dashed border is labelled Feature-Based Explanation and contains two child boxes, LIME Explanation and SHAP Explanation, each with its own source-count badge. Curved edges leave the group and each child and converge on a node labelled Overreliance on AI. Their labels read increases with 3 sources, no effect with 4 sources, and increases with 14 sources. Below, a separate node labelled Example-Based Explanation connects to the same target through an edge whose label is a stack of three rows reading increases, No Effect, and Decreases.
Figure 7. The evidence panel for the aggregated edge from Example-Based Explanation to Overreliance on AI . The bars break the 12 triples behind the edge down by relation as a share of the 6 unique papers reporting them; the shares add to more than 100% because a paper that reports two different relations counts toward both. Below, the panel unfolds the aggregate into the specific concept-relations it merged, each with the raw phrasing its paper used next to the canonical relation it was normalized to. Each of these unfolds one step further into the observation and experiment behind it (Figure 8 ). A vertical side panel headed Edge. At the top, two boxes joined by an arrow read Example-Based Explanation and Overreliance on AI. Beneath them, three labelled bars read Increases 83 percent, No Effect 17 percent, and Decreases 17 percent, followed by the line: 12 triplets, 6 sources. The lower part of the panel, headed Specific Concept-Relations, Normalized versus Raw Relations, is a stack of cards. Each card names a more specific source concept, such as Gradient Similarity Explanation or Representer Point Selection Explanation, with the same target, the normalized relation, the raw relation phrases in small chips, and a count of triplets and sources.
Figure 8. Every relation on the canvas traces back to sentences in a paper. This is the last step of the drill-down that starts at an edge in Figure 7 . Left: the evidence behind one triple, showing the observation as it was extracted, the triples drawn from that observation, the claim the authors made from it, and the experiment that produced it. Right: the same paper opened in the reader, with the sentence the observation was extracted from highlighted in place, so a user can check the extraction against the authors’ own wording and read around it. (Example paper ( Morrison et al., 2023 ) ). A two-pane screenshot. The left pane is a scrolling panel headed with a paper title, The Impact of Imperfect XAI on Human-AI Decision-Making, and the note 1 observation. It contains an OBSERVATION paragraph reporting relative self-reliance figures for expert participants, a TRIPLETS section with two extracted concept-relation triples each carrying an Add or Remove button, and CLAIM and EXPERIMENT sections describing what the authors concluded and how the study was run. The right pane shows the rendered PDF of that paper, open at a section headed Deception caused by Imperfect XAI, with one long sentence highlighted in yellow, the same sentence the observation was extracted from.
Figure 9. A saved theory, synthesized from hypotheses kept in the review drawer on the right of Figure 1 . The theory starts with a description that ties the hypotheses together under a single lens. The inline badges in the description reference the evidence behind each claim in the user-curated graph on the left, and hovering one highlights the corresponding edge. The hypotheses listed below can also be hovered to show the parts of the evidence graph they reference. The Conflicting Evidence and Unaccounted For sections report evidence on the canvas that runs against the theory or is not covered by it (based on ( Jansen et al., 2026 ) ). A two-pane screenshot. The right pane is the theory panel. It shows the theory name, Expertise-Modulated Explanation Reliance Theory, a paragraph describing the three interlocking claims it is built from, with small inline badges labelled e2.0 and e4.0 embedded in the text, and buttons reading Restore Full Canvas and Visualize Theory Nodes. Below that, a section headed Hypotheses lists three collapsed rows, each with a name and a tag reading mixed or qualitative. At the bottom, a section headed Conflicting Evidence contains two grey boxes describing findings that run against the theory. The left pane is the canvas, where five concept nodes joined by green and red edges are drawn at full contrast while the remaining nodes are faded out.
Scope
Ours closer
Zhu et al. closer
Tie
Mean best-J (ours)
Mean best-J (Zhu et al.)
Pooled 288 concepts
204 (71%)
55 (19%)
29 (10%)
0.508 [0.484, 0.532]
0.375 [0.354, 0.397]
Averaged 12 ontologies
74.0% [64.3, 83.7]
16.7% [8.2, 25.3]
9.3% [3.1, 15.4]
0.542 [0.477, 0.608]
0.391 [0.350, 0.431]
Table 1. Per-concept best-match Jaccard over the 288 non-trivial survey concepts, ours against the reference method of Zhu et al. ( Zhu et al., 2025 ) . The first row pools all 288 concepts; the second computes each statistic within each ontology and averages across the 12 ontologies (95% CIs over those 12 values).
ID
Min.
Ses.
Theory question
P0
123
10
How does the presence of external members (AI agents, new hires, new friends) affect group dynamics?
P1
109
10
How do different context-management strategies in agentic AI systems affect task performance? (and 2 more)
P5
53
7
How does different AI explanation technique affect users on cognitive load and task performance? (and 1 more)
P6
179
16
How do different grounding strategies (retrieval, tool, human feedback) affect the novelty and rigor of AI-generated scientific discoveries? (and 3 more)
P7
90
5
How do different AI interaction interface designs (conversational, embedded, agentic) affect user adoption, engagement, and task performance? (and 1 more)
P8
76
8
How do different presentations of AI diagnostic advice […] affect clinician overreliance and diagnostic accuracy in clinical decision support settings? (and 1 more)
Table 2. The theory questions each participant worked on. Some created additional graphs during the study (Table 3 ). A session ends when there was 15+ minutes of inactivity. The final four rows are the two case-study teams (§ 7 ), whose engagement ran through meetings and self-directed use outside the deployment protocol, so minutes and sessions are not shown.
P0
P1
P5
P6
P7
P8
P13
P15
P18
P19
μ±σ
Canvases
1
2
2
4
2
1
4
2
1
3
2.2 ± 1.1
Concepts
Added
60
38
32
287
51
33
70
91
33
47
74.2 ± 77.1
Removed
16
0
0
22
0
0
48
62
0
2
15.0 ± 22.7
Zoomed
59
0
0
0
0
0
75
0
0
2
13.6 ± 28.4
Hypotheses
Table 3. Engagement per participant, based on the interaction logs. Zoomed counts the child concepts revealed by zooming in or grouped by zooming out. Removed counts the distinct concepts a participant placed and later took off the canvas, by hand or by clearing it, and Canvases counts the canvases each participant saved.
Figure 10. Two common strategies interacting with Asterism. Top: how often they took different kind of action (smoothed over 3 minutes; with each curve scaled to its own peak.) Bottom: the number of concepts on their canvases. The horizontal axis is engaged time, concatenating multiple sessions. P0 added concepts more aggressively, and curate further by removing and zooming them; P7 added concepts more selectively, and never removed any. Both inspected evidence throughout, and P7 additionally went back to evidence on the canvas after theory creation, while P0 mostly read the evidence included in the theory report (not shown). Interaction traces of P0 and P7 demonstrate two common strategies using \sysname{} each with a horizontal axis of engaged minutes running from zero to about 120. Each panel holds an upper plot of six smooth colored curves, one per action type, and a lower plot with a dark step line for the number of concepts on the canvas. A colored strip along the top of each panel alternates between two shades marking building the evidence graph and generating and triaging hypotheses. In P0's panel the six curves rise and fall repeatedly through the first hundred minutes, and the concepts line climbs to a peak, dips, recovers, and falls again near the end. In P7's panel the curves are sparser, the concepts line rises early and then stays flat, and the amber Read Evidence curve peaks again near minute eighty-five, inside the stretch marked as hypothesis work. A legend below the two panels names the six action colors: Read Evidence, Chat, Add Concept, Remove Concept, Zoom Concept, and Hyp/Theory.
Source
P0
P1
P5
P6
P7
P8
P13
P15
P18
P19
%
Browse List
14
0
1
6
0
2
10
0
0
6
6 ± 8
Concept-Neighbors
10
0
0
9
1
0
28
1
1
0
7 ± 13
Concept-Hypotheses
9
0
0
192
0
0
0
35
22
13
21 ± 27
Chat
27
38
31
80
50
31
32
55
10
28
66 ± 29
/Proposed
/50
/80
/52
/124
/61
/69
/74
/99
/27
/34
Table 4. Participants used different methods to add concepts, ranging from browsing the full list of triples to find familiar concepts, to anchoring on a node to see its relational and ontological neighbors , to adding the concepts mentioned in a pre-generated hypothesis , and to asking the chat agent to propose concepts or triples.
Term
Definition
Kept (outcome)
Hypothesis marked as interesting or added to a theory
Breadth
# of canvas relations cited
LeafConcepts
% of cited concepts at the ontology’s most specific level
AnchorPapers
# of papers behind the best-supported relation cited
AnchorAgreement
% of its relations in the majority direction
Table 5. The model’s outcome and four hypothesis properties.
Figure 16
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Tool
Purpose
Finding concepts and relations
search_suggested_triples
Keyword-search the ranked pool of triples that can be added to the canvas, matching against source and target labels, their aliases, and the relation text. Returns up to 50 triples ranked by how many papers back the relation and how far they agree on its label; called with no arguments it returns the whole addable pool. Triples conflicting with concepts already on the canvas are filtered out.
expand_around_node
Walk one hop outward from a single concept, returning the top-ranked pool triples whose other endpoint is not yet on the canvas. Restricted to incoming relations, outgoing relations, or both.
find_paths_between
Find directed chains of pool triples connecting two concepts through intermediates, for questions of the form “how does A lead to B” and for bridging disconnected regions of the canvas.
Proposing changes to the canvas
propose_concepts
Render a card of up to eight concepts, one clickable row each with a one-line gloss from the agent; the label and paper count on each row are computed from the data rather than written by the model. Clicking a row adds that single concept and opens a relations widget around it.
Appendix
Table 6. The eleven tools available to Asterism’s chat agent. Relation labels in the ranked triple pool are derived from the extracted evidence, so a tool may choose which concept pairs to propose but never what relation holds between them.
Metric
Ours
Zhu et al.
Per-ontology higher
NMI
0.666 [0.597, 0.735]
0.622 [0.557, 0.687]
8 / 12
ARI
0.255 [0.183, 0.327]
0.185 [0.101, 0.270]
10 / 12
Purity
0.644 [0.560, 0.728]
0.585 [0.506, 0.665]
6 / 12
Appendix
Table 7. Partition-level scores under Zhu et al.’s protocol (higher is closer to the survey ontology), with 95% CIs over the 12 ontologies.
Large language models have recently been explored for scientific hypothesis generation, but most prior work relies on unstructured literature and free-form textual claims. We present a pipeline for Earth observation that grounds hypothesis generation directly in the NASA Earth Observation Knowledge Graph. A heterogeneous graph neural network trained on historical co-usage relations ranks candidate dataset pairings, and a three-agent LLM pipeline filters, generates, and evaluates structured research hypotheses. Applied to 1,475 NASA datasets, the system produces 160 hypotheses spanning multiple Earth-science domains, including ecohydrology, glaciology, aerosol--cloud interactions, vegetation phenology, and stratospheric chemistry. Model-predicted novel dataset pairings are rated nearly as plausible as held-out real co-usages from the literature, indicating that the pipeline surfaces scientifically coherent yet unexplored combinations. A 222 factorial experiment across GPT-5.2 and Claude Sonnet 4.6 shows that hypothesis rankings remain stable, while absolute scores depend strongly on judge identity, highlighting limitations of single-judge LLM evaluation.
Scientific breakthroughs often emerge from synthesizing prior ideas into novel contributions. While language models (LMs) show promise in scientific discovery, their ability to perform this targeted, literature-grounded synthesis remains underexplored. We introduce insight anticipation, a generation task in which a model predicts a downstream paper's core insight from its foundational parent papers. To evaluate this capability, we develop GiantsBench, a benchmark of 17k examples across eight scientific domains, where each example consists of a set of parent papers paired with the core insight of a downstream paper. We evaluate models using an LM judge that scores similarity between generated and ground-truth insights, and show that these similarity scores correlate with expert human ratings. Finally, we present GIANTS-4B, an LM trained via reinforcement learning (RL) to optimize insight anticipation using these similarity scores as a proxy reward. Despite its smaller open-source architecture, GIANTS-4B outperforms proprietary baselines and generalizes to unseen domains, achieving a 34% relative improvement in similarity score over gemini-3-pro. Human evaluations further show that GIANTS-4B produces insights that are more conceptually clear than those of the base model. In addition, SciJudge-30B, a third-party model trained to compare research abstracts by likely citation impact, predicts that insights generated by GIANTS-4B are more likely to lead to higher citations, preferring them over the base model in 68% of pairwise comparisons. We release our code, benchmark, and model to support future research in automated scientific discovery.
While large language models (LLMs) show promise in scientific discovery, existing research focuses on inference or feedback-driven training, leaving the direct modeling of the generative reasoning process, P(hypothesis∣background) (P(h∣b)), unexplored. We demonstrate that directly training P(h∣b) is mathematically intractable due to the combinatorial complexity (O(Nk)) inherent in retrieving and composing inspirations from a vast knowledge base. To break this barrier, we introduce MOOSE-Star, a unified framework that enables tractable and scalable training of P(h∣b), while supporting more scalable inference. In the best case, MOOSE-Star reduces complexity from exponential to logarithmic (O(logN)) by (1) training on decomposed subtasks derived from the probabilistic equation of discovery, (2) employing motivation-guided hierarchical search to enable logarithmic retrieval and prune irrelevant subspaces, and (3) utilizing bounded composition for robustness against retrieval noise. To facilitate this, we release TOMATO-Star, a dataset of 108,717 decomposed papers (38,400 GPU hours) for training. Empirically, MOOSE-Star scales continuously with training data and inference budget, whereas direct brute-force sampling hits a complexity wall.