cs.CLApr 10, 2026

GIANTS: Generative Insight Anticipation from Scientific Literature

Authors: Joy He-Yueya, Anikait Singh, Ge Gao, Michael Y. Li, Sherry Yang, Chelsea Finn, Emma Brunskill, Noah D. Goodman

Organizations: Stanford University · New York University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Scientific breakthroughs often emerge from synthesizing prior ideas into novel contributions. While language models (LMs) show promise in scientific discovery, their ability to perform this targeted, literature-grounded synthesis remains underexplored. We introduce insight anticipation, a generation task in which a model predicts a downstream paper's core insight from its foundational parent papers. To evaluate this capability, we develop GiantsBench, a benchmark of 17k examples across eight scientific domains, where each example consists of a set of parent papers paired with the core insight of a downstream paper. We evaluate models using an LM judge that scores similarity between generated and ground-truth insights, and show that these similarity scores correlate with expert human ratings. Finally, we present GIANTS-4B, an LM trained via reinforcement learning (RL) to optimize insight anticipation using these similarity scores as a proxy reward. Despite its smaller open-source architecture, GIANTS-4B outperforms proprietary baselines and generalizes to unseen domains, achieving a 34% relative improvement in similarity score over gemini-3-pro. Human evaluations further show that GIANTS-4B produces insights that are more conceptually clear than those of the base model. In addition, SciJudge-30B, a third-party model trained to compare research abstracts by likely citation impact, predicts that insights generated by GIANTS-4B are more likely to lead to higher citations, preferring them over the base model in 68% of pairwise comparisons. We release our code, benchmark, and model to support future research in automated scientific discovery.

Figures & tables

Figure 1: Overview of GiantsBench and GIANTS-4B . (top) For dataset construction, we use an LM as a research analyst to process each paper PDF to identify two parent papers whose ideas are synergistically combined to produce the paper’s key insight, which is extracted as the ground-truth target y∗y^{*} . A second LM summarizes each parent paper. (bottom) Given the two parent summaries, the model generates candidate insights, which an LM judge scores by similarity to y∗y^{*} on a 1–10 scale. These scores serve as a proxy reward for RL training, teaching the model to anticipate insights that more closely match real downstream papers.
Figure 2: LM judgements of insight similarity correlate with human judgements. We ask both human annotators and the LM judge ( gemini-3-pro ) to score the similarity between a model-generated insight and a ground-truth downstream insight on a 1–10 scale. The LM’s scores are positively correlated with average human ratings (Spearman ρ=0.761\rho=0.761 , p<0.001p<0.001 , n=60n=60 ).
Figure 3
Figure 5: GIANTS-4B remains strongest under test-time scaling. As the number of samples per example increases, GIANTS-4B consistently outperforms the base model, gemini-2.5-pro , and SciThinker-4B [ 24 ] , which is a scientific-ideation model trained using a citation-preference reward model. Error bars show 95% confidence intervals.
Figure 6: GIANTS-4B achieves a 68% overall win rate against the base model under SciJudge-30B. We evaluate pairwise preferences using SciJudge-30B [ 24 ] , a third-party judge trained to compare research abstracts by likely citation impact. This provides complementary evidence that optimizing for insight anticipation also improves performance under an independent, impact-oriented evaluation signal. Error bars show standard errors.
Figure 9: Qualitative comparison of insights derived from NeurIPS 2025 award-winning papers. (left) GIANTS-4B identifies a more concrete cross-paper mechanism than the base model ( Qwen3-4B ). (right) GIANTS-4B produces a grounded, more plausible interaction, while the base model makes a broader conjecture that is less directly grounded in the parent papers. We show a one-sentence summary of each parent paper for readability. These are illustrative abbreviations only and are not the full parent-paper summaries used as model input.
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

DomainarXiv Labels
Languagecs.CL
ML/AIcs.LG, cs.AI, cs.NE, cs.MA
Roboticscs.RO
Visioncs.CV, cs.GR, cs.MM, cs.SD
Theorycs.CC, cs.DS, cs.FL, cs.LO, cs.DM, cs.CG, cs.GT, cs.CR, cs.IT
Systemscs.AR, cs.OS, cs.DC, cs.NI, cs.PF, cs.SY, cs.PL, cs.SE, cs.DB, cs.IR, cs.SI
Appendix
Table 1: Mapping from domains to arXiv category labels.
DomainBaseSFTSFT-thinkgemini-2.5-progemini-3-proGIANTS-4B
Economics4.66±0.214.66_{{\pm 0.21}}4.78±0.204.78_{{\pm 0.20}}4.94±0.214.94_{{\pm 0.21}}4.75±0.224.75_{{\pm 0.22}}4.50±0.204.50_{{\pm 0.20}}5.47±0.225.47_{{\pm 0.22}}
Language4.50±0.104.50_{{\pm 0.10}}4.75±0.104.75_{{\pm 0.10}}4.84±0.114.84_{{\pm 0.11}}4.45±0.114.45_{{\pm 0.11}}4.14±0.104.14_{{\pm 0.10}}5.50±0.115.50_{{\pm 0.11}}
Vision4.28±0.104.28_{{\pm 0.10}}4.59±0.104.59_{{\pm 0.10}}4.57±0.104.57_{{\pm 0.10}}4.11±0.104.11_{{\pm 0.10}}4.14±0.104.14_{{\pm 0.10}}5.52±0.125.52_{{\pm 0.12}}
Theory4.54±0.104.54_{{\pm 0.10}}4.72±0.104.72_{{\pm 0.10}}4.80±0.114.80_{{\pm 0.11}}4.67±0.114.67_{{\pm 0.11}}4.58±0.114.58_{{\pm 0.11}}5.63±0.115.63_{{\pm 0.11}}
Mathematics4.64±0.114.64_{{\pm 0.11}}4.73±0.104.73_{{\pm 0.10}}4.81±0.104.81_{{\pm 0.10}}4.76±0.114.76_{{\pm 0.11}}4.58±0.104.58_{{\pm 0.10}}5.66±0.115.66_{{\pm 0.11}}
Quant. Fin.4.82±0.254.82_{{\pm 0.25}}5.15±0.235.15_{{\pm 0.23}}5.31±0.255.31_{{\pm 0.25}}5.04±0.265.04_{{\pm 0.26}}4.45±0.244.45_{{\pm 0.24}}5.75±0.265.75_{{\pm 0.26}}
Appendix
Table 2: Average similarity scores (mean ±\pm standard error) per domain on GiantsBench . The judge LM is gemini-3-pro .
DomainBaseSFTSFT-thinkgemini-2.5-progemini-3-proGIANTS-4B
Economics4.63±0.224.63_{{\pm 0.22}}4.73±0.214.73_{{\pm 0.21}}4.93±0.214.93_{{\pm 0.21}}4.76±0.234.76_{{\pm 0.23}}4.55±0.214.55_{{\pm 0.21}}5.50±0.235.50_{{\pm 0.23}}
Language4.59±0.194.59_{{\pm 0.19}}4.79±0.194.79_{{\pm 0.19}}4.96±0.204.96_{{\pm 0.20}}4.52±0.194.52_{{\pm 0.19}}4.28±0.184.28_{{\pm 0.18}}5.62±0.205.62_{{\pm 0.20}}
Vision4.45±0.144.45_{{\pm 0.14}}4.57±0.154.57_{{\pm 0.15}}4.64±0.144.64_{{\pm 0.14}}4.14±0.144.14_{{\pm 0.14}}4.33±0.154.33_{{\pm 0.15}}5.71±0.175.71_{{\pm 0.17}}
Theory4.63±0.124.63_{{\pm 0.12}}4.80±0.124.80_{{\pm 0.12}}4.76±0.124.76_{{\pm 0.12}}4.84±0.134.84_{{\pm 0.13}}4.75±0.134.75_{{\pm 0.13}}5.71±0.135.71_{{\pm 0.13}}
Mathematics4.63±0.114.63_{{\pm 0.11}}4.72±0.104.72_{{\pm 0.10}}4.80±0.104.80_{{\pm 0.10}}4.74±0.114.74_{{\pm 0.11}}4.57±0.104.57_{{\pm 0.10}}5.63±0.115.63_{{\pm 0.11}}
Quant. Fin.4.76±0.274.76_{{\pm 0.27}}5.18±0.255.18_{{\pm 0.25}}5.30±0.265.30_{{\pm 0.26}}4.89±0.274.89_{{\pm 0.27}}4.48±0.254.48_{{\pm 0.25}}5.77±0.275.77_{{\pm 0.27}}
Appendix
Table 3: Average similarity scores (mean ±\pm standard error) per domain on Test-unseen-parents , which indicates the subset of GiantsBench test set with only unseen parent papers. The judge LM is gemini-3-pro .
Figure 10: Prompt for insight anticipation. The summaries of two parent papers are in red .
Figure 11: Prompt for identifying two parent papers whose ideas are synergistically combined to produce the given paper’s key insight, which is extracted as the ground-truth target y∗y^{*} .
Figure 12: Prompt for assessing the similarity between the ground-truth insight y∗y^{*} (in red ) and model-generated insight y^\hat{y} (in orange ). The scale is bolded .
Figure 13: Prompt for rewriting the insight as a standalone statement. The summaries of two parent papers are in red . The raw synergy explanation is in orange .
HyperparameterValue
AlgorithmGRPO [ 15 ]
Training steps400
Train batch size64
Group size8
Max prompt length3000
Max response length1296
Appendix
Table 4: Key reinforcement learning hyperparameters used in our experiments.
HyperparameterValue
AlgorithmSFT
Epochs10
Train batch size64
Max length (input + output)8192
Learning rate1×10−61\times 10^{-6}
Gradient checkpointingTrue
Appendix
Table 5: Key supervised fine-tuning hyperparameters used in our experiments.
Figure 14: Similarity scores on Test-unseen-parents . This is the subset of GiantsBench test set with only unseen parent papers. Across all three LM judges, GIANTS-4B is ranked as the top-performing model.
Figure 15: Human eval labeling interface for assessing similarity. This is the Gradio interface that was used by human annotators to measure the consistency of LM similarity scores with human ratings. The rubric provided to the humans exactly matched that provided to the LM judge for consistency (with markdown formatting for human interpretability).
Figure 16: Human eval labeling app for assessing feasibility. This is the Gradio interface that was used by human annotators to measure the feasibility of a research insight along two dimensions: engineering/algorithmic complexity and conceptual clarity.
More in this topic

Large Language Model-Guided Scientific Discovery

Large language model-guided scientific discovery uses AI to propose, test, and refine hypotheses, models, experiments, and designs.

See all 274 papers

Explore similar work

CardsList
May 28, 2026

ProjectionBench: Evaluating Scientific Hypothesis Generation in LLMs Under Progressive Information Disclosure

Scientific discovery is an inherently creative and uncertain process, requiring reasoning beyond the recall of known knowledge. While many benchmarks have been proposed to evaluate large language model (LLM) performance on deep research tasks via multi-hop retrieval, their innovative reasoning abilities essential for true scientific discovery remain largely untested. We introduce a benchmark framework for evaluating model performance in scientific discovery and reasoning, building up from a raw problem to the classical null hypothesis test. In our framework, models initially receive only the topic and research question from a recent paper, with technical details progressively revealed. At each stage of information disclosure, the model is tasked with generating hypotheses that address the research question, which is compared with the conclusions from the original paper and evaluated via automated semantic similarity of constituent atomic claims. This progressive evaluation of semantic divergence from ground-truth conclusions enables assessment of a model's innovativeness (under minimal information) to grounded reasoning capabilities (under full experimental details), both critical for using LLMs for scientific discovery purposes. Our framework provides a foundation for systematically evaluating scientific reasoning and discovery capabilities in LLMs, crucial for advancing the development of next-generation AI scientist/co-scientist systems. Specifically, here we evaluate GPT-5, GPT-5.4, Gemini 2.5 pro, and Gemini 3.1 pro preview across 45 papers spanning bioactive materials, mechanical materials, and nanomaterials. We find that GPT-5.4 and Gemini 3.1 pro outperform their previous generation counterparts as expected, and GPT-5.4 in particular maintains 0.7 F1 score alignment with ground truth conclusions even under minimal context.

A. J. Lew, Y. Cao, M. J. Buehler

Unreasonable Labs · Unreasonable Labs Mountain View, CA 94043, USA

LLM EvaluationScientific ReasoningScientific Hypothesis GenerationResearch Idea GenerationScientific Reasoning in Language ModelsLarge Language Model-Guided Scientific Discovery
AskPDF
May 27, 2024

Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders

The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across disciplines. Modern artificial intelligence (AI) systems offer new opportunities for scientific ideation, but how compelling are AI-generated ideas, and how can their quality be improved? Here, we introduce SciMuse, which generates personalized research ideas using a knowledge graph of 58 million papers and a large language model (LLM). A central focus of this work is to understand how interesting these ideas are. Therefore, we conducted a large-scale evaluation in which more than 100 research group leaders -- spanning the natural sciences to the humanities -- rated over 4,400 personalized ideas according to their level of interest. Overall, expert ratings were modest (mean 2.40 on a 5-point scale, most common rating 1), while 24.9% of ideas were rated 4 or 5. We find that supplying concept pairs selected using the knowledge graph does not improve expert-rated interest over a titles-only GPT baseline. High-citation-predicted pairs even showed a weak tendency (1.94σσ) toward lower interest than random pairs. Nevertheless, graph features can be used to control properties of ideas, and, using this unique evaluation dataset, we show that idea interest can be predicted with both a supervised neural network based on graph features and a zero-shot ranking approach based on an LLM. Our work provides an AI methodology for generating scientific ideas and a large-scale interdisciplinary expert evaluation, paving the way to study and improve difficult-to-measure metrics such as expert-perceived scientific interestingness.

Xuemei Gu, Mario Krenn

Max Planck Institute for the Science of Light, Staudtstrasse 2, 91058 Erlangen, Germany

AI-Assisted Scientific ResearchScientific Hypothesis GenerationResearch Idea GenerationLarge Language Model-Guided Scientific DiscoveryGenerative AI EvaluationScientific Idea Evaluation
AskPDF
Oct 1, 2026

ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research

What makes great scientists great? Even as AI systems start to make progress on open problems, scientists remain far ahead of them at sensing which prior idea, buried in an ever-growing archive of research, a new problem needs. To study this skill, we draw on researchers who know firsthand which earlier work advanced their completed projects, with papers serving as pointers to the ideas within. Using our automated pipeline that makes author annotation scalable, we build ScholarCatalyst by having 184 lead authors of 207 recent computer science papers label which candidates did or could have advanced their project, each with a detailed rationale. We introduce a retrieval task with author-provided judgments: given an initial research question, retrieve these papers from only the literature available when the project began. Agentic search does no better than embedding retrieval (0.42 vs. 0.48 Recall@20) despite calling that same retriever as a tool. Even an agent built on Claude Fable 5.1, which may have seen the completed papers during training, reaches only 0.51 R@20. These results highlight the need for new training recipes that equip models with expert intuition for searching broad corpora. We envision ScholarCatalyst as a step toward scientific agents that can take a half-formed idea and point to the prior research it needs.

Sohyeon Kim, Yoonho Lee, Bo Liu +11

Stanford University · Seoul National University · University of Washington +3

AI Agents for Scientific DiscoveryAgentic SearchAI Agent BenchmarksInformation RetrievalIR EvaluationScientific Document Retrieval
AskPDF