Training Reasoning Models to Generate Diverse Scientific Ideas

Manya Wadhwa; Pranav Narayanan Venkit; Hiroaki Hayashi; Yu Li; Tuhin Chakrabarty; Chien-Sheng Wu

cs.CLOct 15, 2026

Training Reasoning Models to Generate Diverse Scientific Ideas

Authors: Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi, Yu Li, Tuhin Chakrabarty, Chien-Sheng Wu

Organizations: New York University · Salesforce AI Research · Stony Brook University

Abstract

One of the first steps of a research pipeline is generating ideas. Large Language Models (LLMs) are increasingly used to automate this step, but most of this work optimizes the quality of individual ideas, while overlooking diversity. In our work, we study document-grounded ideation, focusing on increasing the diversity of LLM generated ideas and ensuring they are high quality. Given prior papers, a model must propose a set of ideas that are each high-quality and clearly distinct from one another, while also being grounded in the input documents. To optimize for this, we propose training with a set-level reinforcement learning objective. In our setup, in a single generation, the model writes K structured idea cards grounded in the source papers, and the set is scored jointly, where each idea's quality is weighted by its distance from the ideas written before it. This teaches the model to steer each new idea away from its predecessors. We show that our trained 8B model achieves higher creative utility (a measure combining quality and diversity) than Claude Opus 4.8 and GPT-6 Astra on our contamination-free benchmark. In a pilot human study with four experts, our sets are judged as promising as Claude Opus 4.8's and contain marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, our model's success@k is comparable to Claude Opus 4.8's, so we see that the added diversity does not cost any executability.

Figures & tables

Figure 1: Overview of our training pipeline. Given a set of documents, the policy generates a set of ideas

U=\{u_{1},\dots,u_{k}\}

; for reasoning models, the whole set is generated after a single shared reasoning prefix. Each idea

u_{i}

is scored by its quality

q_{i}

and its distance

d_{i}

to the closest earlier idea (

d_{1}=1

), giving a per-item reward

r_{i}=q_{i}\cdot d_{i}

. The set reward

R(U)=\sum_{i}r_{i}

is used as the RL signal to update the policy, encouraging sets that are both high-quality and diverse.

Model	Creative Utility (CU) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
General models
Claude Opus 4.8	$0.38_{(0.22)}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra	$0.28_{(0.24)}$	0.15	$0.86_{(0.06)}$
Nemotron 120B	$0.14_{(0.18)}$	0.05	$0.79_{(0.10)}$
Competing objectives
DARLING	$0.52_{(0.17)}$	0.38	$0.67_{(0.13)}$

Table 1: Results on our test set. We report Creative Utility (

CU

) with

\gamma=0.9

. Win Rate is the per-idea impact against the gold idea, corrected for length. Avg. Distance is the mean pairwise distance between the ideas in a set. Values in parentheses are standard deviations across instances. Proposed (no think) has the highest creative utility; Proposed (think) is on par with DARLING and the no-SFT ablation.

Figure 2: Creative Utility (CU) against average grounding ( $g$ ) for the systems also mentioned in Table 1 . The highest-CU systems are less grounded than Claude and the SFT-only models.

Model	Win Rate $\uparrow$	Avg. distance $\uparrow$
GIANTS-4B	18.3%	0.025
Proposed	68.3%	0.939

Table 2: GiantsBench results on 2,374 test papers.

Method	$H_{\mathrm{norm}}$ $\uparrow$	TVD vs human $\downarrow$
Qwen3-8B	0.77	0.27
Claude Opus 4.8	0.77	0.23
DARLING	0.01	0.72
Proposed (no think)	0.65	0.43
Proposed (think)	0.73	0.30
Human proposals	0.87	–

Table 3: Taxonomy diversity on IdeaSeed (500x3 ideas), Method Paradigm axis.

System	Promising	Unique	Unique Promising
Qwen3-8B	$1.52_{0.83}$	$2.52_{0.62}$	$1.42_{0.81}$
Claude Opus 4.8	$\mathbf{2.20_{0.73}}$	$2.72_{0.52}$	$2.03_{0.76}$
Proposed	$2.17_{0.72}$	$\mathbf{2.83_{0.38}}$	$\mathbf{2.05_{0.75}}$

Table 4: Human study results with 4 annotators, 15 papers each. We see our proposed model generates ideas that are as promising and diverse as Claude-Opus-4.8. Proposed is significantly better than Qwen3-8B on all three metrics.

Arm	$k=1$	$k=2$	$k=3$
Claude Opus-4.8	13%	20%	27%
Proposed (no think)	11%	22%	33%
Proposed (think)	10%	19%	29%

Table 5: Execution results of 132 ideas by a coding agent.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

In-text context	Source	Label
“Recent work Gilardi et al. ( 2023 ) ; He et al. ( 2023a ); Ding et al. ( 2022 ) has looked at the annotation accuracy for LLMs and found them to be at par with crowd-worker annotators.”	ChatGPT outperforms crowd workers for text-annotation tasks	motivates
“ MuRAG Chen et al. ( 2022 ) requires 3 million image-text pairs for pre-training to achieve decent results.”	MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text	exposes_gap
“The global strategy involves using a trained image caption model to generate text descriptions of the macro scenes of the image Changpinyo et al. ( 2022 ) ; Yang et al. ( 2022b )”	All You May Need for VQA are Image Captions	foundation
“This paper is an extended work of an earlier version presented at SOMET 2022 (Shirafuji et al., 2022 ) ”	Prompt Sensitivity of Language Model for Solving Programming Problems	extends

Appendix

Table 6: Representative source citations, one per functional label. The in-text context is the sentence in the target paper that cites the source; the label is the role assigned by the importance classifier

Figure 3: Distribution of (a) number of sources kept per-target post source filtering (b) the different citation/source types post-filtering.

Figure 4: Statistics around the training and test datasets used in this work. Our training and test distributions contain varying number of source documents as inputs in order for the model to generate ideas, we also maintain a strict contamination boundary for the models we are training and evaluating. The legend counts (train $n{=}8{,}007$ , test $n{=}1{,}741$ ) are the full pools after the source-count filter; the models are SFT’d on 7,172 of the training instances (after some filtering) and evaluated on 1,740 test instances.

Figure 5: Distribution of domains in the test dataset.

	SFT	GRPO
Init	Qwen3-8B	SFT checkpoint (after 1 epoch)
Training instances	7,172 (same instances for SFT and GRPO), plus 100 validation
Learning rate	$5\times 10^{-6}$	$1\times 10^{-6}$
Batch size	64 sequences	12 prompts $\times$ 4 rollouts
PPO mini-batch (prompts)	–	12 (one update per step)
Epochs / steps	1 epoch (112 steps)	600 steps / 1 epoch

Appendix

Table 7: Hyperparameters for the main Qwen3-8B run.

Impact judge	win-rate
SciJudge-30B ‡	0.65
gpt-5.4-mini †	0.74
claude-sonnet-5	0.74

Appendix

Table 8: Impact-judge win-rate. † evaluation judge; ‡ training-reward judge.

Impact judge	Length (tokens)		Strict win-rate
	orig	matched	orig	matched
gpt-5.4-mini †	642	258	0.253	0.284
claude-sonnet-5	642	258	0.090	0.133
claude-sonnet-4.6	642	258	0.165	0.205

Appendix

Table 9: Impact-judge performance with length changes. † evaluation judge.

Field	HH	gpt-oss-120b	gemini-3.5-flash
	$\rho$	$\rho$	$\rho$
problem gap	0.30	0.31	0.44
core claim	0.36	0.29	0.30
method	0.28	0.38	0.53
Distance $d\in[0,1]$	0.36	0.37	0.50

Appendix

Table 10: Distance-metric validation on

n{=}100

within-set idea pairs rated by both human annotators, over the three scored fields and their aggregated distance

d\in[0,1]

(Eq. 4 ). All entries are Spearman

\rho

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
General models
Claude Opus 4.8	$0.38_{(0.22)}$	$0.38_{(0.48)}$	$\mathbf{0.98_{(0.08)}}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra	$0.28_{(0.24)}$	$0.27_{(0.44)}$	$0.94_{(0.11)}$	0.15	$0.86_{(0.06)}$
Nemotron 120B	$0.14_{(0.18)}$	$0.08_{(0.28)}$	$0.90_{(0.18)}$	0.05	$0.79_{(0.10)}$
Qwen3-8B	$0.03_{(0.10)}$	$0.02_{(0.13)}$	$0.91_{(0.21)}$	0.01	$0.80_{(0.11)}$
Competing objectives

Appendix

Table 11: Results on our contamination-free test set (1,740 instances), with two extra columns: number of instances which at least one win in their set and grounding. Values in parentheses are standard deviations across instances.

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
Claude Opus 4.8
Sampled ( $k{=}1,n{=}3$ )	$0.34_{(0.14)}$	$0.59_{(0.49)}$	$0.99_{(0.06)}$	0.33	$0.24_{(0.21)}$
Set ( $k{=}3,n{=}1$ )	$0.38_{(0.22)}$	$0.38_{(0.48)}$	$0.98_{(0.08)}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra
Sampled ( $k{=}1,n{=}3$ )	$0.25_{(0.17)}$	$0.37_{(0.48)}$	$0.96_{(0.09)}$	0.23	$0.28_{(0.23)}$
Set ( $k{=}3,n{=}1$ )	$0.28_{(0.24)}$	$0.27_{(0.44)}$	$0.94_{(0.11)}$	0.15	$0.86_{(0.06)}$

Appendix

Table 12: Results on our contamination-free test set (1,740 instances). Set draws all

K=3

ideas in one response (as in Table 11 ); sampled draws them as three independent single-idea responses. We can see most models trade off quality for diversity when switching from sampling to set-based generation. Values in parentheses are standard deviations across instances.

Figure 6: Creative Utility(CU) for two baselines and our proposed models, across 16 domains in our test-set.

ARR Area	$n$ papers (%)	Promising ideas $\uparrow$	Unique ideas $\uparrow$	Unique promising ideas $\uparrow$
		(Qwen3-8B / Claude Opus 4.8 / Ours per column)
Human-Centered NLP and Human-AI Interaction	18 (30%)	$1.67/\textbf{2.39}/\textbf{2.33}$	$2.56/2.78/\textbf{2.83}$	$1.61/2.28/2.11$
LLM agents	11 (18%)	$1.36/\textbf{2.00}/\textbf{2.00}$	$2.36/\textbf{2.82}/\textbf{2.82}$	$1.18/1.91/\textbf{2.00}$
Machine Learning for NLP	7 (12%)	$2.14/2.14/\textbf{2.43}$	$\textbf{2.86}/2.71/\textbf{2.86}$	$2.14/2.14/\textbf{2.29}$
Resources and Evaluation	5 (8%)	$1.40/\textbf{2.20}/1.40$	$2.20/\textbf{3.00}/2.80$	$1.20/\textbf{2.20}/1.40$
Ethics, Bias, and Fairness	5 (8%)	$0.80/\textbf{2.80}/2.40$	$2.40/2.40/\textbf{2.80}$	$0.60/1.80/\textbf{2.20}$

Appendix

Table 13: Human study results broken down by the ACL Rolling Review (ARR) subarea of the target paper (§ 6.1 ).

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
Base	$0.10_{(0.16)}$	$0.06_{(0.25)}$	$0.73_{(0.27)}$	0.03	$0.79_{(0.12)}$
SFT (no think)	$0.17_{(0.18)}$	$0.11_{(0.31)}$	$0.84_{(0.22)}$	0.05	$0.77_{(0.10)}$
Proposed (no think)	$0.56_{(0.23)}$	$0.67_{(0.47)}$	$0.66_{(0.27)}$	0.39	$0.83_{(0.09)}$
SFT (think)	$0.12_{(0.15)}$	$0.06_{(0.23)}$	$0.86_{(0.22)}$	0.03	$0.75_{(0.10)}$
Proposed (think)	$0.54_{(0.24)}$	$0.63_{(0.48)}$	$0.73_{(0.25)}$	0.34	$0.83_{(0.08)}$

Appendix

Table 14: Results on our contamination-free test set (1,740 instances), showing the recipe applied to Olmo-3-7B-Think-SFT. Values in parentheses are standard deviations across instances.

	Win-Rate			Grounding ( $g$ )
Model	1	2	3	1	2	3
Proposed (no think)	0.69	0.72	0.65	0.79	0.56	0.48
Proposed (think)	0.62	0.54	0.52	0.88	0.80	0.75

Appendix

Table 15: How the Impact/Win-rate and grounding changes with the position at which an idea is generated. We report the raw win-rate (instead of the strict length-controlled) here instead of length normalized

Annotator	System	$n$	Promising	Unique	Unique & Promising
Annotator 1	Qwen3-8B	15	$1.73_{(0.88)}$	$2.67_{(0.49)}$	$1.67_{(0.90)}$
	Claude Opus 4.8	15	$1.93_{(0.59)}$	$2.67_{(0.49)}$	$1.87_{(0.64)}$
	Proposed	15	$2.20_{(0.56)}$	$2.87_{(0.35)}$	$2.20_{(0.56)}$
Annotator 2	Qwen3-8B	15	$1.27_{(0.46)}$	$2.40_{(0.74)}$	$1.13_{(0.35)}$
	Claude Opus 4.8	15	$1.93_{(0.80)}$	$2.80_{(0.56)}$	$1.87_{(0.83)}$
	Proposed	15	$1.67_{(0.49)}$	$2.73_{(0.46)}$	$1.53_{(0.52)}$

Appendix

Table 16: Human annotations per annotator. For each portfolio of

K=3

ideas, annotators marked the ideas that are worthwhile, the distinct representatives, and the distinct representatives that are also worthwhile. We report the mean number per portfolio (standard deviation in parentheses);

n

is the number of portfolios.

Method	$H_{\mathrm{norm}}$ $\uparrow$	TVD vs h $\downarrow$	JSD vs h $\downarrow$	% unique/instance $\uparrow$	Ideas
Opportunity types
Claude Opus 4.8 (sampled, $k=1,n=3$ )	0.86	0.35	0.12	63.4% (n=498)	1494
Qwen3-8B (sampled, $k=1,n=3$ )	0.56	0.60	0.30	54.6% (n=500)	1500
Claude Opus 4.8 (set, $k=3,n=1$ )	0.89	0.27	0.08	89.0% (n=499)	1497
Qwen3-8B (set, $k=3,n=1$ )	0.80	0.30	0.10	89.3% (n=499)	1275
DARLING (sampled, $k=1,n=3$ )	0.05	0.80	0.58	35.0% (n=500)	1500

Appendix

Table 17: Taxonomy diversity on IdeaSeed (500 papers: 250 ML, 250 natural science). Every generated idea and every human proposal is labeled by GPT-5.4-mini with one Opportunity Pattern and one Method Paradigm from the TasteGap taxonomy ( Chen et al., 2026 ) .

Figure 7: Distribution of ‘opportunity’ and ‘method’ labels on the IdeaSeed benchmark, as generated by different models compared to the proposed model.

Verdict	Claim	Proof of concept (summary)	Results (summary)
Supports Per-form robustness map for AI-text detectors think model; coverage: most	A benchmark of many text-evasion forms shows detectors resist some attacks and fail on others, and the perform map lets practitioners choose a detector per setting.	6 detectors $\times$ 12 conditions on human vs. ChatGPT text (HC3), 150 texts per class per condition. Four zero-shot statistical scores, a RoBERTa detector and a char n-gram classifier. Forms: paraphrase, style shift, typos and homoglyphs at three strengths, domain shift, regeneration by two other LLMs. TPR at 5% FPR. Oracle per-form choice replaced by a held-out split-half test (400 splits).	Near ceiling on clean text (TPR $0.92$ – $0.99$ ) but divergent by form: log-likelihood $1.00$ under domain shift, $0.11$ under paraphrase, $0.00$ under typos and homoglyphs (non-overlapping CIs). Rankings barely agree across forms (Kendall $\tau=0.095$ ). Held-out per-form selection beats the best single detector by $+0.107$ TPR (95% CI $[0.064,0.137]$ , 400/400 splits). Not reached: commercial or watermark detectors, white-box attacks, larger domain grid.
Refutes One cross-task faithfulness evaluator no-think model; coverage: most	Faithfulness is task-agnostic and measurable uniformly by one lightweight evaluator; models are far more unfaithful than accuracy suggests, with a larger gap for reasoning than retrieval.	6 task conditions in one evidence format (GSM8K, KG-QA, HotpotQA, SQuAD, two TAT-QA), two Llama solvers, 430 traces. One fixed evaluator prompt on four lightweight evaluators plus a Claude-Sonnet-5 reference. Ground truth from paired corruption: a value inserted that appears nowhere in the evidence (329 traces).	Correctness and grounding are separable, and all evaluators beat chance in all six conditions. But the measurement is not uniform: recall $26.6$ – $94.2\%$ , false-positive rate $0$ – $22\%$ , and the headline contrast flips sign between two evaluators. Reasoning-vs-retrieval gap is null within a modality (table $+9.3$ [ $0.0$ , $20.7$ ] vs. $+1.1$ [ $-10.2$ , $14.0$ ]); the effect is concentrated in KG-QA ( $43.5\%$ of correct answers unsupported vs. $0$ – $11.5\%$ ). A universal claim falsified by a subset.
Inconclusive Content-level citation auditing no-think model; coverage: most; blocker: noise	Many citation errors are content-level, and an auditor reasoning over the claim and cited text catches them far more reliably than a metadata-only verifier, with no index needed.	Citation-Integrity corpus (biomedical, expert labels): 770 test pairs (abstract-level cited text), 399-pair stratified slice, precision reweighted to natural prevalence. Four LLM auditors (sub-claim decomposition then grounding) vs. a metadata-only verifier (OpenAlex/Crossref) and TF-IDF.	Direction holds: content errors are $21.0\%$ of pairs, and the metadata verifier has no content signal (precision $0.26$ vs. $0.21$ base rate). Magnitude does not: AUROC $0.62$ – $0.67$ (all errors), $0.73$ – $0.79$ (core), best F1 $0.37$ – $0.43$ , recall $0.50$ only at precision $0.30$ ; between the pre-registered bars. Auditors agree with each other ( $\kappa$ $0.42$ – $0.73$ ) more than with gold ( $0.15$ – $0.25$ ), within the published expert band. Not reached: full-text papers, other domains, fine-tuned auditors.

Appendix

Table 18: Example autoresearch cards, one per verdict. Each card is an idea’s frozen claim, a scaled-down proof of concept run by the coding agent under a 90-minute, $30, CPU-only budget, and the agent’s structured verdict. Summaries are condensed from the agent’s own report.

Diverse Text Generation

Diverse text generation studies how to produce varied, useful language outputs without sacrificing coherence, accuracy, or task performance.

See all 50 papers

Jul 24, 2026

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years. However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity. This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts. In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality-Diversity (QD) search. In line with this perspective, we introduce IDEAgent, a multi-agent framework that manages the evolution of ideas through lineages. We jointly drive Quality using multi-objective feedback for dedicated repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, their historical ancestors, and rejected proposals. To systematically evaluate this QD conjunction, we develop Yield, a joint metric that computes the largest set of mutually diverse ideas that satisfy a predetermined quality threshold. Finally, through evaluations across 32 topics spanning 8 domains of Computer Science, we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non-zero Yield on 8x more topics. We further corroborate these findings through an analysis of quality improvements, showing that repair and refinement are crucial for building logical rigor and clarity while preserving non-obviousness. To encourage future research on QD-search-based ideation, we open-source IDEAgent at https://github.com/declare-lab/IDEAgent.

Varun Gumma, Navonil Majumder, Soumitra Sinhahajari +1

DeCLaRe Lab, Nanyang Technological University, Singapore

Multi-Agent LLM Systems Quality-Diversity Optimization Research Idea Generation Large Language Model-Guided Scientific Discovery

Ask PDF

Oct 6, 2026

IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas

Scientific research often begins by synthesizing ideas from a set of related papers to identify gaps and formulate new directions. However, training language models to perform this form of literature-grounded ideation remains challenging, as existing approaches based on prompting or feedback lack structured supervision for how papers should be synthesized. We introduce IdeaAnchor, a paradigm for training LLMs to perform research ideation using structured specifications as privileged signals. Each IdeaAnchor instance encodes how each input paper should be synthesized into a successful idea, including their functional roles, relationships, and target synthesis criteria. We build this paradigm by mining instances from published papers, capturing how real ideas emerge from prior literature. We then train models via demonstration, self-distillation, and reinforcement learning, and further enhance generation with retrieval at inference time. Experiments show consistent improvements in ideation quality. Our analysis reveals a functional decomposition: anchor-based training strengthens creative synthesis, retrieval enhances detail elaboration, and combining both yields the best performance.

Ziyu Chen, Yilun Zhao, Jiashuo Sun +3

The Unversity of Chicago · Yale University · University of Illinois Urbana-Champaign +1

Research Idea Generation Large Language Model-Guided Scientific Discovery

Ask PDF

Dec 19, 2024

LDC: Learning to Generate Research Idea with Dynamic Control

Recent advancements in large language models (LLMs) have demonstrated their potential in automating the scientific research ideation. Existing approaches primarily focus on prompting techniques, often producing ideas misaligned with expert standards - novelty, feasibility, and effectiveness, which are widely recognized by the research community as the three key subdimensions of high-quality ideas. Also, balancing these dimensions remains challenging due to their inherent trade-offs. To address these limitations, we propose the first framework that employs a two-stage approach combining Supervised Fine-Tuning (SFT) and controllable Reinforcement Learning (RL) for the task. In the SFT stage, the model learns foundational patterns from pairs of research papers and their corresponding follow-up ideas. In the RL stage, multi-dimensional reward models guided by fine-grained feedback evaluate and optimize the model across key dimensions. During inference, dimensional controllers coordinated by a sentence-level decoder enable dynamic context-aware steering of the idea generation process. Our framework provides a balanced approach to research idea generation, achieving high-quality outcomes in the experiment by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.

Ruochen Li, Liqiang Jing, Chi Han +2

University of Texas at Dallas · UIUC · Stony Brook University

Controllable Text Generation Research Idea Generation Large Language Model-Guided Scientific Discovery

Ask PDF

Abstract

Diverse Text Generation

Explore similar work

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas

LDC: Learning to Generate Research Idea with Dynamic Control