Training Reasoning Models to Generate Diverse Scientific Ideas

Manya Wadhwa; Pranav Narayanan Venkit; Hiroaki Hayashi; Yu Li; Tuhin Chakrabarty; Chien-Sheng Wu

cs.CLOct 15, 2026

Training Reasoning Models to Generate Diverse Scientific Ideas

Authors: Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi, Yu Li, Tuhin Chakrabarty, Chien-Sheng Wu

Organizations: New York University · Salesforce AI Research · Stony Brook University

Abstract

One of the first steps of a research pipeline is generating ideas. Large Language Models (LLMs) are increasingly used to automate this step, but most of this work optimizes the quality of individual ideas, while overlooking diversity. In our work, we study document-grounded ideation, focusing on increasing the diversity of LLM generated ideas and ensuring they are high quality. Given prior papers, a model must propose a set of ideas that are each high-quality and clearly distinct from one another, while also being grounded in the input documents. To optimize for this, we propose training with a set-level reinforcement learning objective. In our setup, in a single generation, the model writes K structured idea cards grounded in the source papers, and the set is scored jointly, where each idea's quality is weighted by its distance from the ideas written before it. This teaches the model to steer each new idea away from its predecessors. We show that our trained 8B model achieves higher creative utility (a measure combining quality and diversity) than Claude Opus 4.8 and GPT-6 Astra on our contamination-free benchmark. In a pilot human study with four experts, our sets are judged as promising as Claude Opus 4.8's and contain marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, our model's success@k is comparable to Claude Opus 4.8's, so we see that the added diversity does not cost any executability.

Figures & tables

Figure 1: Overview of our training pipeline. Given a set of documents, the policy generates a set of ideas

U=\{u_{1},\dots,u_{k}\}

; for reasoning models, the whole set is generated after a single shared reasoning prefix. Each idea

u_{i}

is scored by its quality

q_{i}

and its distance

d_{i}

to the closest earlier idea (

d_{1}=1

), giving a per-item reward

r_{i}=q_{i}\cdot d_{i}

. The set reward

R(U)=\sum_{i}r_{i}

is used as the RL signal to update the policy, encouraging sets that are both high-quality and diverse.

Model	Creative Utility (CU) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
General models
Claude Opus 4.8	$0.38_{(0.22)}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra	$0.28_{(0.24)}$	0.15	$0.86_{(0.06)}$
Nemotron 120B	$0.14_{(0.18)}$	0.05	$0.79_{(0.10)}$
Competing objectives
DARLING	$0.52_{(0.17)}$	0.38	$0.67_{(0.13)}$

Table 1: Results on our test set. We report Creative Utility (

CU

) with

\gamma=0.9

. Win Rate is the per-idea impact against the gold idea, corrected for length. Avg. Distance is the mean pairwise distance between the ideas in a set. Values in parentheses are standard deviations across instances. Proposed (no think) has the highest creative utility; Proposed (think) is on par with DARLING and the no-SFT ablation.

Figure 2: Creative Utility (CU) against average grounding ( $g$ ) for the systems also mentioned in Table 1 . The highest-CU systems are less grounded than Claude and the SFT-only models.

Model	Win Rate $\uparrow$	Avg. distance $\uparrow$
GIANTS-4B	18.3%	0.025
Proposed	68.3%	0.939

Table 2: GiantsBench results on 2,374 test papers.

Method	$H_{\mathrm{norm}}$ $\uparrow$	TVD vs human $\downarrow$
Qwen3-8B	0.77	0.27
Claude Opus 4.8	0.77	0.23
DARLING	0.01	0.72
Proposed (no think)	0.65	0.43
Proposed (think)	0.73	0.30
Human proposals	0.87	–

Table 3: Taxonomy diversity on IdeaSeed (500x3 ideas), Method Paradigm axis.

System	Promising	Unique	Unique Promising
Qwen3-8B	$1.52_{0.83}$	$2.52_{0.62}$	$1.42_{0.81}$
Claude Opus 4.8	$\mathbf{2.20_{0.73}}$	$2.72_{0.52}$	$2.03_{0.76}$
Proposed	$2.17_{0.72}$	$\mathbf{2.83_{0.38}}$	$\mathbf{2.05_{0.75}}$

Table 4: Human study results with 4 annotators, 15 papers each. We see our proposed model generates ideas that are as promising and diverse as Claude-Opus-4.8. Proposed is significantly better than Qwen3-8B on all three metrics.

Arm	$k=1$	$k=2$	$k=3$
Claude Opus-4.8	13%	20%	27%
Proposed (no think)	11%	22%	33%
Proposed (think)	10%	19%	29%

Table 5: Execution results of 132 ideas by a coding agent.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

In-text context	Source	Label
“Recent work Gilardi et al. ( 2023 ) ; He et al. ( 2023a ); Ding et al. ( 2022 ) has looked at the annotation accuracy for LLMs and found them to be at par with crowd-worker annotators.”	ChatGPT outperforms crowd workers for text-annotation tasks	motivates
“ MuRAG Chen et al. ( 2022 ) requires 3 million image-text pairs for pre-training to achieve decent results.”	MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text	exposes_gap
“The global strategy involves using a trained image caption model to generate text descriptions of the macro scenes of the image Changpinyo et al. ( 2022 ) ; Yang et al. ( 2022b )”	All You May Need for VQA are Image Captions	foundation
“This paper is an extended work of an earlier version presented at SOMET 2022 (Shirafuji et al., 2022 ) ”	Prompt Sensitivity of Language Model for Solving Programming Problems	extends

Appendix

Table 6: Representative source citations, one per functional label. The in-text context is the sentence in the target paper that cites the source; the label is the role assigned by the importance classifier

Figure 3: Distribution of (a) number of sources kept per-target post source filtering (b) the different citation/source types post-filtering.

Figure 4: Statistics around the training and test datasets used in this work. Our training and test distributions contain varying number of source documents as inputs in order for the model to generate ideas, we also maintain a strict contamination boundary for the models we are training and evaluating. The legend counts (train $n{=}8{,}007$ , test $n{=}1{,}741$ ) are the full pools after the source-count filter; the models are SFT’d on 7,172 of the training instances (after some filtering) and evaluated on 1,740 test instances.

Figure 5: Distribution of domains in the test dataset.

	SFT	GRPO
Init	Qwen3-8B	SFT checkpoint (after 1 epoch)
Training instances	7,172 (same instances for SFT and GRPO), plus 100 validation
Learning rate	$5\times 10^{-6}$	$1\times 10^{-6}$
Batch size	64 sequences	12 prompts $\times$ 4 rollouts
PPO mini-batch (prompts)	–	12 (one update per step)
Epochs / steps	1 epoch (112 steps)	600 steps / 1 epoch

Appendix

Table 7: Hyperparameters for the main Qwen3-8B run.

Impact judge	win-rate
SciJudge-30B ‡	0.65
gpt-5.4-mini †	0.74
claude-sonnet-5	0.74

Appendix

Table 8: Impact-judge win-rate. † evaluation judge; ‡ training-reward judge.

Impact judge	Length (tokens)		Strict win-rate
	orig	matched	orig	matched
gpt-5.4-mini †	642	258	0.253	0.284
claude-sonnet-5	642	258	0.090	0.133
claude-sonnet-4.6	642	258	0.165	0.205

Appendix

Table 9: Impact-judge performance with length changes. † evaluation judge.

Field	HH	gpt-oss-120b	gemini-3.5-flash
	$\rho$	$\rho$	$\rho$
problem gap	0.30	0.31	0.44
core claim	0.36	0.29	0.30
method	0.28	0.38	0.53
Distance $d\in[0,1]$	0.36	0.37	0.50

Appendix

Table 10: Distance-metric validation on

n{=}100

within-set idea pairs rated by both human annotators, over the three scored fields and their aggregated distance

d\in[0,1]

(Eq. 4 ). All entries are Spearman

\rho

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
General models
Claude Opus 4.8	$0.38_{(0.22)}$	$0.38_{(0.48)}$	$\mathbf{0.98_{(0.08)}}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra	$0.28_{(0.24)}$	$0.27_{(0.44)}$	$0.94_{(0.11)}$	0.15	$0.86_{(0.06)}$
Nemotron 120B	$0.14_{(0.18)}$	$0.08_{(0.28)}$	$0.90_{(0.18)}$	0.05	$0.79_{(0.10)}$
Qwen3-8B	$0.03_{(0.10)}$	$0.02_{(0.13)}$	$0.91_{(0.21)}$	0.01	$0.80_{(0.11)}$
Competing objectives

Appendix

Table 11: Results on our contamination-free test set (1,740 instances), with two extra columns: number of instances which at least one win in their set and grounding. Values in parentheses are standard deviations across instances.

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
Claude Opus 4.8
Sampled ( $k{=}1,n{=}3$ )	$0.34_{(0.14)}$	$0.59_{(0.49)}$	$0.99_{(0.06)}$	0.33	$0.24_{(0.21)}$
Set ( $k{=}3,n{=}1$ )	$0.38_{(0.22)}$	$0.38_{(0.48)}$	$0.98_{(0.08)}$	0.21	$0.80_{(0.07)}$
GPT-6 Astra
Sampled ( $k{=}1,n{=}3$ )	$0.25_{(0.17)}$	$0.37_{(0.48)}$	$0.96_{(0.09)}$	0.23	$0.28_{(0.23)}$
Set ( $k{=}3,n{=}1$ )	$0.28_{(0.24)}$	$0.27_{(0.44)}$	$0.94_{(0.11)}$	0.15	$0.86_{(0.06)}$

Appendix

Table 12: Results on our contamination-free test set (1,740 instances). Set draws all

K=3

ideas in one response (as in Table 11 ); sampled draws them as three independent single-idea responses. We can see most models trade off quality for diversity when switching from sampling to set-based generation. Values in parentheses are standard deviations across instances.

Figure 6: Creative Utility(CU) for two baselines and our proposed models, across 16 domains in our test-set.

ARR Area	$n$ papers (%)	Promising ideas $\uparrow$	Unique ideas $\uparrow$	Unique promising ideas $\uparrow$
		(Qwen3-8B / Claude Opus 4.8 / Ours per column)
Human-Centered NLP and Human-AI Interaction	18 (30%)	$1.67/\textbf{2.39}/\textbf{2.33}$	$2.56/2.78/\textbf{2.83}$	$1.61/2.28/2.11$
LLM agents	11 (18%)	$1.36/\textbf{2.00}/\textbf{2.00}$	$2.36/\textbf{2.82}/\textbf{2.82}$	$1.18/1.91/\textbf{2.00}$
Machine Learning for NLP	7 (12%)	$2.14/2.14/\textbf{2.43}$	$\textbf{2.86}/2.71/\textbf{2.86}$	$2.14/2.14/\textbf{2.29}$
Resources and Evaluation	5 (8%)	$1.40/\textbf{2.20}/1.40$	$2.20/\textbf{3.00}/2.80$	$1.20/\textbf{2.20}/1.40$
Ethics, Bias, and Fairness	5 (8%)	$0.80/\textbf{2.80}/2.40$	$2.40/2.40/\textbf{2.80}$	$0.60/1.80/\textbf{2.20}$

Appendix

Table 13: Human study results broken down by the ACL Rolling Review (ARR) subarea of the target paper (§ 6.1 ).

Model	Utility (CU) $\uparrow$	$\geq$ 1 Win $\uparrow$	Grounding ( $g$ ) $\uparrow$	LC Win Rate ( $q$ ) $\uparrow$	Avg. Distance ( $d$ ) $\uparrow$
Base	$0.10_{(0.16)}$	$0.06_{(0.25)}$	$0.73_{(0.27)}$	0.03	$0.79_{(0.12)}$
SFT (no think)	$0.17_{(0.18)}$	$0.11_{(0.31)}$	$0.84_{(0.22)}$	0.05	$0.77_{(0.10)}$
Proposed (no think)	$0.56_{(0.23)}$	$0.67_{(0.47)}$	$0.66_{(0.27)}$	0.39	$0.83_{(0.09)}$
SFT (think)	$0.12_{(0.15)}$	$0.06_{(0.23)}$	$0.86_{(0.22)}$	0.03	$0.75_{(0.10)}$
Proposed (think)	$0.54_{(0.24)}$	$0.63_{(0.48)}$	$0.73_{(0.25)}$	0.34	$0.83_{(0.08)}$

Appendix

Table 14: Results on our contamination-free test set (1,740 instances), showing the recipe applied to Olmo-3-7B-Think-SFT. Values in parentheses are standard deviations across instances.

	Win-Rate			Grounding ( $g$ )
Model	1	2	3	1	2	3
Proposed (no think)	0.69	0.72	0.65	0.79	0.56	0.48
Proposed (think)	0.62	0.54	0.52	0.88	0.80	0.75

Appendix

Table 15: How the Impact/Win-rate and grounding changes with the position at which an idea is generated. We report the raw win-rate (instead of the strict length-controlled) here instead of length normalized

Annotator	System	$n$	Promising	Unique	Unique & Promising
Annotator 1	Qwen3-8B	15	$1.73_{(0.88)}$	$2.67_{(0.49)}$	$1.67_{(0.90)}$
	Claude Opus 4.8	15	$1.93_{(0.59)}$	$2.67_{(0.49)}$	$1.87_{(0.64)}$
	Proposed	15	$2.20_{(0.56)}$	$2.87_{(0.35)}$	$2.20_{(0.56)}$
Annotator 2	Qwen3-8B	15	$1.27_{(0.46)}$	$2.40_{(0.74)}$	$1.13_{(0.35)}$
	Claude Opus 4.8	15	$1.93_{(0.80)}$	$2.80_{(0.56)}$	$1.87_{(0.83)}$
	Proposed	15	$1.67_{(0.49)}$	$2.73_{(0.46)}$	$1.53_{(0.52)}$

Appendix

Table 16: Human annotations per annotator. For each portfolio of

K=3

ideas, annotators marked the ideas that are worthwhile, the distinct representatives, and the distinct representatives that are also worthwhile. We report the mean number per portfolio (standard deviation in parentheses);

n

is the number of portfolios.

Method	$H_{\mathrm{norm}}$ $\uparrow$	TVD vs h $\downarrow$	JSD vs h $\downarrow$	% unique/instance $\uparrow$	Ideas
Opportunity types
Claude Opus 4.8 (sampled, $k=1,n=3$ )	0.86	0.35	0.12	63.4% (n=498)	1494
Qwen3-8B (sampled, $k=1,n=3$ )	0.56	0.60	0.30	54.6% (n=500)	1500
Claude Opus 4.8 (set, $k=3,n=1$ )	0.89	0.27	0.08	89.0% (n=499)	1497
Qwen3-8B (set, $k=3,n=1$ )	0.80	0.30	0.10	89.3% (n=499)	1275
DARLING (sampled, $k=1,n=3$ )	0.05	0.80	0.58	35.0% (n=500)	1500

Appendix

Table 17: Taxonomy diversity on IdeaSeed (500 papers: 250 ML, 250 natural science). Every generated idea and every human proposal is labeled by GPT-5.4-mini with one Opportunity Pattern and one Method Paradigm from the TasteGap taxonomy ( Chen et al., 2026 ) .

Figure 7: Distribution of ‘opportunity’ and ‘method’ labels on the IdeaSeed benchmark, as generated by different models compared to the proposed model.

Verdict	Claim	Proof of concept (summary)	Results (summary)
Supports Per-form robustness map for AI-text detectors think model; coverage: most	A benchmark of many text-evasion forms shows detectors resist some attacks and fail on others, and the perform map lets practitioners choose a detector per setting.	6 detectors $\times$ 12 conditions on human vs. ChatGPT text (HC3), 150 texts per class per condition. Four zero-shot statistical scores, a RoBERTa detector and a char n-gram classifier. Forms: paraphrase, style shift, typos and homoglyphs at three strengths, domain shift, regeneration by two other LLMs. TPR at 5% FPR. Oracle per-form choice replaced by a held-out split-half test (400 splits).	Near ceiling on clean text (TPR $0.92$ – $0.99$ ) but divergent by form: log-likelihood $1.00$ under domain shift, $0.11$ under paraphrase, $0.00$ under typos and homoglyphs (non-overlapping CIs). Rankings barely agree across forms (Kendall $\tau=0.095$ ). Held-out per-form selection beats the best single detector by $+0.107$ TPR (95% CI $[0.064,0.137]$ , 400/400 splits). Not reached: commercial or watermark detectors, white-box attacks, larger domain grid.
Refutes One cross-task faithfulness evaluator no-think model; coverage: most	Faithfulness is task-agnostic and measurable uniformly by one lightweight evaluator; models are far more unfaithful than accuracy suggests, with a larger gap for reasoning than retrieval.	6 task conditions in one evidence format (GSM8K, KG-QA, HotpotQA, SQuAD, two TAT-QA), two Llama solvers, 430 traces. One fixed evaluator prompt on four lightweight evaluators plus a Claude-Sonnet-5 reference. Ground truth from paired corruption: a value inserted that appears nowhere in the evidence (329 traces).	Correctness and grounding are separable, and all evaluators beat chance in all six conditions. But the measurement is not uniform: recall $26.6$ – $94.2\%$ , false-positive rate $0$ – $22\%$ , and the headline contrast flips sign between two evaluators. Reasoning-vs-retrieval gap is null within a modality (table $+9.3$ [ $0.0$ , $20.7$ ] vs. $+1.1$ [ $-10.2$ , $14.0$ ]); the effect is concentrated in KG-QA ( $43.5\%$ of correct answers unsupported vs. $0$ – $11.5\%$ ). A universal claim falsified by a subset.
Inconclusive Content-level citation auditing no-think model; coverage: most; blocker: noise	Many citation errors are content-level, and an auditor reasoning over the claim and cited text catches them far more reliably than a metadata-only verifier, with no index needed.	Citation-Integrity corpus (biomedical, expert labels): 770 test pairs (abstract-level cited text), 399-pair stratified slice, precision reweighted to natural prevalence. Four LLM auditors (sub-claim decomposition then grounding) vs. a metadata-only verifier (OpenAlex/Crossref) and TF-IDF.	Direction holds: content errors are $21.0\%$ of pairs, and the metadata verifier has no content signal (precision $0.26$ vs. $0.21$ base rate). Magnitude does not: AUROC $0.62$ – $0.67$ (all errors), $0.73$ – $0.79$ (core), best F1 $0.37$ – $0.43$ , recall $0.50$ only at precision $0.30$ ; between the pre-registered bars. Auditors agree with each other ( $\kappa$ $0.42$ – $0.73$ ) more than with gold ( $0.15$ – $0.25$ ), within the published expert band. Not reached: full-text papers, other domains, fine-tuned auditors.

Appendix

Table 18: Example autoresearch cards, one per verdict. Each card is an idea’s frozen claim, a scaled-down proof of concept run by the coding agent under a 90-minute, $30, CPU-only budget, and the agent’s structured verdict. Summaries are condensed from the agent’s own report.

Diverse Text Generation

Diverse text generation studies how to produce varied, useful language outputs without sacrificing coherence, accuracy, or task performance.

See all 50 papers

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
Jul 24, 2026Varun Gumma, Navonil Majumder, Soumitra Sinhahajari +1Multi-Agent LLM Systems Quality-Diversity Optimization
DeCLaRe Lab, Nanyang Technological University, Singapore
IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas
Oct 6, 2026Ziyu Chen, Yilun Zhao, Jiashuo Sun +3Research Idea Generation Large Language Model-Guided Scientific Discovery
The Unversity of Chicago · Yale University · University of Illinois Urbana-Champaign +1
LDC: Learning to Generate Research Idea with Dynamic Control
Dec 19, 2024Ruochen Li, Liqiang Jing, Chi Han +2Controllable Text Generation Research Idea Generation
University of Texas at Dallas · UIUC · Stony Brook University

Jul 24, 2026J/K move · Enter open · S save

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

Varun Gumma, Navonil Majumder, Soumitra Sinhahajari, Soujanya Poria

DeCLaRe Lab, Nanyang Technological University, Singapore

Multi-Agent LLM Systems Quality-Diversity Optimization Research Idea Generation Large Language Model-Guided Scientific Discovery

Open paper AskPDF arXiv

Large Language Models (LLMs) have significantly advanced the automation of scientific discovery. However, existing systems share a core limitation: they optimize ideas independently for either Quality or Diversity. This often results in closely related ideas or large sets of trivial, unsound, or unclear concepts. We instead argue that research ideation should be framed as a conjunction of Quality and Diversity objectives, i.e., a Quality-Diversity (QD) search. Accordingly, we introduce IDEAgent, a multi-agent framework that tracks idea evolution through lineages. We jointly drive Quality using multi-objective feedback for repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, historical variants, and rejected proposals. To evaluate this joint objective, we develop Yield, a metric that computes the largest mutually diverse set of ideas satisfying predefined quality thresholds. Across 32 topics spanning 8 CS domains, IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving Yield of at least 2 on 8x as many topics. We further analyze quality improvements, showing that repair and refinement improve logical rigor and clarity while preserving non-obviousness. Lastly, we extend IDEAgent to two algorithm discovery tasks for verifiable quality judgment and find that it still consistently outperforms all baselines, and yields the largest set of high-quality and diverse programs that beat a trivial/standard baseline. To encourage future research on QD-search-based ideation, we open-source IDEAgent at https://github.com/declare-lab/IDEAgent.

Abstract

Diverse Text Generation

Explore similar work

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas

LDC: Learning to Generate Research Idea with Dynamic Control