Scientific discovery requires ideas that are original, useful, and non-obvious. Yet AI scientists are typically given as much task information as possible during idea generation, which can anchor generation toward familiar approaches. Inspired by the cognitive framework of Blind Variation and Selective Retention (BVSR), we propose a discovery framework that separates these two roles of information: it generates proposals under deliberately limited task information to broaden exploration, then learns from proposals previously verified on the same task and their outcomes to retain promising candidates for costly empirical evaluation. We evaluate BVSR across five proposer models and 12 scientific tasks. Limiting task information substantially increases proposal originality and non-obviousness, and produces a larger share of highly creative and high-utility proposals than existing state-of-the-art prompting-based baselines. Learning from prior proposal outcomes on the task further improves candidate selection, discarding half of the candidates while retaining most top-performing proposals. When incorporated into existing AI-scientist workflows, BVSR achieves stronger discoveries across gene expression analysis, drug response prediction, anomaly detection, scaling law discovery, and brain activity prediction. These results suggest that effective AI scientists should use information differently across stages of discovery: constrain task-specific information during exploration while exploiting accumulated empirical experience on the task.
Figures & tables
Figure 1: Overview of Blind Variation and Selective Retention (BVSR). (a) Limiting task information during idea generation encourages creative proposals. (b) Selective retention reduces the number of candidates requiring empirical verification while preserving top proposals. (c) BVSR’s best programs borrow mechanisms from diverse fields, outperforming the best AI scientist on biology and ML tasks.
Figure 2: Effects of task information on proposal creativity. Each dot summarizes one batch of proposals for a benchmark, information level, and model. The trend line follows the 95th percentile (a, b) or the mean (c, d) of the dots. Percentages give its change from 0 to 140 bits. As information increases, the relative frequency of proposals ranked in the benchmark’s top 1% for creativity or utility generally decreases, while mean originality and mean non-obviousness decline substantially.
Method
Creativity ( ↑ )
Utility ( ↑ )
Originality ( ↑ )
Non-obviousness ( ↑ )
BV (Limited Information)
0.530±0.043
0.571±0.041
0.955±0.002
0.713±0.012
BV (Full Information)
0.462±0.045
0.524±0.043
0.926±0.004
0.609±0.015
Analogical Reasoning
0.429±0.049
0.508±0.050
0.914±0.007
0.492±0.037
Cross-Domain
0.369±0.052
0.531±0.045
0.871±0.012
0.263±0.020
No-Domain
0.117±0.025
0.530±0.050
0.598±0.020
0.104±0.015
Table 1: Comparison of BV with baselines. The highest is shown in bold. Values are mean ± SE.
Figure 3: Preserving top candidates during selective retention. Fraction of top proposals kept as discard ratio varies. Selective retention keeps the most top candidates, ahead of SFT regression and prompt-based judges.
Figure 4: Integrating BVSR into SimpleTES. Best task score as a function of verification steps on five scientific tasks. BVSR improves final score and reaches strong solutions with fewer steps. Numbers give how many times fewer verified proposals SimpleTES + BVSR needs to reach SimpleTES’s final score (dotted line).
Task
Ours
SimpleTES
Best Baseline
Gene Expression Analysis
0.918
0.906
0.853 [ 60 ]
Anomaly Detection
0.679
0.629
0.491 [ 23 ]
Scaling Law Discovery
0.967
0.940
N/A
Table 2: Best programs of our methods and of SimpleTES on example tasks (higher is better). Best baseline: SPARK-X for gene expression analysis, Isolation Forest for anomaly detection.
Figure 6: Proposal quality across domains. Normalized top-1% fractions by creativity and utility.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Agreement across models on estimated information levels. Each cell shows the Pearson correlation between two models’ estimates across 32 contexts. Bold model names indicate the three models used in our estimator.
Four fixed classifiers on MNIST, Fashion-MNIST, Madelon, Breast Cancer
ECE, Brier, NLL vs. baselines; geometric mean
Temperature scaling 0.2935
180 s
Appendix
Table 3: The twelve discovery tasks across four scientific domains. The time limit is the CPU wall-clock evaluation limit per candidate (for ZAPBench , the GPU training budget on an RTX PRO 6000).
Figure 8: Example page of the originality evaluation form. The evaluator sees a task and the mechanism descriptions of two proposals written for it, and judges whether the two express essentially the same underlying mechanism (Yes, No, or Unsure).
Figure 9: Example page of the non-obviousness evaluation form. The evaluator sees a task, a summary of the paper the proposal draws on, and a summary of the proposed method, and rates how obvious it is that the method would work for the task, from 1 (obvious: the literature already shows it working on this kind of task) to 5 (not obvious at all).
Research idea generation uses computational methods, especially language models, to propose and assess new questions, hypotheses, and research directions.
AI research agents now support large-scale AI-assisted scientific discovery. We examine whether AI-generated ideas broaden scientific exploration or primarily reinforce existing work. Using five agent frameworks and five large language models, we generate 219,655 ideas for different scientific fields. Across experiments, four consistent patterns emerge. First, AI-generated ideas are more concentrated than human-authored papers within the same research area. Second, they remain much closer to starting literature than later human follow-on work does. Third, AI-generated ideas align less with future human research. Last, AI-generated ideas are located in lower-impact regions of the historical scientific landscape. Overall, current AI research agents appear better suited to local elaboration than to broadening scientific exploration.
Yixuan Tang, Yi Yang
1*ISOM, The Hong Kong University of Science and Technology.
Despite rapid progress in automating scientific research, generating promising and well grounded research solutions remains a central challenge. We isolate research ideation as a standalone task and build our solution on the intuition that a challenge in one field can often be addressed by a mechanism that solved an analogous challenge in another. Accordingly, we introduce IdeaScientist, which decomposes ideation into gap finding, innovation, and report writing, and trains each role with reinforcement learning. These roles identify limitations in related work, draw solution intuitions from analogous problem settings, and develop those intuitions into complete research proposals. To facilitate discovery of insights across domains, we construct the Svalbard Idea Vault, a corpus of 2.77M decomposed research ideas for retrieval, training, and temporally controlled evaluation. Our evaluation restricts access to literature available before a cutoff date and assesses how closely proposed directions align with those later explored in 15K papers authored by human researchers. On Qwen3.6-27B, IdeaScientist outperforms the strongest open-source autoresearch baseline by 14.0%, driven mainly by gains in novelty. On this 27B open backbone, IdeaScientist even outperforms Claude Code SDK with Claude-4.8-Opus and Codex SDK with GPT-5.4, by up to 5.9%.
Jiarui Liu, Renjie Tao, Yiwei Liao +15
Carnegie Mellon University · Meta Reality Labs · FAIR at Meta +5
Scientific ideation unfolds over multiple stages, including literature search, paper reading, tool use, claim checking, cross-paper synthesis, brainstorming, rejection of weak directions, and iterative writing. Yet most existing resources capture isolated components or final artifacts rather than the process connecting them. We introduce IdeaTrail, a dataset of 1,170 multi-turn trajectories for scientific ideation and proposal generation. Each trajectory follows a research process from evidence gathering to either idea selection or proposal construction, jointly recording tool use, acquired evidence, intermediate artifacts, and reasoning. IdeaTrail is synthesized from human-selected research papers and proposal artifacts through a Generator--Advisor loop. The Generator produces the visible sequence of actions, observations, and artifact edits, while the Advisor uses the full generation context to check grounding, causal order, naturalness, and leakage from hidden targets. This reverse-to-forward design keeps trajectories aligned with real scientific artifacts while retaining the uncertainty, evidence use, and staged convergence characteristic of research practice. IdeaTrail provides both reusable process supervision and a general recipe for constructing scientific-research-agent data.