FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Exploration
Authors: Qiyao Wang, Hongbo Wang, Longze Chen, Zhihao Yang, Guhong Chen, Hamid Alinejad-Rokny, Hui Li, Yuan Lin, Min Yang
Organizations: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Dalian University of Technology · UNSW Sydney · Xiamen University · Shenzhen University of Advanced Technology
Scientific idea generation (SIG) is critical to AI-driven autonomous research, yet existing approaches are often constrained by a static retrieval-then-generation paradigm, leading to homogeneous and insufficiently divergent ideas. In this work, we propose FlowPIE, a tightly coupled retrieval-generation framework that treats literature exploration and idea generation as a co-evolving process. FlowPIE expands literature trajectories via a flow-guided Monte Carlo Tree Search (MCTS) inspired by GFlowNets, using the quality of current ideas assessed by an LLM-based generative reward model (GRM) as a supervised signal to guide adaptive retrieval and construct a diverse, high-quality initial population. Based on this population, FlowPIE models idea generation as a test-time idea evolution process, applying selection, crossover, and mutation with the isolation island paradigm and GRM-based fitness computation to incorporate cross-domain knowledge. It effectively mitigates the information cocoons arising from over-reliance on parametric knowledge and static literature. Extensive evaluations demonstrate that FlowPIE consistently produces ideas with higher novelty, feasibility and diversity compared to strong LLM-based and agent-based frameworks, while enabling reward scaling during test time.
Figures & tables
Figure 1: Comparison of traditional literature-based SIG frameworks and our FlowPIE.
Figure 2: Test-time idea evolution scaling for reward. Initial Ideation couples literature exploration with idea generation via flow-guided MCTS, where higher reward, reflecting better initial ideas, amplify the weight of corresponding literature. Idea Evolution uses various evolutionary operators to guide ideas toward regions of higher rewards and stable convergence.
Figure 3: Overview of FlowPIE. Left: Idea initialization based on flow-guided MCTS: forward exploration via flow-guided UCB (Eq. 1 ) for nodes selection and expansion, and backward updating (Eq. 2 ) to enforce local and global flow constraints; this dual process dynamically adjusts literature weights with GRM-based rewards on generated ideas. Right: Test-time idea evolution, supervised by GRM-based fitness computation, leveraging various operators, including crossover on core technical features and isolation-island-enhanced mutation, to evolve ideas.
Method
I2T
I2I
IMCQ
Motivation
Exp Plan
Motivation
Exp Plan
Motivation
Exp Plan
SCIPIP
4.18 ± 0.662
—
3.68 ± 0.383
—
0.464
—
Research Agent
4.56 ± 0.679
3.59 ± 0.670
3.78 ± 0.491
3.45 ± 0.619
0.510
0.497
Chain-of-Ideas
4.63 ± 0.703
3.79 ± 0.566
3.74 ± 0.504
3.50 ± 0.653
0.507
0.433
VirSci
4.32 ± 0.652
3.68 ± 0.565
3.95 ± 0.410
3.84 ± 0.410
0.560
0.580
FlowPIE ( Ours )
4.64 ± 0.606
3.72 ± 0.481
4.44 ± 0.318
3.85 ± 0.347
0.780
0.635
Table 1: Main results on AI-Idea-Bench-2025. We generate three ideas with each method and report using the mean ± std for I2T and I2I. Bold indicates the best score, and underlining denotes the second-best score.
Method
Simi.
Overlap
NI
FI
SCIPIP
0.526
5.03
0.816
0.133
Research Agent
0.558
6.66
0.722
0.138
Chain-of-Ideas
0.482
7.24
0.926
0.095
VirSci
0.521
6.24
0.716
0.075
FlowPIE( Ours )
0.559
7.76
0.825
0.105
Initial Population
0.532
7.64
0.750
0.136
Table 2: IdeaBench Results. Simi. denotes Semantic Similarity, while Overlap denotes Idea Overlap. NI and FI denote the Novelty and Feasibility Insight scores.
Method
Reward
Human Evaluation
N
F
Avg.
N
F
E
EE
Avg.
SCIPIP
0.50
0.73
0.62
0.38
0.37
0.30
0.29
0.34
Research Agent
0.49
0.66
0.58
0.22
0.19
0.13
0.13
0.17
Chain-of-Ideas
0.59
0.66
0.63
0.34
0.29
0.31
0.29
0.31
VirSci
0.60
0.65
0.63
0.27
0.17
0.20
0.20
0.21
FlowPIE ( Ours )
0.75
0.76
0.76
0.45
0.36
0.38
0.37
0.39
Table 3: Reward and human evaluation results. N: Novelty, F: Feasibility, E: Excitement, EE: Expected Effectiveness, and Avg.: Average.
Method
AI
HM
GMB
ES
NCS
TE
SBS
MS
PHP
SCIPIP
0.62
0.60
0.64
0.63
0.63
0.62
0.61
0.67
0.60
Research Agent
0.58
0.63
0.66
0.65
0.65
0.63
0.60
0.69
0.58
Chain-of-Ideas
0.63
0.67
0.71
0.71
0.70
0.70
0.66
0.73
0.67
VirSci
0.60
0.65
0.69
0.66
0.67
0.67
0.61
0.70
0.60
FlowPIE( Ours )
0.75
0.81
0.80
0.83
0.81
0.77
0.76
0.88
0.78
Initial Population
0.68
0.72
0.67
0.78
0.70
0.68
0.67
0.83
0.69
Table 4: Domain-specific reward performance across nine domains.
Categories
Metric
N
F
E
EE
Human and Human
Spearman ρ
0.49
0.54
0.32
0.47
Human and Reward
Spearman ρ
0.60
0.87
–
–
Table 5: Inter-expert agreement of human evaluation and correlation between our reward and the average human score on novelty and feasibility.
Figure 4: (a) Distribution of explored literature count; (b) Diversity score distribution.
Method
AR ↑
I2T ↑
I2I ↑
IMCQ ↑
SCIPIP
0.61
4.18
3.68
0.464
w/ Idea Evolution
0.63 (+3.3%)
4.24 (+1.4%)
3.70 (+0.5%)
0.475 (+2.4%)
Chain-of-Ideas
0.59
4.21
3.62
0.437
w/ Idea Evolution
0.64 (+8.5%)
4.23 (+0.5%)
3.63 (+0.3%)
0.448 (+2.5%)
FlowPIE ( initial )
0.68
4.08
3.94
0.688
w/ Idea Evolution
0.75 (+10.1%)
4.18 (+2.5%)
4.15 (+5.3%)
0.708 (+2.9%)
Table 6: Idea evolution results with initial ideas from different baselines. We report the average reward (AR) and average score of three tasks in AI Idea Bench 2025.
Input topic : The topic of this paper is improving reasoning capabilities in Large Language Models (LLMs).
Final Idea Title : Dynamic Macro-Guided Verification (DMGV): A Modular Reasoning Augmentation Framework for LLMs
Summary : The proposed Dynamic Macro-Guided Verification (DMGV), a framework that converts recurrent reasoning subchains in CoT traces into reusable reasoning macros , and attaches lightweight verifiers to validate their outputs . Instead of repeatedly regenerating similar reasoning steps, the model composes solutions by invoking validated macros and verifying their outputs. The core insight is that many reasoning traces contain repeated sub-structures that can be compressed into reusable operators. By validating macro outputs through interface-level verification rather than re-evaluating entire reasoning chains, DMGV aims to reduce both error accumulation and token-level redundancy in LLM reasoning . It consists of 3 stages: (1) Macro discovery : reasoning traces are segmented and clustered to identify recurrent subchains, which are distilled into parameterized macros with explicit input-output interfaces. (2) Macro-guided reasoning : during inference, the generator composes reasoning steps by invoking macros whenever a subgoal matches a macro signature. (3) Targeted verification : a lightweight verifier checks macro outputs using symbolic, arithmetic, or structural consistency tests; failed cases trigger localized fallback re-synthesis.
Table 7: Demonstration of an idea generated by FlowPIE. Due to space limitations, we present only a concise summary of the final idea. The complete version of this idea is provided in Table 11 .
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Prompt Tokens
Avg. Tokens / Idea
Avg. Cost / Idea
SCIPIP
4062.50
921.33
$0.038
Research Agent
1755.44
477.53
$0.028
Chain-of-Ideas
4526.92
650.43
$0.052
Virsci
1458.36
789.60
$0.061
FlowPIE ( Ours )
4123.43
924.25
$0.056
Appendix
Table 8: API cost comparison for idea generation across different methods. All methods use GPT-4o-mini as the idea generator.
Section
Count
Percentage (%)
A
1293
8.06
B
980
6.11
C
710
4.42
D
26
0.16
E
123
0.77
F
208
1.30
Appendix
Table 9: Distribution of IPC sections.
Figure 5: Dataset overlap distribution between our literature and two benchmarks across different similarity levels. The overlap ratio rapidly decreases as similarity increases and approaches zero in the high-similarity region, indicating minimal risk of data leakage.
Figure 6: Human evaluation criteria. Top: The definition and guideline of novelty score. Bottom: The definition and guideline of feasibility score.
Figure 7: Human evaluation criteria. Top: The definition and guideline of excitement score. Bottom: The definition and guideline of expected effectiveness score.
Table 10: Performance comparison across different LLM backbones on the AI Idea Bench 2025.
Figure 8: Prompt for the initial idea population generation.
Figure 9: Prompt for the parent ideas crossover operator.
Figure 10: Prompt for the parent ideas mutation operator.
Figure 11: Prompt for the generative reward model for novelty and feasibility evaluation.
Domain : Health & Medicine
Final Idea Title : A Tumor-Core–Activated Therapeutic Strategy
Summary : The proposed strategy introduces a tumor-core–activated neutrophil reprogramming blockade that selectively prevents infiltrating neutrophils from converging into the pro-angiogenic dcTRAIL-R1 + T3 state identified in the target paper. Rather than depleting neutrophils systemically or blocking VEGF broadly, this approach acts upstream at the level of fate determination , using the hypoxic–glycolytic tumor niche as a spatial trigger to locally release an inhibitor against a critical epigenetic or transcriptional effector of the T3 program. The central advantage is that it suppresses tumor-promoting neutrophil plasticity with spatial precision while preserving peripheral neutrophil host-defense functions , thereby overcoming the major limitations of pan-neutrophil depletion and conventional anti-angiogenic therapy. Conceptually, the strategy combines (1) microenvironment-selective activation , ensuring that the payload is engaged primarily within the tumor core; (2) neutrophil fate interception , blocking the chromatin remodeling and transcriptional induction required for dcTRAIL-R1 and VEGF α upregulation; and (3) functional antitumor consequence , reducing T3 accumulation, intratumoral angiogenesis, and tumor growth. By targeting the newly defined deterministic reprogramming axis rather than only its downstream effector output, this framework offers a more mechanism-based, selective, and potentially safer therapeutic paradigm for modulating tumor-associated neutrophils in solid cancers.
Appendix
Table 12: Demonstration of an idea generated by FlowPIE in the Health & Medicine domain.
Domain : Genetics & Molecular Biology
Final Idea Title : A system-and-method for "sticker-resolved flow-activation spectroscopy" of biomolecular condensates
Summary : The proposed framework introduces sticker-resolved flow-activation spectroscopy (SRFAS) , a combined experimental–computational system that quantitatively separates the energetic barrier governing condensate network rearrangement from the chain-length–dependent kinetic prefactors that control relaxation dynamics . Instead of relying on conventional rheology that conflates binding energetics with polymer architecture, SRFAS integrates temperature-dependent microrheology, sequence-tunable transient probe tracers, and Bayesian inversion of coarse-grained transient-network models to independently infer two key parameters: the flow activation energy (Ea) , reflecting sticker binding free energies, and the kinetic prefactor (τ0(N,v,c)) , reflecting chain length, valence, and network topology. This separation enables direct recovery of molecular-scale interaction energetics from purely bulk and tracer-level rheological observables , providing predictive control over macroscopic properties such as viscosity, relaxation time, and molecular diffusion in biomolecular condensates. By coupling experimentally measurable viscoelastic spectra, tracer diffusion, and residence-time distributions with model-based inference, the system delivers experimentally verifiable, sequence-to-rheology mapping that surpasses conventional microrheological approaches in both parameter identifiability and predictive accuracy. As a result, SRFAS establishes a general platform for rational design, screening, and mechanistic analysis of biomolecular condensates and associative polymer materials , with applications ranging from understanding disease-related condensate mutations to engineering synthetic phase-separating biomaterials with programmable dynamic properties.
Appendix
Table 13: Demonstration of an idea generated by FlowPIE in the Genetics & Molecular Biology domain.
Domain : Environmental Sciences
Final Idea Title : A transferable Diagnostic-and-Prediction system for ecosystem functional loss under rare short-term droughts
Summary : The proposed DP-Extreme (Diagnostic-and-Prediction system for Extreme drought impacts) introduces a transferable framework that quantitatively predicts ecosystem productivity loss under rare short-term drought events and guides optimized mitigation decisions. Unlike conventional precipitation–productivity models that assume linear sensitivity to rainfall deficits, DP-Extreme explicitly incorporates drought rarity (return period), pre-drought resource stocks (soil moisture and nutrients), and community resilience traits (biodiversity and rooting depth) into a hierarchical scaling law that captures the nonlinear sensitivity of aboveground net primary production (ANPP) to extreme pulses. By combining standardized pulse-exclusion experiments across environmental gradients with Bayesian parameter inference, the system estimates site-specific parameters that generate an Extreme Sensitivity Index (ESI) —a predictive indicator of expected ANPP loss for rare drought scenarios (e.g., 1-in-100-year events). This enables threshold-triggered mitigation strategies such as targeted irrigation or grazing control that activate only when predicted loss exceeds critical levels. The key advantage is that DP-Extreme decouples drought rarity effects from ecological buffering traits , producing substantially more accurate forecasts of ecosystem productivity loss and enabling resource-efficient interventions. In validation experiments, the system is designed to demonstrate significantly lower prediction error and improved decision efficiency compared with conventional climate-only or linear sensitivity models, establishing a practical decision-support platform for managing ecosystem resilience under increasingly frequent extreme climate events.
Appendix
Table 14: Demonstration of an idea generated by FlowPIE in the Environmental Sciences domain.
Research idea generation uses computational methods, especially language models, to propose and assess new questions, hypotheses, and research directions.
Generating novel research ideas is fundamental to scientific progress. While Large Language Models (LLMs) show promise in assisting this process, existing approaches often exhibit semantic convergence, resulting in limited diversity and novelty. To address this, we introduce EvoGens, an evolution-inspired framework that recasts scientific idea generation as an evolutionary search over a population of ideas. EvoGens iteratively applies rank-based mutation with differentiated retrieval planning to incorporate external knowledge, and semantic-aware crossover to fuse complementary concepts for conceptual reorganization. A lightweight evaluation signal guides the selection process, encouraging sustained exploration while mitigating premature convergence. Extensive experiments demonstrate that EvoGens substantially enhances exploration capabilities compared to state-of-the-art baselines. Specifically, it improves the Novelty from 0.1 to 0.4 and the Diversity from 0.24 to 0.55, while maintaining comparable idea quality under the current automatic evaluation protocol. These findings suggest that evolutionary mechanisms can serve as a useful framework for exploration-oriented research ideation, especially for broadening the novelty and diversity of candidate ideas under a shared automatic evaluation setting.
Xu Li, Hanzhe Tu, Xinyi Li +3
Southwest Petroleum University, Chengdu, China · Sichuan Police College, Luzhou, China
Despite rapid progress in automating scientific research, generating promising and well grounded research solutions remains a central challenge. We isolate research ideation as a standalone task and build our solution on the intuition that a challenge in one field can often be addressed by a mechanism that solved an analogous challenge in another. Accordingly, we introduce IdeaScientist, which decomposes ideation into gap finding, innovation, and report writing, and trains each role with reinforcement learning. These roles identify limitations in related work, draw solution intuitions from analogous problem settings, and develop those intuitions into complete research proposals. To facilitate discovery of insights across domains, we construct the Svalbard Idea Vault, a corpus of 2.77M decomposed research ideas for retrieval, training, and temporally controlled evaluation. Our evaluation restricts access to literature available before a cutoff date and assesses how closely proposed directions align with those later explored in 15K papers authored by human researchers. On Qwen3.6-27B, IdeaScientist outperforms the strongest open-source autoresearch baseline by 14.0%, driven mainly by gains in novelty. On this 27B open backbone, IdeaScientist even outperforms Claude Code SDK with Claude-4.8-Opus and Codex SDK with GPT-5.4, by up to 5.9%.
Jiarui Liu, Renjie Tao, Yiwei Liao +15
Carnegie Mellon University · Meta Reality Labs · FAIR at Meta +5
Recent advancements in large language models (LLMs) have demonstrated their potential in automating the scientific research ideation. Existing approaches primarily focus on prompting techniques, often producing ideas misaligned with expert standards - novelty, feasibility, and effectiveness, which are widely recognized by the research community as the three key subdimensions of high-quality ideas. Also, balancing these dimensions remains challenging due to their inherent trade-offs. To address these limitations, we propose the first framework that employs a two-stage approach combining Supervised Fine-Tuning (SFT) and controllable Reinforcement Learning (RL) for the task. In the SFT stage, the model learns foundational patterns from pairs of research papers and their corresponding follow-up ideas. In the RL stage, multi-dimensional reward models guided by fine-grained feedback evaluate and optimize the model across key dimensions. During inference, dimensional controllers coordinated by a sentence-level decoder enable dynamic context-aware steering of the idea generation process. Our framework provides a balanced approach to research idea generation, achieving high-quality outcomes in the experiment by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.
Ruochen Li, Liqiang Jing, Chi Han +2
University of Texas at Dallas · UIUC · Stony Brook University