cs.AINov 2, 2026

PRIME: Predictive Routing of Interventions by Modeling Effects for Scientific Ideation

Authors: Fengxian Ji, Xie Yu, Zhexuan Cui, Yuke Li, Zirui Song, Fan Zhang, Qian Jiang, Jingpu Yang, 

Organizations: School of Artificial Intelligence, Wuhan University · MBZUAI · Northeastern University · Zhongguancun Academy

Abstract

Feedback-driven iteration is widely used to improve LLM-generated scientific ideas, but existing systems either leave revision decisions implicit in generation, align revisions with feedback-localized weaknesses, or restrict action selection to system-specific operations. This is problematic because problems, hypotheses, and solutions are interdependent: the location where a weakness manifests does not itself determine the revision scope that will produce a better successor. We term this phenomenon diagnosis-intervention mismatch. We introduce Predictive Routing of Interventions by Modeling Effects (PRIME), a plug-and-play feedback-to-revision decision layer for existing scientific ideators. PRIME operates in three stages: (1) Structural Attribution interprets feedback over the problem-hypothesis-solution structure without fixing a revision target; (2) Counterfactual Routing constructs alternative scope-conditioned interventions and compares their projected successor utility and risks through lightweight rollouts; and (3) Intervention Execution compiles the selected intervention into an ordered generator-facing plan with semantic actions, preservation constraints, and dependency requirements for successor generation. We validate diagnosis intervention mismatch on 112 human-annotated idea-feedback pairs. Across four benchmarks, two ideators, and fixed- and open-problem settings, PRIME improves successor quality and routing reliability over strong baselines; removing counterfactual routing reduces quality from 3.463 to 1.960 and increases route regret from 0.426 to 1.929, confirming cross-scope effect modeling as the primary source of its gains.

Figures & tables

Explore similar work

Sep 24, 2026

Learning to Ideate for Scientific Impact

Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately judgeable proxies such as novelty, clarity, and feasibility. This leaves open whether delayed signals of scientific uptake can be used as feedback for steering models toward research directions with higher expected impact. We study this question using citation-normalized impact as a noisy but scalable proxy for scholarly uptake. We construct a large-scale dataset from over 100K computer science papers by extracting goal-conditioned idea descriptions and assigning each paper an ordinal, year-normalized citation label. We then train a goal-conditioned reward model to predict citation-impact labels from research goal and idea pairs, and use this reward to align an idea generator through supervised fine-tuning followed by reinforcement learning. To reduce circularity, we evaluate generated ideas with a held-out, reference-grounded protocol that compares model outputs against historical ideas under the same research goal and weights judgments by the reference idea's citation-impact label. Experiments show that our RL-tuned model consistently produces ideas with higher estimated impact than both the base model and supervised fine-tuning baselines. Our findings position scientific impact as a practical, outcome-grounded feedback signal for aligning LLMs in open-ended scientific discovery.
Mar 23, 2026

EvoIdeator: Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learning

Scientific idea generation is a cornerstone of autonomous knowledge discovery, yet the iterative evolution required to transform initial concepts into high-quality research proposals remains a formidable challenge for Large Language Models (LLMs). Existing Reinforcement Learning (RL) paradigms often rely on rubric-based scalar rewards that provide global quality scores but lack actionable granularity. Conversely, language-based refinement methods are typically confined to inference-time prompting, targeting models that are not explicitly optimized to internalize such critiques. To bridge this gap, we propose EvoIdeator, a framework that facilitates the evolution of scientific ideas by aligning the RL training objective with checklist-grounded feedback. EvoIdeator leverages a structured judge model to generate two synergistic signals: (1) lexicographic rewards for multi-dimensional optimization, and (2) fine-grained language feedback that offers span-level critiques regarding grounding, feasibility, and methodological rigor. By integrating these signals into the RL loop, we condition the policy to systematically utilize precise feedback during both optimization and inference. Extensive experiments demonstrate that EvoIdeator, built on Qwen3-4B, significantly outperforms much larger frontier models across key scientific metrics. Crucially, the learned policy exhibits strong generalization to diverse external feedback sources without further fine-tuning, offering a scalable and rigorous path toward self-refining autonomous ideation.
May 1, 2026

Structure Liberates: How Constrained Sensemaking Produces More Novel Research Output

Scientific discovery is an extended process of ideation -- surveying prior work, forming hypotheses, and refining reasoning -- yet existing LLM research agents treat this phase as a brief preamble despite its central role in research. We introduce SciSense, a sensemaking-grounded framework that operationalizes ideation as a structured sequence of eight cognitive stages (Pirolli and Card, 2005). We construct SciSense-Traj, a dataset of 120K citation-conditioned research trajectories in two modes: Hindsight, where an LLM reconstructs the ideation path from a paper's cited works to the paper itself, and Foresight, where the LLM proposes novel directions from the same citations. We distill these into SciSense-LM, a family of sensemaking LLMs spanning 3B to 70B parameters. Contrary to the assumption that looser supervision promotes exploration, Hindsight-trained models improve trajectory quality by 2.0% over Foresight-trained models while also producing more novel and diverse outputs, across all eight model families we test. The advantage propagates downstream: coding agents conditioned on Hindsight trajectories produce research artifacts with higher executability, scientific grounding, and utility than those conditioned on Foresight trajectories. This suggests that outcome-anchored ideation reduces the cognitive burden on downstream agents, freeing them to explore more creatively. SciSense offers both a practical tool for augmenting LLM-driven research workflows and a principled testbed for studying how planning shapes scientific discovery.