Feedback-driven iteration is widely used to improve LLM-generated scientific ideas, but existing systems either leave revision decisions implicit in generation, align revisions with feedback-localized weaknesses, or restrict action selection to system-specific operations. This is problematic because problems, hypotheses, and solutions are interdependent: the location where a weakness manifests does not itself determine the revision scope that will produce a better successor. We term this phenomenon diagnosis-intervention mismatch. We introduce Predictive Routing of Interventions by Modeling Effects (PRIME), a plug-and-play feedback-to-revision decision layer for existing scientific ideators. PRIME operates in three stages: (1) Structural Attribution interprets feedback over the problem-hypothesis-solution structure without fixing a revision target; (2) Counterfactual Routing constructs alternative scope-conditioned interventions and compares their projected successor utility and risks through lightweight rollouts; and (3) Intervention Execution compiles the selected intervention into an ordered generator-facing plan with semantic actions, preservation constraints, and dependency requirements for successor generation. We validate diagnosis intervention mismatch on 112 human-annotated idea-feedback pairs. Across four benchmarks, two ideators, and fixed- and open-problem settings, PRIME improves successor quality and routing reliability over strong baselines; removing counterfactual routing reduces quality from 3.463 to 1.960 and increases route regret from 0.426 to 1.929, confirming cross-scope effect modeling as the primary source of its gains.
Figures & tables
Method
Feedback update
Explicit action
Editable state
Output search
Scope-effect routing
Plug-in layer
Mismatch audit
Self-Refine / Reflexion
✓
×
×
×
×
×
×
ResearchAgent / EvoIdeator
✓
✓
×
×
×
×
×
IdeaSynth
✓
✓
✓
×
×
×
×
Deep Ideation / VirSci
✓
✓
×
×
×
×
×
AI Scientist / Co-Scientist
✓
×
×
✓
×
×
×
Nova / Chain of Ideas
✓
×
×
✓
×
×
×
Table 1: Positioning PRIME relative to output-search and feedback-driven revision methods. Output-level search selects among generated ideas or trajectories, whereas PRIME selects among alternative revisions for the same idea–feedback pair before final generation.
Figure 1: Position–Execute Iterative Refinement. PRIME structures each idea into problem (P), hypothesis (H), and solution (S), positions feedback across their components and relations, selects an action, remove, revise, or replace; rewrites the prompt with explicit reasoning to iteratively generate improved ideas.
Figure 2: Feedback-guided revision through structured reasoning. (a) Holistic and localized revision: decisions are implicit in generation or tied to feedback-localized weaknesses. (b) Workflow-coupled revision: actions are selected from a system-specific operation space. (c) Feedback-reasoned revision: diagnosis revision mismatch is explicitly modeled to guide revision.
Update Policy
Ideator
AI Idea Bench
IdeaBench
LiveIdeaBench
MLR-Bench
I2I
I2T
IMCQ
SemSim
Overlap
NIS
FIS
Orig.
Feas.
Clar.
Cons.
Clar.
Nov.
Feas.
Sig.
Overall
Flat Feedback
Direct Prompting
2.656
4.690
0.744
0.647
1.774
0.885
0.165
5.506
4.810
5.112
7.192
6.338
6.657
5.689
7.061
6.411
SciMON
2.648
4.560
0.734
0.599
1.540
0.996
0.029
6.50
2.528
3.776
7.839
6.533
7.676
5.113
7.673
6.561
Problem-First H/S
Direct Prompting
2.685
4.716
0.759
0.647
2.011
0.937
0.164
6.366
4.683
5.582
7.734
6.657
6.905
6.237
7.567
6.896
SciMON
2.621
4.469
0.707
0.642
1.660
0.997
0.017
6.938
2.426
3.637
7.240
6.144
7.774
4.506
7.508
6.273
Problem Regeneration
Direct Prompting
2.519
4.556
0.766
0.647
1.699
0.864
0.186
6.180
4.171
4.916
7.577
6.660
7.052
5.703
7.595
6.750
Table 2: Main results across four scientific idea-generation benchmarks. Rows compare feedback-to-revision policies across two ideators, with the Graph-Based block additionally reporting reasoning and PRIME-enhanced variants. PRIME-Refinement fixes the problem reference, whereas PRIME-Evolution permits context-grounded problem switching. All methods use the same experimental settings within each benchmark. All metrics are higher-is-better.
(a) Diagnostic–intervention location mismatch
Source or Setting
N
M@0
M@1
M@2 [95% CI]
M@5
Many@2 [95% CI]
Rep. M@2 [95% CI]
Rdiag
95% CI
p
Human Groundtruth
112
73.2
72.3
65.2 [56.2, 74.1]
41.1
40.2 [31.2, 49.1]
43.8 [34.8, 52.7]
3.79
[ 2.74 , 4.93]
.0001
Model Attribution
112
73.2
73.2
61.6 [52.7, 70.5]
38.4
42.9 [33.9, 51.8]
25.0 [17.0, 33.0]
1.13
[ −0.06 , 2.42]
.0386
Human 1
112
73.4
73.4
68.8 [59.6, 77.1]
38.5
65.1 [56.0, 74.3]
28.4 [20.2, 37.6]
1.15
[ 0.21 , 2.15]
.0137
Human 2
112
76.6
76.6
71.2 [62.2, 79.3]
38.7
68.5 [59.5, 77.5]
29.7 [21.6, 38.7]
1.03
[ 0.05 , 2.02]
.0212
Human–Human Reliability
77
67.5
66.2
62.3 [51.9, 72.7]
29.9
62.3 [51.9, 72.8]
20.8 [11.7, 29.9]
0.05
[ −1.01 , 1.14]
.4356
Table 3: Diagnosis intervention location mismatch and feedback-to-intervention routing. Panel (a) establishes whether the diagnostic location identifies the most effective intervention location. Panel (b) evaluates how six update policies convert the same feedback into intervention actions on the 112 complete annotated instances.
Update Policy
I2I ↑
I2T ↑
IMCQ ↑
(a) Base-model transfer: Llama with Direct Prompting
Flat Feedback
3.114
4.374
0.787
Problem-First H/S
3.094
4.231
0.821
Problem Regeneration
2.876
4.238
0.816
Direct Feedback Decomp.
3.122
4.415
0.821
Feedback Reasoning
3.573
4.568
0.883
Table 4: Generalization results on AI Idea Bench. Panel (a) replaces the base model with Llama under Direct Prompting, while Panel (b) replaces Direct Prompting with CoI-Agent using the original base model.
Variant
Quality ↑
Regret ↓
Cons. ↑
PRIME
3.463
0.426
3.173
w/o Counterfactual Routing
1.960
1.929
1.773
w/o Dependency Closure
3.490
0.399
3.211
w/o Upstream Candidates
3.177
0.712
2.909
w/o Problem Candidates
3.110
0.779
2.831
w/o Risk Penalty
3.459
0.430
3.158
Table 5: Ablation results for PRIME. Quality, Regret, and Cons. denote successor utility, regret relative to the post-hoc best intervention, and structural consistency, respectively. Higher Quality and Cons. and lower Regret indicate better performance.