Scientific feedback reveals productive regimes beyond training conventions

Yoon Pyo Lee; Samrendra Roy; Souvik Chakraborty; Syed Bahauddin Alam

cs.LGDec 12, 2025

Scientific feedback reveals productive regimes beyond training conventions

Authors: Yoon Pyo Lee, Samrendra Roy, Souvik Chakraborty, Syed Bahauddin Alam

Organizations: Department of Nuclear Engineering, Hanyang University, 222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea.

Abstract

Human-imposed design conventions can restrict the alternatives represented in scientific training data and shape the search preferences learned by generative models. We show that scientific performance feedback can redirect the learned search distribution toward productive regions beyond those conventions, without explicitly rewarding departures from them. We test this process in reactor lattice design with simulated feedback and chemical reaction-condition selection with measured yields, using a pretrained sequence policy and a small non-language policy, respectively. In both settings, adaptation improves scientific performance while increasing generation of choices excluded from the corresponding supervised data. In measured reactions, probability on task-specific ligand choices excluded from training rises from 0.29 to 0.70-0.71 as expected yield error decreases. Controls in the lattice setting show that the productive region identified during adaptation provides reusable information to random search, while aligned generation produces useful candidates more frequently than informed random sampling. Removing the target penalty redirects four of five matched runs, linking the direction of adaptation to the scientific objective. Together, these results show how scientific feedback reshapes inherited search preferences and reveals productive alternatives beyond training conventions, with attribution controls distinguishing learned redistribution from candidate availability and reusable search information.

Figures & tables

Figure 1: Scientific feedback beyond training conventions in two implementations and the attribution of the adaptation. a Reactor lattices. One of 10,000 training lattices, all with 16 absorber rods, is serialized into 289 symbols for fuel ( f ), absorber ( g ) and guide channel ( c ). The histogram gives the corpus multiplication factor, whose nearest lattice lies 11,192 pcm above the target of 1.05. Gemma 3 with 270 million parameters, trained on this corpus, generates text for the prompt $k=1.05$ that is corrected and evaluated by OpenMC. The generated lattice is the selected DPO design with 33 absorbers. Maps give the thermal neutron flux ( $E<0.625$ eV) relative to its maximum and the pin fission rate relative to the cell mean ( $k=1.0492$ ). b Reaction conditions. Filled cells mark the one anchor ligand per aryl-halide task kept in the supervised corpus from 3,955 measured Buchwald–Hartwig reactions. A categorical policy selects ligand, additive and base among measured rows for the coupling with p-toluidine. Colours give measured yields for 3-bromopyridine with MTBD over the four ligands (rows, XPhos, t-BuXPhos, t-BuBrettPhos and AdBrettPhos) and 22 additives, and the reward is the negative absolute yield error. c Attribution on the inventory axis. Outlined bars give the inventory of 5,000 corrected outputs from the five supervised policies before alignment (Methods), 88.6% at exactly 16 and none above 18. The triangle marks the corpus and the tallest bar is clipped. Grey bars give the inventory of 10,000 corrected, unselected outputs from the ten aligned checkpoints, 5.2% of them above 64. The dashed line is the inventory-uniform density over 0–264 (availability). The upper bracket marks 28–35, the inventories of the ten selected layouts (redistribution). The arrow marks objective removal, after which four of five matched runs select zero absorbers. The lower bracket marks the interval 20–40 supplied to informed random search (information value). Right, best objectives after 2,000 evaluations (five-run mean and standard deviation) and the exact off-corpus ligand probability and expected yield error at requested yield 90 (five-seed means), with the full-catalog uniform sampler dashed.

Figure 2: Scientific feedback identifies and populates a productive design regime. a Inventory of best-so-far layouts during DPO and GRPO alignment, with one line per training seed. The horizontal reference marks the 16-absorber corpus convention. GRPO is displayed at group-completion times. b Signed deviations from the requested multiplication factor 1.05 after independent reevaluation of all ten selected layouts. Each evaluation uses 20 million active histories. Error bars show one Monte Carlo standard deviation. Every layout remains within 500 pcm of the target. c Empirical cumulative distributions of inventory in 5,000 corrected, unselected outputs per method, with 1,000 per checkpoint at temperature 1.0. d Target-window incidence among 500 randomly selected corrected outputs, with 50 per checkpoint, and 10,000 candidates from each random sampler. DPO contributes 33/250 hits and GRPO 68/250, giving 101/500 in the pooled aligned sample. Informed random sampling over inventories 20–40 gives 521/10,000 hits. Inventory-uniform sampling over 0–264 gives 36/10,000. Methods specifies the sampling procedures.

Procedure	Absorber-count access	Best objective
Constrained GA	Fixed at 16	$12.38\pm 0.06$
Unconstrained GA	0–264	$1.90\pm 0.13$
Inventory-uniform random	0–264	$1.767\pm 0.045$
Informed random	Supplied interval 20–40	$1.65\pm 0.04$
DPO policy	0–264	$1.65\pm 0.04$
GRPO policy	0–264	$1.66\pm 0.05$

Table 1: Best-of-budget objectives across five runs of 2,000 online evaluations. Values are means

\pm

sample standard deviations. The aligned policies also use a shared 10,000-evaluation training corpus. Informed random search receives the interval identified from alignment.

Figure 3: Objective dependence and the value of inventory information. a Chosen-layout inventory trajectories for full-objective DPO and matched runs with the target penalty removed. Thin lines show seeds and thick lines show the five-seed mean after a 25-step rolling mean. b Final-100-step mean inventories for each matched seed. Four removed-penalty runs reach zero inventory and the fifth averages 39.29. c–e Best-of-budget composite objective, absolute target error and peaking-only component from five independent runs of 2,000 online evaluations per method. Points show seeds and bars show means with sample standard deviations. Inventory-uniform random search samples count over 0–264. Informed random search receives the interval [20,40] identified after alignment and recovers comparable objective quality. The dashed reference in d marks the approximate online transport uncertainty of 150 pcm. The fixed-inventory GA uses 16 absorbers.

Figure 4: Scientific rewards select useful reactions beyond task-specific training choices. a,b Expected absolute yield error and off-corpus ligand probability for requested yield 90. c,d Corresponding measurements for ten balanced targets from 10 to 100. The supervised base, DPO and GRPO use CPT

+

SFT initialization. Gray lines connect each of five seeds to its aligned descendants. Large markers and intervals show means and two-sided 95% Student-

t

intervals across seeds. Uniform full-catalog and anchor-catalog samplers provide exact distributional comparisons. Error panels also show the attainable minima in the full and anchor catalogs. All within-policy quantities are exact catalog expectations.

	Measured-reaction catalog	Simulator-evaluated lattices
Policy	Masked categorical autoregressive network, hidden size 48	Gemma 3 sequence backbone, 270 million parameters
Supervised convention	One anchor ligand per aryl-halide task	Exactly 16 absorber rods per lattice
Online candidate space	All 3,955 measured catalog rows, conditioned on task	Fuel/absorber choices at 264 non-guide positions
Feedback	Deterministic lookup of measured yield	Stochastic continuous-energy Monte Carlo transport
Distribution measurement	Exact task-conditioned catalog probabilities	Corrected sample incidence and selected-design trajectories
Quality measurement	Expected target error and catalog optima	Target-window incidence and best-of-budget objective

Table 2: Two experimental settings and their measurements. Each supervised corpus fixes an identifiable feature, while online generation permits alternative values.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Parameter	Value
Geometry	Assembly	$17\times 17$ PWR type
	Lattice pitch	1.26 cm
	Fuel-pellet radius	0.4096 cm
	Cladding inner diameter	0.835 cm
	Cladding outer diameter	0.950 cm
	Cladding material	Zircaloy-4

Appendix

Supplementary Table 1: Parameters of the two-dimensional reactor model. The pin-resolved PWR-type assembly specifies the reflective-lattice, SMR-inspired testbed.

Procedure	All inventories	Inventories 28–35
Inventory-uniform random	36/10,000 (0.36%)	36/281 (12.8%)
Informed random	521/10,000 (5.21%)	521/3,830 (13.6%)
DPO	33/250 (13.2%)	23/109 (21.1%)
GRPO	68/250 (27.2%)	60/173 (34.7%)
Aligned pooled	101/500 (20.2%)	83/282 (29.4%)

Appendix

Supplementary Table 2: Target-window incidence under the three sampling procedures. Each entry gives hits/evaluated candidates and the corresponding percentage. The right column conditions both numerator and denominator on inventories 28–35. Policy candidates are corrected before transport evaluation. The pooled aligned row combines DPO and GRPO. Each random procedure has five runs of 2,000 evaluations, while each policy has five checkpoints with 50 randomly selected outputs per checkpoint.

Supplementary Fig. 1: Additional convergence traces and selected lattice configurations. a,b Cumulative-best composite fitness and the multiplication factor of the incumbent layout for five runs per method. Shading gives pointwise 95% Student- $t$ intervals across seeds after interpolation. Solid lines denote CPT $+$ SFT and dashed lines denote SFT only for the aligned policies. c,d Constructed symmetric references. e–h Best-of-budget layouts from the fixed-inventory GA, variable-inventory GA, DPO and GRPO, respectively. Values beneath each layout are its online evaluation. DPO/GRPO denote the implemented variants, and $k_{\mathrm{eff}}$ denotes $k_{\infty}$ .

Method	Seed	Gd	$k_{\infty}$	$\sigma_{k}$ (pcm)	$F_{q}$	$F_{\Delta H}$	$\mathcal{L}_{\mathrm{fit}}$
DPO	0	33	1.048970	22.0	1.564	1.234	1.535
DPO	1	30	1.051477	19.0	1.549	1.225	1.567
DPO	2	35	1.050032	20.1	1.629	1.145	1.439
DPO	3	28	1.051875	19.3	1.544	1.151	1.574
DPO	4	29	1.050691	19.6	1.483	1.252	1.460
GRPO	0	28	1.052712	18.7	1.480	1.213	1.644

Appendix

Supplementary Table 3: High-statistics reevaluation of the ten selected aligned layouts. Each fixed layout receives one independent OpenMC evaluation with 20 million active histories.

\sigma_{k}

is the OpenMC-reported Monte Carlo standard deviation in pcm, where one pcm is

10^{-5}

k_{\infty}

. Composite fitness uses main-text Equation (1). Seed identifiers refer to the five-seed campaign.

Supplementary Fig. 2: First passage, inventory reversals and seedwise sampling counts. a First batch containing a candidate with at least 20 Gd rods. Points mark batch completion, segments show the unresolved within-batch interval, and numbers identify seeds. DPO, GRPO and GA batches contain 2, 4 and 40 candidates, respectively. b First-passage counts and subsequent changes in incumbent inventory across five runs. c Counts from five full-space random runs with 2,000 evaluations each. All 36 candidates within

|k_{\infty}-1.05|\leq 0.005

contain 28–35 Gd rods. DPO/GRPO denote the implemented variants.

Update	Initialization	Composite fitness	$F_{q}$	$F_{\Delta H}$
DPO	SFT only	$1.72\pm 0.11$	$1.95\pm 0.08$	$1.28\pm 0.05$
DPO	CPT $+$ SFT	$1.65\pm 0.04$	$1.88\pm 0.11$	$1.20\pm 0.06$
GRPO	SFT only	$1.69\pm 0.10$	$1.98\pm 0.19$	$1.21\pm 0.04$
GRPO	CPT $+$ SFT	$1.66\pm 0.05$	$1.87\pm 0.05$	$1.24\pm 0.03$

Appendix

Supplementary Table 4: Best-design metrics for the reactor initialization ablation. Mean

\pm

sample standard deviation across five seeds. DPO and GRPO identify the implemented pairwise and group-relative variants.

Supplementary Fig. 3: Initialization ablation. Solid lines and filled markers denote CPT

+

SFT. Dashed lines and open markers denote SFT only. a,c Single-target cumulative-best fitness. b,d Multi-target per-step fitness with a 25-step rolling mean. e,f Best-design fitness and inventory by seed. Shading gives pointwise Student-

t

intervals across five seeds. CPT reduces single-target fitness variation in these runs, with inventory and multi-target fitness effects that depend on the update variant. DPO/GRPO denote the implemented variants.

	Filtered responses		All responses
Checkpoint condition	Slope	95% interval	Slope	95% interval
CPT $+$ SFT base	$-1.6$	$[-12.2,9.1]$	$0.4$	$[-13.1,13.5]$
CPT $+$ SFT / DPO single	$17.9$	$[-5.1,45.1]$	$22.5$	$[-8.3,55.2]$
CPT $+$ SFT / GRPO single	$-11.2$	$[-42.6,20.1]$	$-21.5$	$[-56.9,12.9]$
CPT $+$ SFT / DPO multi	$-10.4$	$[-39.3,18.9]$	$-14.4$	$[-49.6,20.1]$
CPT $+$ SFT / GRPO multi	$-162.9$	$[-193.7,-134.3]$	$-162.5$	$[-198.4,-127.4]$

Appendix

Supplementary Table 5: Prompt-response slopes and response-sampling intervals for all archived checkpoints. Slopes are rods per unit requested multiplication factor. Each interval uses 10,000 bootstrap resamples within target cells at one fixed checkpoint. The filtered analysis excludes responses with fewer than 50 valid lattice symbols before padding. The all-response analysis retains all 700 responses per checkpoint.

Supplementary Fig. 4: Prompt-conditioned inventory response in archived checkpoints. a,b Mean generated absorber count for CPT

+

SFT and SFT-only checkpoints, respectively, at seven requested multiplication factors. Each condition uses one archived checkpoint from runs preceding the five-seed campaign. Each target begins with 100 responses; exclusion of responses containing fewer than 50 valid cell symbols before padding retains 94–100. Counts follow parsing and length normalization and precede guide-coordinate correction. Bands show two-sided 95% Student-

t

intervals across retained responses. c Slopes for all ten checkpoints with two-sided 95% percentile intervals from 10,000 bootstrap resamples independently within each target cell. The filled point identifies the multi-target GRPO checkpoint with CPT

+

SFT whose interval excludes zero. Supplementary Table 5 includes the all-response sensitivity analysis.

Layout	Initial $k_{\infty}$	Endpoint $k_{\infty}$	Unity crossing	$\Delta k_{\mathrm{Gd}}$
			(MWd kgHM -1 )
DPO	1.0521	0.8361	27.00	$0.00552\pm 0.00140$
GRPO	1.0499	0.8363	26.97	$0.00640\pm 0.00182$
Variable GA	1.0492	0.8380	27.07	$0.00716\pm 0.00209$
16-Gd reference	1.1675	0.8392	27.15	$0.00152\pm 0.00199$
24-Gd reference	1.0861	0.8365	27.12	$0.00900\pm 0.00155$

Appendix

Supplementary Table 6: Depletion characteristics of five selected layouts. Uncertainties in

\Delta k_{\mathrm{Gd}}

are propagated transport standard deviations. Unity-crossing values are point estimates from the prescribed burnup grid, with depletion-discretization uncertainty unquantified.

Supplementary Fig. 5: Depletion assessment beyond the alignment reward. a Reflective-lattice multiplication factors for three selected optimized layouts and two constructed references over 26 CE/CM intervals to 50 MWd kgHM -1 at 35 W gHM -1 . Shading denotes reported transport standard deviation. b Endpoint residual-Gd effect calculated by removing Gd while fixing other endpoint nuclide densities. Error bars denote propagated transport uncertainty. c Initial and endpoint multiplication factors, interpolated unity-crossing burnups and endpoint Gd effects. These diagnostics characterize the selected lattices under the stated depletion conditions.

Setting / initialization	Policy	Absolute error	Off-support mass
Single / CPT $+$ SFT	SFT base	$50.040\pm 0.536$	$0.287\pm 0.042$
	DPO	$29.886\pm 0.246$	$0.701\pm 0.023$
	GRPO	$29.771\pm 0.166$	$0.714\pm 0.018$
Single / SFT only	SFT base	$50.929\pm 0.728$	$0.669\pm 0.012$
	DPO	$30.530\pm 0.725$	$0.728\pm 0.003$
	GRPO	$29.878\pm 0.116$	$0.731\pm 0.002$

Appendix

Supplementary Table 7: Complete chemistry policy endpoints. Mean

\pm

sample standard deviation across five seeds. Absolute yield error is in yield percentage points. All within-policy quantities are exact catalog expectations.

Setting / initialization	Update	Error change	Mass change
Single / CPT $+$ SFT	DPO	$-20.154\pm 0.299$	$+0.414\pm 0.057$
Single / CPT $+$ SFT	GRPO	$-20.269\pm 0.444$	$+0.427\pm 0.025$
Single / SFT only	DPO	$-20.399\pm 0.454$	$+0.059\pm 0.013$
Single / SFT only	GRPO	$-21.052\pm 0.746$	$+0.062\pm 0.010$
Multi / CPT $+$ SFT	DPO	$-4.688\pm 0.444$	$+0.044\pm 0.023$
Multi / CPT $+$ SFT	GRPO	$-4.861\pm 0.289$	$+0.038\pm 0.015$

Appendix

Supplementary Table 8: Paired chemistry changes from the supervised base. Mean

\pm

sample standard deviation of five within-seed changes. Every error change is negative and every off-support change positive at the seed level.

AI for Science

AI for Science applies machine learning and AI agents to scientific data, models, experiments, and discovery workflows.

See all 157 papers

SciDesignBench: Benchmarking and Improving Language Models for Scientific Inverse Design
Mar 13, 2026David van Dijk, Ivan VrkicInverse Design Large Language Model-Guided Optimization
CellType Inc.
Learning to Ideate for Scientific Impact
Sep 24, 2026Shubham Kale, Aniketh Garikaparthi, Manasi PatwardhanScientific Hypothesis Generation Large Language Model-Guided Scientific Discovery
TCS Research, India.
Can AI Scientist Agents Learn from Lab-in-the-Loop Feedback? Evidence from Iterative Perturbation Discovery
Mar 27, 2026Gilles Wainrib, Barbara Bodinier, Haitem Dakhli +5AI Agents for Scientific Discovery Large Language Model-Guided Scientific Discovery
Owkin Inc.

Mar 13, 2026J/K move · Enter open · S save

SciDesignBench: Benchmarking and Improving Language Models for Scientific Inverse Design

David van Dijk, Ivan Vrkic

CellType Inc.

Inverse Design Large Language Model-Guided Optimization Scientific Reasoning in Language Models Benchmark Design Fine-Tuning

Open paper AskPDF arXiv

Many of the most important problems in science and engineering are inverse problems: given a desired outcome, find a design that achieves it. Evaluating whether a candidate meets the spec is often routine; a binding energy can be computed, a reactor yield simulated, a pharmacokinetic profile predicted. But searching a combinatorial design space for inputs that satisfy those targets is fundamentally harder. We introduce SciDesignBench, a benchmark of 520 simulator-grounded tasks across 14 scientific domains and five settings spanning single-shot design, short-horizon feedback, long-horizon refinement, and seed-design optimization. On the 10-domain shared-core subset, the best zero-shot model reaches only 29.0% success despite substantially higher parse rates. Simulator feedback helps, but the leaderboard changes with horizon: Sonnet 4.5 is strongest in one-turn de novo design, whereas Opus 4.6 is strongest after 20 turns of simulator-grounded refinement. Providing a starting seed design reshuffles the leaderboard again, demonstrating that constrained modification requires a fundamentally different capability from unconstrained de novo generation. We then introduce RLSF, a simulator-feedback training recipe. An RLSF-tuned 8B model raises single-turn success rates by 8-17 percentage points across three domains. Together, these results position simulator-grounded inverse design as both a benchmark for scientific reasoning and a practical substrate for amortizing expensive test-time compute into model weights.

Abstract

AI for Science

Explore similar work

SciDesignBench: Benchmarking and Improving Language Models for Scientific Inverse Design

Learning to Ideate for Scientific Impact

Can AI Scientist Agents Learn from Lab-in-the-Loop Feedback? Evidence from Iterative Perturbation Discovery