Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

Zixun Lan; Zhenfu Liu; Yi Li; Xinheng Wang; Mingmin Chi; Qibiao Wang; Jie Zhu; Xiaohui Tian

cs.AIOct 10, 2026

Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

Authors: Zixun Lan, Zhenfu Liu, Yi Li, Xinheng Wang, Mingmin Chi, Qibiao Wang, Jie Zhu, Xiaohui Tian,

Organizations: Xi’an Jiaotong-Liverpool University, Suzhou 215000, China · Sichuan University of Science & Engineering, Zigong 643000, China · Fudan University, Shanghai 200433, China · Sinochem Digital Intelligence Technology Co., Ltd., Beijing, China

Abstract

Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Reference-based training gives no explicit positive signal for unrecorded alternatives, and exact-match evaluation rejects non-matching predictions regardless of chemical plausibility. We propose Retrosynthesis via Reinforcement Learning from Forward Feedback (RetroRLFF), which optimizes reactant generation using both reference agreement and forward consistency with the target product. Starting from a supervised decoder-only Transformer, RetroRLFF generates complete reactant candidates and uses a frozen forward reaction model to predict products. A sequence-level reward combines reference agreement and forward consistency while penalizing invalid outputs. Direct Advantage-Based Policy Optimization updates the policy without human preference annotations or a preference-trained reward model. On USPTO-50K, RetroRLFF outperforms all 14 baselines in candidate-level round-trip accuracy at k = 1, 3, 5, and 10. With approximately 50 million parameters, it achieves 90.1% top-1 round-trip accuracy, exceeding R-SMILES, the strongest competing baseline on this metric, by 8.5 percentage points. Compared with its supervised policy, round-trip accuracy increases from 83.1% to 90.1%, while exact-match accuracy is maintained at 57.9% versus 57.5%. In an anonymized expert assessment of 131 randomly selected test cases, organic synthesis experts prefer RetroRLFF over EditRetro in 67.9% of comparisons. These results support forward feedback as a training signal for improving forward consistency while retaining top-1 reference recovery. Expert assessment provides complementary evidence of synthetic feasibility on the selected cases, although neither forward consistency nor expert preference establishes experimental feasibility.

Figures & tables

Fig. 1: Normalized comparison of RetroRLFF (Ours) with three state-of-the-art retrosynthesis models on USPTO-50K across four prediction metrics and a parameter index. The parameter index is proportional to the reciprocal of the retrosynthesis model’s parameter count; higher values indicate smaller models. Higher scores are better on all axes. RetroRLFF achieves higher round-trip consistency with fewer parameters.

Fig. 2: Overview of RetroRLFF. (a) Single-step retrosynthesis is a one-to-many inverse problem: plausible reactant alternatives may differ from the recorded reference. (b) A supervised decoder-only policy samples complete reactant candidates, which are evaluated by a frozen forward model. Canonicalized reference agreement and forward consistency determine a sequence-level reward, with penalties for invalid evaluation paths. Direct Advantage-Based Policy Optimization (DAPO) updates the policy using offline advantages while keeping the critic and supervised reference policy fixed. (c) The optimized policy generates ranked reactant candidates. On USPTO-50K, RLFF improves top-1 round-trip accuracy from 83.1% to 90.1% while retaining exact-match accuracy (57.5% versus 57.9%) with the same policy architecture. Forward consistency is a proxy for synthetic feasibility, rather than experimental validation.

Template-Based
Model	Top- $k$ Performance (%)
	RoundTrip				ExactMatch				MaxFrag
	1	3	5	10	1	3	5	10	1	3	5	10
RetroSim	61.1	55.3	52.4	47.6	37.3	54.7	63.3	74.1	40.5	58.9	67.9	78.7
NeuralSym	66.2	67.2	64.8	51.4	44.4	65.3	72.4	78.9	53.6	74.6	81.5	87.8
GLN	66.0	63.2	61.4	57.4	52.5	74.6	80.5	86.9	57.2	72.4	78.2	84.0

TABLE I: Overall performance on the USPTO-50K test set. RoundTrip reports candidate-level forward-model consistency among the first

k

predictions (

\mathrm{RT@}k

); ExactMatch and MaxFrag report the proportions of products with at least one matching prediction among the first

k

(

\mathrm{EM@}k

and

\mathrm{MaxFrag@}k

). All entries are percentages. Bold values indicate the highest result in each column.

Fig. 3: Candidate-level round-trip accuracy (

\mathrm{RT@}k

, %) on USPTO-50K for non-ring, ring-opening, and ring-forming reactions at

k\in\{1,3,5,10\}

Configuration	$\boldsymbol{\alpha}$	$\boldsymbol{\beta}$	ExactMatch	RoundTrip
ST (w/o RL)	—	—	57.5%	83.1%
ST+RL	1.8	1.2	57.9%	90.1%
ST+RL	2	1	48.7%	93.8%
ST+RL	2	2	52.4%	88.3%

TABLE II: Top-1 exact-match and round-trip accuracy on USPTO-50K for the supervised policy and three RLFF reward configurations. All ST+RL configurations use

\gamma=\delta=r_{\mathrm{inv}}=-1

. Coefficient entries are not applicable to ST.

Fig. 4: Sensitivity of RetroRLFF to model size, layer count, and attention head count on USPTO-50K: (a) candidate-level round-trip accuracy (

\mathrm{RT@}k

) and (b) product-level exact matching accuracy (

\mathrm{EM@}k

), for

k\in\{1,3,5,10\}

. The legend specifies each architecture; the default configuration has approximately 50M parameters, six layers, and eight attention heads.

Fig. 5: Expert-assigned rank positions for RetroRLFF (Ours), recorded reference reactants (GroundTruth), and EditRetro on 131 selected test cases. Top 1, Top 2, and Top 3 denote individual rank positions, not cumulative top-

k

accuracy. Labels report percentages and case counts; equal ranks are permitted.

Fig. 6: Pairwise expert preferences derived from the rankings of 131 test cases. Outcomes are reported from the perspective of the first named reactant set. Green, gray, and red indicate wins, ties, and losses, respectively. Percentages use 131 as the denominator; counts appear in parentheses where labeled.

Fig. 7: Retrosynthetic suggestions for (a) lenalidomide, (b) nirmatrelvir, (c) DDR1-032, (d) febuxostat, (e) DDR1-037, and (f) cimetidine. Arrows point to proposed precursors; STEP and RANK indicate the expansion step and candidate position. Colors highlight structural changes. Panel (e) expands both precursor branches.

Fig. 8: Five USPTO-50K test cases. Rows show target products, recorded ground-truth reactants, and RetroRLFF predictions. Columns illustrate enone construction, secondary-amine formation, amino-diol deprotection, heterocyclic alcohol deprotection, and lactam alkylation.

Retrosynthesis

See all 23 papers

RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction
Mar 13, 2026Hanbum Ko, Chanhui Lee, Ye Rin Kim +4AI-Assisted Scientific Research Language Modeling
Department of Artificial Intelligence, Korea University · Materials Intelligence Lab, LG AI Research · Department of Statistics, Korea University
RETROSPECT: RETROsynthesis via Sequential Prediction, and Chemically Transformed-ranking
Jun 5, 2026Raja Sekhar Pappala, Shreyas Vinaya Sathyanarayana, Ronit Kumar Choudhary +2Transformer Learning to Rank
Mstack AI
Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis
Aug 19, 2026Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev +5Chemical Reaction Prediction Retrosynthesis
Insilico Medicine AI Limited, Abu Dhabi, UAE · Insilico Medicine Canada Inc., Montreal, Quebec, Canada · Insilico Medicine Hong Kong Ltd., Hong Kong SAR, China

Mar 13, 2026J/K move · Enter open · S save

RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction

Hanbum Ko, Chanhui Lee, Ye Rin Kim, Rodrigo Hormazabal+3

Department of Artificial Intelligence, Korea University · Materials Intelligence Lab, LG AI Research · Department of Statistics, Korea University

AI-Assisted Scientific Research Language Modeling LLM Reasoning Retrosynthesis

Open paper AskPDF arXiv

Retrosynthesis prediction aims to identify reactants that can synthesize a given product molecule. Although molecular large language models (LLMs) have recently shown promising results, most existing methods either generate reactants directly or provide only generic product-level analysis, without explicitly reasoning about bond-disconnection strategies that justify specific reactant choices. This paper proposes RetroReasoner, a retrosynthetic reasoning model that captures chemists' strategic disconnection-based thinking. RetroReasoner is trained with supervised fine-tuning and reinforcement learning. For supervised fine-tuning, SyntheticRetro generates structured disconnection rationales paired with reactant predictions. For reinforcement learning, a round-trip reward evaluates predicted reactants by passing them through a forward synthesis model and rewarding predictions that reconstruct the original product. RetroReasoner can also be applied to multi-step retrosynthetic planning by incorporating it into a parallelized Monte Carlo tree search framework, reducing search time while increasing the number and diversity of valid synthetic pathways. Experimental results show that RetroReasoner outperforms prior baselines, including not only molecular LLMs but also retrosynthesis-specific expert models, and generates a broader range of feasible reactant proposals, especially for challenging reaction instances. The code is available at https://github.com/KU-AGI/RetroReasoner.

Abstract

Retrosynthesis

Explore similar work

RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction

RETROSPECT: RETROsynthesis via Sequential Prediction, and Chemically Transformed-ranking

Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis