Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback
Authors: Zixun Lan, Zhenfu Liu, Yi Li, Xinheng Wang, Mingmin Chi, Qibiao Wang, Jie Zhu, Xiaohui Tian, Fei Ma
Organizations: Xi’an Jiaotong-Liverpool University, Suzhou 215000, China · Sichuan University of Science & Engineering, Zigong 643000, China · Fudan University, Shanghai 200433, China · Sinochem Digital Intelligence Technology Co., Ltd., Beijing, China
Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Reference-based training gives no explicit positive signal for unrecorded alternatives, and exact-match evaluation rejects non-matching predictions regardless of chemical plausibility. We propose Retrosynthesis via Reinforcement Learning from Forward Feedback (RetroRLFF), which optimizes reactant generation using both reference agreement and forward consistency with the target product. Starting from a supervised decoder-only Transformer, RetroRLFF generates complete reactant candidates and uses a frozen forward reaction model to predict products. A sequence-level reward combines reference agreement and forward consistency while penalizing invalid outputs. Direct Advantage-Based Policy Optimization updates the policy without human preference annotations or a preference-trained reward model. On USPTO-50K, RetroRLFF outperforms all 14 baselines in candidate-level round-trip accuracy at k = 1, 3, 5, and 10. With approximately 50 million parameters, it achieves 90.1% top-1 round-trip accuracy, exceeding R-SMILES, the strongest competing baseline on this metric, by 8.5 percentage points. Compared with its supervised policy, round-trip accuracy increases from 83.1% to 90.1%, while exact-match accuracy is maintained at 57.9% versus 57.5%. In an anonymized expert assessment of 131 randomly selected test cases, organic synthesis experts prefer RetroRLFF over EditRetro in 67.9% of comparisons. These results support forward feedback as a training signal for improving forward consistency while retaining top-1 reference recovery. Expert assessment provides complementary evidence of synthetic feasibility on the selected cases, although neither forward consistency nor expert preference establishes experimental feasibility.
Figures & tables
Fig. 1: Normalized comparison of RetroRLFF (Ours) with three state-of-the-art retrosynthesis models on USPTO-50K across four prediction metrics and a parameter index. The parameter index is proportional to the reciprocal of the retrosynthesis model’s parameter count; higher values indicate smaller models. Higher scores are better on all axes. RetroRLFF achieves higher round-trip consistency with fewer parameters.
Fig. 2: Overview of RetroRLFF. (a) Single-step retrosynthesis is a one-to-many inverse problem: plausible reactant alternatives may differ from the recorded reference. (b) A supervised decoder-only policy samples complete reactant candidates, which are evaluated by a frozen forward model. Canonicalized reference agreement and forward consistency determine a sequence-level reward, with penalties for invalid evaluation paths. Direct Advantage-Based Policy Optimization (DAPO) updates the policy using offline advantages while keeping the critic and supervised reference policy fixed. (c) The optimized policy generates ranked reactant candidates. On USPTO-50K, RLFF improves top-1 round-trip accuracy from 83.1% to 90.1% while retaining exact-match accuracy (57.5% versus 57.9%) with the same policy architecture. Forward consistency is a proxy for synthetic feasibility, rather than experimental validation.
Model
Top- k Performance (%)
RoundTrip
ExactMatch
MaxFrag
1
3
5
10
1
3
5
10
1
3
5
10
Template-Based
RetroSim
61.1
55.3
52.4
47.6
37.3
54.7
63.3
74.1
40.5
58.9
67.9
78.7
NeuralSym
66.2
67.2
64.8
51.4
44.4
65.3
72.4
78.9
53.6
74.6
81.5
87.8
GLN
66.0
63.2
61.4
57.4
52.5
74.6
80.5
86.9
57.2
72.4
78.2
84.0
TABLE I: Overall performance on the USPTO-50K test set. RoundTrip reports candidate-level forward-model consistency among the first k predictions ( RT@k ); ExactMatch and MaxFrag report the proportions of products with at least one matching prediction among the first k ( EM@k and MaxFrag@k ). All entries are percentages. Bold values indicate the highest result in each column.
Fig. 3: Candidate-level round-trip accuracy ( RT@k , %) on USPTO-50K for non-ring, ring-opening, and ring-forming reactions at k∈{1,3,5,10} .
Configuration
α
β
ExactMatch
RoundTrip
ST (w/o RL)
—
—
57.5%
83.1%
ST+RL
1.8
1.2
57.9%
90.1%
ST+RL
2
1
48.7%
93.8%
ST+RL
2
2
52.4%
88.3%
TABLE II: Top-1 exact-match and round-trip accuracy on USPTO-50K for the supervised policy and three RLFF reward configurations. All ST+RL configurations use γ=δ=rinv=−1 . Coefficient entries are not applicable to ST.
Fig. 4: Sensitivity of RetroRLFF to model size, layer count, and attention head count on USPTO-50K: (a) candidate-level round-trip accuracy ( RT@k ) and (b) product-level exact matching accuracy ( EM@k ), for k∈{1,3,5,10} . The legend specifies each architecture; the default configuration has approximately 50M parameters, six layers, and eight attention heads.
Fig. 5: Expert-assigned rank positions for RetroRLFF (Ours), recorded reference reactants (GroundTruth), and EditRetro on 131 selected test cases. Top 1, Top 2, and Top 3 denote individual rank positions, not cumulative top- k accuracy. Labels report percentages and case counts; equal ranks are permitted.
Fig. 6: Pairwise expert preferences derived from the rankings of 131 test cases. Outcomes are reported from the perspective of the first named reactant set. Green, gray, and red indicate wins, ties, and losses, respectively. Percentages use 131 as the denominator; counts appear in parentheses where labeled.
Fig. 7: Retrosynthetic suggestions for (a) lenalidomide, (b) nirmatrelvir, (c) DDR1-032, (d) febuxostat, (e) DDR1-037, and (f) cimetidine. Arrows point to proposed precursors; STEP and RANK indicate the expansion step and candidate position. Colors highlight structural changes. Panel (e) expands both precursor branches.
Fig. 8: Five USPTO-50K test cases. Rows show target products, recorded ground-truth reactants, and RetroRLFF predictions. Columns illustrate enone construction, secondary-amine formation, amino-diol deprotection, heterocyclic alcohol deprotection, and lactam alkylation.
Retrosynthesis prediction aims to identify reactants that can synthesize a given product molecule. Although molecular large language models (LLMs) have recently shown promising results, most existing methods either generate reactants directly or provide only generic product-level analysis, without explicitly reasoning about bond-disconnection strategies that justify specific reactant choices. This paper proposes RetroReasoner, a retrosynthetic reasoning model that captures chemists' strategic disconnection-based thinking. RetroReasoner is trained with supervised fine-tuning and reinforcement learning. For supervised fine-tuning, SyntheticRetro generates structured disconnection rationales paired with reactant predictions. For reinforcement learning, a round-trip reward evaluates predicted reactants by passing them through a forward synthesis model and rewarding predictions that reconstruct the original product. RetroReasoner can also be applied to multi-step retrosynthetic planning by incorporating it into a parallelized Monte Carlo tree search framework, reducing search time while increasing the number and diversity of valid synthetic pathways. Experimental results show that RetroReasoner outperforms prior baselines, including not only molecular LLMs but also retrosynthesis-specific expert models, and generates a broader range of feasible reactant proposals, especially for challenging reaction instances. The code is available at https://github.com/KU-AGI/RetroReasoner.
Hanbum Ko, Chanhui Lee, Ye Rin Kim +4
Department of Artificial Intelligence, Korea University · Materials Intelligence Lab, LG AI Research · Department of Statistics, Korea University
Single-step retrosynthesis needs both accurate first-ranked suggestions and candidate lists that are rich enough for downstream selection. We study this as a proposal-selection decomposition. Our system, RETROSPECT, combines a single Transformer proposal model, which we call the ChemAlign Transformer, with a LambdaMART reranker over structural, reaction-template, upstream-score, and optional DFT-derived descriptors. The generator is trained with hybrid root-aligned and random SMILES augmentation, Pre-LayerNorm, tied embeddings, exponential moving average weights, and a differentiable atom-balance auxiliary loss. On the full USPTO-50K test set of 5,007 reactions, the generator reaches 55.00% top-1 and 86.18% top-10 exact-match accuracy with 99.86% top-1 validity. On the merged candidate-pool benchmark used for reranking, which contains 5,007 test products and about 111 candidates per product, a LambdaMART model trained on the structural feature set reaches 59.4% top-1 with 0.7171 mean reciprocal rank. Feature ablations show that upstream proposal score and template-frequency statistics provide most of the reranking signal, while DFT and reaction-center DFT features provide smaller and less consistent gains. These results support a modular view of retrosynthesis: stronger single-model proposal and learned candidate selection are complementary, and the proposal model can serve as a drop-in component for ensemble systems such as RetroChimera (Maziarz et al., 2024)
Raja Sekhar Pappala, Shreyas Vinaya Sathyanarayana, Ronit Kumar Choudhary +2
Single-step retrosynthesis is a central component of computer-aided synthesis planning, yet its intrinsically one-to-many nature is poorly captured by single-answer evaluation and benchmarking protocols. To address this, we introduce Top-K prompting as a robust training and inference paradigm to better capture diverse, plausible reaction predictions. We compile CREED-CCV-2+USPTO-XL, an ultra-large-scale dataset of ~45.6 million verified reactions to train the C3LM (Chemistry Constraint-Consistent Language Model). By integrating fine-tuning with ChemCensor-based and novelty-oriented rewards, our model achieves state-of-the-art performance on the OOD URSA-expert-2026 benchmark. Further analysis of reaction uniqueness shows that LLMs and conventional models explore complementary reaction spaces, motivating ensemble-based retrosynthesis systems. Overall, our results establish Top-K, plausibility-aware training as a practical new direction for robust future LLM-based synthesis planning.
Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev +5
Insilico Medicine AI Limited, Abu Dhabi, UAE · Insilico Medicine Canada Inc., Montreal, Quebec, Canada · Insilico Medicine Hong Kong Ltd., Hong Kong SAR, China