PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

Boqiao Zhang; Godbless James; Sai Krishna Gottipati; Andrew Fitzgibbon

cs.LGAug 19, 2026

PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

Authors: Boqiao Zhang, Godbless James, Sai Krishna Gottipati, Andrew Fitzgibbon

Organizations: University of Cambridge, United Kingdom · Graphcore, United Kingdom

Abstract

Improving molecular properties, such as drug-likeness or binding affinity, is a recurring task in early-stage drug discovery. However, molecules optimized in an unconstrained chemical space have limited practical value if they cannot be synthesized. Policy Gradient for Forward Synthesis (PGFS) is a synthesis-aware reinforcement learning method for molecular improvement, but its use of reactant embedding prediction makes reactant selection indirect, which, as we show, limits learning effectiveness. We first develop PGFS+, in which reaction templates and second reactants are represented by trainable embedding lookup tables. Combined with a more effective scoring function and RL algorithm, PGFS+ significantly improves the desired property. However, it exposes a reward-hacking failure mode: a powerful reactant search can map diverse input molecules to the same high-reward magnet molecule, improving the reward while collapsing the output diversity. We therefore introduce PGFS++, a synthesis-aware reinforcement learning framework for input-specific molecular improvement. Given an input molecule, PGFS++ treats it as the start of a forward-synthesis trajectory, applies learned reaction templates with compatible in-stock building blocks, and produces a molecule with improved target properties, an explicit synthesis route, and structural similarity to the input. Experiments on molecular improvement tasks show that PGFS++ improves target properties while preserving high output diversity.

Figures & tables

Figure 1: Reward hacking phenomenon in PGFS+. A , PGFS+ maps hundreds of input molecules to the same output molecule, causing high QED but low output diversity. This is referred to as the magnet output phenomenon. Highlighted atoms indicate the portions of the output molecule that are traced back to the corresponding input molecules. B , Trade-off between mean QED and mean diversity for PGFS, its variants, and PGFS+. Because of the magnet output, PGFS+ achieves high mean QED but has output diversity close to zero.

Figure 2: Workflow comparison between PGFS, PGFS+, and PGFS++. At step $t$ , all methods use an $f$ -network conditioned on the current first reactant $R_{1}^{\prime}(t)$ to predict the reaction template $T$ . PGFS uses the fingerprint representation $F(R_{1}^{\prime}(t))$ directly, whereas PGFS+ and PGFS++ first transform the $R_{1}$ representation through an additional MLP. For second-reactant selection, PGFS represents $R_{2}$ candidates using fixed fingerprints and retrieves candidates by $k$ -nearest neighbours. In contrast, PGFS+ and PGFS++ represent template-compatible $R_{2}$ candidates with a trainable embedding lookup table and score them globally using an attention-like mechanism with query $q=\mathrm{concat}(\phi_{1}(R_{1}^{\prime}),\phi_{T}(T))$ . Once $R_{2}^{\prime}(t)$ is selected, the reaction runner applies the chosen reaction and returns the product, which becomes the next state $R_{1}^{\prime}(t+1)$ .

Figure 3: Performance comparison on the QED improving task among ours (PGFS++) and random search, PGFS, PGFS TAC, PGFS MB, REINVENT4 (mol2mol), SynFlowNet. The comparison with SynFlowNet used the reversible reaction templates provided in the SynFlowNet repository to avoid potential issues arising from irreversible templates. Comparison with other baselines uses a general template set derived from Button et al. (2019) , containing both reversible and irreversible templates.

Figure 4: Performance comparison on the SEH improving task among ours (PGFS++) and random search, PGFS, PGFS TAC, PGFS MB, REINVENT4 (mol2mol), SynFlowNet. The comparison with SynFlowNet used the reversible reaction templates provided in the SynFlowNet repository to avoid potential issues arising from irreversible templates. Comparison with other baselines uses a general template set derived from Button et al. (2019) , containing both reversible and irreversible templates.

Figure 5: Ablation of input–output similarity bonus. We compare PGFS+ without an input–output similarity bonus against PGFS++ with additive and multiplicative similarity bonuses. Bold entries indicate the best value for each metric, and underlined entries indicate the second-best value. Without input–output similarity bonus, PGFS+ suffers from the magnet output problem, as reflected by high mean QED, but very low output diversity, input–output similarity, and RAS.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Value	Symbol	Value
$\alpha$	$3\times 10^{-4}$	$\lambda$	$0.95$
$n_{\mathrm{steps}}$	$2048$	$\epsilon$	$0.2$
$B$	$64$	$c_{e}$	$0.05$
$K$	$10$	$c_{v}$	$0.5$
$\gamma$	$0.99$	$g_{\max}$	$0.5$
		$\mathrm{KL}_{\mathrm{targ}}$	$0.02$

Appendix

Table S1: Additional PPO hyperparameters used in PGFS++ experiments (notations as in Equation ( S1 )).

Method	General templates	Reversible templates
SynFlowNet	$0.766$	$0.856$
Ours	$\mathbf{0.895}$	$\mathbf{0.867}$

Appendix

Table S2: Median QED on the test set under general vs. reversible templates. Ours denotes the matched PGFS++ QED models from the main paper.

Tem.	Obj.	Seed/stat.	Prop. $\uparrow$	Div. $\uparrow$	$1-\mathrm{SA}$ $\uparrow$	I/O sim. $\uparrow$	RAS $\uparrow$
Gen.	QED	$123456$	$0.895$	$0.873$	$0.749$	$0.389$	$0.705$
		$654321$	$0.897$	$0.867$	$0.698$	$0.360$	$0.267$
		$246135$	$0.897$	$0.877$	$0.722$	$0.371$	$0.685$
		Mean	$\mathbf{0.896}$	$\mathbf{0.872}$	$\mathbf{0.723}$	$\mathbf{0.373}$	$\mathbf{0.552}$
		s.d.	$0.001$	$0.005$	$0.026$	$0.015$	$0.247$
Gen.	SEH	$123456$	$0.812$	$0.772$	$0.748$	$0.312$	$0.545$

Appendix

Table S3: Seed-wise median performance of PGFS++. Gen. and Rev. denote general and reversible reaction templates, respectively. The first three rows of each block report the median over generated molecules for each random seed. The final two rows report the mean and sample standard deviation, respectively, across the three seed-wise medians.

Figure S2: Property–novelty trade-offs on the test set. Top: general templates (PGFS++ versus REINVENT4) for QED (left) and SEH (right). Bottom: reversible templates (PGFS++ versus SynFlowNet) for QED (left) and SEH (right). Each point is one episode’s generated molecule. Colored stars mark each method’s median (property, novelty). The black star marks an ideal high-property / high-novelty corner. Numerical values of medians are presented in Table S1 .

Molecular Optimization

Molecular optimization computationally designs or edits chemical structures to improve desired properties while respecting chemical and practical constraints.

See all 101 papers

LLM-Guided Dynamic Action Spaces for Synthesizable Molecular Optimization
Apr 9, 2026Tao Li, Kaiyuan Hou, Tuan Vinh +4Molecular Optimization Large Language Model-Guided Optimization
Department of Computer Science, Emory University · School of Computer Science, Carnegie Mellon University · MRC Brain Network Dynamics Unit, University of Oxford +3
Graph Forward Distribution Matching for Molecular Inverse Design
Sep 25, 2026Yihan Zhu, Yuhan Liu, Brett Savoie +2Molecular Graph Generation Inverse Design
University of Notre Dame
Information-Dense Synthesis for Molecular Discovery
Oct 6, 2026Kasper K. Jakobsen, Eli N. WeinsteinMolecular Optimization Drug Discovery
Department of Chemistry, Technical University of Denmark, Kgs. Lyngby, DK.

Apr 9, 2026J/K move · Enter open · S save

LLM-Guided Dynamic Action Spaces for Synthesizable Molecular Optimization

Tao Li, Kaiyuan Hou, Tuan Vinh, Fanglei Xue+3

Department of Computer Science, Emory University · School of Computer Science, Carnegie Mellon University · MRC Brain Network Dynamics Unit, University of Oxford+3

Molecular Optimization Large Language Model-Guided Optimization

Open paper AskPDF arXiv

Synthesizable molecular optimization seeks to improve target properties while ensuring that molecular modifications follow feasible synthetic pathways. Existing synthesis-aware methods typically rely on exploring a large space of candidate transformations defined by reaction templates and purchasable building blocks. This search becomes even more challenging when property improvement requires multiple reaction steps, as the space expands further along the pathway. To address this challenge, we introduce MolReAct, which reformulates molecular optimization as search over compact reaction spaces proposed by a tool-augmented large language model (LLM). At each step, the LLM combines its prior chemical knowledge with cheminformatics tools to identify a molecule-specific set of compatible reactions, preserving synthesizability while making multi-step optimization feasible. Given this compact action space, we further leverage Group Relative Policy Optimization (GRPO) with the terminal oracle reward to improve long-term decision-making over multiple reaction steps. Across diverse molecular optimization tasks, MolReAct achieves the highest Top-10 score on 11 of 14 tasks and the best sample efficiency on 12 of 14 tasks, outperforming existing baselines under limited oracle budgets. Beyond these gains, MolReAct also provides each optimized molecule with a template-grounded synthetic pathway.

Abstract

Molecular Optimization

Explore similar work

LLM-Guided Dynamic Action Spaces for Synthesizable Molecular Optimization

Graph Forward Distribution Matching for Molecular Inverse Design

Information-Dense Synthesis for Molecular Discovery