PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization

Gor Simonyan; Tatevik Abrahamyan; Narek Abelyan; Tigran Fahradyan; Hrant Khachatrian

cs.LGOct 13, 2026

PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization

Authors: Gor Simonyan, Tatevik Abrahamyan, Narek Abelyan, Tigran Fahradyan, Hrant Khachatrian

Organizations: Yerevan State University · University of Cambridge

Abstract

The Practical Molecular Optimization (PMO) benchmark standardized evaluation in molecular optimization, but it is built on simple property-based oracles that do not capture structure-based drug design. As the field has moved to docking-based objectives, evaluation practice has become unstandardized, and different methods report results on different docking tasks, often without a shared protocol for fair comparison or sample-efficiency constraints. To address this, we introduce Practical Molecular Optimization for Docking (PMO-Dock), a benchmark and protocol for docking-based optimization consisting of 25 tasks covering hit generation, lead optimization, and a new specificity task requiring strong on-target binding while penalizing off-target interactions. The protocol separates model development from final evaluation: validation tasks are used for hyperparameter tuning, and test tasks are held out for final evaluation and are not used during development. Evaluation on the test tasks is done under a strict oracle budget to reflect realistic drug-discovery settings. We benchmark four diverse high-performing methods spanning different optimization paradigms, Saturn (reinforcement learning), GenMol (discrete diffusion), Genetic-guided GFlowNet, and ChemLlama (LLM-based evolutionary search). Our results show no universally best method across tasks, additional analyses of hyperparameter transfer (cross-task and cross-category) and sensitivity show that transfer is often limited and varies by method, motivating method-dependent selection strategies. This benchmark provides a concrete standard for evaluating generalization to unseen tasks and sample efficiency under a limited test-time oracle budget.

Figures & tables

Figure 1: Overview of all 25 PMO-Dock tasks organized by oracle category. Each task is shown as a separate cell, color-coded by split: validation tasks (light blue) are used for algorithm development and hyperparameter tuning, while test tasks (light red) are strictly held out for evaluation with limited evaluation budgets. Hit Generation tasks maximize the fraction of molecules meeting docking thresholds, Lead Optimization tasks optimize docking scores while maintaining similarity to seed molecules and Specificity Binding tasks focus on selective target engagement.

Figure 2: Validation performance under category-specific hyperparameter selection. For each task, hyperparameters are chosen to maximize average performance on the other validation tasks in the same category (hit generation, lead optimization, or specificity). The last three columns are means over the tasks in each category (Hit Avg, Lead Avg, Spec Avg). Colors are normalized separately by category, so intensities should only be compared within category. More intense colors indicate higher performance relative to other entries in the same category. The alternative all-but-me rule is in Appendix D .

Figure 3: Test task leaderboard. Rows: methods, columns: test tasks by category. Hyperparameters selected per Section 3.3 . Best per task highlighted. Colors are normalized separately by category, so intensities should only be compared within category. More intense colors indicate higher performance relative to other entries in the same category.

Figure 4: Validation performance under per-task best hyperparameters (one row per method). For each task column, the reported value uses the hyperparameter setting that maximizes the validation metric on that task, corresponding to an unrestricted oracle-call scenario for model selection on the validation split. Layout and per-category coloring match Figure 2 .

Figure 5: Category-level cross-task bootstrap win percentages. Left three panels: $4\times 4$ matrices for hit ( $K{=}20$ ), lead ( $K{=}27$ ), and specificity ( $K{=}9$ ) (task–seed pool sizes). Rows and columns are methods. Cell $(i,j)$ is the percentage of $B=10{,}000$ bootstrap replicates in which method $i$ ’s resampled mean strictly exceeds method $j$ ’s after jointly resampling task blocks and seeds within the category. Colour encodes percentage on $[0,100]$ . Rightmost panel ( $\Sigma$ ): per method summing row totals across the three category matrices.

Figure 6: Cross-target lead optimization performance (parp1 vs 5ht1b,

\delta=0.6

). One panel per method, colors represent different seed-molecule pairs. Points are aggregated across random seeds, and legends report Pearson correlations and sample sizes. Cross-target lead transfer is generally weak for all four methods.

Figure 7: Hit vs lead (averaged across proteins, lead at

\delta{=}0.6

). One panel per method, legends report Pearson correlations and sample sizes. Saturn shows the strongest alignment, GenMol is near zero. Per-protein breakdown in Figure 8 .

Figure 8: Hit rate versus lead optimization score (

\delta=0.6

) by protein. Each protein is shown in a different color within each method panel. Scores are aggregated across random seeds and seed molecules (for lead tasks), legends report Pearson correlations and sample sizes. Saturn shows positive alignment on all three targets, other methods vary.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Hyperparameter transfer matrices for Saturn (left) and GenMol (right). Cell $(i,j)$ shows the change in task- $j$ performance when using hyperparameters selected from source task $i$ instead of task- $j$ -optimal hyperparameters. Scores are normalized by task type before comparison (hit rates direct, lead scores divided by 20, specificity scores divided by 5). Because entries are measured relative to the target-task optimum, values are expected to be non-positive, values near 0 indicate little transfer loss, while more negative values indicate larger degradation. These matrices summarize relative transfer rather than absolute task performance. Thick separators distinguish individual tasks from aggregate rows and columns (hit_avg, lead_avg, spec_avg, all_avg).

Panel of Figure 10 — Figure 10: Hyperparameter transfer matrices for Genetic GFlowNet (left) and ChemLlama (right). As in Figure 9 , each cell $(i,j)$ reports the change in task- $j$ performance when using hyperparameters selected from source task $i$ instead of task- $j$ -optimal hyperparameters. Scores are normalized by task type before comparison (hit rates direct, lead scores divided by 20, specificity scores divided by 5). Values are expected to be non-positive because the reference is the target-task optimum, entries near 0 indicate little transfer loss and more negative entries indicate larger degradation.

Figure 11: Validation performance under all-but-me hyperparameter selection. For each task, hyperparameters maximize average performance on every other validation task. The last three columns are means over the tasks in each category (Hit Avg, Lead Avg, Spec Avg). Colors are normalized separately by category.

Figure 12: Per-task bootstrap win percentages. One $4\times 4$ heatmap per test task (fourteen panels: two hit, nine lead, three specificity). Rows and columns are methods (ChemLlama, Genetic GFN, GenMol, Saturn), cell $(i,j)$ is the percentage of $B=10{,}000$ bootstrap replicates in which the row method’s resampled mean strictly exceeds the column method’s under seed resampling for that fixed task. Each replicate resamples seeds with replacement from the task’s empirical seed set and averages each method’s score over the resampled seeds. Colour encodes percentage on a common $0$ – $100\%$ scale across panels (darker = higher).

Figure 13: Bootstrap distributions of mean hit rate under seed resampling. Rows are the three task summaries ( braf , jak2 , both ), columns are methods. Each panel shows a normalized histogram of

\bar{x}_{m}^{(b)}

over

B=10{,}000

replicates, the vertical orange line is the bootstrap mean and the shaded band is

\mathrm{CI}_{95}(\bar{x}_{m})

Figure 14: Bootstrap distributions of paired mean differences (Saturn

-

competitor). Rows are the three task summaries ( braf , jak2 , both ), columns are competitors

m\neq\text{Saturn}

. Each panel histograms

\bar{\Delta}_{m}^{(b)}

over

B=10{,}000

replicates. The black line marks zero, the orange line marks the bootstrap mean, and the shaded band is

\mathrm{CI}_{95}(\bar{\Delta}_{m})

Figure 15: Cross-target hit generation performance: parp1 vs 5ht1b. Each point represents a hyperparameter configuration, averaged across random seeds. One panel per method, legends report Pearson correlations and sample sizes.

Figure 16: Cross-target hit generation performance: parp1 vs fa7. Each point represents a hyperparameter configuration, averaged across random seeds.

Figure 17: Cross-target hit generation performance: 5ht1b vs fa7. Each point represents a hyperparameter configuration, averaged across random seeds.

Figure 18: Lead optimization performance comparison across threshold values (

\delta=0.4

\delta=0.6

). Each color represents a specific (target, seed molecule) pair. Legends report Pearson correlations and sample sizes. Labels showing

r=\text{undefined}

correspond to subsets where one threshold produces constant scores across experiments, making Pearson correlation undefined.

Figure 19: Comparison of specificity task performance (6nzp-5ut5 vs 6nzp-7uyt). Axes: mean top-10 specificity score (delta) for each pair. Legends report Pearson correlations and sample sizes. Correlation is method-dependent, indicating incomplete transfer across antitarget choices.

Figure 20: Held-out target transfer analysis: 5ht1b hit performance versus other validation tasks. Top row: 5ht1b hit vs average hit on fa7+parp1. Bottom row: 5ht1b hit vs combined hit+lead on fa7+parp1 (normalized). Each column represents one algorithm. Legends report Pearson correlations and sample sizes. Saturn shows substantially stronger positive alignment than the other methods.

Figure 21: Exhaustiveness and replication analysis for specificity docking. Red boxes highlight cases where variability across individual runs most strongly affected the aggregated outcome.

Figure 22: Representative binding poses of a molecule generated by ChemLlama from the specificity optimization task. A) Binding pose of the generated molecule in complex with the anti-target TYK2 JH1 (7UYT). B) Binding pose of the same molecule in complex with the target TYK2 JH2 (6NZP).

Target	Hit rate (ex $=1$ )	Hit rate (ex $=8$ )
PARP1	$0.4570\pm 0.0791$	$0.5534\pm 0.0901$
FA7	$0.3693\pm 0.0774$	$0.3776\pm 0.0917$
5HT1B	$0.4681\pm 0.0662$	$0.6405\pm 0.0476$

Appendix

Table 1: Hit generation hit rates for ChemLlama at QuickVina exhaustiveness

1

versus

8

. Entries are mean

\pm

standard deviation.

Target	Top-1 DS (ex $=1$ )	Top-1 DS (ex $=8$ )
PARP1	$12.490\pm 0.532$	$12.830\pm 0.754$
FA7	$10.110\pm 0.367$	$10.090\pm 0.335$
5HT1B	$11.780\pm 0.461$	$12.240\pm 0.744$

Appendix

Table 2: Hit generation top-1 docking scores for ChemLlama at QuickVina exhaustiveness

1

versus

8

. Entries are mean

\pm

standard deviation.

Task	Top-1 DS (ex $=1$ )	Top-1 DS (ex $=8$ )
lead.5ht1b_06_0	$11.667\pm 0.153$	$12.300\pm 0.900$
lead.5ht1b_06_1	$10.667\pm 0.764$	$11.300\pm 0.700$
lead.fa7_06_0	$9.000\pm 0.200$	$9.133\pm 0.231$
lead.fa7_06_1	$8.367\pm 0.379$	$8.333\pm 0.404$
lead.parp1_06_0	$10.933\pm 0.577$	$10.733\pm 0.153$
lead.parp1_06_1	$12.267\pm 0.231$	$12.800\pm 0.200$

Appendix

Table 3: Lead optimization top-1 docking scores for ChemLlama at QuickVina exhaustiveness

1

versus

8

(

\delta=0.6

). Entries are mean

\pm

standard deviation.

Method	Hit all	Hit successful	Lead all	Lead successful
ChemLlama	$0.827$	$0.845$	$0.676$	$0.886$
Saturn	$0.962$	$0.928$	$0.882$	$0.886$
Genetic GFN	$0.972$	$0.924$	$0.959$	$0.885$
GenMol	$0.922$	$0.913$	$0.875$	$0.680$

Appendix

Table 4: Pearson correlation between QuickVina and AutoDock Vina docking scores for all generated molecules and for successful molecules, by method and task category.

Method	Pass rate
Genetic GFN	$60.6\%$
GenMol	$57.7\%$
Saturn	$55.0\%$
ChemLlama	$50.8\%$

Appendix

Table 5: PoseBusters pass rate for the highest-scoring molecules of each method. A molecule passes only if it satisfies every PoseBusters check.

Molecular Optimization

Molecular optimization computationally designs or edits chemical structures to improve desired properties while respecting chemical and practical constraints.

See all 103 papers

Jul 14, 2026

Sample Efficient Generative Optimization for Molecular Design

Molecular optimization in drug discovery, materials design, and catalysis requires searching vast chemical spaces under tight evaluation budgets, since high-fidelity oracles and experimental measurements are costly. The practical impact of an optimization method therefore hinges on its sample efficiency: how few evaluations it needs to find strong candidates. We introduce Sample Efficient Generative Optimization (SEGO), a framework for Bayesian optimization on adaptively generated molecules. In SEGO, a probabilistic surrogate model forms a hypothesis about where hits lie in chemical space, a generative model is steered to propose candidates in that region, the most promising candidate is selected via an acquisition function, and the resulting oracle call is used both to sharpen the surrogate and to anchor the generator in real reward. SEGO attains state-of-the-art performance on the practical molecular optimization (PMO) benchmark using only one tenth of the oracle calls consumed by other methods, and on a multiparameter docking task it reaches ten hits in roughly half the oracle calls of existing approaches. These gains move molecular optimization closer to campaigns driven by direct experimental feedback.

Sarina Kopf, Cristina Nevado, Philippe Schwaller

LIAC, ISIC, EPFL, Lausanne, Switzerland · Department of Chemistry, University of Zurich, Zurich, Switzerland · Centre of Competence in Research (NCCR) Catalysis, Zurich, Switzerland

Molecular Optimization Surrogate-Assisted Optimization Bayesian Optimization Molecular Generation Drug Discovery

Ask PDF

Jun 29, 2026

Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark

Generative molecular design is shaped by simple proxy benchmarks for drug-like properties and models pretrained on large pharmaceutical datasets. This combination yields strong benchmark metrics but limits transferability to domains structurally distinct from drug discovery. To overcome this limitation and drive discovery toward real, scientifically grounded targets, we introduce the Nanotechnology Molecular Optimization (NMO) Benchmark, which bridges machine learning (ML) and quantum materials science. NMO acts simultaneously as a rigorous testbed for the ML community and a discovery engine for nanotechnology research. The suite replaces proxy oracles with quantum simulations and introduces strict protocols that prioritize scientific utility over leaderboard-oriented overfitting. The physics-based NMO tasks impose hard structural constraints and rugged fitness landscapes, posing fundamentally new requirements on generative models. Notably, advanced molecular optimization methods underperform much simpler approaches on the NMO tasks. We develop a new baseline method identifying the critical components to solve the NMO tasks, including a novel representation for modeling structural constraints and a domain-agnostic pretraining strategy to eliminate pharmaceutical dataset bias. Our results surpass state-of-the-art physical properties and reveal previously unknown structural motifs, offering new insights for the nanotechnology community and demonstrating that ML can drive genuine scientific discovery.

Matthias Blaschke, Daniel Kienzle, Zsuzsanna Koczor-Benda +3

University of Augsburg · CAAPS† · Germany +2

Molecular Optimization Benchmark Design Molecular Generation Molecular Representation Learning Computational Materials Science

Ask PDF

Mar 18, 2026

MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models

Recent reasoning-based large language models have shown strong performance on tasks with verifiable outcomes, but their use in de novo molecular generation remains limited by the lack of training environments where rewards can be computed without reference molecules. We introduce MolRGen, a benchmark and molecular verifier for training and evaluating reasoning LLMs on de novo molecular generation. MolRGen contains approximately 4,500 protein-pocket targets, resulting in 50k multi-objective optimization prompts combining docking scores with molecular properties such as QED, synthetic accessibility, logP, and physicochemical descriptors. Unlike caption-based generation or molecule-editing benchmarks, MolRGen evaluates molecules proposed from scratch by computing rewards at generation time. We benchmark general-purpose and chemistry-specialized open-source LLMs and introduce a diversity-aware top-k metric to measure whether models can generate a diverse set of high-scoring molecules. Finally, we use the verifier to fine-tune a 128B LLM with GRPO, showing improved performance, at the cost of a diversity-exploitation trade-off. MolRGen provides a scalable testbed for studying verifier-based reasoning and reinforcement learning in molecular design.

Philippe Formont, Maxime Darrin, Ismail Ben Ayed +1

Universit´e Paris-Saclay · ´ETS Montr´eal · ILLS – International Laboratory on Learning Systems +4

Molecular Generation Molecule Design Molecular Optimization RL for Language Model Reasoning

Ask PDF

Abstract

Molecular Optimization

Explore similar work

Sample Efficient Generative Optimization for Molecular Design

Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark

MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models