Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry

Yiming Huang; Yujie Zeng; Vijay Prakash Dwivedi; Simone Foti; Jianmin Wang; Jure Leskovec; Tolga Birdal

cs.LGOct 1, 2026

Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry

Authors: Yiming Huang, Yujie Zeng, Vijay Prakash Dwivedi, Simone Foti, Jianmin Wang, Jure Leskovec, Tolga Birdal

Organizations: Department of Computing, Imperial College London, London, UK · School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK · Department of Computer Science, Stanford University, Stanford, CA, USA · Department of Integrative Biotechnology, Yonsei University, Incheon, Korea

Abstract

Molecular learning models are strongly shaped by their underlying representations. Yet standard sequential and graph formalisms struggle to explicitly encode higher-order topology, such as ring systems and recurring motifs. Existing higher-order representations can capture these structures directly, but they are often computationally demanding and difficult to decode into valid molecules. Here, we introduce Higher-order Grammar Representation (HGR), a principled, topology-aware framework that lifts molecules to combinatorial complexes and parses each complex into a compact sequence of production rules under a context-free higher-order grammar. By serialising higher-order topology into rule sequences, HGR makes these structures directly compatible with standard sequence models, avoiding the computational overhead of explicit higher-order encodings while preserving topological expressiveness. To reduce benchmark bias towards simple ring systems, we construct RingDiv, a ring-enriched benchmark containing 1.18 million molecules, including the curated RingDiv300k subset, and introduce the ring diversity index (RDI) to quantify ring-system coverage. In molecular generation, HGR-based models uniquely combine 100% validity by construction with leading distributional alignment, ranking first in FCD on all five generation benchmarks. In representation learning, HGR-FM achieves the highest mean AUC across seven MoleculeNet benchmarks under both transfer protocols, improving on the strongest baseline by 8.3 and 3.3 AUC points under probing and full fine-tuning, respectively. Collectively, these results establish HGR as an efficient higher-order representation for molecular generation and transferable representation learning.

Figures & tables

Figure 1: Overview of molecular representations and our HGR framework. a , Example molecule used throughout the schematic. b , Representative molecular representations, illustrated by a SMILES string, a pairwise atom–bond graph and higher-order boundary matrices such as $B_{0,1}$ , $B_{0,2}$ and $B_{1,2}$ , which encode incidence relations. c , Higher-order lifting produces a combinatorial complex that encodes the molecule’s hierarchical higher-order topology, with its cells forming a contraction hierarchy. For clarity, each cell is annotated only with the right-hand-side graph $\mathcal{G}^{R}_{i}$ of its associated rule. d , Each production rule $r_{i}:\mathcal{G}^{L}_{i}\rightarrow\mathcal{G}^{R}_{i}$ maps a left-hand-side interface graph that specifies a non-terminal site and its ordered anchor interface to the corresponding right-hand-side local graph. Distinct rules induced across the molecular library form the shared corpus $\mathcal{P}=\{r_{i}\mid i=1,\ldots,|\mathcal{P}|\}$ . e , HGR serialises each complex as an ordered string $\bm{r}$ of reusable rule tokens and thereby provides a common input representation for diverse molecular-learning models, such as generative and foundation models. f , Starting from the start symbol $S$ , successive rule applications replace selected non-terminal sites with their corresponding $\mathcal{G}^{R}_{i}$ and reconnect boundary bonds through the anchor interfaces, thereby reconstructing the combinatorial complex and its associated molecular graph.

Figure 2: Ring-system sparsity and grammar-rule statistics. a , Scale of experimentally accessible chemical collections (left axis) compared with ring-system diversity observed in practice (right axis). b , Growth of the grammar corpus on ZINC250k, quantified as the number of unique rules discovered as molecules are progressively added during grammar induction. c , Frequency distributions of grammar rules induced on ZINC250k.

Figure 3: Ring-system diversity across molecular benchmarks. a , Ring Diversity Index (RDI). b , Number of unique ring systems. c , Quantitative Ring Complexity Index (QRCI) distributions. Higher QRCI values indicate greater ring complexity and topological diversity. The light-blue region (QRCI = 1.8–5) denotes the range typically observed for approved drugs, balancing structural diversity and synthetic feasibility. Values above this range indicate more complex ring systems that may reduce drug-likeness and synthetic accessibility. d , Fractions of molecules containing each ring-size or ring-connectivity category, together with the unique-scaffold ratio.

Method	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	FCD score $\uparrow$	KL score $\uparrow$
DiGress [ 42 ]	85.2	100.0	99.9	68.0	92.9
DeFoG [ 43 ]	99.0	99.0	–	73.8	97.7
HoG-Diff [ 23 ]	99.1	100.0	97.0	78.3	96.9
SELFIES-VAE [ 14 ]	100.0	100.0	99.7	39.9	88.0
HGR-VAE	100.0	100.0	99.4	83.7	93.8
HGR-LDF	100.0	100.0	98.9	84.4	95.7

Table 1: Generation performance across molecular benchmarks.

Figure 4: Generation fidelity and chemical-space coverage of molecular generative models on RingDiv300k. a , b , Functional-group prevalence in the RingDiv300k test set and HGR-generated molecules for common functional groups ( a ) and less frequent or special motifs ( b ). Bars show motif prevalence; error bars denote s.d. across three generation seeds. c , d , Size-resolved Jensen–Shannon (JS) divergence between generated and test-set distributions for ring-size distributions ( c ) and functional-group composition ( d ), with molecules binned by heavy-atom count. Lower JS indicates closer agreement with the test distribution; grey dotted curves show the train-versus-test reference. e , Lines show size-resolved synthetic-accessibility (SA) scores; shaded bands denote s.d. across generation seeds. Lower SA indicates easier synthesis. f , Chemical-space coverage in Morgan-fingerprint Uniform Manifold Approximation and Projection (UMAP) space. Filled densities show generated molecules for each model (HGR variants, green; baselines, blue), and black contours show the test-set density.

Figure 5: Workflow and transfer performance of the HGR-based molecular foundation model. a , Overview of HGR-based pretraining and downstream transfer. A pretrained Transformer with rank-aware rotary position encoding (RoPE) and an HGR attention bias yields a molecular fingerprint, jointly optimised with atom-, substructure-, 3D- and global-level pretext tasks and transferred to downstream tasks via lightweight multilayer-perceptron (MLP) task heads. b , Normalised AUCs after ZINC2m pretraining under probing (left) and full fine-tuning (right) across seven MoleculeNet benchmarks. AUCs are min–max normalised per task across all methods; annotated values are the raw HGR-FM AUCs. c , Ablation of higher-order rule tokens and attention bias. Curves show validation loss during pretraining for the full HGR-FM model and variants without each component. Inset, mean probing AUC across seven downstream tasks. d , Comparison of training regimes: no pretraining, probing and full fine-tuning. e , Effect of pretraining data source, comparing RingDiv, ZINC2m and their combination. Left, mean probing AUC; right, number of downstream tasks for which each setting ranks first or second.

Symbol	Description
$\mathcal{G}=(\mathcal{V},\mathcal{E})$	Molecular graph with vertex set $\mathcal{V}$ (atoms) and edge set $\mathcal{E}$ (bonds).
$U\sim_{\mathcal{G}}W$	Adjacency between vertex subsets: $\exists(u,w)\in\mathcal{E}$ with $u\in U$ and $w\in W$ .
$\mathcal{C}=(\mathcal{V},\mathcal{X},\operatorname{rk})$	Combinatorial complex with cell family $\mathcal{X}$ (atom and lifted motif cells) and rank function $\operatorname{rk}$ ; the lifting variant is written $\mathcal{C}_{\eta}$ , $\eta\in\{\mathrm{MIG},\mathrm{RSG}\}$ , and dropped when clear.
$\mathcal{M}$	Motif vocabulary mined from the molecule library to score merge candidates in lifting.
$\Gamma=(\Sigma,\mathcal{N},S,\mathcal{P})$	Higher-order grammar with terminal set $\Sigma$ , non-terminal set $\mathcal{N}$ , start symbol $S\in\mathcal{N}$ , and production-rule corpus $\mathcal{P}$ .
$N^{*}\in\mathcal{N}$	A generic non-terminal symbol marking replaceable sites; $N_{k}^{}$ denotes the site marked by $N^{}$ and created at parsing step $k$ .

Supplementary Table 1: Notation summary.

Panel of Figure 8 — Supplementary Algorithm 1 MIG lifting

Supplementary Fig. 1: Illustration of the grammar induction process. Cells are successively contracted into non-terminal sites, shown as squares, until only the start symbol

S

remains. Applying the recorded rules in reverse order reconstructs the lifted complex and its associated molecular graph, whereas grammar-constrained derivation supports generation.

Panel of Figure 10 — Supplementary Algorithm 2 ParseComplex : bottom-up parsing of a lifted complex into an HGR string

Supplementary Fig. 2: Workflow of HGR-based molecular generative models. Molecules are parsed into HGR strings, encoded by the grammar encoder and reconstructed by the grammar decoder. An optional latent diffusion module enables sampling in the learned latent space.

Variant	$N$	SRing	NRing	MRing	Spiro	Fused	Condensed	Bridged
RingDiv	1,183,434	157,542	1,160,912	69,061	104,198	522,461	582,747	113,423
RingDiv300k	299,819	42,703	290,985	33,591	32,026	147,500	164,171	37,660

Supplementary Table 2: Ring-topology composition of the released RingDiv variants. Each cell reports the number of molecules containing at least one ring of the indicated type.

Category	Metrics
Basic metrics	Validity, Uniqueness, Novelty, V.U.N.
Chemistry-based metrics	FCD, KL score and SA distribution
Topology-based metrics	NSPDK, SNN, Scaffold, and curvature-filtration descriptors ( $\kappa_{\mathrm{OR}}$ , $\kappa_{\mathrm{BF}}$ )

Supplementary Table 3: Categorization of evaluation metrics for molecular generative models.

Supplementary Fig. 3: Functional groups and SMARTS patterns for molecular profiling. Representative structures and RDKit-compatible SMARTS queries used to annotate functional groups in RingDiv300k reference and generated molecules. Categories follow a previous functional-group diversity analysis and include aryl and halogen motifs, nitrogen-containing groups, oxygen- and carbonyl-containing groups, aromatic motifs, sulfur-containing groups and strained small rings. SMARTS patterns were adapted from prior functional-group profiling work [ 87 ] and applied as molecule-level, non-exclusive queries.

Architecture
	QM9	ZINC250k	RingDiv300k	MOSES	GuacaMol
Max rule-sequence length	17	72	137	52	175
Production rules $\|\mathcal{P}\|$	280	403	6,205	320	2,385
Motif vocabulary	500	500	1,000	1,000	1,000
Latent dimension $d$	32	256	256	256	768
Rule-token dimension	64	256	768	512	768

Supplementary Table 4: Grammar variational autoencoder (HGR-VAE) configuration for the five generation benchmarks under the RSG-induced grammar.

Architecture
	ZINC250k	RingDiv300k	MOSES	GuacaMol
Latent dimension $d$	256	256	256	768
Tokens	8	16	32	8
Patch dimension	32	16	8	96
Hidden width	768	512	512	1,024
Layers	8	8	8	6

Supplementary Table 5: Latent diffusion (HGR-LDF) configuration. Tokens

\times

patch dimension

=d

Pretraining
Setting	Value
Corpus (size)	ZINC2m
Grammar / production rules	RSG / 825
Backbone (Grammar Transformer)	width 512, 4 layers, 8 heads
Dropout	0.1
Max grammar-tree path distance	3

Supplementary Table 6: HGR-FM pretraining and fine-tuning configuration.

Supplementary Fig. 4: Grammar statistics on QM9 dataset. a , Grammar corpus growth on QM9, showing the number of unique rules discovered as progressively more molecules are included during grammar induction. b , Rule-frequency distributions on QM9.

Model	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	V.U.N.	KL score $\uparrow$	FCD $\downarrow$	NSPDK $\downarrow$	SA dist. $\downarrow$	$\kappa_{\text{BF}}\downarrow$	$\kappa_{\text{OR}}\downarrow$
Training	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	N/A	N/A	99.8 $\pm$ 0.0	0.1737 $\pm$ 0.0024	0.0001 $\pm$ 0.0000	0.0123 $\pm$ 0.0040	0.1312 $\pm$ 0.0577	0.1203 $\pm$ 0.0644
DiGress	98.9 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	98.90 $\pm$ 0.04	98.2 $\pm$ 0.1	0.6300 $\pm$ 0.0110	0.00041 $\pm$ 0.00006	0.4171 $\pm$ 0.0247	0.1021 $\pm$ 0.0316	0.2656 $\pm$ 0.0582
DeFoG	98.3 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	98.27 $\pm$ 0.04	98.7 $\pm$ 0.1	0.8575 $\pm$ 0.0087	0.00109 $\pm$ 0.00002	0.1298 $\pm$ 0.0081	0.4735 $\pm$ 0.0373	0.4045 $\pm$ 0.0088
HoG-Diff	100.0 $\pm$ 0.0	99.9 $\pm$ 0.0	99.6 $\pm$ 0.1	99.57 $\pm$ 0.1	98.2 $\pm$ 0.1	0.9250 $\pm$ 0.0236	0.00134 $\pm$ 0.00026	0.1435 $\pm$ 0.0333	0.1810 $\pm$ 0.0326	0.1729 $\pm$ 0.03247
SMILES-VAE	88.1 $\pm$ 0.1	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	88.1 $\pm$ 0.1	97.6 $\pm$ 0.0	0.6642 $\pm$ 0.0156	0.00072 $\pm$ 0.00005	0.1653 $\pm$ 0.0117	0.1556 $\pm$ 0.0061	0.2375 $\pm$ 0.0502
SELFIES-VAE	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	92.8 $\pm$ 0.4	2.4266 $\pm$ 0.0526	0.00286 $\pm$ 0.00005	0.6077 $\pm$ 0.0071	0.6891 $\pm$ 0.0687	0.3874 $\pm$ 0.0738

Supplementary Table 7: Generation performance on the RingDiv300k benchmark.

Model	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	V.U.N.	FCD score $\uparrow$	KL score $\uparrow$	$\kappa_{\text{BF}}\downarrow$	$\kappa_{\text{OR}}\downarrow$
Training	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	N/A	N/A	96.12 $\pm$ 0.09	99.71 $\pm$ 0.00	0.1276 $\pm$ 0.0819	0.0573 $\pm$ 0.0304
DiGress	85.2	100.0	99.9	-	68.0	92.9	-	-
Disco	86.6	86.6	86.5	-	59.7	92.6	-	-
Cometh	98.9	98.9	97.6	-	72.7	96.7	-	-
DeFoG	99.0	99.0	-	97.9	73.8	97.7	-	-
HoG-Diff	99.1 $\pm$ 0.1	100.0 $\pm$ 0.0	97.0 $\pm$ 0.3	95.9 $\pm$ 0.2	78.3 $\pm$ 0.3	96.9 $\pm$ 0.3	0.1726 $\pm$ 0.0937	0.0944 $\pm$ 0.0423

Supplementary Table 8: Generation performance on the GuacaMol benchmark.

Method	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	V.U.N. $\uparrow$	FCD $\downarrow$	NSPDK $\downarrow$	$\kappa_{\text{BF}}\downarrow$	$\kappa_{\text{OR}}\downarrow$
Training	100.00 $\pm$ 0.00	99.99 $\pm$ 0.00	N/A	N/A	0.074 $\pm$ 0.004	0.0002 $\pm$ 0.0000	0.1337 $\pm$ 0.0560	0.2380 $\pm$ 0.0762
MiCaM	99.93	93.89	83.25	–	1.045	0.001	–	–
MoFlow	91.36 $\pm$ 1.23	98.65 $\pm$ 0.57	94.72 $\pm$ 0.77	–	4.467 $\pm$ 0.595	0.017 $\pm$ 0.003	–	–
EDP-GNN	47.52 $\pm$ 3.60	99.25 $\pm$ 0.05	86.58 $\pm$ 1.85	–	2.680 $\pm$ 0.221	0.005 $\pm$ 0.001	–	–
GraphEBM	8.22 $\pm$ 2.24	97.90 $\pm$ 0.05	97.01 $\pm$ 0.17	–	6.143 $\pm$ 0.411	0.030 $\pm$ 0.004	–	–
GDSS	95.72 $\pm$ 1.94	98.46 $\pm$ 0.61	86.27 $\pm$ 2.29	86.48	2.900 $\pm$ 0.282	0.003 $\pm$ 0.000	0.9252	0.6012

Supplementary Table 9: Generation performance on the QM9 benchmark.

Method	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	V.U.N. $\uparrow$	FCD $\downarrow$	NSPDK $\downarrow$	$\kappa_{\text{BF}}\downarrow$	$\kappa_{\text{OR}}\downarrow$
Training	100.00 $\pm$ 0.00	99.98 $\pm$ 0.01	N/A	N/A	0.201 $\pm$ 0.005	0.0001 $\pm$ 0.0000	0.1007 $\pm$ 0.0563	0.0478 $\pm$ 0.0190
MiCaM	100.00	88.48	99.98	–	32.395	0.166	–	–
MoFlow	63.11 $\pm$ 5.17	99.99 $\pm$ 0.01	100.00 $\pm$ 0.00	–	20.931 $\pm$ 0.184	0.046 $\pm$ 0.002	–	–
EDP-GNN	82.97 $\pm$ 2.73	99.79 $\pm$ 0.08	100.00 $\pm$ 0.00	–	16.737 $\pm$ 1.300	0.049 $\pm$ 0.006	–	–
GraphEBM	5.29 $\pm$ 3.83	98.79 $\pm$ 0.15	100.00 $\pm$ 0.00	–	35.471 $\pm$ 5.331	0.212 $\pm$ 0.075	–	–
GDSS	97.01 $\pm$ 0.77	99.64 $\pm$ 0.13	100.00 $\pm$ 0.00	–	14.656 $\pm$ 0.680	0.019 $\pm$ 0.001	–	–

Supplementary Table 10: Generation performance on the ZINC250k benchmark.

Method	Val. $\uparrow$	Unique. $\uparrow$	Nov. $\uparrow$	V.U.N. $\uparrow$	FCD $\downarrow$	Filters $\uparrow$	SNN $\uparrow$	Scaf. $\uparrow$	$\kappa_{\text{BF}}\downarrow$	$\kappa_{\text{OR}}\downarrow$
Training	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	-	-	0.518 $\pm$ 0.007	100.0 $\pm$ 0.1	0.586 $\pm$ 0.000	-	0.1138 $\pm$ 0.0303	0.1365 $\pm$ 0.01592
GraphINVENT	96.4	99.8	-	-	1.22	95.0	0.54	12.7	-	-
DiGress	85.7	100.0	95.0	-	1.19	97.1	0.52	14.8	-	-
DisCo	88.3	100.0	97.7	-	1.44	95.6	0.50	15.1	-	-
Cometh	90.5	99.9	92.6	-	1.27	99.1	0.54	16.0	-	-
DeFoG	92.8	99.9	92.1	-	1.95	98.9	0.55	14.4	-	-

Supplementary Table 11: Generation performance on the MOSES benchmark.

Method BBBP Tox21 ToxCast SIDER ClinTox HIV BACE avg

\uparrow

a. Probing with frozen encoder Infomax 60.8

\pm

0.38 67.0

\pm

0.40 58.3

\pm

0.24 58.2

\pm

0.76 62.6

\pm

0.28 71.3

\pm

1.36 65.1

\pm

0.67 63.33 EdgePred 52.7

\pm

1.45 63.0

\pm

0.77 54.1

\pm

0.39 51.7

\pm

0.99 48.2

\pm

5.46 65.2

\pm

1.06 58.6

\pm

1.76 56.21 AttrMasking 51.8

\pm

0.23 69.3

\pm

0.04 57.7

\pm

0.08 51.3

\pm

0.09 54.5

\pm

0.44 60.5

\pm

0.31 61.8

\pm

0.62 58.13 ContextPred 58.8

\pm

0.57 68.3

\pm

0.24 58.8

\pm

0.36 59.2

\pm

0.19 40.0

\pm

1.51 67.0

\pm

0.62 59.6

\pm

2.53 58.81 GraphCL 63.0

\pm

0.29 67.6

\pm

0.39 57.4

\pm

0.48 52.8

\pm

0.95 54.7

\pm

5.10 64.8

\pm

1.69 66.3

\pm

0.29 60.94 GraphLoG 54.5

\pm

0.30 66.8

\pm

0.17 57.4

\pm

0.17 58.0

\pm

0.58 57.6

\pm

1.29 65.2

\pm

0.54 72.4

\pm

0.65 61.70 GraphMAE 56.5

\pm

0.41 66.7

\pm

0.45 57.6

\pm

0.10 52.0

\pm

0.82 44.3

\pm

0.57 60.5

\pm

0.54 61.8

\pm

5.53 57.06 GraphMVP 57.9

\pm

0.68 66.9

\pm

0.17 58.5

\pm

0.09 56.0

\pm

1.16 42.7

\pm

1.99 67.1

\pm

0.83 65.3

\pm

0.20 59.20 MORE 67.9

\pm

0.28 70.6

\pm

0.33 62.2

\pm

0.16 60.7

\pm

0.28 63.5

\pm

1.26 72.8

\pm

0.32 80.0

\pm

0.62 68.24 HGR-FM 74.4

\pm

0.96 78.5

\pm

0.57 68.9

\pm

0.11 63.0

\pm

0.58 89.7

\pm

1.59 76.9

\pm

1.02 84.1

\pm

0.56 76.51 b. Full fine-tuning Infomax 68.0

\pm

1.05 75.0

\pm

0.55 62.4

\pm

0.57 59.7

\pm

0.47 71.1

\pm

3.88 76.8

\pm

1.38 76.8

\pm

1.32 69.97 EdgePred 65.3

\pm

1.96 76.3

\pm

0.26 63.6

\pm

0.28 61.3

\pm

0.54 64.5

\pm

2.67 75.7

\pm

1.05 79.9

\pm

1.27 69.51 AttrMasking 63.4

\pm

1.45 76.4

\pm

0.19 63.4

\pm

0.52 60.0

\pm

0.87 71.1

\pm

2.46 76.3

\pm

0.28 79.7

\pm

0.47 70.04 ContextPred 67.1

\pm

0.81 74.4

\pm

0.12 63.7

\pm

0.12 60.9

\pm

0.78 59.0

\pm

1.99 76.6

\pm

0.45 79.1

\pm

2.13 68.69 GraphCL 68.0

\pm

2.05 74.5

\pm

0.12 62.4

\pm

0.59 59.2

\pm

1.12 75.3

\pm

3.67 76.3

\pm

1.09 76.8

\pm

0.36 70.36 GraphLoG 66.8

\pm

2.55 74.7

\pm

0.25 62.4

\pm

0.55 59.6

\pm

0.67 64.2

\pm

1.27 76.6

\pm

0.77 82.3

\pm

0.42 69.51 GraphMAE 68.0

\pm

2.54 75.6

\pm

0.27 63.4

\pm

0.14 60.2

\pm

0.49 70.8

\pm

4.15 76.6

\pm

0.87 82.1

\pm

1.40 70.96 GraphMVP 71.3

\pm

1.13 75.0

\pm

0.37 63.4

\pm

0.26 62.9

\pm

0.29 68.2

\pm

5.95 75.8

\pm

1.44 78.8

\pm

4.61 70.77 MORE 71.9

\pm

0.94 75.6

\pm

0.54 64.6

\pm

0.58 60.9

\pm

0.62 81.0

\pm

0.65 77.0

\pm

0.74 82.8

\pm

1.33 73.40 HGR-FM 74.3

\pm

1.08 79.0

\pm

0.41 70.8

\pm

0.12 62.1

\pm

1.05 89.4

\pm

1.00 77.5

\pm

0.21 84.0

\pm

0.43 76.73

Supplementary Table 12: Transfer performance of pretrained molecular encoders.

Supplementary Fig. 5: Accelerated evaluation of generation metrics. a , Wall-clock execution time of representative evaluation metrics on the RingDiv300k benchmark (Scaf, SNN, Frag, FCD, NSPDK,

\kappa_{BF}

\kappa_{OR}

). Dark bars indicate the optimised implementation; light shading denotes time saved relative to the original pipeline. Blue bold numbers above bars denote the speed-up factor, and grey numbers report the post-optimisation runtime. b , Runtime of the two slowest metrics (

\kappa_{BF}

and

\kappa_{OR}

) across additional standard datasets (QM9, ZINC250k, MOSES and GuacaMol), highlighting consistent acceleration under diverse data regimes. Data represent mean values for ten independent replicates.

Training time (h)
Method	QM9	ZINC250k	MOSES	GuacaMol	RingDiv300k
DiGress	-	-	-	-	63.0
DeFoG	-	-	-	-	53.1
HoG-Diff	3.1	6.5	7.8	30.0	8.1
HGR-VAE (RSG)	0.3	3.1	1.4	8.4	3.0
Inference time (min)

Supplementary Table 13: Training and inference runtime across molecular datasets.

Explore similar work

Controllable Molecular Generative Foundation Models
May 14, 2026Yihan Zhu, Yuhan Liu, Weijiang Li +2Molecular Generation Physics-Guided Diffusion
University of Notre Dame
Augmenting Molecular Language Models with Local $n$ -gram Memory
Jun 10, 2026Xinni Zhang, Zijing Liu, He Cao +2Unigram Tokenizer Language Modeling
1The Chinese University of Hong Kong · 2International Digital Economy Academy
SMolLM: Small Language Models Learn Small Molecular Grammar
May 7, 2026Akhil Jindal, Harang JuMolecular Generation Chemistry
Moku

May 14, 2026, cs.LGJ/K move · Enter open · S save

Controllable Molecular Generative Foundation Models

Yihan Zhu, Yuhan Liu, Weijiang Li, Tengfei Luo+1

University of Notre Dame

Molecular Generation Physics-Guided Diffusion Generative Models Graph Foundation Model Wireless Foundation Models Foundation Model+2

Open paper AskPDF arXiv

Despite the success of foundation models in language and vision, molecular graph generation still lacks a unified framework for heterogeneous design tasks with reliable controllability. While reinforcement learning (RL) offers a natural post-training mechanism for task-specific optimization, applying it to graph generative models is hindered by the vast atom-wise action spaces and chemically invalid intermediate states. We propose \textbf{Co}ntrollable \textbf{Mole}cular Generative Foundation Models (CoMole), built with a unified motif-aware graph diffusion pipeline. By learning a motif-aware graph space, CoMole transfers pretrained structural priors into controllable generation, where RL optimizes conditional reverse policies over chemically meaningful decisions. We theoretically characterize the bottleneck of atom-level RL and justify motif-aware policy optimization. Across three heterogeneous benchmarks spanning materials and drug discovery, CoMole ranks first in controllability on all nine targets, reduces MAE by up to 48.2% relative to the strongest baselines, and maintains validity above 0.94 without rule-based correction or post-hoc filtering. We further show that CoMole transfers controllability to unseen properties by optimizing only task embeddings with the generator frozen, achieving performance competitive with strong task-specific baselines.

Abstract

Explore similar work

Controllable Molecular Generative Foundation Models

Augmenting Molecular Language Models with Local nnn-gram Memory

SMolLM: Small Language Models Learn Small Molecular Grammar

Augmenting Molecular Language Models with Local $n$ -gram Memory