Chemical language models can struggle to generate new molecules when trained on small datasets, where a model may reproduce training examples rather than learn a useful generative distribution. Randomized SMILES augmentation varies atom traversal, whereas Sequential Attachment-based Fragment Embedding (SAFE) also allows the order of fragment blocks to vary. How these distinct sources of representational variation affect low-data molecular generation has not been clearly studied. Here, using 1,000 seed molecules to train an autoregressive sequence model, we compare canonical SAFE with randomized SAFE at increasing corpus sizes, isolate atom-order and fragment-order axes of SAFE randomization, and compare fully randomized SAFE with randomized SMILES at molecule-by-molecule matched training corpora. Three findings emerge. First, randomization of SAFE alone raises the genuine yield of novel molecules more than 450-fold, turning a model that merely reproduces its training set into a practical molecular generator. Second, the two axes of randomization act jointly rather than additively: atom order alone reaches a tenth, and fragment order alone a seventh, of the genuine yield obtained when both vary, which is four times their combined contribution. Third, at matched corpus size and compute, randomized SAFE delivers 2.8 times the genuine yield of randomized SMILES. Welch tests with Games-Howell post-hoc comparisons resolve every difference on the primary outcomes. In the low-data regime, thus, what one randomizes rather than only how much determines whether augmentation produces new molecules or merely new spellings of the training set.
Figures & tables
Figure 1: The SAFE representation . The same molecule (acetanilide) is shown three ways: as a SMILES string (top), a 2D structure (middle), and a SAFE string (bottom). SMILES encodes a molecule by an atom-by-atom traversal. SAFE instead decomposes the molecule into fragments, here cleaved by the BRICS algorithm into a phenyl ring (blue), the central nitrogen (orange), and an acetyl group (green), with dashed lines marking the cleaved bonds. Each fragment gets written as a SMILES-like block. The blocks are concatenated with a . separator, and the bonds that originally joined the fragments are preserved as matching ring-bond attachment digits shared across blocks (e.g. the shared 2 and 3 ). Colors link each fragment to its corresponding block.
Figure 2: The axes of randomization of a SAFE string . All four SAFE strings encode the same molecule, acetanilide (top) and decode back to it. Colors mark the three BRICS fragments (phenyl:blue, acetyl:green, and nitrogen:orange). Canonical : the single deterministic SAFE string. Dot-randomized (fragment-order) : the fragment blocks are permuted while each block’s internal spelling is unchanged, meaning the atom traversal within a fragment follows the same order as in the canonical representation. Atom-randomized (atom-order) : the atom traversal within each fragment is rewritten, and the resulting blocks are then arranged in deterministic sorted order rather than a random one, so that randomness enters only through the spelling of each fragment and not through their sequence. Full-randomized (dot + atom) : both axes vary jointly, the default for SAFE randomizer. Because a molecule’s fragments can be reordered and each fragment respelled independently, SAFE exposes two orthogonal randomization axes. SMILES randomization varies only the atom-order axis, making the fragment-order axis unique to SAFE.
Figure 3: Token-length distributions of the training corpora . Kernel density estimates of tokens per training string, excluding special tokens, with each arm under its own tokenizer. Distributions are closely matched within the augmentation sweep (left) and the axis ablation (middle), but SAFE strings are longer than randomized SMILES (right). See Table 1 for summary statistics.
Family
Arm
Strings
Median
Mean
SD
Max
Augmentation sweep
safe-1k
1,000
55
56.8
17.8
132
Augmentation sweep
safe-25k
24,972
56
57.5
17.7
132
Augmentation sweep
safe-50k
49,450
56
57.8
17.6
134
Randomization axis ablation
safe-atom
32,152
60
63.2
17.4
134
Randomization axis ablation
safe-dot
32,152
60
62.6
17.4
132
Randomization axis ablation
safe-full
32,152
60
63.2
17.4
134
Table 1: Summary statistics of token-length distributions of the training corpora . Lengths are counts of tokenizer tokens per training string, excluding special tokens. SAFE and SMILES arms use their respective tokenizers.
Metric
Definition
Validity
∣V∣/n . Fraction of sampled strings that decode to a valid molecule.
Uniqueness
∣U∣/∣V∣ . Fraction of valid molecules that are distinct.
Novelty
∣N∣/∣U∣ . Fraction of distinct valid molecules absent from the reference set (train plus validation).
Yield
∣N∣=n×Validity×Uniqueness×Novelty , reported per 10,000 samples. Count of novel, distinct, valid molecules that composes the three metrics above into one end-to-end measure.
Genuine yield
∣{g∈N:nn(g)<thresh}∣ per 10,000 samples. It excludes novel molecules that are minor edits of a training molecule.
Seed recovery
∣U∩T∣/∣T∣ . Fraction of the 1,000 training molecules the model regenerates exactly. Near 1 indicates memorization.
Table 2: Evaluation metrics . For one generated set, n=10,000 strings are sampled and each is decoded to a molecule, keeping its largest connected component. V is the resulting valid molecules, U the distinct molecules in V (compared as canonical SMILES), T the 1,000 training molecules, R the reference set of T plus the 200 validation molecules, and N=U∖R the novel molecules. Throughout, ∣⋅∣ denotes the number of items in a set, A∖B the items in A that are not in B , and A∩B the items in both. Tc(⋅,⋅) denotes Tanimoto similarity of ECFP4 fingerprints (radius 2, 2048 bits), and nn(g)=maxs∈TTc(g,s) the similarity of a generated molecule g to its most similar training molecule s . The Tanimoto similarity threshold for genuine yield is thresh=0.7 .
Arm
safe-1k
safe-25k
safe-50k
Canonical
Randomized
Randomized
Training corpus
(1,000)
( ≈ 25,000)
( ≈ 50,000)
Validity
0.9977 ± 0.0002
0.9326 ± 0.0025
0.9238 ± 0.0029
Uniqueness
0.1022 ± 0.0002
0.6257 ± 0.0204
0.8649 ± 0.0031
Novelty
0.0198 ± 0.0021
0.8466 ± 0.0069
0.9371 ± 0.0007
Yield
20 ± 2
4,941 ± 189
7,488 ± 31
Table 3: Effect of corpus size on SAFE generation (augmentation sweep) . Values are mean ± SD over five independently trained models per arm and each model’s value is the mean of three generation sets of 10,000 samples. Yield and genuine yield are counts per 10,000 samples while the remaining rows are fractions.
Figure 4: Physicochemical property distributions of the novel generated molecules . Kernel density estimates of molecular weight, logP, QED, and synthetic accessibility score (SAS) for the novel molecules of safe-25k (top) and safe-50k (bottom) against the 1,000 training molecules. Each panel shows one representative generation set from a single trained model per arm. The safe-1k arm is omitted because it yields only about 20 novel molecules per generation set.
Arm
safe-atom
safe-dot
safe-full
Randomized axis
Atom order
Fragment order
Both
Validity
0.9769 ± 0.0017
0.9874 ± 0.0006
0.9114 ± 0.0031
Uniqueness
0.1974 ± 0.0034
0.2171 ± 0.0051
0.8674 ± 0.0047
Novelty
0.5345 ± 0.0071
0.5801 ± 0.0087
0.9301 ± 0.0021
Yield
1,031 ± 30
1,244 ± 47
7,353 ± 43
Genuine yield
712 ± 20
1,026 ± 42
6,977 ± 43
Table 4: Contribution of each randomization axis (axis ablation) . All three arms are composition-matched to Nmatch=32,152 strings per arm by Eqs. ( 1 ) and ( 2 ), so they differ only in which axis is randomized. Values are mean ± SD over five independently trained models per arm and each model’s value is the mean of three generation sets of 10,000 samples.
Figure 5: Per-run yield (left) and genuine yield (right) for the three axis-ablation arms . Each point is one trained model, the mean of its three generation sets of 10,000 samples (five models per arm). Horizontal bars show the mean across models and whiskers show the standard deviations. The standard deviations (20 to 47 molecules per 10,000 samples) are small relative to the between-arm differences and are barely visible on the logarithmic y -axis.
Figure 6: Physicochemical property distributions of the novel generated molecules . Kernel density estimates of molecular weight, logP, QED, and synthetic accessibility score (SAS) for the novel molecules of safe-atom (top), safe-dot (middle), and safe-full (bottom) against the 1,000 training molecules. Each panel shows one representative generation set from a single trained model per arm.
Arm
smiles-rand
safe-rand
Randomized axis
Atom order
Both
Validity
0.9484 ± 0.0012
0.9230 ± 0.0012
Uniqueness
0.5096 ± 0.0117
0.8650 ± 0.0016
Novelty
0.7993 ± 0.0054
0.9365 ± 0.0012
Yield
3,864 ± 119
7,477 ± 24
Genuine yield
2,562 ± 90
7,067 ± 32
Table 5: Randomized SMILES versus randomized SAFE at matched corpus size of 49,209 strings . The arm safe-rand randomizes both axes (fragment order and atom order, i.e. full ). Entries are mean ± SD over five independently trained models per arm and each model’s value is the mean of three generation sets of 10,000 samples.
Figure 7: Physicochemical property distributions of the novel molecules at matched corpus size . Kernel density estimates of molecular weight, logP, QED, and synthetic accessibility score (SAS) for the novel molecules of smiles-rand (top) and safe-rand (bottom), each plotted against the 1,000 training molecules. Each panel shows one representative generation set from a single trained model per arm.
Figure 8: Generated molecules nearest to and furthest from a single, randomly sampled training molecule (top). Middle row shows the four safe-rand molecules with the highest Tanimoto similarity to it and bottom row shows the four with the lowest. The value above each structure is its Tanimoto similarity (Morgan fingerprints, radius 2, 2048 bits) from the training molecule.
Figure 9: Sample novel molecules generated by safe-rand . Nine molecules drawn uniformly at random from the generated set.
Molecular generation develops computational methods to create chemical structures and conformations for discovery, optimization, and structure elucidation.
We introduce Fraglingo, an autoregressive molecular generator that constructs molecules step by step from chemically meaningful fragments connected through predefined attachment sites. At each generation step, Fraglingo jointly predicts which fragment to add and how it should attach by producing an attachment-aware fragment embedding and retrieving the nearest fragment through latent-space search. A wildcard-anchored readout represents both the growing molecule and candidate fragments relative to their attachment sites, enabling a single latent prediction to determine both fragment identity and attachment configuration. Because prediction operates in a continuous embedding space rather than over fixed fragment identifiers, larger fragment libraries can be introduced at inference time without retraining. This retrieval-based formulation provides a unified generation primitive for molecule generation, scaffold generation, scaffold decoration, and molecule optimization. Fraglingo also supports property-conditional generation, allowing desired molecular properties to guide the generation process. On controlled property-conditional benchmarks, Fraglingo achieves stronger joint property control than comparably trained baselines while maintaining competitive validity, uniqueness, and novelty. It also generalizes to fragment libraries up to four times larger than those used during training without retraining. Code is available at: https://anonymous.4open.science/r/FragLingo-3551.
Thao Nguyen, Jeonghwan Kim, Zhenhailong Wang +1
Siebel School of Computing and Data Science University of Illinois Urbana-Champaign Urbana, IL 61801, USA
Language models for molecular design have scaled to hundreds of millions of parameters, yet how they learn chemical grammar is poorly understood. We train SMolLM, a 53K-parameter weight-shared transformer, to generate novel SMILES with 95% validity on the ZINC-250K drug-like-molecule benchmark, outperforming a standard GPT with 10 times more parameters. Mechanistically, the same block resolves SMILES constraints across passes in a fixed hierarchy: brackets first, rings second, and valence last, as shown by error classification and linear probing, with ablation isolating the bracket-matching head. Together, these results yield a compact, mechanistically interpretable molecular generator and a testbed for studying iterative computation in formal-language domains.
Three-dimensional (3D) molecule generation has been dominated by diffusion models, which achieve strong generation quality but typically require the molecular size to be specified a priori. Recent autoregressive approaches have substantially narrowed the performance gap while naturally supporting variable-length generation and conditioning on partial molecular context. However, balancing unconditional and context-conditioned generation remains challenging. We introduce KRONOS, a latent autoregressive diffusion framework that generates molecules in the latent space of a pre-trained autoencoder, jointly modeling molecular graph topology and geometry, while retaining the flexibility of autoregressive generation. We further introduce a mixed training strategy inspired by Fill-in-the Middle (FIM) paradigm, enabling both unconditional and fragment-conditioned molecular generation within a single left-to-right autoregressive model. Experiments on QM9 and GEOM-Drugs demonstrate that KRONOS achieves leading unconditional generation performance among autoregressive methods, while remaining competitive with diffusion models. Moreover, fragment-conditioned generation is achieved with negligible impact on unconditional generation performance, demonstrating that both generation paradigms can be supported within a single architecture.
Federico Ottomano, Gaopeng Ren, Yingzhen Li +2
Department of Chemistry Imperial College London · Department of Computing Imperial College London