Generative Chemical Language Models for Energetic Materials Discovery

Andrew Salij; R. Seaton Ullberg; Megan C. Davis; Marc J. Cawkwell; Christopher J. Snyder; Cristina Garcia Cardona; Ivana Matanovic; Wilton J. M. Kort-Kamp

physics.chem-phMar 30, 2026

Generative Chemical Language Models for Energetic Materials Discovery

Authors: Andrew Salij, R. Seaton Ullberg, Megan C. Davis, Marc J. Cawkwell, Christopher J. Snyder, Cristina Garcia Cardona, Ivana Matanovic, Wilton J. M. Kort-Kamp

Organizations: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA · Weapon Stockpile Modernization Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA · Computing and Artificial Intelligence Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA

Abstract

The discovery of new energetic materials remains a pressing challenge hindered by limited availability of high-quality data. To address this, we have developed generative molecular language models that have been pretrained on extensive chemical data and then fine-tuned with curated energetic materials datasets. This transfer-learning strategy extends the chemical language model capabilities beyond the pharmacological space in which they have been predominantly developed, offering a framework applicable to other data-spare discovery problems. Furthermore, we discuss the benefits of fragment-based molecular encodings for chemical language models, in particular in constructing synthetically accessible structures. Together, these advances provide a foundation for accelerating the design of next-generation energetic materials with demanding performance requirements.

Figures & tables

Figure 1: a) Training pipeline for GPT models, staged into pretraining to produce a wide variety of molecules and fine-tuning, which produces many C-, N-, O-containing compounds. b) Scheme of GPT model architecture and data processing.

Model (Encoding)	Validity ( $\uparrow$ )	Uniqueness ( $\uparrow$ )	Diversity ( $\uparrow$ )	Novelty ( $\uparrow$ )
MolGPT Bagal et al. (2021) (SMILES)	$0.994$	$\mathbf{1.0}$	$0.857$	$0.797$
GenMol Lee et al. (2025) (SMILES)	$\mathbf{1.000\pm 0.000}$	$0.997\pm 0.001$	$0.818\pm 0.001$	-
Group-VAE-125 Cheng et al. (2023)	$\mathbf{1.0(0)}$	$0.9985(4)$	$\mathbf{0.8587(1)}$	$0.7187(11)$
SELFIES-VAE-125 Cheng et al. (2023)	$\mathbf{1.0(0)}$	$0.9986(4)$	$0.8579(1)$	$0.7345(16)$
$\chi$ hem-GPT (SELFIES)	$\mathbf{0.999\pm 0.001}$	$\mathbf{1.000\pm 0.000}$	$0.820\pm 0.001$	$\mathbf{0.998\pm 0.001}$
(LARGE) (GSELFIES $i$ )	$0.911\pm 0.006$	$0.996\pm 0.002$	$0.821\pm 0.001$	$0.995\pm 0.001$

Table 1: Benchmark metrics for the large and small

\chi

hem-GPT models encoded via SELFIES and GroupSELFIES

i

compared to prior art in the literature. For

\chi

hem-GPT models,

5000

molecules have been generated in

N=5

isolated runs of

1000

molecules each. Validity, uniqueness, novelty, and diversity are determined as scalars ranging from

0-1

for each isolated run, while the other metrics have their means and variances pooled. GroupSELFIES

i

is GroupSELFIES that has its index alphabet enforced to be identical to SELFIES. Cells for literature models source from their respective papers, and cells are blank where the corresponding metric does not exist.

Figure 2: a) Synthetic accessibility (SA) score Ertl and Schuffenhauer (2009) distributions for unconditioned molecular outputs of pretrained

\chi

hem- and fine-tuned X-GPT models with a) number of heavy atoms generated and b) predicted detonation velocities via ChemProp Heid et al. (2023) surrogate. All subfigures are normalized such that the highest histogram bin is 1.

Temp	Validity ( $\uparrow$ )	Diversity ( $\uparrow$ )	SA Score ( $\downarrow$ )	$N$ Radicals ( $\downarrow)$	$N$ Heavy Atoms
$0.8$	$\mathbf{0.999\pm 0.001}$	$0.811\pm 0.003$	$\mathbf{4.27\pm 1.16}$	$\mathbf{0.13\pm 0.47}$	$25.04\pm 11.79$
$1.0$	$\mathbf{0.999\pm 0.001}$	$0.820\pm 0.001$	$4.63\pm 1.18$	$0.33\pm 0.85$	$23.57\pm 9.03$
$1.2$	$0.994\pm 0.003$	$0.839\pm 0.003$	$4.97\pm 1.18$	$0.67\pm 1.21$	$22.35\pm 9.37$
$2.0$	$0.892\pm 0.005$	$\mathbf{0.943\pm 0.002}$	$6.56\pm 1.19$	$3.27\pm 3.45$	$13.84\pm 9.40$

Table 2: Temperature screening for the large

\chi

hem-GPT model trained using SELFIES over benchmark metrics. Runs and statistics performed equivalently to in Table 1 .

Table 3: Comparison of generated outputs between the base and fine-tuned GPT models. a) General metrics from all generated molecular texts across multiple machines (

N=5000

). b) Kamlet-Jacobs Kamlet and Jacobs (1968) (KJ) metrics as characterized on a subset of the molecules in (a), filtered by converged calculations (

185\leq N\leq 530

). The condition for high

P

v

corresponds to

(p,v)=(40\ \text{GPa},10\ \text{km/s})

. Standard fine-tuning is where the first and final transformer layers are unfrozen alongside the token output layer, while LoRA defines low-rank adaptation matrices with rank dimension

r=8

and

\alpha=32

. Statistics follow the procedure outlined for Table 1 .

Figure 3: Comparison of estimated detonation velocities and pressures from Kamlet-Jacobs equations Kamlet and Jacobs (1968) for unconditioned (left column) and conditioned generation (right) for fine-tuned GroupSELFIES models compared to the base model. All subfigures have been normalized to an identical maximum value.

Figure 4: Distributions of a) number of nitrogen-oxygen bonds, b) number of nitrogen-nitrogen bonds, c) quantitative estimation of druglikeness (QED), and d) synthetic accessibility score (SA Score) in large SELFIES-based GPT models.

Figure 5: Common substructures of generated output from chemical language models that have been manually selected for diversity. Substructures were chosen as representative samples from the 200 most common subgraphs between size

5-10

as obtained via RDKit RDKit development team (2025) .

Name	SELFIES Size	GroupSELFIES Size	Sources
SAFE-8M	$7897563$	$7905990$	Noutahi et al. (2024) ; Irwin and Shoichet (2005) ; Chambers et al. (2013)
X-17K	$17407$	$17389$	Allen (2002)

Table S1: Overview of datasets used in this study. Note that for the dataset size, this is of the entire dataset prior to train/val/test splitting, and final batches smaller than the batch size are dropped during training. Note that datasets differ trivially in exact size and composition due to rare parsing issues in constructing the datasets from initial SMILES.

Figure S1: Distribution of synthetic accessibility to heavy atoms in molecule for SAFE-8M.

Figure S2: Distribution of druglikeness to molecular weight in molecule for SAFE-8M.

Figure S3: Calculated velocities and pressures from CHEETAH Fried (1994) for X-17K.

Model	Small (SELFIES)	Small (GSELFIES)	Large (SELFIES)	Large (GSELFIES)
Parameters	$40$ M	$40$ M	$150$ M	$160$ M
$N$ Layers	$12$	$12$	$12$	$12$
$d_{embed}$	$512$	$512$	$1028$	$1028$
$w_{context}$	$256$	$256$	$512$	$512$
$n_{head}$	$4$	$4$	$4$	$4$
vocab. size	$975$	$1617$	$975$	$1617$

Table S2: Model architecture parameters. Parameters are rounded to the nearest

10

million and obtained via a summation of all registered parameters and vocabulary size includes special tokens.

Figure S4: Training and validation loss for $\chi$ hem-GPT (LARGE, SELFIES).

Figure S5: Training and validation loss for $\chi$ hem-GPT (LARGE, GroupSELFIES).

Figure S6: Training and validation loss for $\chi$ hem-GPT (SMALL, SELFIES).

Figure S7: Training and validation loss for $\chi$ hem-GPT (SMALL, GroupSELFIES).

Figure S8: Validation of ChemProp model for detonation a) velocities and b) pressures for the X-17K dataset.

Figure S9: Distributions of a) number of nitrogen-oxygen bonds, b) number of nitrogen-nitrogen bonds, c) quantitative estimation of druglikeness (QED), and d) synthetic accessibility score (SA Score) in large GroupSELFIES

i

-based GPT models.

Figure S10: Comparison of estimated detonation velocities and pressures via XChemProp for unconditioned generation of

\chi

hem- and X-GPT (standard fine-tuning) models. All subfigures normalized to an identical maximum value.

Figure S11: Comparison of CHEETAH calculated (not estimated) detonation properties for $\chi$ hem- and X-GPT GroupSELFIES $i$ models. All subfigures normalized to an identical maximum value.

Figure S12: Comparison of CHEETAH calculated (not estimated) detonation properties for $\chi$ hem- and X-GPT SELFIES models. All subfigures normalized to an identical maximum value.

Figure S13: Comparison of Kamlet-Jacobs calculated (not estimated) detonation properties for $\chi$ hem- and X-GPT SELFIES models. All subfigures normalized to an identical maximum value.

Molecular Generation

Molecular generation develops computational methods to create chemical structures and conformations for discovery, optimization, and structure elucidation.

See all 234 papers

LLM-Driven Discovery of High-Entropy Catalysts via Retrieval-Augmented Generation
Mar 16, 2026AI Scientists, Xinyi Lin, Danqing Yin +1Materials Discovery Materials Science
School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pokfulam, Hong Kong SAR, 999077, China · Laboratory of Data Discovery for Health Limited (D24H), Pak Shek Kok, Hong Kong SAR, 999077, China · Room 312, Lau Chung Him Building, 8 Castle Peak Road, Tuen Mun, Hong Kong
Atomistic Language Models Understand and Generate Materials
Jun 19, 2026Sathya Edamadaka, Krithik Ramesh, Ju Li +1Crystal Structure Prediction Language Modeling
Massachusetts Institute of Technology · Lyra Labs · Lila Sciences
Probing Chemical Language Models: Effects of Pre-training and Fine-tuning
Jul 2, 2026Anna Karnysheva, Dietrich Klakow, Ji-Ung LeeLanguage Model Pretraining Chemical Language Models
1RTG Neuroexplicit Models · 2Spoken Language Systems · Saarland University +1

Mar 16, 2026J/K move · Enter open · S save

LLM-Driven Discovery of High-Entropy Catalysts via Retrieval-Augmented Generation

AI Scientists, Xinyi Lin, Danqing Yin, Ying Guo

School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pokfulam, Hong Kong SAR, 999077, China · Laboratory of Data Discovery for Health Limited (D24H), Pak Shek Kok, Hong Kong SAR, 999077, China · Room 312, Lau Chung Him Building, 8 Castle Peak Road, Tuen Mun, Hong Kong

Materials Discovery Materials Science Large Language Model-Guided Scientific Discovery

Open paper AskPDF arXiv

CO2 reduction requires efficient catalysts, yet materials discovery remains bottlenecked by 10-20 year development cycles requiring deep domain expertise. This paper demonstrates how large language models can assist the catalyst discovery process by helping researchers explore chemical spaces and interpret results when augmented with retrieval-based grounding. We introduce a retrieval-augmented generation framework that enables GPT-4 to navigate chemical space by accessing a database of 50,000+ known materials, adapting general-purpose language understanding for high-throughput materials design. Our approach generated over 250 catalyst candidates with an 82% thermodynamic stability rate while addressing multi-objective constraints: 68% achieved <

100/kg cost with metallic conductivity (band gap<0.1eV) and mechanical stability (B/G>1.75). The best-performing Fe0.2Co0.2Ni0.2Ir0.1Ru0.3 achieves 0.285V limiting potential (25% improvement over IrO2), while Cr0.2Fe0.2Co0.3Ni0.2Mo0.1 optimally balances performance-cost trade-offs at

18/kg. Volcano plot analysis confirms that 78% of LLM-generated catalysts cluster near the theoretical activity optimum, while our system achieves 200x computational efficiency compared to traditional high-throughput screening. By demonstrating that retrieval-augmented generation can ground AI creativity in physical constraints without sacrificing exploration, this work demonstrates an approach where natural language interfaces can streamline materials discovery workflows, enabling researchers to explore chemical spaces more efficiently while the LLM assists in result interpretation and hypothesis generation.

Abstract

Molecular Generation

Explore similar work

LLM-Driven Discovery of High-Entropy Catalysts via Retrieval-Augmented Generation

Atomistic Language Models Understand and Generate Materials

Probing Chemical Language Models: Effects of Pre-training and Fine-tuning