Chemical Language Models

Latest papers

8 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Does Pretraining Teach Chemical Language Models Chemistry?
Oct 13, 2026Leif Sieben, Yoel ZimmermannChemical Language Models Language Model Pretraining
Harvard University, Cambridge, MA, USA
Probing Chemical Language Models: Effects of Pre-training and Fine-tuning
Jul 2, 2026Anna Karnysheva, Dietrich Klakow, Ji-Ung LeeLanguage Model Pretraining Chemical Language Models
1RTG Neuroexplicit Models · 2Spoken Language Systems · Saarland University +1
Uncertainty-aware reinforcement learning for chemical language models
Jun 23, 2026Borja Medina, Jon Paul JanetChemical Language Models Molecular Generation
Molecular AI, Discovery Sciences, BioPharmaceuticals R&D AstraZeneca AB Gothenburg, Sweden
What Does a Chemical Language Model Know About Molecules?
Jun 22, 2026Christian Kenneth, Etowah Adams, Liam Bai +1Chemical Language Models Mechanistic Interpretability
Independent · Department of Systems Biology, Columbia University, New York · Generate:Biomedicines, Massachusetts +1
From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models
May 11, 2026Zehao Li, Yasuhiro Yoshikai, Shumpei Nemoto +2Chemical Language Models Transformer Interpretability
Laboratory of Molecular Pharmacokinetics, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, Japan · The Institute of Statistical Mathematics (ISM), Research Organization of Information and Systems, 190-8562 Tachikawa, Tokyo, Japan
A chemical language model for reticular materials design
Mar 20, 2026Dhruv Menon, Vivek Singh, Xu Chen +7Chemical Language Models Materials Science
Department of Chemical Engineering & Biotechnology, University of Cambridge, Cambridge, UK · Department of Chemistry, University of California – Berkeley, Berkeley, CA, USA · Bakar Institute of Digital Materials for the Planet, Berkeley, CA, USA +2
BERTology of Molecular Property Prediction
Mar 13, 2026Mohammad Mostafanejad, Paul Saxe, T. Daniel CrawfordChemical Language Models Molecular Property Prediction
Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA · Molecular Sciences Software Institute, Blacksburg, Virginia 24060, USA
A Large Scale Investigation of Scaling Limits in Chemical Language Models
Aug 19, 2025Roshan Balaji, Kamran Chitsaz, Quentin Fournier +2Chemical Language Models Language Model Scaling Laws
BioSystems Engineering and Control Lab · Wadhwani School of Data Science and AI, IIT Madras · The Centre for Integrative Biology and Systems medicinE (IBSE) +5

Oct 13, 2026J/K move · Enter open · S save

Does Pretraining Teach Chemical Language Models Chemistry?

Leif Sieben, Yoel Zimmermann

Harvard University, Cambridge, MA, USA

Chemical Language Models Language Model Pretraining Molecular Representation Learning Language Modeling Molecular Property Prediction

Open paper AskPDF arXiv

Chemical language models (CLMs) are pretrained on large corpora of molecules with the aim of learning representations that transfer to diverse downstream tasks. Here, we systematically study the effects of pretraining using a series of such CLMs across 64 molecular property prediction datasets. We find that the downstream gains of pretraining in CLMs do not scale and performance saturates below gradient-boosting baselines. Across the regimes evaluated, only supervised pretraining on traditional physico-chemical descriptors consistently improves downstream performance for the CLMs tested whereas some forms of pretraining result in models worse than a randomly initialized encoder. While models can resolve molecular structure well, they do so no better than a randomly initialized model, suggesting that, at best, CLMs learn descriptor-like properties in addition to low-level token statistics during pretraining, but surprisingly little chemistry beyond what classical molecular features already capture.