cs.LGDec 5, 2025

NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation

Authors: Daniel Rose, Roxane Axel Jacob, Johannes Kirchmair, Thierry Langer

Organizations: Department of Pharmaceutical Sciences, University of Vienna · Christian Doppler Laboratory for Molecular Informatics in the Biosciences · Vienna Doctoral School of Pharmaceutical, Nutritional and Sport Sciences Josef-Holaubek-Platz 2, 1090 Vienna, Austria

Abstract

Transformer-based autoregressive models offer an efficient alternative to diffusion- and flow-based approaches for generating 3D molecules. One challenge remains: standard transformer architectures require a sequential ordering of tokens, which is not inherently defined for the atoms in a molecule. Prior works have addressed this by using canonical atom orderings. However, these approaches are not permutation invariant w.r.t. atoms and bias next-token prediction towards ordering conventions. We overcome this limitation by introducing a novel neighborhood-guided training strategy. Our model, NEAT (Neighborhood-Guided, Efficient, Autoregressive Set Transformer) treats molecular graphs as sets of atoms and learns an order-agnostic distribution over admissible tokens at the graph boundary, thereby ensuring atom-level permutation invariance. NEAT achieves state-of-the-art generation quality on the QM9 and GEOM-Drugs datasets, enables seamless completion of molecular prefixes, and offers a significant speed advantage over existing baselines.

Figures & tables

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Fixed-Dimensional Latent Flow for Generating Variable-Size 3D Molecules

    Sep 8, 2026Weichi Yao, Cameron Gruich, Bryan R. Goldsmith +13D Molecular GenerationMolecular Generation

  2. Autoregressive latent diffusion for 3D molecule generation

    Jul 10, 2026Federico Ottomano, Gaopeng Ren, Yingzhen Li +2Latent Diffusion ModelsMolecular Graph Generation

  3. NEAT-POCKET: Pocket-Conditioned Autoregressive 3D Molecular Generation with a Neighborhood-Guided Set Transformer

    Sep 4, 2026Roxane Axel Jacob, Daniel Rose, Thierry Langer +13D Molecular GenerationMolecular Generation