Authors: Andreas Burger, Malte Franke, Luka Mucko, Kjell Jorner, Alan Aspuru-Guzik
Organizations: University of Toronto NVIDIA Vector Institute Toronto, Canada · NVIDIA · ETH Zurich NCCR Catalysis Zurich, Switzerland · University of Toronto Vector Institute Toronto, Canada · University of Toronto NVIDIA Vector Institute Acceleration Consortium CIFAR Toronto, Canada
We introduce Grand Canonical Generators (GCG), a generative framework that extends Boltzmann generators to the grand canonical ensemble. We present two designs. The first conditions a variable-size generative model on the chemical potential, sampling particle number and configuration jointly. The second factorizes the grand canonical distribution into a particle-number distribution and the corresponding canonical Boltzmann density. This factorized formulation can use any existing Boltzmann generator for the canonical component, encodes the known linear chemical-potential dependence analytically, and yields a tractable likelihood that supports self-normalized importance sampling (SNIS). Empirically, GCG accurately reproduces grand canonical observables on a Lennard--Jones fluid and methane adsorption in a zeolite, demonstrating generalization across chemical potentials and correction via SNIS and grand canonical Monte Carlo.
Figures & tables
Figure 1: Grand Canonical Generators learn to sample from the μVT ensemble, which defines a distribution over both the particle number and geometries, conditioned on the chemical potential μ .
Figure 2: Lennard–Jones. Curves and bands show the mean and range over three training seeds.
Figure 3: Chemical-potential extrapolation. Mean and ±1 std over three training seeds.
Figure 4: Inference-time correction of Factorized Grand Canonical Generators on LJ with self-normalized importance sampling (top) and grand canonical Monte Carlo (bottom). Bands show 95% bootstrap intervals (200 resamples for self-normalized importance sampling; 2,000 for grand canonical Monte Carlo over five seeds).
Figure 5: Methane adsorption in MFI zeolite for a Factorized Grand Canonical Generators model, compared to grand canonical Monte Carlo.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: One-coordinate marginals for the periodic ideal gas. Lines are the mean density over the three coordinate axes and bands are their range. Dashed is the analytic uniform density. Morph develops a sharp central peak at eβμ=1 . Joint Grand Canonical Generators and Factorized Grand Canonical Generators remain close to the true marginal at all activities.
Figure 7: Ideal-gas errors across activity. Dashed: finite-sample error from independent uniform configurations.
Figure 8: Particle-number transfer to unseen chemical potentials.
Figure 9: Normalized effective sample size ESS/M of the stratified pool in Fig. 4 . (a) Per fixed- N sector at 400 configurations, with the largest normalized weight (dashed). (b) Pooled over all sectors for conditional self-normalized importance sampling and grand canonical self-normalized importance sampling as a function of target-energy evaluations.
Figure 10: Inference-time correction with conditional self-normalized importance sampling, grand canonical self-normalized importance sampling, and Bennett acceptance ratio between N and N+1 particles. Bennett acceptance ratio combines insertion and deletion energy changes to reconstruct particle-number probabilities.
Morph
Joint GCG
Factorized GCG
Factorized GCG
System
LJ
LJ
LJ
Zeolite
Temperature
1.5
1.5
1.5
298 K
Box
6.53
6.53
6.53
40.04×39.80×26.77 Å
Batch
1024
1024
1024
128
Steps
60k
60k
60k
200k
Backbone
8×512
8×512
8×512
8×768
Appendix
Table 1: Model hyperparameters for the Lennard–Jones (LJ) fluid and the MFI zeolite.
Efficient sampling of molecular systems at thermodynamic equilibrium is a hallmark challenge in statistical physics. This challenge has driven the development of Boltzmann Generators (BGs), which allow rapid generation of uncorrelated equilibrium samples by combining a generative model with exact likelihoods and an importance sampling correction. However, modern BGs predominantly rely on normalizing flows (NFs), which either suffer from limited expressivity due to strict invertibility constraints (discrete time) or computationally expensive likelihoods (continuous time). In this paper, we propose Autoregressive Boltzmann Generators (ArBG) -- a novel autoregressive modelling framework -- that overcomes these limitations by departing from the flow-based BG paradigm. ArBG circumvents the topological constraints of flows and enables sequential inference-time interventions, while offering enhanced scalability by leveraging architectures effective in Large Language Models. We empirically demonstrate that ArBG leads to significant improvements over flow-based models across all benchmarks, but particularly in larger peptide systems such as the 10-residue Chignolin. Furthermore, we introduce Robin, a 132 million parameter transferable model trained with the ArBG framework which improves over the previous state-of-the-art, reducing the zero-shot energy error, E-W2, on 8-residue systems by over 60%. The code can be found at the following link: https://github.com/danyalrehman/autobg.
Danyal Rehman, Charlie B. Tan, Yoshua Bengio +2
Mila – Québec AI Institute · Broad Institute of MIT & Harvard · Aithyra +4
Predicting the thermodynamic properties of an alloy requires sampling its configurations across many chemical conditions and recovering free energies on a common absolute scale. Markov chain Monte Carlo (MCMC) is the standard tool, but it requires separate simulations at different conditions, and auxiliary free-energy methods such as thermodynamic integration are used to place results on a common absolute scale. Modern discrete neural samplers typically use reverse KL divergence as the objective and can be mode-seeking or biased. We present Free energy Offering Generative Sampler (FrOGS), a hybrid discrete neural sampler that couples an autoregressive model to a continuous-time Markov chain (CTMC) to be trained jointly under a single shared loss. FrOGS draws i.i.d. configurations, returns an unbiased estimate of the partition function, and gives consistent estimates of thermodynamic observables. We train a single model across a wide range of chemical conditions to produce estimates on a common absolute free-energy scale. FrOGS matches exact finite-size results on the 2D Ising model and reference phase diagrams for AgPd and CuAu, without mode collapse. We additionally compare to SEGAL, a published autoregressive baseline, and find that only FrOGS recovers the stability range of the CuAu3 phase.
Kyucheol Min, Elyssa Hofgard, Tess Smidt
Massachusetts Institute of Technology Cambridge, MA 02139
Sampling high-dimensional probability distributions is a central task in scientific computing, with applications ranging from Bayesian inference to statistical physics and molecular simulation. Despite decades of methodological developments, two major challenges remain: scaling to high dimensions and efficiently exploring multimodal distributions characterized by metastable states. Classical approaches such as Markov chain Monte Carlo, tempering methods, or enhanced sampling based on collective variables have achieved major successes, but they also face intrinsic limitations. This tutorial review explores a new paradigm that has recently emerged at the interface of machine learning and computational statistical physics: the use of generative models as tools for sampling. In this context, models such as normalizing flows and diffusion models are not used in their traditional data-driven setting, but rather as flexible probabilistic models that can assist the sampling of distributions known only up to a normalization constant. This manuscript reviews the early development of this rapidly evolving field and discusses several methodological directions, including exact samplers based on generative models and strategies to train such models in the absence of data. While an exhaustive survey of the literature is not attempted, we present a selection of key ideas and methods, along with a discussion of their strengths and limitations. The review is intended to be an accessible tutorial for both physics and machine learning audiences, and it aims to provide a starting point for researchers interested in exploring this exciting area of research.
Marylou Gabrié
Laboratoire de Physique Statistique, École normale supérieure, PSL Research University, 24 rue Lhomond, 75005 Paris, France