Sampling the XY model with an attention-based neural network

Piotr Białas; Christian D. Morales; Tomasz Stebel; Dawid Zapolski

cond-mat.dis-nnOct 16, 2026

Sampling the XY model with an attention-based neural network

Authors: Piotr Białas, Christian D. Morales, Tomasz Stebel, Dawid Zapolski

Organizations: Institute of Applied Computer Science, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · V. N. Karazin Kharkiv National University, 6 Nezalezhnosti Ave, Kharkiv 61022, Ukraine · Institute of Theoretical Physics, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland

Abstract

We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to $96\times96$ , verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.

Boltzmann Sampling

See all 17 papers

Sampling two-dimensional spin systems with transformers
Apr 30, 2026Piotr Białas, Piotr Korcyl, Tomasz Stebel +2Transformer Ising Model
Institute of Applied Computer Science, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · Institute of Theoretical Physics, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland
Scaling Autoregressive Models for Lattice Thermodynamics
Mar 16, 2026Xiaochen Du, Juno Nam, Sulin Liu +1Autoregressive Modeling Computational Materials Science
Department of Chemical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Diffusion-warm sampling of the XY model enables fast thermalization at scale
Jun 29, 2026Sehmimul Hoque, Roger Melko, Pooya RonaghMarkov Chain Monte Carlo Diffusion Sampling
Institute for Quantum Computing, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada · Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada · Perimeter Institute for Theoretical Physics, Waterloo, Ontario N2L 2Y5, Canada +1

Apr 30, 2026J/K move · Enter open · S save

Sampling two-dimensional spin systems with transformers

Piotr Białas, Piotr Korcyl, Tomasz Stebel, Adam Stefański+1

Institute of Applied Computer Science, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · Institute of Theoretical Physics, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, ul. Łojasiewicza 11, 30-348 Kraków, Poland

Transformer Ising Model Autoregressive Modeling

Open paper AskPDF arXiv

Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to

180 \times 180

spins in case of the Ising model. The Effective Sample Size of our sampler is

\sim 20

times larger than that of the previous state-of-the-art neural sampler when trained for the

128 \times 128

Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train

64\times 64

spin systems.

Abstract

Boltzmann Sampling

Explore similar work

Sampling two-dimensional spin systems with transformers

Scaling Autoregressive Models for Lattice Thermodynamics

Diffusion-warm sampling of the XY model enables fast thermalization at scale