Geometry-Induced Long-Range Correlations in Recurrent Neural Network Quantum States
Authors: Asif Bin Ayub, Amine Mohamed Aboussalah, Mohamed Hibat-Allah
Organizations: Department of Applied Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada · Perimeter Institute for Theoretical Physics, 31 Caroline St N, Waterloo, ON N2L 2Y5, Canada · NYU Tandon School of Engineering, 6 MetroTech Center, Brooklyn, New York 11201, United States · Vector Institute, Toronto, Ontario, M5G 0C6, Canada
Neural Quantum States based on autoregressive recurrent neural network (RNN) wave functions enable efficient sampling without Markov-chain autocorrelation, but standard RNN architectures are biased toward finite-length correlations and can fail on states with long-range dependencies. A common response is to adopt transformer-style self-attention, but this typically comes with substantially higher computational and memory overhead. Here we introduce dilated RNN wave functions, where recurrent units access distant sites through dilated connections, injecting an explicit long-range inductive bias while retaining a favorable O(NlogN) forward pass scaling. We show analytically that dilation changes the correlation geometry and can induce power-law correlation scaling in a simplified linearized and perturbative setting. Numerically, for the critical 1D transverse-field Ising model, dilated RNNs reproduce the expected power-law connected two-point correlations in contrast to the exponential decay typical of conventional RNN ansätze. We further show that the dilated RNN accurately approximates the one-dimensional Cluster state, a paradigmatic example with long-range conditional correlations that has previously been reported to be challenging for RNN-based wave functions. These results highlight dilation as a simple geometric mechanism for building correlation-aware autoregressive neural quantum states.
Figures & tables
Figure 1: Schematic representation of the RNN architectures used. (a) RNN wave function which sequentially generates the conditional probabilities pn and conditional phases ϕn by taking the one-hot encoded spin σn−1 and hidden vector hn−1 as inputs at the nth step. (b) A dilated RNN illustration for a spin configuration with length N=5 . For the nth site, the network uses the hidden vector hn−2l−1(l) from the (n−2l−1)th site at the same layer l , in addition to hn(l−1) from the preceding layer. If the dilation length is larger than the spin position n , a zero initialization h0 is used instead.
Figure 2: Illustration of the shortest path (in bold green arrows) between the last RNN cell at site xn and the first site x1 of (a) the vanilla RNN and (b) the dilated RNN given a system size N . For the vanilla RNN, the shortest path between sites separated by lag n is given by O(n) , whereas for the dilated RNN, the shortest path has worst-case scaling O(log(n)) .
Figure 3: (a) A plot of the connected two-point correlation function C(r) as a function of distance r computed by dilated RNNs for different numbers of layers using the 1D periodic TFIM with N=100 spins as a testbed. We observe that for l≥4 layers, the expected power-law behavior is captured by our dilated RNN ansatz. Error bars correspond to the one-standard deviation [ 25 ] . (b) A plot of the critical exponent η versus different numbers of dilation layers obtained from fitting C(r) against the chord length Lr and the corresponding R2 values. η approaches the theoretical value of 0.25 for l≥4 layers. The goodness of the fit, indicated by R2 , is close to unity for those layers. Error bars on η denote the one-standard-error uncertainty from the covariance matrix of the least-squares fit.
Figure 4: RNN energy expectation values during training on the Cluster state benchmark for a single-layer RNN and a dilated RNN with 6 layers. Note that the dilated architecture converges to the true ground state energy EG=−64 indicated by the dashed line.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figures
Hyperparameter
Value
Fig. 3
Architecture
1D positive RNN wave function
Number of samples used for training
Ns=100
Number of samples used for estimating correlations
Ns=100,000
Number of memory units
dh=32
Training iterations
100,000
Learning rate
10−4
Appendix
Table 1: Hyperparameters used to obtain the results reported in this paper.
Neural-network quantum states have emerged as a powerful variational framework for quantum many-body systems, with recent progress often driven by massively parallel architectures such as transformers. Recurrent neural network quantum states, however, are frequently regarded as intrinsically sequential and therefore less scalable. Here we revisit this view by showing that modern recurrent architectures can support fast, accurate, and computationally accessible neural quantum state simulations. Using autoregressive recurrent wave functions together with recent advances in parallelizable recurrence, we develop variational ansätze, called parallel scan recurrent neural quantum states (PSR-NQS), which can be trained efficiently within variational Monte Carlo in one and two spatial dimensions. We demonstrate accurate benchmark results and show that, with iterative retraining, our approach reaches two-dimensional spin lattices as large as 52×52 while remaining in agreement with available quantum Monte Carlo data. Our results establish recurrent architectures as a practical and promising route toward scalable neural quantum state simulations with modest computational resources.
Ejaaz Merali, Mohamed Hibat-Allah, Mohammad Kohandel +2
Department of Physics and Astronomy, University of California, Davis, California 95616, USA · Department of Applied Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada · Vector Institute, Toronto, Ontario, M5G 0C6, Canada +1
Neural quantum states based on modern deep learning architectures have emerged as powerful representations for quantum many-body systems. In particular, Transformer-based neural quantum states provide expressive models capable of capturing long-range correlations, and their empirical generalization performance has recently been demonstrated. However, a theoretical understanding of their generalization behavior remains largely unexplored. In this paper, we develop a theoretical framework to analyze the generalization properties of Transformer-based neural quantum states under in-context learning. We establish a rigorous inference-time generalization error bound in terms of mean squared error (MSE), showing that the pointwise prediction error decreases inversely with both the number of in-context examples and the depth of the Transformer. We further show that the Transformer depth required to achieve this guarantee scales only linearly with the system size--namely, the number of particles in continuous systems or the number of qudits in discrete systems. Building on this result, we extend our analysis to full quantum states formulated as rank-one density operators, and derive MSE-based generalization bounds over both continuous and discrete domains under physical constraints. Finally, numerical simulations corroborate our theoretical analysis.
Zhen Qin, Qing Qu, Alfred O. Hero
Michigan Institute for Computational Discovery and Engineering, Department of Electrical Engineering and Computer Science and Department of Statistics, University of Michigan, Ann Arbor, MI 48109 USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109 USA
In the first part of this work, we construct the first type of two-dimensional (2D) hyperbolic neural quantum state (NQS) in the form of the Lorentz 2DRNN (Recurrent Neural Network) and benchmark its performance against the Euclidean 2DRNN in the paradigmatic N×N 2D Transverse Field Ising Model (2DTFIM) setting with different lattice sizes up to N = 12 at different transverse magnetic field strengths. We find that hyperbolic Lorentz 2DRNN NQS robustly outperforms Euclidean 2DRNN NQS in our experiments when the system is at the phase transition point, where the physics can be described by a conformal field theory (CFT) - known via the AdS/CFT holographic duality to possess an underlying hyperbolic geometry. In the second part of this work, we benchmark the performances of recently introduced one-dimensional (1D) hyperbolic NQS ansatzes (including Poincare RNN/GRU and Lorentz RNN/GRU) against their Euclidean versions in N×N 2DTFIM, converted to a one-dimensional sequence. The findings extend our previous results showing that 1D hyperbolic NQS robustly outperforms 1D Euclidean NQS, thanks to the combined effects of the induced 1st and Nth neighbor interaction hierarchy and the CFT physics at criticality. While studies on larger system sizes remain future work, this study serves as a proof-of-concept for the utility, effectiveness, and superior performance of 1D and 2D hyperbolic NQS ansatzes compared to conventional Euclidean NQS models in many-body quantum systems - particularly those exhibiting structural hierarchy, criticality, or a combination of both.