Variational Monte Carlo for Quantum Excited States via Nested Low-Rank Approximation
Authors: Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell
Organizations: Center for AI and Natural Sciences, KIAS, Seoul 02455, South Korea · Department of CSE, Penn State University, University Park, PA 16802, USA · Kim Jaechul Graduate School of AI, KAIST, Daejeon 34141, South Korea · Department of EECS, MIT, Cambridge, MA 02139, USA
The electronic Schrödinger equation encodes a molecule's electronic structure and quantum properties, but solving it scales exponentially with the number of electrons. Solving this equation with neural-network variational Monte Carlo (VMC) has reached chemical accuracy on the ground state of small molecules; extending this success to excited states is constrained by two existing paradigms. Penalty-based methods need K Laplacians per step but require tuning the overlap-penalty weights against the energy gaps they aim to compute. Natural excited states VMC (NES-VMC) is penalty-free, but it combines the K ansatzes into one determinant, whose local energy requires K2 Laplacians per step. We resolve this trade-off via low-rank approximation (LoRA) reparameterization of the rank-K variational objective. The reparameterized objective admits a VMC-compatible stochastic gradient estimator via MCMC sampling from the per-state Born densities, and a sequential nesting scheme recovers the lowest K eigenstates in eigenvalue order. The resulting algorithm, NestedLoRA-VMC, inherits a global optimality guarantee from the LoRA principle. On the first-row atoms Li through Ne at K=10, its accuracy is comparable to that of NES-VMC and of a state-of-the-art penalty-based method.
Figures & tables
NES-VMC [ 27 ]
Penalty-based [ 3 , 37 ]
NestedLoRA-VMC
Objective
Energy of the extended system
Energies + overlap penalty
Rank- K approximation of −\mathpzcH
Overlap-penalty weights
-
αkℓ>Ek−Eℓ , per pair
-
Sampling
One chain over Knel electrons
K chains over nel electrons
K chains over nel electrons
Laplacians per step
K2
K
K
Theoretical scaling
O(K2nel4+K3)
O(Knel4+K2nel3)
O(Knel4+K2nel3)
Wall-time scaling on Ne
O(K2.5)†
O(K1.1)†
≈ penalty-based (Figure 6 )
Table 1: Comparison of NestedLoRA-VMC with the two existing excited-state VMC approaches. Theoretical scaling excludes spin-penalty overhead (Appendix I.2 ). † Measured by Szabó et al. [37, Figure 1] for K∈{1,…,30} on Ne.
Figure 1: First-row K=10 energy-level diagram (Li–Ne). Per-atom recovered spectra (NestedLoRA-VMC) on top of NIST term centroids [ 32 ] (horizontal segments). Markers are colored by total spin S (per-network measured ⟨S^2⟩ ). Within each subplot, markers are placed left-to-right in order of energy, so states close in energy separate horizontally. For an atom with several seeds in Table 2 , each marker is the mean over seeds of the energy and ⟨S^2⟩ of the state at that rank. Table 9 gives the term and configuration of each state.
Method
Li
Be
B
C
N
O
F
Ne
NES-VMC [ 27 ]
0.02
0.04
0.54
0.09
0.16
0.26
1.90
1.95
Szabó et al. [37] (Penalty-based)
1.06
0.20
0.63
0.59
2.95
1.43
1.40
2.86
NestedLoRA-VMC
0.06±0.01
0.03±0.01
0.64±0.23
0.18±0.01
0.28
0.27
0.65
1.55±0.56
Table 2: Per-atom mean absolute error (mHa) of K=10 excited-state energies against (2J+1) -weighted NIST term centroids [ 32 ] . An entry m±s is the mean m and standard deviation s over random seeds; the others are from one run.
Figure 2: Normalized overlap S~kℓ of every pair of networks within a spin sector at the final training step of the seed- 1 run of each atom from Li to F, with the pairs of an atom ordered by S~kℓ . Each value comes from two long Metropolis chains (Appendix F.2.2 ), and Table 4 gives their convergence. Error bars are bootstrap standard errors (Appendix F.2.2 ).
Figure 3: Carbon runs of Section 4.4 , with Eref,k=0 or with the shift Eref,k=−λk+5σk , for training seed 1 (a, c) and seed 2 (b, d). (a, b) Batch mean of the clipped local energy of the five networks of the 1D term; the thick line is the network furthest above the others with Eref,k=0 . (c, d) Normalized overlap from the unclipped batch mean, averaged over the 10 pairs of these networks. Every curve is a mean over a sliding window of 100 steps, taken before the absolute value in (c, d).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Particle and walker variables
nel,nnu
Number of electrons / nuclei in the molecule
K
Rank of the LoRA approximation; number of states, the ground state included
ri,RI
3D positions of the i -th electron, i∈[nel] , and of the I -th nucleus, I∈[nnu]
x=(r1,…,rnel)
Single-walker nel -electron configuration
X=(x1,…,xK)
K electron configurations of NES-VMC, of which xi is the i -th
Appendix
Table 3: Notation summary.
Figure 4: Overlap ratio R=ψℓ/ψk of networks k and ℓ of the Li ansatz, on the walkers of ψk . Panels (a) and (b) are at training step 1,000 and number the networks by ascending energy at that step; panels (c) and (d) are at step 100,000 and number them as in Table 9 . Dotted lines in (a) mark the clip widths of (b); error bars in (b) are standard errors over six batches; the axis of R in (d) is logarithmic beyond ±0.01 . Panel (b) uses the restarted chains (Appendix F.2.2 ).
Atom
Pairs
Converged
Short
Differ
Unresolved
Largest S~kℓ
Li
45
29
14
0
2
0.037
Be
21
21
0
0
0
0.037
B
24
18
3
2
1
0.032
C
18
18
0
0
0
0.026
N
29
28
0
1
0
0.021
O
18
18
0
0
0
0.017
Appendix
Table 4: Pairs of networks of each atom in Figure 2 by the convergence of their two long chains, and the largest S~kℓ over all pairs of the atom.
Input of electron i
ri−RI and ∥ri−RI∥ for every nucleus I , each scaled by log(1+∥ri−RI∥)/∥ri−RI∥ , and the spin σi=±1
Embedding
linear map without bias to dimension 256
Layers
4 , each a self-attention over all electrons with 4 heads followed by a two-layer tanh MLP of width 256 , both with residual sums
Orbitals
linear map without bias from the embedding of electron i to fdij
Determinants
ndet=16 , each over all nel electrons, summed with unit weights
Envelopes
πdjIσ and ζdjIσ initialized to 1
Jastrow factor
J=−∑i<jcijα2/(α+∥ri−rj∥) with cij=1/4 for equal and 1/2 for opposite spins, and one trained α for each case, initialized to 1
Appendix
Table 5: Settings of each network ψk .
Figure 5: Computation of logψ with the envelope shift, from the electron positions and spins and the trained parameters θ . The constant cdi , held fixed under differentiation, is subtracted from the exponents in row i of Ad and its sum over i from logdetA~d .
atom
mS
K=10 block
CAS (Ne,No)
Li
1/2
2S(1),2P∘(3),2S(1),2P∘(3),2D(2)
(1,13)
Be
0
1S(1),1P∘(3),1S(1),1D(1)
(2,8)
1
3P∘(3),3S(1)
B
1/2
2P∘(3),2S(1),2D(3)
(3,8)
3/2
4P(3)
C
0
1D(5),1S(1)
(4,4)
Appendix
Table 6: Per-atom physical configuration. Each row lists the terms one spin sector mS=S (Section 4.1 ) holds, in NIST term-centroid order, with the number of networks assigned to each term in parentheses. A fully covered term gets its 2L+1 orbital sub-states; a term the K=10 window cuts keeps fewer (e.g. Li’s 2D(2) keeps 2 of its 5 ). The CAS (Ne,No) is chosen per atom by counting the orbitals of the K=10 configurations (Li, C, Ne) or by enlarging the valence shell by {3s,3p} (Be, B, N, O, F).
Component
Joint backbone, K heads
K disjoint networks
Order
NES-VMC
Laplacian
3nel(KCbb+K2Cpost)
3nelK2Ceval
K2nel4
Sampling
nMH(KCbb+K2Cpost)
nMHK2Ceval
nMHK2nel3
Inverse of Ψ
K3
K3
K3
NestedLoRA-VMC, penalty
Laplacian
3nelKCeval
3nelKCeval
Knel4
Sampling
nMHKCeval
nMHKCeval
nMHKnel3
Overlaps
K2Cpost
K2Ceval
K2nel3
Appendix
Table 7: Cost of one training step by component. Cbb is one forward evaluation of the backbone, Cpost that of one state’s orbitals, envelopes and determinants, and Ceval=Cbb+Cpost ; a training step takes nMH Metropolis steps. The last column gives the leading order in K and nel . Bold: the Laplacian, the largest single evaluation.
Figure 6: Empirical per-step wall time of NestedLoRA-VMC’s main optimization loop on Ne, 8× RTX 5090, FP32, at batches of 4,096 and 64 . Dashed lines are least-squares power-law fits. The spin penalty is off.
Atom
Seed
Steps
MAE
Blocks of 100
Blocks of 300
Li
1
100,000
0.046
[0.034, 0.073]
[0.036, 0.070]
Li
2
100,000
0.066
[0.049, 0.093]
[0.051, 0.086]
Li
3
100,000
0.059
[0.046, 0.083]
[0.046, 0.086]
Be
1
100,000
0.019
[0.015, 0.042]
[0.015, 0.042]
Be
2
100,000
0.035
[0.028, 0.054]
[0.025, 0.056]
Be
3
100,000
0.036
[0.022, 0.064]
[0.026, 0.059]
Appendix
Table 8: MAE (mHa) of each run entering Table 2 , with its 95% block-bootstrap interval for blocks of 100 and of 300 training steps over the last 3,000 steps ( 2,000 for Ne seed 1 ). Steps is the number of training steps the run had completed when it was read.
This work
NIST
Baselines (mHa)
System
k
Config.
Term
E (Ha)
⟨S^2⟩
ΔE (Ha)
ΔENIST (Ha)
Err (mHa)
NES-VMC [ 27 ]
Szabó [ 37 ]
Li
1
2s
2S
-7.47805(1)
0.750
0.00000
0.00000
+0.00
—
—
Li
2
2p
2P∘
-7.41015(2)
0.750
0.06790(2)
0.06791
+0.00
+0.00
+0.05
Li
3
2p
2P∘
-7.41014(2)
0.750
0.06791(2)
0.06791
+0.00
+0.01
+0.07
Li
4
2p
2P∘
-7.41010(1)
0.750
0.06795(2)
0.06791
+0.04
+0.01
+0.14
Li
5
3s
2S
-7.35408(3)
0.750
0.12397(3)
0.12396
+0.01
+0.01
+0.22
Appendix
Table 9: Per-state K=10 results across first-row atoms (Li–Ne), one run per atom: seed 1 (Ne: seed 2 ), read at the step given in Table 8 . Within an atom, the rows list the target terms by ascending ΔENIST , each repeated by its number of networks in Table 6 , against the run’s ten lowest states in ascending energy. E is the per-state mean defined above, with its stderr in parentheses. Errors in mHa against the NIST term centroids of Section 4.1 ; baseline errors from Pfau et al. [27, Table S2] and Szabó et al. [37, SI Table S1] . Config. omits the closed 1s 2 shell (Ne: 1s 2 2s 2 ); each column is rounded from unrounded values.
Variational Monte Carlo (VMC) is a central algorithm in electronic structure theory and has gained renewed importance through modern neural-network ansätze such as FermiNet. At its core, VMC seeks ground states by minimizing the Rayleigh quotient by stochastic optimization. In this work, we show that the resulting stochastic optimization problem is intrinsically governed by the nodal geometry of the underlying wave function. More precisely, we establish that properties of the nodal set determine the integrability of the local energy and gradient estimators that drive VMC. For broad and practically relevant ansatz classes, including Slater-Jastrow wave functions with variable-exponent Slater-type orbitals, we prove that these estimators are generically heavy-tailed and fail to admit higher moments. At the same time, for general analytic ansätze, we prove weak moment bounds for the relevant estimators and identify precise low-moment regimes, showing how generic and degenerate nodal structures lead to different integrability thresholds. Building on this analysis, we introduce a new robust variant of VMC \unicodex2013 coined PS-Clip-VMC \unicodex2013 which is based on clipping both the local energy and the gradient random variable. We prove that PS-Clip-VMC converges both in expectation and with high probability in the weak moment regime of VMC. Preliminary experiments for training FermiNet on Atoms with up to 18 electrons suggest that PS-Clip-VMC is significantly more robust than standard methods.
Philipp Grohs, Davide Nobile
Faculty of Mathematics, University of Vienna · RICAM, Austrian Academy of Sciences
Neural-network wave functions in Variational Monte Carlo (VMC) have achieved great success in accurately representing both ground and excited states. However, achieving sufficient numerical accuracy in state overlaps requires increasing the number of Monte Carlo samples, and consequently the computational cost, with the number of states. We present a nearly constant sample-size approach, Multi-State Importance Sampling (MSIS), that leverages samples from all states to estimate pairwise overlap. To efficiently evaluate all states for all samples, we introduce Excited Pfaffians. Inspired by Hartree-Fock, this architecture represents many states within a single neural network. Excited Pfaffians also serve as generalized wave functions, allowing a single model to represent multi-state potential energy surfaces. On the carbon dimer, we match the O(Ns4)-scaling natural excited states while training >200× faster and modeling 50% more states. Our favorable scaling enables us to be the first to use neural networks to find all distinct energy levels of the beryllium atom. Finally, we demonstrate that a single wave function can represent excited states across various molecules.
Nicholas Gao, Till Grutschus, Frank Noé +1
Department of Computer Science & Munich Data Science Institute, Technical University of Munich · CuspAI · Free University of Berlin +2
Neural-network quantum states have emerged as a powerful variational framework for quantum many-body systems, with recent progress often driven by massively parallel architectures such as transformers. Recurrent neural network quantum states, however, are frequently regarded as intrinsically sequential and therefore less scalable. Here we revisit this view by showing that modern recurrent architectures can support fast, accurate, and computationally accessible neural quantum state simulations. Using autoregressive recurrent wave functions together with recent advances in parallelizable recurrence, we develop variational ansätze, called parallel scan recurrent neural quantum states (PSR-NQS), which can be trained efficiently within variational Monte Carlo in one and two spatial dimensions. We demonstrate accurate benchmark results and show that, with iterative retraining, our approach reaches two-dimensional spin lattices as large as 52×52 while remaining in agreement with available quantum Monte Carlo data. Our results establish recurrent architectures as a practical and promising route toward scalable neural quantum state simulations with modest computational resources.
Ejaaz Merali, Mohamed Hibat-Allah, Mohammad Kohandel +2
Department of Physics and Astronomy, University of California, Davis, California 95616, USA · Department of Applied Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada · Vector Institute, Toronto, Ontario, M5G 0C6, Canada +1