Variational Monte Carlo for Quantum Excited States via Nested Low-Rank Approximation
Authors: Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell
Organizations: Center for AI and Natural Sciences, KIAS, Seoul 02455, South Korea · Department of CSE, Penn State University, University Park, PA 16802, USA · Kim Jaechul Graduate School of AI, KAIST, Daejeon 34141, South Korea · Department of EECS, MIT, Cambridge, MA 02139, USA
The electronic Schrödinger equation encodes a molecule's electronic structure and quantum properties, but solving it scales exponentially with the number of electrons. Solving this equation with neural-network variational Monte Carlo (VMC) has reached chemical accuracy on the ground state of small molecules; extending this success to excited states is constrained by two existing paradigms. Penalty-based methods need K Laplacians per step but require tuning the overlap-penalty weights against the energy gaps they aim to compute. Natural excited states VMC (NES-VMC) is penalty-free, but it combines the K ansatzes into one determinant, whose local energy requires K2 Laplacians per step. We resolve this trade-off via low-rank approximation (LoRA) reparameterization of the rank-K variational objective. The reparameterized objective admits a VMC-compatible stochastic gradient estimator via MCMC sampling from the per-state Born densities, and a sequential nesting scheme recovers the lowest K eigenstates in eigenvalue order. The resulting algorithm, NestedLoRA-VMC, inherits a global optimality guarantee from the LoRA principle. On the first-row atoms Li through Ne at K=10, its accuracy is comparable to that of NES-VMC and of a state-of-the-art penalty-based method.
Figures & tables
NES-VMC [ 27 ]
Penalty-based [ 3 , 37 ]
NestedLoRA-VMC
Objective
Energy of the extended system
Energies + overlap penalty
Rank- K approximation of −\mathpzcH
Overlap-penalty weights
-
αkℓ>Ek−Eℓ , per pair
-
Sampling
One chain over Knel electrons
K chains over nel electrons
K chains over nel electrons
Laplacians per step
K2
K
K
Theoretical scaling
O(K2nel4+K3)
O(Knel4+K2nel3)
O(Knel4+K2nel3)
Wall-time scaling on Ne
O(K2.5)†
O(K1.1)†
≈ penalty-based (Figure 6 )
Table 1: Comparison of NestedLoRA-VMC with the two existing excited-state VMC approaches. Theoretical scaling excludes spin-penalty overhead (Appendix I.2 ). † Measured by Szabó et al. [37, Figure 1] for K∈{1,…,30} on Ne.
Figure 1: First-row K=10 energy-level diagram (Li–Ne). Per-atom recovered spectra (NestedLoRA-VMC) on top of NIST term centroids [ 32 ] (horizontal segments). Markers are colored by total spin S (per-network measured ⟨S^2⟩ ). Within each subplot, markers are placed left-to-right in order of energy, so states close in energy separate horizontally. For an atom with several seeds in Table 2 , each marker is the mean over seeds of the energy and ⟨S^2⟩ of the state at that rank. Table 9 gives the term and configuration of each state.
Method
Li
Be
B
C
N
O
F
Ne
NES-VMC [ 27 ]
0.02
0.04
0.54
0.09
0.16
0.26
1.90
1.95
Szabó et al. [37] (Penalty-based)
1.06
0.20
0.63
0.59
2.95
1.43
1.40
2.86
NestedLoRA-VMC
0.06±0.01
0.03±0.01
0.64±0.23
0.18±0.01
0.28
0.27
0.65
1.55±0.56
Table 2: Per-atom mean absolute error (mHa) of K=10 excited-state energies against (2J+1) -weighted NIST term centroids [ 32 ] . An entry m±s is the mean m and standard deviation s over random seeds; the others are from one run.
Figure 2: Normalized overlap S~kℓ of every pair of networks within a spin sector at the final training step of the seed- 1 run of each atom from Li to F, with the pairs of an atom ordered by S~kℓ . Each value comes from two long Metropolis chains (Appendix F.2.2 ), and Table 4 gives their convergence. Error bars are bootstrap standard errors (Appendix F.2.2 ).
Figure 3: Carbon runs of Section 4.4 , with Eref,k=0 or with the shift Eref,k=−λk+5σk , for training seed 1 (a, c) and seed 2 (b, d). (a, b) Batch mean of the clipped local energy of the five networks of the 1D term; the thick line is the network furthest above the others with Eref,k=0 . (c, d) Normalized overlap from the unclipped batch mean, averaged over the 10 pairs of these networks. Every curve is a mean over a sliding window of 100 steps, taken before the absolute value in (c, d).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Particle and walker variables
nel,nnu
Number of electrons / nuclei in the molecule
K
Rank of the LoRA approximation; number of states, the ground state included
ri,RI
3D positions of the i -th electron, i∈[nel] , and of the I -th nucleus, I∈[nnu]
x=(r1,…,rnel)
Single-walker nel -electron configuration
X=(x1,…,xK)
K electron configurations of NES-VMC, of which xi is the i -th
Appendix
Table 3: Notation summary.
Figure 4: Overlap ratio R=ψℓ/ψk of networks k and ℓ of the Li ansatz, on the walkers of ψk . Panels (a) and (b) are at training step 1,000 and number the networks by ascending energy at that step; panels (c) and (d) are at step 100,000 and number them as in Table 9 . Dotted lines in (a) mark the clip widths of (b); error bars in (b) are standard errors over six batches; the axis of R in (d) is logarithmic beyond ±0.01 . Panel (b) uses the restarted chains (Appendix F.2.2 ).
Atom
Pairs
Converged
Short
Differ
Unresolved
Largest S~kℓ
Li
45
29
14
0
2
0.037
Be
21
21
0
0
0
0.037
B
24
18
3
2
1
0.032
C
18
18
0
0
0
0.026
N
29
28
0
1
0
0.021
O
18
18
0
0
0
0.017
Appendix
Table 4: Pairs of networks of each atom in Figure 2 by the convergence of their two long chains, and the largest S~kℓ over all pairs of the atom.
Input of electron i
ri−RI and ∥ri−RI∥ for every nucleus I , each scaled by log(1+∥ri−RI∥)/∥ri−RI∥ , and the spin σi=±1
Embedding
linear map without bias to dimension 256
Layers
4 , each a self-attention over all electrons with 4 heads followed by a two-layer tanh MLP of width 256 , both with residual sums
Orbitals
linear map without bias from the embedding of electron i to fdij
Determinants
ndet=16 , each over all nel electrons, summed with unit weights
Envelopes
πdjIσ and ζdjIσ initialized to 1
Jastrow factor
J=−∑i<jcijα2/(α+∥ri−rj∥) with cij=1/4 for equal and 1/2 for opposite spins, and one trained α for each case, initialized to 1
Appendix
Table 5: Settings of each network ψk .
Figure 5: Computation of logψ with the envelope shift, from the electron positions and spins and the trained parameters θ . The constant cdi , held fixed under differentiation, is subtracted from the exponents in row i of Ad and its sum over i from logdetA~d .
atom
mS
K=10 block
CAS (Ne,No)
Li
1/2
2S(1),2P∘(3),2S(1),2P∘(3),2D(2)
(1,13)
Be
0
1S(1),1P∘(3),1S(1),1D(1)
(2,8)
1
3P∘(3),3S(1)
B
1/2
2P∘(3),2S(1),2D(3)
(3,8)
3/2
4P(3)
C
0
1D(5),1S(1)
(4,4)
Appendix
Table 6: Per-atom physical configuration. Each row lists the terms one spin sector mS=S (Section 4.1 ) holds, in NIST term-centroid order, with the number of networks assigned to each term in parentheses. A fully covered term gets its 2L+1 orbital sub-states; a term the K=10 window cuts keeps fewer (e.g. Li’s 2D(2) keeps 2 of its 5 ). The CAS (Ne,No) is chosen per atom by counting the orbitals of the K=10 configurations (Li, C, Ne) or by enlarging the valence shell by {3s,3p} (Be, B, N, O, F).
Component
Joint backbone, K heads
K disjoint networks
Order
NES-VMC
Laplacian
3nel(KCbb+K2Cpost)
3nelK2Ceval
K2nel4
Sampling
nMH(KCbb+K2Cpost)
nMHK2Ceval
nMHK2nel3
Inverse of Ψ
K3
K3
K3
NestedLoRA-VMC, penalty
Laplacian
3nelKCeval
3nelKCeval
Knel4
Sampling
nMHKCeval
nMHKCeval
nMHKnel3
Overlaps
K2Cpost
K2Ceval
K2nel3
Appendix
Table 7: Cost of one training step by component. Cbb is one forward evaluation of the backbone, Cpost that of one state’s orbitals, envelopes and determinants, and Ceval=Cbb+Cpost ; a training step takes nMH Metropolis steps. The last column gives the leading order in K and nel . Bold: the Laplacian, the largest single evaluation.
Figure 6: Empirical per-step wall time of NestedLoRA-VMC’s main optimization loop on Ne, 8× RTX 5090, FP32, at batches of 4,096 and 64 . Dashed lines are least-squares power-law fits. The spin penalty is off.
Atom
Seed
Steps
MAE
Blocks of 100
Blocks of 300
Li
1
100,000
0.046
[0.034, 0.073]
[0.036, 0.070]
Li
2
100,000
0.066
[0.049, 0.093]
[0.051, 0.086]
Li
3
100,000
0.059
[0.046, 0.083]
[0.046, 0.086]
Be
1
100,000
0.019
[0.015, 0.042]
[0.015, 0.042]
Be
2
100,000
0.035
[0.028, 0.054]
[0.025, 0.056]
Be
3
100,000
0.036
[0.022, 0.064]
[0.026, 0.059]
Appendix
Table 8: MAE (mHa) of each run entering Table 2 , with its 95% block-bootstrap interval for blocks of 100 and of 300 training steps over the last 3,000 steps ( 2,000 for Ne seed 1 ). Steps is the number of training steps the run had completed when it was read.
This work
NIST
Baselines (mHa)
System
k
Config.
Term
E (Ha)
⟨S^2⟩
ΔE (Ha)
ΔENIST (Ha)
Err (mHa)
NES-VMC [ 27 ]
Szabó [ 37 ]
Li
1
2s
2S
-7.47805(1)
0.750
0.00000
0.00000
+0.00
—
—
Li
2
2p
2P∘
-7.41015(2)
0.750
0.06790(2)
0.06791
+0.00
+0.00
+0.05
Li
3
2p
2P∘
-7.41014(2)
0.750
0.06791(2)
0.06791
+0.00
+0.01
+0.07
Li
4
2p
2P∘
-7.41010(1)
0.750
0.06795(2)
0.06791
+0.04
+0.01
+0.14
Li
5
3s
2S
-7.35408(3)
0.750
0.12397(3)
0.12396
+0.01
+0.01
+0.22
Appendix
Table 9: Per-state K=10 results across first-row atoms (Li–Ne), one run per atom: seed 1 (Ne: seed 2 ), read at the step given in Table 8 . Within an atom, the rows list the target terms by ascending ΔENIST , each repeated by its number of networks in Table 6 , against the run’s ten lowest states in ascending energy. E is the per-state mean defined above, with its stderr in parentheses. Errors in mHa against the NIST term centroids of Section 4.1 ; baseline errors from Pfau et al. [27, Table S2] and Szabó et al. [37, SI Table S1] . Config. omits the closed 1s 2 shell (Ne: 1s 2 2s 2 ); each column is rounded from unrounded values.
Department of Physics and Astronomy, University of California, Davis, California 95616, USA · Department of Applied Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada · Vector Institute, Toronto, Ontario, M5G 0C6, Canada +1