Coupled Cluster con MōLe: Molecular Orbital Learning for Neural Wavefunctions
Authors: Luca Thiede, Abdulrahman Aldossary, Andreas Burger, Jorge Arturo Campos-Gonzalez-Angulo, Ning Wang, Alexander Zook, Melisa Alkan, Kouhei Nakaji, Taylor Lee Patti, Jérôme Florian Gonthier, Mohammad Ghazi Vakili, Alán Aspuru-Guzik
Organizations: Department of Computer Science, University of Toronto, Sandford Fleming Building, 10 King’s College Road, ON M5S 3G4, Toronto, Canada · Vector Institute for Artificial Intelligence, 661 University Ave. Suite 710, ON M5G 1M1, Toronto, Canada · Department of Chemistry, University of Toronto, Lash Miller Chemical Laboratories, 80 St. George Street, ON M5S 3H6, Toronto, Canada · Key Lab of Electromagnetic Processing of Materials, Ministry of Education, Northeastern University, 314 Mailbox, Shenyang, 110819, People’s Republic of China · NVIDIA, 431 King St W #6th, M5V 1K4, Toronto, Canada · Department of Materials Science & Engineering, University of Toronto, 184 College St., M5S 3E4, Toronto, Canada · Department of Chemical Engineering & Applied Chemistry, University of Toronto, 200 College St. ON M5S 3E5, Toronto, Canada · Acceleration Consortium, 700 University Ave., M7A 2S4, Toronto, Canada · Senior Fellow, Canadian Institute for Advanced Research (CIFAR), 661 University Ave., M5G 1M1, Toronto, Canada
Density functional theory (DFT) is the most widely used method for calculating molecular properties; however, its accuracy is often insufficient for quantitative predictions. Coupled-cluster (CC) theory is the most successful method for achieving accuracy beyond DFT and for predicting properties that closely align with experiment. It is known as the ''gold standard'' of quantum chemistry. Unfortunately, the high computational cost of CC limits its widespread applicability. In this work, we present the Molecular Orbital Learning (MōLe) architecture, an equivariant machine learning model that directly predicts CC's core mathematical objects, the excitation amplitudes, from the mean-field Hartree-Fock molecular orbitals as inputs. We test various aspects of our model and demonstrate its remarkable data efficiency and out-of-distribution generalization to larger molecules and off-equilibrium geometries, despite being trained only on small equilibrium geometries. Finally, we also examine its ability to reduce the number of cycles required to converge CC calculations. MōLe can set the foundations for high-accuracy wavefunction-based ML architectures to accelerate molecular design and complement force-field approaches.
Figures & tables
Figure 1: Given a molecule, a Hartree-Fock calculation provides the molecular orbitals represented by their coefficient matrix C . The coefficient vector is padded for each atom to ensure that they all have the same number of basis coefficients, enabling their embedding in an equivariant neural network. The model then alternates message passing to mix information within the MOs and attention layers to mix information between MOs. Finally, the embeddings are read out by “outer product-like” operations, outputting the T1 and T2 amplitudes.
Figure 2: On the left, we are summarizing the key components of the MōLe architecture, consisting of a padding and embedding layer, (T) transformer layers, and finally the T1 and T2 readout. On the right, we are further detailing the attention mechanism, as well as the T1 and T2 readout of our architecture.
Train set
Size extrapolation
Out-of-equilibrium
Model
Complexity
QM7
Amino acids
PubChem
Diels-Alder
Dihedral scan
Chair-to-boat
MACE-100
O(N) – O(N2)
19.15
26.41
64.89
15.01
15.37
26.56
MACE-100 ( Δ -MP2)
O(N5)
1.64
5.53
5.29
3.17
0.79
1.16
eSEN-100 ( Δ -MP2)
O(N5)
1.48
7.43
17.41
5.55
2.74
2.53
MōLe-100
O(N5)
0.66
0.67
2.80
1.50
0.33
0.24
MACE
O(N) – O(N2)
1.83
10.60
17.74
7.05
4.61
4.50
Table 1: Mean absolute error of energy predictions in mHa. "MACE ( Δ -MP2)" and "eSEN ( Δ -MP2)" models are trained to predict ΔE=ECCSD−EMP2 while "MACE" is trained to predict ECCSD directly. MōLe was trained to predict Δtijab=tij,CCSDab−tij,MP2ab . We train each model on a 100 molecule subset (denoted “Model”-100), and on the full training split (5732 molecules) of QM7 (denoted “Model”). The 100 molecule models are averaged over three seeds. Pure MACE without MP2 scales square up to a certain system size, and then linear due to cutoff radii. Similar cutoffs can be made for wavefunction methods as well but are harder to implement, which is why we list them as the fifth power here.
Figure 3: The energy error of MōLe, MP2+MACE (i.e., Δ -MP2), and MACE along two scans. In the inset, the potential energy surface is shown, with the black line indicating the ground truth energies. MōLe achieves lower error particularly for the transition state region, while MACE would overestimate the activation energies.
Model
Amino acids
PubChem
MP2
0.59
0.90
MōLe
0.13
0.36
Table 2: The Frobenius norm of the error between the density matrix of MP2 and MōLe.
Figure 4: The electron density error of MP2 and MōLe on L-Arginine amino acid. The error is plotted at the 95% percentile of the MP2 error.
Model
Amino acids
PubChem
Avg. cycles
Num. unconverged
Avg. cycles
Num. unconverged
MP2 (default guess)
7.61
0
10.11
3
MōLe
3.83
0
6.3
1
Table 3: The average CCSD cycles needed for convergence and number of unconverged systems with default MP2 vs. MōLe amplitudes. The convergence criterion is set to obtain an energy error of ≲0.1 mHa compared to a very tightly converged calculation.
Figure 5: Left: Scaling the transformer’s depth monotonically decreases the prediction error up to four layers. Right: Scaling the transformer width also decreases the prediction error monotonically.
Figure 6: We are timing CC as implemented by GPU4PySCF Li et al. (2024) ; Wu et al. (2024) and MōLe for increasingly larger alkane systems, confirming the expected O(N5) and O(N6) scalings.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
QM7
MōLe - canonical
0.27
MōLe - localized
0.16
Appendix
Table S1 : Ablation of the orbital localization. We train a smaller model with 64 irreps features in the transformer with canonical and localized orbitals. We see that localized orbitals reduce the error significantly.
Figure S1 : The prediction of MōLe and ground truth for the slice (tCCSD−tMP2)i=2,j=2a,b of Methanol. We see that MōLe correctly predicts the highly non-trivial correction between MP2 and CCSD.
Category
Configuration
Model / Geometry
Basis set
def2-SVP
Number of layers
4
Transformer irrep dimensions
Hidden irreps
128x0e + 128x1o + 128x2e
Edge irreps
1x0e + 1x1o + 1x2e
Appendix
Table S2: Model architecture and training hyperparameters used in our experiments.
Category
Configuration
Model Architecture
Number of interactions
3
Hidden irreps
128x0e + 128x1o
Number of radial basis
8
Cutoff radius
6.0 Å
Training
Appendix
Table S3: Model architecture and training hyperparameters for MACE.
Category
Configuration
Model Architecture
Backbone
eSEN-MD
Number of layers
4
Hidden channels
128
Sphere channels
128
ℓmax
1
Appendix
Table S4: Model architecture and training hyperparameters for eSEN.
Coupled-cluster (CC) theory is often considered the gold standard of quantum chemistry, but its high computational cost limits routine access to accurate energies, forces and response properties. While the right-hand T-amplitudes determine the correlated wavefunction, many practically important observables additionally require the left-hand Λ-amplitudes. We introduce MōLe-Λ, an extension of Molecular Orbital Learning (MōLe) that predicts the full ground-state coupled-cluster singles and doubles (CCSD) response state by jointly learning right-hand amplitudes (T1,T2) and left-hand amplitudes (Λ1,Λ2) from localized Hartree--Fock molecular orbitals. Architecturally, MōLe-Λ extends MōLe with Λ1 and Λ2 readouts that mirror the symmetry constraints of the T1 and T2 heads, while preserving the original equivariant orbital encoder, odd sign-equivariant decoding, locality and size-extensivity. The resulting model yields accurate CC-quality energies and forces, while simultaneously recovering dipoles, quadrupoles, polarizabilities, the electron density, and 2-electron observables such as the pair density. We show that MōLe-Λ further extends the speed advantage of MōLe over full CCSD while substantially expanding the accessible properties, providing a route to wavefunction-level surrogate models for correlated quantum chemistry.
Andreas Burger, Luca Thiede, Abdulrahman Aldossary +4
University of Toronto, Canada · Vector Institute for Artificial Intelligence, Toronto, Canada · NVIDIA, Toronto, Canada +2
Coupled-cluster theory defines the accuracy standard for molecular electronic-structure properties but scales too steeply for routine application, whereas density-functional theory is affordable yet systematically biased. We resolve this trade-off with a single equivariant network, HARP (Hamiltonian Read-out for Properties), that predicts an effective one-electron Hamiltonian from one inexpensive B3LYP/def2-SVP calculation and derives a broad suite of properties from it (energy, optical gap, dipole, quadrupole, polarizability, Mulliken atomic charges, and Mayer bond orders) at coupled-cluster accuracy across nine main-group elements, including the under-served phosphorus, sulfur, and chlorine chemistries. The model is trained on a new in-house dataset of multi-property labels computed at the CCSD(T) level for all nine elements. On a held-out test set, it reduces the error of every property by a factor of 3.8 to 270 relative to semi-local, hybrid, and double-hybrid DFT (referenced to composite CCSD(T)/cc-pVTZ), while adding only ~0.1 s wall time per molecule, delivering coupled-cluster-quality predictions at the cost of a single DFT calculation. Critically, deriving every property from a predicted Hamiltonian rather than pooling per-atom features builds the correct size-scaling into the model architecture: on pi-conjugated oligothiophenes it matches finite-field CCSD polarizability to ~1% and the EOM-CCSD optical gap to ~3% at the largest sizes where those references remain affordable (44 and 37 atoms, where a single CCSD field point already costs ~500x the model's entire inference) and extrapolates the corrected trends to 58-atom chains, a regime where pooling-based architectures fail by construction. Accurate extrapolation is therefore set by the model's inductive bias rather than by the training data.
Wenhao He, Xu Chen, Noah Song +10
Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Chemistry, Emory University, Atlanta, GA 30322, USA · Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +2
Machine-learned (ML) operator models can be trained to predict density functional theory (DFT) Hamiltonian/density matrices at significantly reduced computational cost, thus extending electronic-structure calculations to previously unfeasible scales. Here, we introduce MALOQ (Massively Accelerated Learning of Operators for Quantum Transport), an application built to train on and predict electronic-structure matrices for systems made of few to 100k atoms, described by large basis sets, and covering a wide range of atomic elements. Based on a state-of-the-art, SO(2)-equivariant backbone architecture, MALOQ provides (i) custom data-processing kernels to handle high-rank Hamiltonian matrix data and (ii) a scalable edge-wise distribution of atomic graph(s). Trained on the largest molecular Hamiltonian datasets available today, it reduces time-per-epoch by over 30% compared to a molecule-wise-distributed framework, and enables inference on material graphs of arbitrary size. We demonstrate scalable training and inference for 3,000-12,000 atoms on the Alps supercomputer, up to 192 GPUs and 256 GPUs, respectively.
Manasa Kaniselvan, Alexander Maeder, Denghui Lu +2