Coupled Cluster con MōLe: Molecular Orbital Learning for Neural Wavefunctions
Authors: Luca Thiede, Abdulrahman Aldossary, Andreas Burger, Jorge Arturo Campos-Gonzalez-Angulo, Ning Wang, Alexander Zook, Melisa Alkan, Kouhei Nakaji, Taylor Lee Patti, Jérôme Florian Gonthier, Mohammad Ghazi Vakili, Alán Aspuru-Guzik
Organizations: Department of Computer Science, University of Toronto, Sandford Fleming Building, 10 King’s College Road, ON M5S 3G4, Toronto, Canada · Vector Institute for Artificial Intelligence, 661 University Ave. Suite 710, ON M5G 1M1, Toronto, Canada · Department of Chemistry, University of Toronto, Lash Miller Chemical Laboratories, 80 St. George Street, ON M5S 3H6, Toronto, Canada · Key Lab of Electromagnetic Processing of Materials, Ministry of Education, Northeastern University, 314 Mailbox, Shenyang, 110819, People’s Republic of China · NVIDIA, 431 King St W #6th, M5V 1K4, Toronto, Canada · Department of Materials Science & Engineering, University of Toronto, 184 College St., M5S 3E4, Toronto, Canada · Department of Chemical Engineering & Applied Chemistry, University of Toronto, 200 College St. ON M5S 3E5, Toronto, Canada · Acceleration Consortium, 700 University Ave., M7A 2S4, Toronto, Canada · Senior Fellow, Canadian Institute for Advanced Research (CIFAR), 661 University Ave., M5G 1M1, Toronto, Canada
Density functional theory (DFT) is the most widely used method for calculating molecular properties; however, its accuracy is often insufficient for quantitative predictions. Coupled-cluster (CC) theory is the most successful method for achieving accuracy beyond DFT and for predicting properties that closely align with experiment. It is known as the ''gold standard'' of quantum chemistry. Unfortunately, the high computational cost of CC limits its widespread applicability. In this work, we present the Molecular Orbital Learning (MōLe) architecture, an equivariant machine learning model that directly predicts CC's core mathematical objects, the excitation amplitudes, from the mean-field Hartree-Fock molecular orbitals as inputs. We test various aspects of our model and demonstrate its remarkable data efficiency and out-of-distribution generalization to larger molecules and off-equilibrium geometries, despite being trained only on small equilibrium geometries. Finally, we also examine its ability to reduce the number of cycles required to converge CC calculations. MōLe can set the foundations for high-accuracy wavefunction-based ML architectures to accelerate molecular design and complement force-field approaches.
Figures & tables
Figure 1: Given a molecule, a Hartree-Fock calculation provides the molecular orbitals represented by their coefficient matrix C . The coefficient vector is padded for each atom to ensure that they all have the same number of basis coefficients, enabling their embedding in an equivariant neural network. The model then alternates message passing to mix information within the MOs and attention layers to mix information between MOs. Finally, the embeddings are read out by “outer product-like” operations, outputting the T1 and T2 amplitudes.
Figure 2: On the left, we are summarizing the key components of the MōLe architecture, consisting of a padding and embedding layer, (T) transformer layers, and finally the T1 and T2 readout. On the right, we are further detailing the attention mechanism, as well as the T1 and T2 readout of our architecture.
Train set
Size extrapolation
Out-of-equilibrium
Model
Complexity
QM7
Amino acids
PubChem
Diels-Alder
Dihedral scan
Chair-to-boat
MACE-100
O(N) – O(N2)
19.15
26.41
64.89
15.01
15.37
26.56
MACE-100 ( Δ -MP2)
O(N5)
1.64
5.53
5.29
3.17
0.79
1.16
eSEN-100 ( Δ -MP2)
O(N5)
1.48
7.43
17.41
5.55
2.74
2.53
MōLe-100
O(N5)
0.66
0.67
2.80
1.50
0.33
0.24
MACE
O(N) – O(N2)
1.83
10.60
17.74
7.05
4.61
4.50
Table 1: Mean absolute error of energy predictions in mHa. "MACE ( Δ -MP2)" and "eSEN ( Δ -MP2)" models are trained to predict ΔE=ECCSD−EMP2 while "MACE" is trained to predict ECCSD directly. MōLe was trained to predict Δtijab=tij,CCSDab−tij,MP2ab . We train each model on a 100 molecule subset (denoted “Model”-100), and on the full training split (5732 molecules) of QM7 (denoted “Model”). The 100 molecule models are averaged over three seeds. Pure MACE without MP2 scales square up to a certain system size, and then linear due to cutoff radii. Similar cutoffs can be made for wavefunction methods as well but are harder to implement, which is why we list them as the fifth power here.
Figure 3: The energy error of MōLe, MP2+MACE (i.e., Δ -MP2), and MACE along two scans. In the inset, the potential energy surface is shown, with the black line indicating the ground truth energies. MōLe achieves lower error particularly for the transition state region, while MACE would overestimate the activation energies.
Model
Amino acids
PubChem
MP2
0.59
0.90
MōLe
0.13
0.36
Table 2: The Frobenius norm of the error between the density matrix of MP2 and MōLe.
Figure 4: The electron density error of MP2 and MōLe on L-Arginine amino acid. The error is plotted at the 95% percentile of the MP2 error.
Model
Amino acids
PubChem
Avg. cycles
Num. unconverged
Avg. cycles
Num. unconverged
MP2 (default guess)
7.61
0
10.11
3
MōLe
3.83
0
6.3
1
Table 3: The average CCSD cycles needed for convergence and number of unconverged systems with default MP2 vs. MōLe amplitudes. The convergence criterion is set to obtain an energy error of ≲0.1 mHa compared to a very tightly converged calculation.
Figure 5: Left: Scaling the transformer’s depth monotonically decreases the prediction error up to four layers. Right: Scaling the transformer width also decreases the prediction error monotonically.
Figure 6: We are timing CC as implemented by GPU4PySCF Li et al. (2024) ; Wu et al. (2024) and MōLe for increasingly larger alkane systems, confirming the expected O(N5) and O(N6) scalings.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
QM7
MōLe - canonical
0.27
MōLe - localized
0.16
Appendix
Table S1 : Ablation of the orbital localization. We train a smaller model with 64 irreps features in the transformer with canonical and localized orbitals. We see that localized orbitals reduce the error significantly.
Figure S1 : The prediction of MōLe and ground truth for the slice (tCCSD−tMP2)i=2,j=2a,b of Methanol. We see that MōLe correctly predicts the highly non-trivial correction between MP2 and CCSD.
Category
Configuration
Model / Geometry
Basis set
def2-SVP
Number of layers
4
Transformer irrep dimensions
Hidden irreps
128x0e + 128x1o + 128x2e
Edge irreps
1x0e + 1x1o + 1x2e
Appendix
Table S2: Model architecture and training hyperparameters used in our experiments.
Category
Configuration
Model Architecture
Number of interactions
3
Hidden irreps
128x0e + 128x1o
Number of radial basis
8
Cutoff radius
6.0 Å
Training
Appendix
Table S3: Model architecture and training hyperparameters for MACE.
Category
Configuration
Model Architecture
Backbone
eSEN-MD
Number of layers
4
Hidden channels
128
Sphere channels
128
ℓmax
1
Appendix
Table S4: Model architecture and training hyperparameters for eSEN.
Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Chemistry, Emory University, Atlanta, GA 30322, USA · Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +2