Multi-objective optimization and quantum hybridization of equivariant deep learning interatomic potentials

G. Laskaris; D. Morozov; D. Tarpanov; A. Seth; J. Procelewska; G. Sai Gautam; A. Sagingalieva; R. Brasher

cond-mat.mtrl-sciFeb 18, 2026

Multi-objective optimization and quantum hybridization of equivariant deep learning interatomic potentials

Authors: G. Laskaris, D. Morozov, D. Tarpanov, A. Seth, J. Procelewska, G. Sai Gautam, A. Sagingalieva, R. Brasher, +1 more

Organizations: Terra Quantum AG, Kornhausstrasse 25, 9000 St. Gallen, Switzerland · LIACS, Leiden University, Netherlands · Nanoscience Center and Department of Chemistry, University of Jyväskylä, P.O. Box 35, 40014 Jyväskylä, Finland · Department of Materials Engineering, Indian Institute of Science, Bengaluru 560012, India · Schaeffler Technologies AG & Co. KG, Herzogenaurach, Germany

Abstract

Allegro is a machine learning interatomic potential model designed to predict atomic properties in molecules using E(3) equivariant neural networks. When training this model, there tends to be a trade-off between accuracy and inference time. For this reason, we apply multi-objective hyperparameter optimization to both objectives. Additionally, we experiment with modified architectures by constructing variants of Allegro: one extended with additional classical layers and one incorporating quantum-classical hybrid layers. We evaluate all models on QM9, rMD17-aspirin, rMD17-benzene, and a self-generated dataset of copper-lithium structures. As results, both variants surpass Allegro in force prediction accuracy across multiple datasets. The classical variant consistently improves over the baseline, while the quantum-classical hybrid variant achieves the best overall force prediction accuracy on the Cu-Li dataset, where it was fully optimized, outperforming the classical variant by approximately 13%. Notably, the hybrid variant also achieves competitive results on the remaining datasets despite using hyperparameters transferred from Cu-Li without dataset-specific optimization, suggesting that quantum-classical hybridization is a promising direction for enhancing MLIP architectures.

Figures & tables

Figure 1: Quantum depth-infused layer

Panel of Figure 2 — Figure 2: Pareto fronts as they generated by our implementation of SAMO-COBRA during multi-objective optimization of the Allegro model and its variants, minimizing both the forces mean absolute errors on the validation set in (eV/Å) and the inference time for each dataset tested. The x-axis shows the force mean absolute error (F MAE) on the validation set in eV/Å, and the y-axis shows the total inference time on the entire test set in seconds.

Allegro	$r_{max}$	lr	batch	P	Layers	edge MLP
rMD17 aspirin	7.3	5.36e-03	3	6.9	4	32
rMD17 benzene	12.0	1e-03	1	5.0	3	128
QM9	12.0	1e-03	5	10.0	3	32
CuLi	12.0	2e-03	2	5.0	2	128

Table 1: Best parameters from Allegro HPO

Allegro+MLP	$r_{max}$	lr	batch	P	Layers	edge MLP	final MLP
rMD17 aspirin	7.4	2.93e-03	2	6.7	3	16	[1024,1024]
rMD17 benzene	12.0	1e-03	1	5.0	3	128	[32,4]
QM9	6.2	1e-03	2	5.0	2	64	[32,16]
CuLi	5.0	9.35e-04	1	10.3	3	16	[4,16]

Table 2: Best parameters from Allegro+MLP HPO

Allegro+QDI	$r_{\mathrm{max}}$	lr	batch	P	Layers	edge MLP	final MLP	QDI depth	QDI input features
rMD17 aspirin	9.5	1e-03	1	6.0	4	16	4	4	21
rMD17 benzene	12.0	1e-03	1	5.0	3	32	16	1	4
QM9	12.0	1e-03	5	10.0	3	32	128	3	4
CuLi	5.0	4.02e-03	5	12.0	3	4	128	3	4

Table 3: Best parameters from Allegro+QDI HPO

$\mathrm{Val}_{\mathrm{f_{mae}}}$	Allegro	Allegro [ 12 ]	Allegro+MLP	Allegro+QDI
aspirin	10.2	9.8	9.0	9.9
benzene	0.43	0.37	0.28	0.30
CuLi	45.9	-	43.3	37.8

Table 4:

\mathrm{Val}_{\mathrm{f_{mae}}}

values for best runs in (meV/Å)

$\mathrm{Val}_{\mathrm{e_{mae}}}$	Allegro	Allegro [ 12 ]	Allegro+MLP	Allegro+QDI
aspirin	3.6	2.3	2.6	3.2
benzene	0.21	0.20	0.21	0.19
CuLi	809.0	-	698.9	495.7
QM9	10.3	13.4	21.4	23.3

Table 5:

\mathrm{Val}_{\mathrm{e_{mae}}}

values for best runs in (meV/atom) for up to 300 epochs on QM9, 100 for CuLi and 3000 for aspirin and benzene.

	rMD17-aspirin		rMD17-benzene		QM9
Model	$\mathrm{F_{MAE}}$	$\mathrm{E_{MAE}}$	$\mathrm{F_{MAE}}$	$\mathrm{E_{MAE}}$	$\mathrm{E_{MAE}}$
	(meV/Å)	(meV/atom)	(meV/Å)	(meV/atom)	(meV/atom)
NequIP [ 11 ]	8.2	2.3	0.3	0.04	-
MACE [ 45 ]	6.6	2.2	0.3	0.4	-
BOTNet [ 46 ]	8.5	2.3	0.3	0.03	-
Allegro [ 12 ]	7.3	2.3	0.2	0.3	13.4

Table 6: Comparison with state-of-the-art MLIP models on rMD17 and QM9 benchmarks. Values for NequIP, MACE, and BOTNet are taken from Table 1 of Ref. [ 45 ] , trained on 950 configurations with 50 for validation. ∗ Allegro+QDI results on rMD17 and QM9 use transferred hyperparameters from Cu-Li (see text).

Figure 3: (a) Circuit’s ZX graph with original parameters. (b) Circuit’s ZX graph after simplification.

Figure 4: (a) The histogram of the Fisher eigenspectrum. There are around 78% of parameters with near-zero eigenvalues, but this margin does not go higher than 95% trainability threshold. Other parameters have great impact, contributing to circuit’s trainability. (b) Average Fisher Information Matrix. The diagonal elements show parameters having mostly high gradient values, showing their impact on the circuit. Meanwhile non-diagonal ones are almost non-existent, indicating parameters’ independence. Also there is no evident single-parameter dominance, showing high trainability. (c) Average FIM rank. There are three layers presented, although only one is used in the original circuit. This clearly shows how rank goes on the plateau, which means the model is in the overparameterization mode. Thus, no extra layers are needed as the circuit already has a sufficient number of parameters. (d) Real and imaginary part of Fourier coefficients

Figure 5: Training curves of all Allegro variants on QM9 dataset using the best hyperparameters, as achieved from our hyperparameter optimization. Among the variants, we include the Allegro Anders et al. training curve, which corresponds to QM9 Allegro hyperparameters as reported in [ 12 ] . To smooth the resulting curves we use the time-weighted exponential moving average (EMA) [ 56 ] smoothing technique with 0.75 smoothing factor.

Panel of Figure 6 — Figure 6: Training curves for all Allegro variants including Allegro with the parameters reported by Anders et al. [ 12 ] . We report mean absolute errors both forces and energies for all datasets up to 3000 epochs (except from QM9 and CuLi which is up to 300 and 100 epochs respectively). Moreover, for the energy sub-figures (b) and (d) we use the time weighted EMA [ 56 ] smoothing technique with smoother factors $0.9$ and $0.8$ respectively. All graphs are plotted with semi-logarithmic scaling in y-axis.

Figure 7: An example generated (a) Li (110) and (b) Cu (110) slabs with a thickness of 20 Å and 30 Å and vacuum spacing of 15 Å. Interface generated between the (c) Li (110) and Cu (110) slabs and (d) Li (111) and Cu (111) slabs with the same vacuum spacing of 15 Å.

Explore similar work

Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials
Jul 2, 2026Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng +4Machine Learning Interatomic Potentials Neural Network Optimization
John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA · Robert Bosch LLC Research and Technology Center, Watertown, MA, USA
Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs
May 22, 2026Ihor Neporozhnii, Sjoerd Hoogland, Oleksandr VoznyyMachine Learning Interatomic Potentials Multi-Task Learning
Department of Physical and Environmental Sciences, University of Toronto, 1065 Military Trail, Toronto, M1C 1A4, Ontario, Canada. · The Alliance for AI-Accelerated Materials Discovery, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada. · Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada.
MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials
Mar 2, 2026Yuanchang Zhou, Siyu Hu, Xiangyu Zhang +3Machine Learning Interatomic Potentials Computational Materials Science
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, UCAS

Jul 2, 2026J/K move · Enter open · S save

Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi+3

John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA · Robert Bosch LLC Research and Technology Center, Watertown, MA, USA

Machine Learning Interatomic Potentials Neural Network Optimization Muon Optimizer Matrix Optimization

Open paper AskPDF arXiv

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.

Abstract

Explore similar work

Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs

MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials