AI for Parameter Selection in Sample-based Quantum Diagonalization
Authors: Doriela Grabocka, Dimitrios Christofidellis, Federico Zipoli, Anna van Elst, Patrick Ruch, Antonio Foncubierta-Rodríguez, Siddhartha Mishra, Matteo Manica
Organizations: IBM Research · ETH Zürich · NCCR Catalysis, Switzerland
Sample-based Quantum Diagonalization (SQD) methods provide a way to compute molecular ground-state energies, but their accuracy and runtime depend sensitively on configuration choices such as the subspace dimension, propagation timestep, and the allocation of a fixed sampling budget across circuits, for which no formal selection method exists. In this work, we explore whether oracle models can predict the quantum runtime and approximation error from the Hamiltonian and the candidate configuration. Using synthetic Hamiltonians with controllable spectral properties, we show that machine-learned models can map a Hamiltonian directly to the best-ranked configuration within a predefined grid for the SKQD algorithm, validated against classical Krylov propagation, achieving strong rank correlations between true and predicted error and runtime, Spearman ρ=0.969 and ρ=0.905, respectively. Extending the approach to 50 molecular Hamiltonians (20-50 qubits), with reference ground-state energies from classical calculations, we train regressors that predict the simulation error and runtime for a given molecule--configuration pair ahead of execution, and classifiers to distinguish, from 600 SqDRIFT quantum-hardware runs, whether a configuration achieves a ground-state energy within chemical accuracy (1.6×10−3 Ha ≈1 kcal/mol). For the molecular Hamiltonians, the models additionally provide an a priori feasibility and resource estimate, predicting whether a molecule can reach chemical accuracy on current hardware and at what runtime, thus correctly classifying held-out molecules at the molecule level, achieving balanced accuracy 61.8% in-distribution and 100% for five larger held-out molecules.
Figures & tables
Figure 1: The main pipeline. The dataset consists of Hamiltonians that are decomposed into Pauli terms. The Pauli terms are used as input for the encoder, that creates 768-dimensional embeddings for each Hamiltonian. The embeddings are paired with all the configurations. The Hamiltonians and the configurations are executed on the quantum computer, yielding runtime and ground state energy approximation error . The embeddings of the Hamiltonians, combined with the configurations and the results from the quantum runs, create the dataset for training the error and runtime models to predict the ground state energy error and the quantum processing unit runtime.
Figure 2: The encoder model. Part A shows the pre-processing of a Hamiltonian H . H is expressed as a sum of Pauli terms, from which features are extracted. These Pauli terms’ features are then concatenated with the global system features and qubit features, which are tokenized and input into the encoder model, which in turn generates the embeddings. For more details on the encoder architecture, refer to Supporting Information E . Part B shows how the embeddings of Hamiltonians with different spectra are scattered in a low-dimensional space using t-SNE. The spatial coherence of colour reflects how well the encoder groups spectrally similar matrices. Indices denote regions of a continuous spectral family rather than distinct physical classes; t-SNE axes are in arbitrary units (perplexity set to 20). Matrices sharing an eigenvalue spectrum are mapped to near-coincident points despite differing in structure (mean pairwise cosine distance 0.017, n=1268 spectral classes over 44,514 matrices), while matrices with distinct spectra are well separated (0.681).
Figure 3: Summary results on the synthetic-Hamiltonian dataset. Panel A shows the test-set results for both the state-propagation error and runtime predictions. For training, we transform the targets: the runtime is log-transformed, while the error is mapped to a normal distribution using the QuantileTransformer from scikit-learn [ 55 , 51 ] . We report the mean per-Hamiltonian Spearman correlation coefficient ρ between true and predicted values, together with the mean per-Hamiltonian median absolute percentage error, on the original and transformed scales, to assess how well the transformed results transfer to the original scale. The vertical dashed line marks the dimensions present only in the test set. The third-column panels show scatter plots of predicted against true values. Panel B shows the results of the classical SKQD simulations using the statevector method of the AerSimulator . The plots compare the error and runtime of the model-suggested configuration ( Minimum Predicted Error ) with those of a Theoretical Δt configuration and a Random Configuration .
Figure 4: Summary results on the molecular-Hamiltonian dataset. Panel A shows the predicted versus true runtime and absolute error for the two test sets, in-distribution (ID) and out-of-distribution (OOD). The shaded area in the error plot highlights the samples whose true and predicted errors fall beyond chemical accuracy (CA =1.6×10−3 Ha). Panel B reports, for the runtime, the Spearman correlation coefficient and the median absolute percentage error (med. APE) between true and predicted runtimes on the ID and OOD test sets; and, for the error, the Spearman correlation coefficient between true and predicted absolute errors together with the balanced accuracy for classifying whether a configuration yields a prediction below or above chemical accuracy.
Figure 5: Out-of-grid circuit parameter generalization. Part A shows the drift of the predicted runtime from the true runtime observed during the hardware execution. Part B plots the predicted error and the observed error, expressed as distance from chemical accuracy. The labels represent the number of qubits of the molecule. Part C is the table with the details of the out-of-grid circuit parameter configurations.
Feature set
Composition
Dimension
Estimator
Baseline
m , Δt
2
Random forest
Matrix features
130 matrix descriptors +m , Δt
132
Random forest
Encoder features
PCA-50 of the 768-dim embedding +m , Δt
52
Random forest
Table 1: Feature representations and estimators compared on the synthetic Hamiltonians dataset. Every representation is concatenated with the two configuration parameters, the Krylov subspace dimension m and the propagation timestep Δt . The encoder embedding is reduced by PCA fitted on the training split only.
Table 2: Feature representations and estimators compared on the molecular Hamiltonians. The physics block holds four molecule features ( n_electrons , n_orbitals , n_qubits and log10 of the configuration-interaction dimension); the config block holds the six SqDRIFT circuit features: num_groups , num_rand , and four entries ( t1 – t4 ) representing a fixed-length encoding of the evolution-time list. The two time configurations considered are [1,2,3] and [1,2,3,4] ; the former is padded to [1,2,3,0] so that both configurations have the same dimensionality. In both cases, t1-t3 are the same, thus reducing the number of effective variables to 3 informative ones ( num_groups , num_rand , m), with m∈{3,4} being the Krylov dimension and Δt=1 . The 768-dimensional Hamiltonian embedding enters either in full or reduced by PCA to eight components, fitted on the training split only. Every regression feature set includes the config block; the classification panel additionally fits two feature sets that isolate it.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Split
Idx
Class
Formula
Norb
Nelec
Nq
Circuit depth
TD GS est. (Ha)
Train
30
Alkanes and cycloalkanes
C 4 H 8
16
22
32
716±237
−156.180
Train
51
Alkenes and dienes
C 2 H 4
10
10
20
492±188
−78.129
Train
63
Alkenes and dienes
C 5 H 8
16
22
32
489±193
−194.074
Train
101
Alkynes
C 3 H 4
12
14
24
317±139
−115.994
Train
102
Alkynes
C 4 H 6
14
18
28
354±148
−155.021
Train
106
Alkynes
C 4 H 6
14
18
28
547±202
−155.033
Appendix
Table S1: The 50 molecular Hamiltonians. Columns: dataset split; global molecule index; chemical class; molecular formula; number of active-space spatial orbitals ( Norb ); number of active electrons ( Nelec ); number of qubits ( Nq ); mean SqDRIFT two-qubit gate depth with standard deviation; Truncated Davidson ground-state energy estimate (Hartree). The estimated error of this reference energy is between 10−6 and 10−4 , thus below chemical accuracy at 1.6×10−3 Ha.
Median APE (%)
Feature set
Train
Original
Transf.
ρ
p
Target: state-propagation error
Baseline ( m , Δt )
all- N
84.9
116.0
0.660
<0.01
Baseline ( m , Δt )
OOD
68.4
84.7
0.681
<0.01
Matrix features (132)
all- N
12.3
8.5
0.990
<0.01
Matrix features (132)
OOD
15.8
10.4
0.983
<0.01
Appendix
Table S2 : Model and feature comparison. For each target we report the median absolute percentage error (APE) on the original scale and on the transformed scale in which the regressors were fitted (quantile-transformed error, log10 runtime), together with Spearman’s rank correlation ρ and its two-sided p -value. All numbers are computed on the held-out test set, pooled over all matrix dimensions N∈{8,…,1024} . Train indicates the training split: all- N uses all dimensions, OOD restricts training to N≤128 , leaving N∈{256,512,1024} entirely unseen ( Table S3 ). All estimators are random forests (500 trees, maximum depth 20). Note that the two training variants fit their own quantile transform, so transformed-scale APE is comparable within a variant but not across them; original-scale APE and ρ are comparable throughout.
all- N
OOD
Train
Test
Train
Test
N
Samples
Matrices
Samples
Matrices
Samples
Matrices
Samples
Matrices
8
54 504
3 535
15 740
1 010
54 504
3 535
15 740
1 010
16
101 472
3 469
28 117
959
101 472
3 469
28 117
959
32
118 592
2 750
34 235
791
118 592
2 750
34 235
791
64
104 851
2 496
27 914
664
104 851
2 496
27 914
664
Appendix
Table S3 : Composition of the synthetic-Hamiltonian dataset by matrix dimension N . all- N covers every dimension, while OOD restricts the training matrices to N≤128 , leaving the three largest dimensions unseen during training. Both variants are evaluated on the same test split. Training and test matrices are disjoint.
Figure S1 : Borda counts from the classical SKQD simulations. The first plot ranks the configurations, per dimension, by the state-propagation error ϵprop of the runs. The second plot shows the configuration rankings by the runtime of each configuration.
N
n
median m
mode m
median Δt
mode Δt
16
10
15
15
0.5
0.5
32
10
25
25
0.5
0.5
64
10
12.5
10
0.5
0.5
128
10
17.5
15, 20
0.5
0.5
256
10
25
25
0.5
0.5
512
10
11
7
1.0
1.0
Appendix
Table S4 : Configurations selected by the surrogate on the synthetic Hamiltonians. For each matrix dimension N we take the n=10 test matrices used in the classical Krylov-Arnoldi simulations and record the Minimum Predicted Error configuration, the (m,Δt) pair minimizing the predicted classical state-propagation error ϵ^prop . We report the median and the most frequent (mode) values of the selected m and Δt . The selected Δt clusters at 0.5 – 1.0 rather than at the grid minimum 0.01 .
nonzeros per row (mean, std, min, max); per column (mean, std, min, max, median, q25 , q75 , IQR, CV)
13
Diagonal
S[diag(H)]
12
Off-diagonal (signed)
S[Hi=j] , plus mean and max of ∣Hi=j∣
14
Off-diagonal magnitude
S[∣Hi=j∣]
12
Off-diagonal real part
S[ℜHi=j]
12
Appendix
Table S5 : The 130 handcrafted matrix descriptors. S[⋅] denotes the block of twelve summary statistics applied to the entries of its argument: mean, standard deviation, variance, minimum, maximum, skewness, excess kurtosis and the percentiles p5,p25,p50,p75,p95 . Together with the two grid parameters (m,Δt) these form the 132 inputs of the matrix-feature surrogate. Of the 130 descriptors, 112 take distinct values on the benchmark; the remainder are constant or coincide exactly with another descriptor.
Config
Num. groups
Num. rand
m ( Δt=1 )
Num. samples
Total samples
A1
[15]
1000
[1.0, 2.0, 3.0]
500
1.5M
A2
[15]
1000
[1.0, 2.0, 3.0, 4.0]
375
1.5M
A3
[20]
1000
[1.0, 2.0, 3.0]
500
1.5M
A4
[20]
1000
[1.0, 2.0, 3.0, 4.0]
375
1.5M
B1
[15]
100
[1.0, 2.0, 3.0]
5000
1.5M
B2
[15]
100
[1.0, 2.0, 3.0, 4.0]
3750
1.5M
Appendix
Table S6 : SqDRIFT configuration grid. The detailed combinations of parameters used in the benchmarking-molecules experiments.
Figure S2 : The quantum hardware runs. The first column of the figure shows the distribution of the ground state approximation errors from the SqDRIFT circuits, the error distribution per configuration and the error distribution per the number of qubits. The second column shows the runtime distributions, overall, per SqDRIFT configuration and per number of qubits.
Nq
FCI dimension
Recovered (median)
Coverage
20
63,504
61,287
0.97
22
213,444
167,450
0.78
24
627,264
305,138
0.49
26
1.7×106
472,286
0.29
28
4.0×106
649,173
0.16
30
9.0×106
665,982
0.07
Appendix
Table S7 : Recovered-subspace coverage of the FCI space, by qubit count. For each system size we report a representative full configuration-interaction (FCI) dimension, (N↑Norb)(N↓Norb) , the median recovered-subspace dimension (merged recovered configurations passed to the SQD diagonalization step, final iteration), and their ratio. All runs use a fixed budget of 1.5×106 samples. Coverage falls from near-unity at 20 qubits to ∼10−5 for the largest molecules, producing the two modes of the error distribution in Figure S2 .
Figure S3 : t-SNE plot of the embeddings of the 50 molecular Hamiltonians.
Balanced accuracy (chance 0.500 )
Spearman ρ (null 0 )
Feature set
Estimator
ID test
OOD test
ID test
OOD test
Physics + config
OLS
0.727
0.521
0.730
0.846
Ridge
N/A
N/A
0.723
0.856
RF
0.617
0.542
0.642
0.643
XGBoost
0.650
0.528
0.626
0.774
LightGBM
0.655
0.500
0.608
0.699
Appendix
Table S8 : Molecular Hamiltonians model selection. The classification task learns and predicts whether a configuration yields an error below or above chemical accuracy at 1.6×10−3 Ha and is evaluated using Balanced accuracy . The regression task learns and predicts the QPU runtime t and the error ϵ , and is evaluated using Spearman ρ , the rank correlation between true and predicted values. Feature sets all include the six circuit configuration parameters (config) unless otherwise noted; “physics” features include molecule data such as the no. of electrons, the no. of orbitals and the log of the full Hilbert space dimension. OLS pairs the regressor with its normal-CDF classification head, so the two columns of that row share one underlying model. Bold marks the models carried forward, † a ρ not significant at α=0.05 . The last two rows are the feature-free classification baselines; a balanced accuracy of exactly 0.500 means the classifier assigns every sample to one class.
Figure S4 : Summary results on the molecular Hamiltonians’ dataset . Panel A shows the predicted vs true runtime and absolute values of errors for the three sets: train, in-distribution test and out-of-distribution test set. The shaded area in the errors plot highlights the samples whose true and predicted errors fall beyond chemical accuracy (CA= 1.6×10−3 ). Panel B shows the Spearman correlation coefficient and the median absolute percentage error (med. APE) between the true and predicted runtimes, and the Spearman correlation coefficient between the true and predicted absolute errors on the in-distribution and out-of-distribution test sets, together with the balanced accuracy for classifying whether a configuration’s prediction falls below or above chemical accuracy.
IBM Quantum, IBM Japan, Tokyo 103-8510, Japan · Mitsubishi Chemical Corporation, Science & Innovation Center, 1000 Kamoshida-cho, Aoba-ku, Yokohama 227-8502, Japan · Quantum Computing Center, Keio University, Hiyoshi 3-14-1, Kohoku, Yokohama 223-8522, Japan +5
Qclairvoyance Quantum Labs, Secunderabad, TG 500094, India. · The University of Arizona, Tucson, AZ 85721, USA. · Indian Institute of Technology Bombay, Mumbai, MH 400076, India.