Latent Style-based Quantum Wasserstein GAN for Drug Design
Authors: Julien Baglio, Yacine Haddad, Richard Polifka
Organizations: Center for Quantum Computing and Quantum Coherence (QC2), Department of Physics, University of Basel, Klingelbergstrasse 82, CH-4056 Basel, Switzerland · QuantumBasel, Schorenweg 44b, CH-4144 Arlesheim, Switzerland · Laboratory for High Energy Physics (LHEP), Physics Institute, Faculty of Sciences, University of Bern, Sidlerstrasse 5, CH-3012 Bern, Switzerland · Open Quantum Institute and Quantum Technology Initiative, CERN, Esplanade des Particules 1, CH-1211 Geneva 23, Switzerland
The development of new drugs is a tedious, time-consuming, and expensive process, for which the average costs are estimated to be up to around $2.5 billion. The first step in this long process is the design of the new drug, for which de novo drug design, assisted by artificial intelligence, has blossomed in recent years and revolutionized the field. In particular, generative artificial intelligence has delivered promising results in drug discovery and development, reducing costs and the time to solution. However, classical generative models, such as generative adversarial networks (GANs), are difficult to train due to barren plateaus and prone to mode collapse. Quantum computing may be an avenue to overcome these issues and provide models with fewer parameters, thereby enhancing the generalizability of GANs. We propose a new style-based quantum GAN (QGAN) architecture for drug design that implements noise encoding at every rotational gate of the circuit and a gradient penalty in the loss function to mitigate mode collapse. Our pipeline employs a variational autoencoder to represent the molecular structure in a latent space, which is then used as input to our QGAN. Our baseline model runs on up to 15 qubits to validate our architecture on quantum simulators, and a 156-qubit IBM Heron quantum computer in the five-qubit setup is used for inference to investigate the effects of using real quantum hardware on the analysis. We benchmark our results against classical models as provided by the MOSES benchmark suite.
Figures & tables
Figure 1: Schematic representation of the quantum-augmented drug-design pipeline. The model comprises a pre-trained VAE on the MOSES dataset using SMILES input (in purple), that embeds the original molecules into a low-dimensional latent space, and a quantum GAN with a quantum generator (in red) and a classical discriminator (in blue). The features extracted by the VAE from the input dataset are used for the training of the GAN. The final generated SMILES molecules are reconstructed by inversely transforming the features generated by the quantum generator using the pre-trained decoder of the VAE, and finally validated using RDKit.
Figure 2: Styled simple circuit with five qubits and one layer.
Figure 3: Styled basic entangled layer (BEL) circuit with five qubits and one layer. In addition there is a final set of one-qubit RY gates on each qubit to rotate all qubits in the computational basis before performing the final measurement operation.
Metrics
Nominal VAE
Tuned VAE
Learning rate (lr)
3×10−4
Scheduled 10−3→10−5
Encoder nl
1
1
Encoder Dh
256
512
Decoder nl
3
4
Decoder Dh
512
512
Decoder dropout
0
0.75
Table 1: VAE hyperparameter settings comparison between nominal (MOSES default) and tuned settings.
Metrics
Train set
Nominal VAE
Tuned VAE
IntDiv
0.898
0.896
0.896
W(LogP)
0.091
0.087
0.079
W(SA)
0.268
0.275
0.251
W(QED)
0.027
0.025
0.022
W(Weight)
42.8
43.9
41.8
Nparams
–
4,271,250
6,472,082
Table 2: Metrics derived from the training set and sets generated by the nominal and tuned VAE. Internal diversity is computed as the mean over the respective samples, whereas the remaining four metrics are the average Wasserstein distances between the sample in question and the test sample for the corresponding metric.
Figure 4: Distributions of metrics derived from the input train set and the generated sets using the nominal and tuned VAE decoder.
Figure 5: Comparison of the loss functions for the nominal (left) and tuned (right) GANs. Each model was run with five different seeds, the shaded band denotes a ±1σ deviation.
Metrics
Train set
Tuned VAE
Nominal GAN
Tuned GAN
ϵd
0.982
0.605 ± 0.017
0.510 ± 0.007
0.481 ± 0.030
ϵv
1.000
0.868 ± 0.006
0.911 ± 0.006
0.917 ± 0.012
ϵu
1.000
0.993 ± 0.001
0.986 ± 0.002
0.986 ± 0.003
Novelty
0.000
0.461 ± 0.026
0.501 ± 0.015
0.536 ± 0.015
Filters
0.734
0.728 ± 0.002
0.723 ± 0.003
0.721 ± 0.011
IntDiv
0.898
0.896 ± 0.000
0.902 ± 0.002
0.898 ± 0.004
Table 3: Metrics results for the VAE and classical GANs. All GANs use the tuned VAE for the latent space representation. Unlike the other cases, the nominal GAN uses the nominal GAN hyperparameter settings as discussed in the text. The quoted standard deviation comes from running each scenario five times with different seeds for the initialization of the trainable parameters.
Figure 6: Comparison of the metrics for the models used in the analysis. For the VAE, the tuned version is used, for the GAN, not only the tuned VAE is used but also the GAN is tuned itself. Each model was run with five different seeds, the shaded band is defined as ±1σ deviation.
Figure 7: Comparison of the loss functions for the nominal (left) and tuned (right) quantum GANs with the BEL circuit. Each model was run with five different seeds, the shaded band is defined as ±1σ deviation.
Metrics
Tuned GAN
Simple QGAN
BEL QGAN
Nparams
705,162
20
110
ϵd
0.481±0.031
0.529±0.003
0.523±0.003
ϵv
0.917±0.012
0.831±0.002
0.875±0.003
ϵu
0.986±0.003
0.994±0.000
0.993±0.001
Novelty
0.536±0.015
0.572±0.003
0.548±0.003
IntDiv
0.898±0.004
0.882±0.000
0.883±0.000
Table 4: Comparison of the performance of the two types of quantum GANs with respect to the classical GAN. All models use tuned VAE and GAN training parameters. The quantum circuit use five qubits, two layers and dual readout to produce a latent vector of dimension 10.
Figure 8: Comparison of the metrics for the two quantum circuits and the classical GAN. All models use tuned VAE and GAN training parameters. The quantum circuit uses five qubits, two layers and dual readout to produce a latent vector of dimension 10. Each model was run with five different seeds and the shaded band is defined as ±1σ deviation.
Figure 9: Distribution of latent vectors for the train, test, classical GAN and BEL QGAN samples.
Latent Dimension
GAN
Simple QGAN
BEL QGAN
( Nparams )
Readout
nqb
nl=2
nl=4
nl=2
nl=4
nl=6
( npar )
( npar )
( npar )
( npar )
( npar )
10
Reference
Dual
5
-1.00
-1.01
-0.26
-0.27
(20)
(40)
(110)
(210)
(705,162)
Single
10
-0.84
-0.83
-0.10
-0.08
-0.11
(40)
(80)
(220)
(420)
(620)
Table 5: Average ⟨Z0⟩ comparison of different scenarios between classical GANs and QGANs using different setup for the QGANs. The numbers in parenthesis in the first column are the number of trainable parameters and reflect the model capacity (i.e. complexity).
Metrics
Classical GAN
QGAN (noiseless)
QGAN (ibm_kingston)
ϵd
0.689
0.892
0.892
ϵv
0.947
0.934
0.955
ϵu
0.990
1.000
1.000
Novelty
0.310
0.338
0.336
IntDiv
0.915
0.881
0.890
Filters
0.719
0.706
0.729
Table 6: Metrics comparison calculated with 2,500 generated molecules, between classical GAN, the style QGAN on noiseless simulator, and the style QGAN inference on ibm_kingston. The quantum generator uses nqb=5 , a number of layers nl=2 , and dual readout.
Figure 10: Distributions of LogP, SA, and QED metrics for the generation of 2,500 molecule using a classical GAN, the style QGAN ran noiseless simulator, and the style QGAN ran on ibm_kingston. The quantum generator uses nqb=5 , a number of layers nl=2 , and dual readout.
Figure 11: Random selection of the 2,500 generated molecules sampled on ibm_kingston, passing the RDKit filters and all new compared to both the training and test dataset. The purple cluster contains molecules with LogP values compatible with injectable drugs; the pink cluster contains molecules with LogP values ideal for oral ingestion; the green cluster contains all other molecules with LogP not in the previous categories and still viable drug candidates with ∣LogP∣<5 .
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Comparison of metrics distributions between the input train and test sets.
Metrics
Train set
Test set
IntDiv
0.898
0.889
W(LogP)
0.091
–
W(SA)
0.268
–
W(QED)
0.027
–
W(Weight)
42.8
–
Appendix
Table 7: Comparison of metrics between the input train and test sets.
Figure 13: VAE loss function as a function of number of training epochs for the nominal VAE.
Nep
100
250
800
1,000
GPU runtime
9m 10s
17m 24s
48m 49s
52m 35s
IntDiv
0.892
0.919
0.896
0.896
W(LogP)
0.190
0.030
0.048
0.048
W(SA)
0.258
0.2646
0.2796
0.2751
W(QED)
0.084
0.090
0.096
0.097
W(Weight)
67.6
51.2
43.8
43.9
Appendix
Table 8: VAE metrics comparison for the four Nep points. The Wasserstein distance is measured between the test and generated samples. While the internal diversity is maximized, all other metrics are minimized. We also report the GPU runtime for comparison.
Figure 14: Metrics distributions for models trained with different number of epochs. Stabilization of the results starts with 800 epochs.
Ngenerated
30,000
300,000
1,000,000
GPU runtime
52m
1h 37m
2h 12m
Sampling runtime
10m
55m
1h 30m
ϵd
0.479
0.168
0.120
ϵv
0.889
0.704
0.609
ϵu
0.996
0.982
0.977
NCanonical
12,725
34,763
71,118
Appendix
Table 9: Metrics comparison for the nominal VAE training for three different sizes of sets of generated molecules. We also report runtime on the GPU.
lr max
3×10−4
1×10−4
1×10−2
5×10−5
5×10−3
1×10−3
lr min
3×10−4
1×10−6
1×10−4
5×10−5
5×10−3
1×10−5
ϵd
0.479
0.805
0.938
0.842
0.936
0.417
ϵv
0.889
0.660
0.018
0.626
0.041
0.957
ϵu
0.996
0.986
0.956
0.985
0.940
0.997
Novelty
0.237
0.702
0.925
0.768
0.856
0.119
IntDiv
0.896
0.893
0.887
0.891
0.929
0.898
Appendix
Table 10: Comparisons of the results of VAE learning rate tuning, the first column corresponds to the nominal scenario. Individual scenarios are defined by the learning rate minima/maxima at the top of the Table.
Figure 15: Metrics distributions for models trained with different learning rates. Clear breakdown for the 0.01-0.0001 and 0.005 scenarios is visible.
nl
1
2
1
1
Dh
256
256
128
512
Dropout
0.0
0.5
0.0
0.0
ϵd
0.479
0.548
0.475
0.5321
ϵv
0.889
0.838
0.888
0.8552
ϵu
0.996
0.994
0.995
0.9933
Novelty
0.237
0.349
0.238
0.327
Appendix
Table 11: Comparisons of the results of VAE encoder parameter tuning, the first column corresponds to the nominal scenario. The scenario variation is highlighted at the top of the table. The dropout is active only for a scenario with more than one encoder layer, in such case value of 0.5 was used.
Figure 16: Metrics distributions for models trained with different encoder settings. Results are very similar between the different scenarios.
nl
3
2
4
3
3
3
3
Dh
512
512
512
256
1024
512
512
Dropout
0.0
0.0
0.0
0.0
0.0
0.25
0.75
ϵd
0.479
0.590
0.487
0.718
0.416
0.480
0.556
ϵv
0.889
0.818
0.869
0.718
0.944
0.911
0.875
ϵv
0.996
0.990
0.996
0.988
0.997
0.996
0.994
Novelty
0.237
0.429
0.240
0.573
0.126
0.248
0.378
Appendix
Table 12: Comparisons of the results of VAE decoder parameter tuning, the first column corresponds to the nominal scenario. The scenario variation is highlighted at the top of the table. The default decoder dropout is zero regardless the number of layers.
Figure 17: Metrics distributions for models trained with different decoder settings. Results are very similar for the different scenarios.
Figure 18: Distributions of 10 latent vectors used as inputs of the VAE decoder.
Figure 19: Metric distribution obtained from SMILES sets which were generated by the VAE decoder from various latent vector distributions. Additional scenario – VAE(gen) – corresponds to a set which is generated by the standard MOSES sampling procedure. The distributions are normalized to unit area.
Figure 20: Correlations between individual latent vectors for the training set, test set, classical GAN training output, and BEL QGAN training output, all for a latent dimension of 10.
Latent dimension
QGAN
Table
Figures
nqb
nl
Readout
10
5
2
Dual
14
8
10
5
4
Dual
15
21
10
10
2
Single
16
22
10
10
4, 6
Single
17
23
20
10
2
Dual
18
24
Appendix
Table 13: Summary of the different scenarios and links to the corresponding individual tables and figures.
Metrics
Classical tuned
Styled simple QGAN
Z0
styled BEL QGAN
Z0
Nparams
705,162
20
110
ϵd
0.481±0.031
0.529±0.003
+1.56
0.523±0.003
+1.36
ϵv
0.917±0.012
0.831±0.002
−7.21
0.875±0.003
−3.43
ϵu
0.986±0.003
0.994±0.000
+3.50
0.993±0.001
+2.90
Novelty
0.536±0.015
0.572±0.003
+2.40
0.548±0.003
+0.82
IntDiv
0.898±0.004
0.882±0.000
−4.60
0.883±0.000
−4.21
Appendix
Table 14: Tuned classical GAN vs QGANs with nqb=5 , nl=2 , and dual readout.
Metrics
Classical tuned
Styled simple QGAN
Z0
Styled BEL QGAN
Z0
Nparams
705,162
40
210
ϵd
0.481±0.031
0.526±0.003
+1.46
0.522±0.004
+1.32
ϵv
0.917±0.012
0.830±0.002
−7.24
0.876±0.004
−3.36
ϵu
0.986±0.003
0.994±0.000
+3.42
0.993±0.001
+2.80
Novelty
0.536±0.015
0.577±0.003
+2.72
0.548±0.004
+0.76
IntDiv
0.898±0.004
0.882±0.000
−4.66
0.883±0.000
−4.21
Appendix
Table 15: Tuned classical GAN vs QGANs with nqb=5 , nl=4 , and dual readout.
Figure 21: Metrics distributions for classical and quantum models. All GANs are trained with a latent dimension of 10. The quantum GANs use nqb=5 , nl=4 , and dual readout.
Metrics
Classical tuned
Styled simple QGAN
Z0
Styled BEL QGAN
Z0
Nparams
716,692
40
220
ϵd
0.481±0.031
0.514±0.004
+1.06
0.514±0.002
+1.06
ϵv
0.917±0.012
0.833±0.003
−6.99
0.887±0.003
−2.45
ϵu
0.986±0.003
0.994±0.000
+3.27
0.992±0.001
+2.45
Novelty
0.536±0.015
0.586±0.003
+3.34
0.548±0.002
+0.82
IntDiv
0.898±0.004
0.882±0.000
−4.68
0.884±0.000
−3.99
Appendix
Table 16: Tuned classical GAN vs QGANs with nqb=10 , nl=2 , and single readout.
Figure 22: Metrics distributions for classical and quantum models. All GANs are trained with a latent dimension of 10. The quantum GANs use nqb=10 , nl=2 , and single readout.
Metrics
Classical tuned
Styled simple QGAN
Z0
Styled BEL QGAN
Z0
styled BEL
Z0
nl
-
4
4
6
Nparams
705,162
80
420
620
ϵd
0.481±0.031
0.515±0.005
+1.08
0.516±0.005
+1.13
0.515±0.003
+1.10
ϵv
0.917±0.012
0.833±0.002
−6.95
0.887±0.003
−2.42
0.888±0.002
−2.41
ϵu
0.986±0.003
0.994±0.000
+3.26
0.992±0.001
+2.48
0.992±0.001
+2.62
Novelty
0.536±0.015
0.586±0.002
+3.37
0.549±0.002
+0.87
0.550±0.001
+0.93
Appendix
Table 17: Tuned classical GAN vs QGANs with nqb=10 , nl=4,6 , and single readout.
Figure 23: Metrics distributions for classical and quantum models. All GANs are trained with a latent dimension of 10. The quantum GANs use nqb=10 , nl=4 and 6, and single readout.
Metrics
Classical tuned
Styled simple QGAN
Z0
Styled BEL QGAN
Z0
Latent dimension
20
20
20
Nparams
716,692
40
220
ϵd
0.526 ± 0.022
0.507 ± 0.003
−0.85
0.519 ± 0.005
−0.32
ϵv
0.901 ± 0.008
0.911 ± 0.004
+1.23
0.910 ± 0.002
+1.22
ϵu
0.990 ± 0.002
0.991 ± 0.001
+0.25
0.991 ± 0.001
+0.05
Novelty
0.488 ± 0.018
0.484 ± 0.001
−0.22
0.484 ± 0.002
−0.25
Appendix
Table 18: Quantum GANs with latent dimension of 20, nqb=10 , nl=2 , and dual readout.
Figure 24: Metrics distributions for classical and quantum models. All GANs are trained with a latent dimension of 20. The quantum GANs use nqb=10 , nl=2 , and dual readout.
Metrics
Classical tuned
Styled simple QGAN
Z0
Styled BEL QGAN
Z0
Latent dimension
30
30
30
Nparams
728,222
60
330
ϵd
0.500 ± 0.066
0.496 ± 0.003
−0.06
0.504 ± 0.002
+0.07
ϵv
0.884 ± 0.032
0.895 ± 0.002
+0.32
0.903 ± 0.003
+0.57
ϵu
0.990 ± 0.004
0.992 ± 0.001
+0.42
0.991 ± 0.001
+0.39
Novelty
0.504 ± 0.018
0.476 ± 0.003
−1.55
0.471 ± 0.003
−1.81
Appendix
Table 19: Quantum GANs with a latent dimension of 30, nqb=15 , nl=2 , and dual readout.
Figure 25: Metrics distributions for classical and quantum models. All GANs are trained with a latent dimension of 30. The quantum GANs use nqb=15 , nl=2 , and dual readout.
Latent Dimension
10
20
Z0
30
Z0
Nparams
705,162
716,692
728,222
ϵd
0.481±0.031
0.526±0.022
+1.19
0.500±0.066
+0.25
ϵv
0.917±0.012
0.901±0.008
−1.16
0.884±0.032
−0.96
ϵu
0.986±0.003
0.990±0.002
+1.59
0.990±0.004
+0.84
Novelty
0.536±0.015
0.488±0.018
−2.04
0.504±0.018
−1.39
IntDiv
0.898±0.004
0.893±0.002
−1.19
0.896±0.003
−0.43
Appendix
Table 20: Classical GAN comparison of different latent dimensions.
Figure 26: Metrics distributions for classical models trained with different latent dimensions.
Molecular generation develops computational methods to create chemical structures and conformations for discovery, optimization, and structure elucidation.
Inovation to Implementation Laboratories, Central Pharmaceutical Research Institute, Japan Tobacco Inc., 1-1 Murasaki-cho, Takatsuki, 569-1125, Japan. · D-Wave Systems Inc., 3033 Beta Ave, Burnaby, V5G 4M9, Canada.
BMW Group, Munich, Germany · Department of Computer Science and Institute of Applied Mathematics, University of British Columbia (UBC), Vancouver, Canada · Stewart Blusson Quantum Matter Institute (QMI), Vancouver, Canada +1
Center for Quantum Information and Quantum Biology, The University of Osaka, 1-2 Machikaneyama, Toyonaka 560-0043, Japan · Graduate School of Engineering Science, The University of Osaka 1-3 Machikaneyama, Toyonaka, Osaka 560-8531, Japan