Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

Harsh Wadhwa; Rahul Bhowmick; Naipunnya Raj; Rajiv Sangle; Ruchira V. Bhat; Krishnakumar Sabapathy

quant-phMar 23, 2026

Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

Authors: Harsh Wadhwa, Rahul Bhowmick, Naipunnya Raj, Rajiv Sangle, Ruchira V. Bhat, Krishnakumar Sabapathy

Organizations: Quantum Lab, Fujitsu Research of India

Abstract

Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox ( $\texttt{QBET}$ ), a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias ( $\texttt{SB}$ ) and Expressivity ( $\texttt{EXP}$ ), for comparing across various models, and extend the analysis of $\texttt{SB}$ to generative and multiclass-classification tasks. We show that $\texttt{QBET}$ enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of $18$ qubits for embeddings ( $6$ qubits each for query, key, and value). We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the $\texttt{SB}$ metric and comparing their relative performance.

Figures & tables

Figure 1: Design and algorithmic workflow of QBET for model selection . The process consists of three stages: (i) Input , where a model (classical or quantum) is selected. This stage is data-agnostic and focuses solely on architectural specification rather than dataset-dependent characteristics; (ii) Execution , which involves converting the model into a Boolean classifier, generating Boolean functions, computing LZ complexity, and producing probability distributions over multiple trials; and (iii) Evaluation , where cumulative distribution functions (CDFs) are constructed, AUC and EXP values are derived to quantify the bias–expressivity trade-off, and top-

k

configurations are selected for downstream performance metric computation based on AUC .

Figure 2: Overview of QBET : input data types (molecular graphs, text sequences, images), tasks (generation and classification), and candidate models (Transformer Encoder, Self-Attention Mechanism-Generative Adversarial Network [SAM-GAN], Transformer Decoder). The model selection workflow involves three steps: (1) choosing the data, task, and model; (2) listing model variants such as classical, quantum, and hybrid approaches, initialization strategies, circuit or network types, and depth/activation choices; and (3) running the toolbox to identify the variant that achieves optimal performance using minimal resources.

Table 1: Top-10 and bottom-10 AUC ranked performance metrics for TF-Encoder image classification model configurations. Performance metrics are averaged across 10 independent initializations.

Model	Parameters Count of SAM block	AUC	EXP	Train Acc. (%)	Test Acc. (%)
Classical	15936	39.5038	0.0781	47.75 $\pm$ 0.40	47.48 $\pm$ 0.70
Hybrid	12924	40.1737	0.0609	47.53 $\pm$ 0.66	47.51 $\pm$ 0.66
Quantum	12954	40.1710	0.0601	47.60 $\pm$ 0.48	47.79 $\pm$ 0.60
Classical	12984	40.1582	0.0602	47.59 $\pm$ 0.44	47.51 $\pm$ 0.64
(HQC)	12984	40.1579	0.0600	47.54 $\pm$ 0.68	47.55 $\pm$ 0.76

Table 2: Performance comparison of classical and hybrid quantum-classical TF-Encoder configurations for image classification, showing parameter count, AUC, expressivity (EXP), and train/test accuracy. hybrid quantum-classical models selected from top-10 AUC values demonstrate competitive performance with comparable parameter efficiency. Performance metrics are averaged over 10 initializations

Figure 3: This heatmap illustrates the strength of correlations between performance metrics and bias-expressivity measures for TF-Encoder model for image classification task. The upper triangle displays the statistical significance of the observed relationships, highlighting the reliability of the experimental findings.

Model	Parameters Count of SAM block	AUC	EXP	Uniqueness (%)	F1 Score
Classical	648	41.9575	0.1355	17.76 $\pm$ 2.21	26.32 $\pm$ 2.50
Hybrid	684	42.7308	0.1089	18.36 $\pm$ 1.11	24.96 $\pm$ 1.38
Quantum	648	42.6167	0.1161	18.83 $\pm$ 1.11	25.56 $\pm$ 1.24
Classical	684	42.5963	0.1133	18.71 $\pm$ 1.59	24.90 $\pm$ 1.88
(HQC)	648	42.5546	0.1188	18.52 $\pm$ 0.82	25.12 $\pm$ 0.96
	684	42.5461	0.1166	19.04 $\pm$ 1.76	26.32 $\pm$ 2.58

Table 3: Performance comparison of classical and top-10 AUC value SAM-GAN model configurations for molecular graph generation, including parameter count, AUC, expressivity (EXP), uniqueness, and F1 score. Hybrid quantum-classical models demonstrate competitive or improved generative performance relative to the classical baseline under similar parameter constraints. Performance metrics are averaged over 10 initializations.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: LZ -Complexity distribution for a classical neural network, as described in the SAM-GAN model (Sec. D.2 ) using only one transformer block and a fully connected final layer with one output neuron. Low complexity functions are seen more frequently, while high complexity functions are fewer in number.

Figure 5: Architecture of the transformer model illustrating the encoder–decoder structure with multi-head attention, feed-forward layers, positional encoding, and output probability computation via linear and softmax layers.

Figure 6: Architecture of the transformer-based encoder for image classification. The workflow includes positional encoding, linear embedding of image patches, transformer encoder layers with single-head attention and multilayer perceptron blocks that outputs the class label. The single head attention can be implemented using classical or quantum self-attention mechanism.

Figure 7: SAM-GAN framework for molecular graph generation. The generator consumes a noise matrix (entropy source) and applies a Transformer encoder block which contains single - head attention (can be quantum or classical) followed by classical fully connected layers to produce hidden representations, which are then decoded through classical node and adjacency layers into a candidate molecular graph. A discriminator, composed of graph convolution and graph aggregation layers with a final fully connected layer, distinguishes real molecular graphs from generated ones. The system jointly trains the generator and discriminator to synthesize chemically plausible molecular structures.

Figure 8: Architecture of a transformer model for molecular SMILES generation. The input SMILES string is tokenized and embedded with positional encoding, followed by processing through a transformer decoder. The model incorporates masked single-head attention, which can be classical or quantum, and outputs probabilities via feed-forward layers and softmax.

Panel of Figure 9 — Figure 9: Quantum circuit block for Feature Mapping encoding scheme. The parametrized $R_{x}$ and $R_{y}$ rotations are applied on each qubit followed by $CNOT$ between neighboring qubits in a linear fashion

Panel of Figure 10 — Figure 10: Quantum circuit block for YZ circular ansatz. The parametrized $R_{y}$ and $R_{z}$ rotations are applied on each qubit followed by $CNOT$ between neighboring qubits in a linear periodic fashion. The blocks are often repeated to increase the entanglement and expressivity.

Figure 11: Workflow of the variants for the transformer encoder/decoder architecture.

Variant	Measurement @ Query/Key	Attention Method
Canonical	Pauli- $Z$ expectation value on a single qubit $\braket{Z_{0}}_{q/k}$	Gaussian Projection Method [ 44 ]
V1	Pauli- $Z$ expectation value on a single qubit $\braket{Z_{0}}_{q/k}$	Gaussian Projection Method [ 44 ]
V2	Pauli- $Z$ expectation value on a single qubit $\braket{Z_{0}}_{q/k}$	Post-processing + Softmax
V3	Expectation values $\braket{P_{i}}$ , $P_{i}\in\{2^{d_{\text{qubits}}}\}$ Pauli operators	Classical Attention (Appendix C.1 )
V4	Expectation values $\braket{P_{i}}$ , $P_{i}\in\{2d+1\}$ anti-commuting Pauli operators	Classical Attention (Appendix C.1 )

Appendix

Table 4: Variants for the transformer encoder/decoder architecture. All variants use the same YZ circular PQC ansatz. The Canonical variant uses feature mapping, while all other variants use amplitude encoding. The variants differ in the measurement strategy and attention computation method.

Figure 12: Workflow of the variants for the SAM+GAN framework.

Variant	Measurement @ Query/Key	Attention Method
Canonical	Pauli- $Z$ expectation value on a single qubit $\braket{Z_{0}}_{q/k}$	Gaussian Projection Method [ 44 ]
V1	Pauli- $Z$ expectation values $\braket{Z_{i}}$ for all qubits $i$	Classical Attention (Appendix C.1 )
V2	Expectation values $\braket{P_{i}}$ , $P_{i}\in\{2d+1\}$ anti-commuting Pauli operators	Classical Attention (Appendix C.1 )
V3	Pauli- $Z$ expectation value on a single qubit $\braket{Z_{0}}_{q/k}$	Post-processing + Softmax

Appendix

Table 5: Variants for the SAM+GAN framework. All variants use angle encoding and the same YZ circular PQC ansatz. The variants differ in the granularity of quantum measurements and the attention computation method.

Figure 13: Architecture illustrating the conversion of a Transformer-based core model into a Boolean classifier. The core model consists of self-attention and fully connected layers, which can be paired with either the Multi-class Classification Block (for performance metric evaluation) or the LZ Test Block (for computing AUC and expressivity). Both blocks are interchangeable depending on the task requirements.

Figure 14: Architecture illustrating the transformation of a Transformer-based core model for generation tasks into a Boolean classifier. The core model includes a attention block (masked attention in case of TF-Decoder) and fully connected layers, which can be paired with either the Generation Block (for performance metric computation) or the LZ Test Block (for AUC and expressivity evaluation). The self-attention block is replaced with an unmasked variant during LZ testing, and both blocks are interchangeable depending on task requirements.

Figure 15: This heatmap illustrates the strength of correlations between performance metrics and bias-expressivity measures for the SAM-GAN. The upper triangle displays the statistical significance of the observed relationships, highlighting the reliability of the experimental findings.

AUC	EXP	Uniqueness (%)	F1 Score
40.4123	0.1805	7.01 $\pm$ 1.57	11.20 $\pm$ 2.30
40.4231	0.1805	5.59 $\pm$ 1.67	8.98 $\pm$ 2.44
40.4237	0.1806	7.81 $\pm$ 3.23	12.34 $\pm$ 4.64
40.4260	0.1805	7.12 $\pm$ 2.08	11.04 $\pm$ 3.34
40.5348	0.1744	11.18 $\pm$ 1.24	19.36 $\pm$ 1.80
40.5581	0.1738	10.45 $\pm$ 0.56	18.22 $\pm$ 0.84

Appendix

Table 6: Bottom-10 SAM-GAN configurations and corresponding performance metrics

Model	Parameters Count of SAM block	AUC	EXP	Validity (%)	Uniqueness (%)	Novelty (%)	F1 Score
Classical	34656	39.1719	0.0808	79.45 $\pm$ 2.01	62.41 $\pm$ 1.29	29.94 $\pm$ 1.19	69.91 $\pm$ 1.54
Hybrid	31680	42.5136	0.0069	79.57 $\pm$ 1.17	62.68 $\pm$ 0.48	30.65 $\pm$ 0.67	70.12 $\pm$ 0.66
Quantum	31656	42.4999	0.0070	79.47 $\pm$ 1.18	62.50 $\pm$ 1.09	30.76 $\pm$ 1.04	69.97 $\pm$ 1.07
Classical	31632	42.4973	0.0093	77.13 $\pm$ 1.54	61.02 $\pm$ 0.75	30.90 $\pm$ 0.62	68.13 $\pm$ 0.97
(HQC)	31656	42.4918	0.0094	76.83 $\pm$ 1.25	61.25 $\pm$ 0.95	31.49 $\pm$ 0.94	68.16 $\pm$ 1.04
	31632	42.4484	0.0074	79.09 $\pm$ 0.76	62.93 $\pm$ 0.63	31.31 $\pm$ 1.40	70.09 $\pm$ 0.49

Appendix

Table 7: Performance metrics for the top-10 selected model configurations for molecular SMILES generation, compared against the corresponding classical variant. The table reports parameter count of the SAM block, AUC , EXP , validity, uniqueness, novelty, and F1 score. The hybrid quantum-classical configurations achieve higher validity, uniqueness, novelty, and F1 scores relative to the classical baseline, while operating under lesser parameter budgets. All performance metrics are averaged over 10 initializations.

Figure 16: This heatmap illustrates the strength of correlations between performance metrics and bias-expressivity measures for TF-Decoder. The upper triangle displays the statistical significance of the observed relationships, highlighting the reliability of the experimental findings.

AUC	EXP	Validity (%)	Uniqueness (%)	Novelty (%)	F1 Score
37.3303	0.1891	76.92 $\pm$ 0.99	61.43 $\pm$ 1.09	31.99 $\pm$ 1.37	68.30 $\pm$ 0.89
37.3305	0.1896	76.44 $\pm$ 1.39	61.10 $\pm$ 1.17	31.69 $\pm$ 1.57	67.91 $\pm$ 1.01
37.3346	0.1913	77.28 $\pm$ 1.27	61.43 $\pm$ 0.92	31.28 $\pm$ 1.04	68.44 $\pm$ 0.76
37.3442	0.1885	77.05 $\pm$ 1.50	61.32 $\pm$ 1.16	31.71 $\pm$ 0.97	68.29 $\pm$ 1.26
37.3473	0.1880	76.96 $\pm$ 1.01	61.33 $\pm$ 1.16	31.53 $\pm$ 1.11	68.25 $\pm$ 0.96
37.3511	0.1896	77.47 $\pm$ 1.39	61.73 $\pm$ 1.06	31.84 $\pm$ 1.17	68.71 $\pm$ 1.08

Appendix

Table 8: Performance Metrics: Validity, Uniqueness, Novelty, and F1 Score for bottom-10 TF-Decoder configurations

Quantum Circuit Expressibility

See all 12 papers

Quantum Adaptive Self-Attention for Quantum Transformer Models
Apr 5, 2025Chi-Sheng Chen, En-Jui KuoHybrid Quantum-Classical ML Transformer Attention
Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Hybrid Quantum Neural Networks: Theory, Implementations, and Applications
Aug 2, 2026Léo Monbroussou, Maniraman Periyasamy, Viacheslav Kuzmin +4Hybrid Quantum-Classical ML Variational Quantum Circuits
Terra Quantum AG, Kornhausstrasse 25, St. Gallen, 9000, Switzerland · Département de Physique Appliquée, Université de Genève, 24 Quai Ernest-Ansermet, Geneva, 1211, Switzerland
Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks
Sep 19, 2026Diego Iacopetta, Andrea GaspariniQuantum Neural Networks Transformer Interpretability
Query Machines B.V. Delft, The Netherlands

Apr 5, 2025J/K move · Enter open · S save

Quantum Adaptive Self-Attention for Quantum Transformer Models

Chi-Sheng Chen, En-Jui Kuo

Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

Hybrid Quantum-Classical ML Transformer Attention Variational Quantum Circuits

Open paper AskPDF arXiv

A recurring weakness in quantum machine learning (QML) is that reported ``quantum advantages'' are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly -- a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware -- which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in quantum machine learning -- a capacity-matched classical bottleneck with the same parameter budget -- and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal's intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum.

Abstract

Quantum Circuit Expressibility

Explore similar work

Quantum Adaptive Self-Attention for Quantum Transformer Models

Hybrid Quantum Neural Networks: Theory, Implementations, and Applications

Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks