quant-phFeb 28, 2025

Quantum-aware Transformer model for state classification

Authors: Przemysław Sekuła, Michał Romaszewski, Przemysław Głomb, Michał Cholewa, Łukasz Pawela

Organizations: Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Bałtycka 5, 44-100 Gliwice, Poland · University of Maryland, College Park, Department of Civil and Environmental Engineering, MD, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Entanglement is a fundamental feature of quantum mechanics, playing a crucial role in quantum information processing. However, classifying entangled states, particularly in the mixed-state regime, remains a challenging problem, especially as system dimensions increase. In this work, we focus on bipartite quantum states and present a data-driven approach to entanglement classification using transformer-based neural networks. Our dataset consists of a diverse set of bipartite states, including pure separable states, Werner entangled states, general entangled states, and maximally entangled states. We pretrain the transformer in an unsupervised fashion by masking elements of vectorized Hermitian matrix representations of quantum states, allowing the model to learn structural properties of quantum density matrices. This approach enables the model to generalize entanglement characteristics across different classes of states. Once trained, our method achieves near-perfect classification accuracy, effectively distinguishing between separable and entangled states. Compared to previous Machine Learning, our method successfully adapts transformers for quantum state analysis, demonstrating their ability to systematically identify entanglement in bipartite systems. These results highlight the potential of modern machine learning techniques in automating entanglement detection and classification, bridging the gap between quantum information theory and artificial intelligence.

Figures & tables

Figure 1: Schematic representation of the set of mixed quantum states Ω\Omega depicting separable states (SEP) ρsep\rho_{\mathrm{sep}} , bound entangled states (BOUND) σbound\sigma_{\mathrm{bound}} and negative partial transpose states (NPT) ξnpt\xi_{\mathrm{npt}} .
 PretrainingClassification
Group nameC2⊗C2\mathbb{C}^{2}\otimes\mathbb{C}^{2}C2⊗C3\mathbb{C}^{2}\otimes\mathbb{C}^{3}C3⊗C3\mathbb{C}^{3}\otimes\mathbb{C}^{3}C2⊗C2\mathbb{C}^{2}\otimes\mathbb{C}^{2}C2⊗C3\mathbb{C}^{2}\otimes\mathbb{C}^{3}C3⊗C3\mathbb{C}^{3}\otimes\mathbb{C}^{3}
sep4,000,0008,000,0006,000,0001,000,0001,000,0001,000,000
general-ent2,000,0008,000,0002,000,000300,0001,000,000500,000
werner-ent2,000,000 2,000,000300,000 500,000
max-ent2,000,000 2,000,000300,000 500,000
horodecki-bound  2,000,000  500,000
Table 1: Training data sizes for Pretraining and Classifier tasks. Numbers indicate samples used for each model configuration and data type. Group names correspond to the type of data (pure separable, general entangled etc.) described in Section 2 .
 UntrainedPretrained
Group nameC2⊗C2\mathbb{C}^{2}\otimes\mathbb{C}^{2}C2⊗C3\mathbb{C}^{2}\otimes\mathbb{C}^{3}C3⊗C3\mathbb{C}^{3}\otimes\mathbb{C}^{3}C2⊗C2\mathbb{C}^{2}\otimes\mathbb{C}^{2}C2⊗C3\mathbb{C}^{2}\otimes\mathbb{C}^{3}C3⊗C3\mathbb{C}^{3}\otimes\mathbb{C}^{3}
sep6.6863.7188.1010.2650.3640.419
general-ent6.7043.7168.0980.1890.1870.167
werner-ent6.645 8.1310.183 0.222
max-ent6.693 8.1020.296 0.449
horodecki-bound  8.104  0.120
Table 2: Averaged Hermitian distances for untrained, and fully pretrained models.
Group nameC2⊗C2\mathbb{C}^{2}\otimes\mathbb{C}^{2}C2⊗C3\mathbb{C}^{2}\otimes\mathbb{C}^{3}C3⊗C3\mathbb{C}^{3}\otimes\mathbb{C}^{3}
sep99.995%99.998%100%
general-ent100%100%100%
werner-ent100% 100%
max-ent100% 100%
horodecki-bound  100%
horodecki-ent  100%
Table 3: Accuracy of binary classification (entangled vs separated) for each data group.
More in this topic

Quantum Machine Learning

Quantum machine learning uses quantum circuits and quantum data to build learning algorithms and study their computational and statistical limits.

See all 256 papers

Explore similar work

CardsList
Oct 2, 2026

Generalization of Transformer-Based Neural Quantum States via In-Context Learning

Neural quantum states based on modern deep learning architectures have emerged as powerful representations for quantum many-body systems. In particular, Transformer-based neural quantum states provide expressive models capable of capturing long-range correlations, and their empirical generalization performance has recently been demonstrated. However, a theoretical understanding of their generalization behavior remains largely unexplored. In this paper, we develop a theoretical framework to analyze the generalization properties of Transformer-based neural quantum states under in-context learning. We establish a rigorous inference-time generalization error bound in terms of mean squared error (MSE), showing that the pointwise prediction error decreases inversely with both the number of in-context examples and the depth of the Transformer. We further show that the Transformer depth required to achieve this guarantee scales only linearly with the system size--namely, the number of particles in continuous systems or the number of qudits in discrete systems. Building on this result, we extend our analysis to full quantum states formulated as rank-one density operators, and derive MSE-based generalization bounds over both continuous and discrete domains under physical constraints. Finally, numerical simulations corroborate our theoretical analysis.

Zhen Qin, Qing Qu, Alfred O. Hero

Michigan Institute for Computational Discovery and Engineering, Department of Electrical Engineering and Computer Science and Department of Statistics, University of Michigan, Ann Arbor, MI 48109 USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109 USA

Neural Quantum StatesNeural Network GeneralizationIn-Context LearningQuantum Many-Body Physics
AskPDF
Sep 19, 2026

Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks

Deep learning models are powerful but opaque. As quantum machine learning matures, the field faces a defining choice: build quantum models that are equally opaque, or exploit the mathematical structure of quantum mechanics to make them inherently interpretable. We show that the latter is possible. By tracking quantum mutual information~(MI), entanglement entropy, and state fidelity through the layers of a Quantum Transformer Block (\qtb{}), a fully-coherent variational circuit with quantum analogues of both attention and feedforward, we gain direct insight into how the model processes information: which tokens it attends to, when correlations form, and why predictions fail. On four tasks with known dependency structure we show that (i)~learned MI matrices align with ground-truth task structure (AUC = 0.69\,{=}\,0.69 on lookup), (ii)~disabling entangling gates collapses accuracy from 100% to 15% while MI→0\to 0, proving entanglement is the mechanism, (iii)~accuracy and MI co-evolve during training (ρ = 0.92ρ\,{=}\,0.92 on lookup), and (iv)per-sample MI predicts prediction correctness on the conditional task with ROC AUC = 0.84\,{=}\,0.84. All results are validated on IBM Quantum hardware (ibm_kingston, Heronr2): the circuit's reasoning process, from product state through structured entanglement, is directly observable on a superconducting processor. These proof-of-concept results, obtained on small synthetic tasks, suggest that the physics of quantum computation can provide intrinsic interpretability signals with no direct classical counterpart, motivating study of whether this advantage persists at scale.

Diego Iacopetta, Andrea Gasparini

Query Machines B.V. Delft, The Netherlands

Quantum Neural NetworksTransformer Interpretability
AskPDF
Mar 23, 2026

Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox (QBET\texttt{QBET}), a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias (SB\texttt{SB}) and Expressivity (EXP\texttt{EXP}), for comparing across various models, and extend the analysis of SB\texttt{SB} to generative and multiclass-classification tasks. We show that QBET\texttt{QBET} enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of 1818 qubits for embeddings (66 qubits each for query, key, and value). We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the SB\texttt{SB} metric and comparing their relative performance.

Harsh Wadhwa, Rahul Bhowmick, Naipunnya Raj +3

Quantum Lab, Fujitsu Research of India

Quantum Circuit ExpressibilityQuantum Neural NetworksHybrid Quantum-Classical ML
AskPDF