Universal Quantum Transformer

Sungyong Chung; Alireza Talebpour

cs.AIApr 29, 2026

Universal Quantum Transformer

Authors: Sungyong Chung, Alireza Talebpour

Organizations: Grainger College of Engineering, Department of Civil and Environmental Engineering, University of Illinois Urbana-Champaign

Abstract

Classical continuous-space neural networks fundamentally struggle to lock into exact mathematical symmetries, such as modular arithmetic and non-commutative algebra. To approximate these discrete logical rules, they often rely on massive parameter scaling, resulting in stochastic instability even after delayed generalization phenomena known as grokking. Here, we introduce the Universal Quantum Transformer (UQT), a fundamentally novel, quantum-native computing architecture that uses the physical properties of multi-qubit systems as a universal inductive bias for exact mathematical and algebraic reasoning. Rather than translating classical neural mechanisms, our framework relies entirely on parameterized geometric phase embedding and $SU(2)$ wave-interference. We demonstrate that the quantum attention circuit, operating on a highly compact 5-qubit substrate, perfectly learns two highly distinct formal classes: cyclic modular arithmetic ( $\mathbb{Z}_{11}$ ) and non-Abelian algebra (the $S_4$ permutation group). While classical attention-based networks exhibit stochastic instability at convergence, the UQT achieves mathematically exact, deterministic generalization. We refer to this phenomenon as crystallization: a step beyond the well-known phenomenon of grokking. Crucially, this framework yields massive computational and memory advantages by theoretically bypassing the quadratic bottleneck of classical self-attention, and by logarithmically compressing the required representation dimension to eliminate the massive over-parameterization inherent to classical networks. Finally, we deploy this architecture on noisy intermediate-scale quantum (NISQ) hardware, proving its viability on current IBM Quantum computers. These results establish parameterized quantum topology as a universally superior physical substrate for exact artificial intelligence.

Explore similar work

Apr 5, 2025quant-ph

Quantum Adaptive Self-Attention for Quantum Transformer Models

A recurring weakness in quantum machine learning (QML) is that reported ``quantum advantages'' are seldom tested against a \emph{capacity-matched} classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly -- a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does \emph{not} help, and validation on real quantum hardware -- which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a \emph{single} encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely \emph{quantum} effect, we introduce a control rarely applied in quantum machine learning -- a capacity-matched classical bottleneck with the same parameter budget -- and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection \emph{bottleneck} (an \emph{architectural parsimony} principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a \emph{competitive} instantiation of this principle: its low-rank compression onto the signal's intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum.

Chi-Sheng Chen, En-Jui Kuo

Quantum Concept Bottleneck Models

Ask PDF

Apr 27, 2026quant-ph

Do Quantum Transformers Help? A Systematic VQC Architecture Comparison on Tabular Benchmarks

Variational quantum circuits (VQCs) are a leading approach to quantum machine learning on near-term devices, yet it remains unclear which circuit architecture yields the best accuracy-parameter trade-off on classical tabular data. We present a systematic empirical comparison of four VQC families -- multi-layer fully-connected (FC-VQC), residual (ResNet-VQC), hybrid quantum-classical transformer (QT), and fully quantum transformer (FQT) -- across five regression and classification benchmarks. Our key findings are: \textbf{(i)}~FC-VQCs achieve 90-96% of the

R^2

of attention-based VQCs while using 40-50% fewer parameters, and consistently outperform equal-capacity MLPs (mean

R^2{=}0.829

vs.\ MLP

_{720}

0.753

on Boston Housing, 3-seed average); \textbf{(ii)}~~FC-VQC's Type~~4 inter-block connectivity provides partial cross-token mixing that approximates the role of attention -- explicit quantum self-attention yields only marginal gains on most datasets while significantly increasing parameter count; \textbf{(iii)}~~expressibility saturates at circuit depth~~

{\approx}\,3

, explaining why shallow VQCs already cover the Hilbert space effectively; \textbf{(iv)}~LayerNorm on the fully quantum transformer improves classification accuracy, suggesting normalization is important when all operations are quantum; \textbf{(v)}~in our noise study on Boston Housing, FQT degrades gracefully under depolarizing noise while QT collapses. All results are validated across three random seeds. These findings provide practical architectural guidance for deploying VQCs on near-term quantum hardware.

Chi-Sheng Chen, En-Jui Kuo

Variational Quantum Circuits Variational Encoder

Ask PDF

Jul 1, 2026cond-mat.str-el

Holographic Quantum Transformer: A Generalist Neuro-Symbolic Architecture for Solving Frustrated Systems via Generative Attention

Simulating two-dimensional frustrated quantum matter is a grand challenge due to the sign problem and exponential Hilbert space complexity. In this work, we introduce the Holographic Quantum Transformer (HQT), a physics-inspired generative architecture that leverages global self-attention to resolve non-local entanglement patterns. We validate HQT on the square lattice

J_1-J_2

Heisenberg model. On the heavily frustrated

8 \times 8

lattice at the quantum critical point (

J_2=0.5

), HQT reaches a ground-state energy per site (

E/N

) of

\mathbf{-0.5001(1)}

, consistent with the expected finite-size scaling trend. Beyond numerical accuracy, HQT exhibits intrinsic physical awareness, autonomously recovering the underlying

J_2

interaction geometry through interpretable attention maps. Our central contribution is ``Holographic Transfer", a zero-shot size-extrapolation protocol with rapid alignment: a model trained on

8 \times 8

systems is directly projected onto larger

10 \times 10

lattices via continuous positional-embedding interpolation and head re-initialization, achieving high-fidelity initialization and rapid convergence. This zero-shot protocol yields an energy of

E/N = \mathbf{-0.49782(3)}

, statistically consistent with the variational state of the art while requiring no from-scratch training on the target lattice. Our results establish generative attention as a scalable paradigm for transferable quantum simulation.

Xingran Guo, Tiaojie Xiao, Jie Liu +1

Two-Dimensional Ising Model

Ask PDF