quant-phOct 25, 2026

Wave Is All You Need: A Quantum Language Model Based on Fully Connected Quantum Boltzmann Machine

Authors: Jun Liu

Organizations: School of Economics, Hunan University of Finance and Economics, Changsha 410205, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This work proposes a self-attention mechanism based on the fully connected quantum Boltzmann machine (FC-QBM), and further constructs a fully connected quantum Boltzmann machine language model (FC-QBM-LM) using the mechanism as the core engine. By performing unitary evolution on semantic quantum states, the mechanism implicitly implements positional encoding and self-attention computation, eliminating the need to decompose continuous semantic streams into discrete matrices for explicit matrix multiplication, with the measurement results naturally satisfying the normalization condition. Specifically, this work rigorously derives the computational principle of the self-attention mechanism based on FC-QBM, and demonstrates that this mechanism is mathematically equivalent to the classical self-attention mechanism. The mathematical principles governing the training and generation of FC-QBM-LM are thoroughly derived, and the two corresponding algorithms are subsequently developed. An 8-qubit FC-QBM-LM is finally constructed and trained on the corpus of Tang poetry. Under the 8-qubit setting, the quantum layer implements the entire positional encoding and self-attention computation pipeline with only 44 parameters. In the noiseless scenario, the model exhibits favorable convergence. Furthermore, the quantum circuit proposed in this work, which terminates with Y-axis rotation gates, achieves better convergence than the counterpart terminating with X-axis rotation gates. Under the small-sample regime and across eight distinct noise intensity levels, the model demonstrates strong noise robustness, retaining over 99% of its noiseless convergence performance. Experiments on five-character quatrain generation reveal that the model preliminarily exhibits three core capabilities: associative and divergent thinking, content generation across imagery and human-aligned word ordering.

Figures & tables

Figure 1: Self-Attention Mechanism Black Box
Figure 2
Figure 3
Figure 2: Quantum Circuit of 8-Qubit Fully Connected Quantum Boltzmann Machine Language Model
 
LayerParameterQuantitySubtotal
 bib_{i}8 
Quantum self-attention layerwijw_{ij}2844
 θi\theta_{i}8 
First classical linear layerb1,kb_{1,k}128 
 w1,k,lw_{1,k,\boldsymbol{l}}3276832896
Table 1: Parameter List of 8-Qubit Fully Connected Quantum Boltzmann Machine Language Model
Figure 3: System Block Diagram of 8-Qubit Fully Connected Quantum Boltzmann Machine Language Model
Figure 4: Computational Results of Model Convergence Experiments
Figure 5: Computational Results of Noise Robustness Experiments
Figure 6: Computational Results of Five-Character Quatrain Generation Experiments
More in this topic

Language Modeling

Language modeling develops systems that learn patterns in text and other sequences to understand, predict, and generate language.

See all 518 papers

Explore similar work

CardsList
Oct 8, 2026

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.

Deniz Yunus Göğüş

Language ModelingQuantum Machine LearningByte-Level Language ModelMemory-Augmented Language Models
AskPDF
Jun 10, 2026

Boltzmann Attention: Learnable Ising Couplings for Cooperative Attention

Attention mechanisms are central to modern sequence models, yet standard attention computes relevance primarily through individual query--key similarities. Although softmax normalization introduces competition among positions, a standard attention layer does not explicitly parameterize learnable interactions between attention decisions. This limits its ability to directly model cooperative or antagonistic co-attention structure within the attention mechanism itself. We propose Boltzmann attention, an energy-based generalization in which attention patterns are governed by an interacting Ising model. The method augments the usual data-dependent local fields with learnable pairwise couplings, allowing the model to represent inter-position correlations beyond those captured by softmax or sigmoid attention. Experiments on character-level language modeling and synthetic bracket matching show that Boltzmann attention consistently improves over standard softmax attention within a standard Transformer architecture, with the advantage becoming more pronounced as sequence length increases. A four-way ablation confirms that the improvement arises from the learnable pairwise couplings. These results suggest that explicit inter-position interactions provide a principled enhancement for attention-based sequence modeling. Moreover, the Ising formulation opens a natural path toward quantum-computing-based sampling strategies: we demonstrate that diabatic quantum annealing provides a practical training method while maintaining competitive performance with exact Boltzmann computation.

Gilhan Kim, Daniel K. Park

Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea · Department of Applied Statistics, Yonsei University, Seoul 03722, Republic of Korea · Department of Quantum Information, Yonsei University, Seoul 03722, Republic of Korea

Quantum AnnealingIsing ModelTransformer AttentionAttention Mechanisms
AskPDF
Apr 5, 2025

Quantum Adaptive Self-Attention for Quantum Transformer Models

A recurring weakness in quantum machine learning (QML) is that reported ``quantum advantages'' are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly -- a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware -- which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in quantum machine learning -- a capacity-matched classical bottleneck with the same parameter budget -- and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal's intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum.

Chi-Sheng Chen, En-Jui Kuo

Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

Hybrid Quantum-Classical MLTransformer AttentionVariational Quantum Circuits
AskPDF