quant-phSep 21, 2026

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

Authors: Xiaoqiang Wang, Mengyang Xiong, Jun Dai, Bang Liu

Abstract

Language models can be adapted by changing the computations applied to individual tokens. Quantum circuits offer one such approach, but evaluating wider circuits inside a large model can be computationally demanding. Here we introduce HyperQ, which adds token-conditioned quantum residual branches to a frozen masked-diffusion language model. A quantum residual branch is a module in each transformer block that reads a token's hidden state, emits the coordinates of that token's circuit, executes it, and adds the measured values back through a residual connection. The backbone remains frozen, and only the added branches are trained. Within each branch, a lightweight circuit hypernetwork emits token-specific rotation angles, coupling strengths, and measurement axes in a shared sparse circuit structure. The required expectation values have an exact classical expression whose evaluation cost grows linearly with the qubit count, enabling circuits from 16 to 64 qubits to be trained within a 1.1-billion-parameter backbone. Across downstream benchmarks, increasing circuit width raises the average score from 47.65 to 54.30. At 64 qubits, HyperQ exceeds the backbone and its low-rank-adapted counterpart by 4.71 and 3.67 points, respectively. HyperQ is fine-tuned on 20,000 prompt-response pairs, compared with 200,000 for the classical baselines. These findings support token-conditioned circuit emission as a tractable architectural approach to quantum-augmented language modelling.

Explore similar work

CardsList
Aug 4, 2026quant-ph

Separating quantum circuits from classical LLMs

Modern large language models - transformers and diffusion language models - are built around two canonical algorithmic tasks: prediction and generation. We prove unconditional separations between low-depth quantum computation and the corresponding bounded-resource classical language-model architectures in both regimes. Concretely, we exhibit the following: 1. Distributional separation. We give a distribution that is sampleable by QNC0\textsf{QNC}^0 circuits (i.e., a family of constant-depth quantum circuits consisting of bounded fan-in gates) that no constant-round diffusion language model (DLM\textsf{DLM}) with shallow scheduling and denoising can sample within constant distance, even when allowed sublinear chain-of-thought and output-token revision/remasking events, the very features modern DLM\textsf{DLM}s rely on. 2. Functional separation. We exhibit a function computable in ∧∘QNC0[log⁡log⁡n]\land \circ \textsf{QNC}^0[\log\log n] (i.e., a family of O(log⁡log⁡n)(\log\log n)-depth QNC0\textsf{QNC}^0 circuits, where nn is the input length, followed by a single classical AND\mathsf{AND} gate) such that any constant-depth decoder-only transformer computing the function must be large: it would have to have width nΩ(1)n^{Ω(1)}. Together, our work initiates the study of quantum advantage in the era of large language models.
Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1
Quantum Circuit SimulationLarge Language Model QuantizationLarge LanguageDiffusion Language ModelsTransformer EncoderCircuits+2
AskPDF
May 7, 2026quant-ph

Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.
Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2
Large Language Model QuantizationQuantum ComputingPhase Transitions
AskPDF
Sep 17, 2026cs.CL

dQwen3.5: Hybrid-Attention Diffusion Language Models

Adapting a pretrained autoregressive (AR) model is a cost-efficient route to a diffusion language model (DLM). While nearly all such adaptations start from a full-attention transformer, AR modeling has shifted toward hybrid architectures that interleave attention and RNN layers. This creates an obstacle for adaptation: unlike attention, RNNs are structurally causal and nontrivial to bidirectionalize. Despite this mismatch, we investigate whether such backbones can become effective DLMs by adapting Qwen3.5 at 0.8B, 2B, 4B, and 9B scales, yielding the dQwen3.5 family. We find that hybrid backbones can be efficient starting points for adaptation: against a full-attention control, the hybrid reaches a given training loss in about half the tokens. Across scales, dQwen3.5 resembles full-attention DLMs in any-order decoding behavior and performs strongly under parallel decoding.
Anton Xue, Litu Rout, Aditya Akella +3
Diffusion Language ModelsAutoregressive ModelRecurrent Neural Networks
AskPDF