quant-phNov 11, 2026

Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM

Authors: Maforikan Amoussou, Borja Aizpurua, Sukhbinder Singh, Román Orús, Hans-Arno Jacobsen

Organizations: Multiverse Computing, Toronto, Ontario, Canada · The Edward S. Rogers Sr. Department of Electrical and Computer Engineering - University of Toronto, Toronto, Ontario, Canada · Multiverse Computing, San Sebasti´an, Spain · Department of Basic Sciences, Tecnun - University of Navarra, San Sebasti´an, Spain · 

Abstract

We introduce the multi-head quantum neural network (QNN), a quantum module compatible with noisy intermediate-scale quantum (NISQ) devices that can replace transformer blocks inside pre-trained large language models (LLMs) with little loss in accuracy and with far fewer parameters than the blocks it replaces. Encoding a model's DD hidden neurons into a qubit register is the obstacle: amplitude encoding scales poorly in circuit depth and angle encoding in qubit count, so neither is practical at production-scale width. We instead partition the hidden space into kk disjoint heads, angle-encode each into an nqn_q-qubit register, process the heads in parallel, and couple them through constrained classical linear layers, the mixers. Performing surgery on Qwen3-30B-A3B-Instruct-2507, we show that a single multi-head QNN with 145 281145\,281 trainable parameters (D=2048D=2048, k=32k=32, nq=8n_q=8, L=4L=4) can replace a contiguous span of eight transformer blocks, displacing 16.33%16.33\% of the model's parameters. Ablations show that, for a single replaced block, the quantum heads (circuits followed by a learned readout), and not the classical mixers alone, recover the displaced function, whereas for fused spans the mixers alone come within 11-3%3\% of the full module in perplexity. Whether this benefit is specific to trained, entangling quantum circuits is not established: controls that switch the entangler off (two seeds), or freeze the circuit at its initial or at uniformly random angles (one seed), beat the full module on held-out perplexity, and a parameter-matched classical multilayer perceptron remains a few percent ahead of the full module. We train classically with exact statevector simulation, then execute the quantum heads of our fusion checkpoints on a 156156-qubit IBM Heron processor, showing that NISQ-scale quantum modules can be installed in a production-scale LLM and run on hardware.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

    May 7, 2026Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2LLM InferenceLLM Fine-Tuning

  2. Variational Quantum Transformer Architecture for Synthetic Language Generation

    Sep 16, 2026Julian Hager, Michael Kölle, Gerhard Stenzel +3Quantum NLPQuantum Machine Learning

  3. Separating quantum circuits from classical LLMs

    Aug 4, 2026Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1Decoder-Only Language ModelsDiffusion Language Models