SQUARE: Structured Quantum Representation Adapters as Compact Quadratic Feature Maps for Frozen Language Models

Emily Jimin Roh; Hyojun Ahn; Hoyeong Lee; Soohyun Park; Sung Whan Yoon; Vaneet Aggarwal; Joongheon Kim

quant-phSep 29, 2026

SQUARE: Structured Quantum Representation Adapters as Compact Quadratic Feature Maps for Frozen Language Models

Authors: Emily Jimin Roh, Hyojun Ahn, Hoyeong Lee, Soohyun Park, Sung Whan Yoon, Vaneet Aggarwal, Joongheon Kim

Organizations: Korea University · Sookmyung Women’s University · Ulsan National Institute of Science and Technology · Purdue University · Seoul National University Hospital

Abstract

Frozen language models (LMs) are increasingly used as fixed feature extractors for downstream reranking, scoring, and preference modeling, raising a practical question: how should a compact module represent interactions among features in a fixed low-dimensional bottleneck? Common linear and low-rank adapters remain linear at the adaptation module itself, whereas explicit second-order alternatives introduce pairwise interactions through direct parameterization or predefined factorizations. We propose SQUARE, a Structured QUAntum REpresentation adapter that amplitude-encodes the bottleneck vector, applies a parameterized quantum circuit, and measures the resulting state. We show that each basis-probability feature is exactly a normalized quadratic form in the bottleneck coordinates, while the additional Pauli- $Z$ readouts are signed linear combinations of these probabilities. The measured map can therefore parameterize interactions over $O(d^2)$ coordinate pairs through a small set of shared circuit parameters, where $d$ is the bottleneck dimension. It provides a structured parameterization within, rather than beyond, the classical normalized-quadratic feature class. In a disjoint same-pipeline evaluation over eight GLUE-derived controlled interaction tasks and five shared seeds, SQUARE achieves an average test accuracy of $0.7565$ , compared with $0.7355$ for an affine normalized-quadratic predictor, $0.7271$ for the evaluated parameter-matched Givens mixing model, $0.6817$ for an MLP, and $0.6155$ for a frozen-circuit control. Under reduced supervision, it also shows consistent gains over the strongest evaluated classical comparator, with the same qualitative pattern across multiple frozen LM backbones. All circuit experiments use simulation, while the learned feature map can be evaluated exactly in batched PyTorch without quantum hardware.

Figures & tables

Figure 1: First-order adaptation and SQUARE over frozen LM bottleneck. (A) A linear baseline aggregates weighted coordinate effects but does not explicitly output pairwise products. (B) SQUARE normalizes and amplitude-encodes $z$ , learns structured circuit mixing, and measures a readout containing normalized $z_{i}^{2}$ and $z_{i}z_{j}$ terms. During training, simulator backpropagation jointly updates the circuit and classical head. For deployment, the learned map is compiled exactly into a batched PyTorch implementation with the same outputs. Shared circuit angles constrain many interaction coefficients, while inference remains fully compatible with classical frozen-LM pipelines.

Figure 2: Learned decision surfaces on MNLI-derived two-dimensional PCA bottlenecks with controlled nonlinear labels.

Method	Params	Acc.	F1	AUC
BitFit	33	0.6502	0.6463	0.6641
LoRA r=4	104	0.6383	0.6283	0.6536
LoRA r=8	200	0.6444	0.6378	0.6576
Prefix	289	0.6595	0.6562	0.6734
MLP	776	0.6869	0.6707	0.7217
SQUARE (Ours)	180	0.7416	0.7536	0.8259

Table 1: Validation performance on the MNLI-derived two-dimensional PCA boundary task.

Method	Params	Boundary	10%	25%	50%	100%
Classical Fourier	210	0.980	0.736 $\pm$ 0.125	0.830 $\pm$ 0.160	0.879 $\pm$ 0.156	0.981 $\pm$ 0.020
Explicit polynomial	386	0.975	0.688 $\pm$ 0.049	0.851 $\pm$ 0.083	0.820 $\pm$ 0.167	0.969 $\pm$ 0.019
MLP	961	0.598	0.515 $\pm$ 0.026	0.599 $\pm$ 0.027	0.563 $\pm$ 0.092	0.616 $\pm$ 0.122
SQUARE (Ours)	281	0.992	0.823 $\pm$ 0.098	0.911 $\pm$ 0.052	0.948 $\pm$ 0.026	0.979 $\pm$ 0.016

Table 2: Controlled interaction recovery on checkerboard and radial-ring tasks. Boundary is the full-supervision mean over both tasks; reduced-supervision results report mean test AUC

\pm

standard deviation over six seeds. Full results are in Appendix B.1 .

Comparator	Gap	Primary contrast
Frozen circuit	$+0.141$	Trained vs. fixed mixing
Head only	$+0.115$	Measured vs. no measured map
MLP	$+0.075$	Interaction-feature architecture
Givens-12	$+0.029$	Matched-angle mixing family
Norm. quadratic	$+0.021$	Coupled vs. affine quadratic score
SQUARE avg. test accuracy: 0.7565

Table 3: Core held-out evidence. Gap is SQUARE minus the comparator in eight-task average test accuracy. Complete scores, parameter counts, and uncertainty summaries are in Appendix B.12 .

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Visualization of the SQUARE parameterized quantum circuit used in the main experiments. The input bottleneck representation is amplitude-encoded into an $n$ -qubit state $|\psi(z)\rangle$ . Each layer applies an $RYRZ$ rotation block, ring CNOT entanglement, and a final $RY$ rotation block before measurement. The shown example uses $n=4$ qubits and one circuit layer.

Model	Coefficient-matrix structure	Stored	Effective family
Full explicit quadratic	any symmetric $A$	$136$	$\leq 136$
Low-rank bilinear (rank $r$ )	$A=\operatorname{Sym}(UV^{\top})$	$32r$	rank $\leq 2r$ ; dim. $\leq\min(136,32r)$
Factorization machine (rank $k$ )	off-diagonal coefficients from $VV^{\top}$	$16k$	Gram-constrained off diagonal
Learned mixing + squares	$A=W\operatorname{diag}(c)W^{\top}$	$256{+}16$	$\leq 136$
Learned orthogonal mixing + squares	spectral form, $W\in O(16)$	$120{+}16$	$\leq 136$
SQUARE, real ( $L=1$ )	restricted orthogonal squares	$8{+}16$	sampled local rank $\leq 8$ before $c$

Appendix

Table 4: Structural comparison at

d=16

. Stored scalars are implementation parameters before a head; the last column gives an effective-family bound or structural constraint.

Characteristic	QAA	QPA	SQUARE (Ours)
Purpose	Adapt states	Generate weights	Relation modeling
Data encoding	Amplitude	None	Amplitude
Hybrid QC	✔	✔	✔
Quantum simulator	✔	✔	✔
Experiment	NLG	Perplexity eval.	Controlled / Reranking

Appendix

Table 5: Quantum-enhanced adaptation methods for LMs.

Work	Conf	Category	Hybrid Q-C	Encoding	Simulator
TensorRL-QAS ( Kundu and Mangini, 2025 )	NeurIPS	RL	✔	Matrix Product State	✔
QVF ( Wang et al., 2025c )	NeurIPS	CV	✔	Amplitude	✔
AQC-DRC ( Xiao et al., 2025 )	NeurIPS	ML	✔	Amplitude	✔
QDSFormer ( Born et al., 2025 )	NeurIPS	CV	✔	Angle	✔
QuanONet ( Wang et al., 2025b )	ICML	ML	✔	Feature Map	✔
QRL ( Meyer et al., 2025 )	ICML	RL	✔	Angle/Re-uploading	✔

Appendix

Table 6: Experimental settings in recent quantum-enhanced learning studies. A checkmark indicates that the corresponding setting is explicitly used or discussed. Recent QML studies commonly adopt hybrid quantum–classical architectures, task-specific classical-to-quantum encodings, and simulator-based evaluation.

Method	Params	Checkerboard	Radial ring	Average
Classical Fourier	210	0.977 $\pm$ 0.033	0.983 $\pm$ 0.004	0.980
Explicit polynomial	386	0.973 $\pm$ 0.001	0.977 $\pm$ 0.003	0.975
MLP (GELU)	961	0.487 $\pm$ 0.009	0.710 $\pm$ 0.041	0.598
Adapter transformation	140	0.505 $\pm$ 0.071	0.638 $\pm$ 0.012	0.571
LoRA transformation	77	0.567 $\pm$ 0.025	0.517 $\pm$ 0.009	0.542
SQUARE (Ours)	281	0.987 $\pm$ 0.009	0.997 $\pm$ 0.002	0.992

Appendix

Table 7: Boundary-specific controlled interaction recovery under an identical validation-selection protocol. Values are mean test AUC

\pm

sample standard deviation over six random seeds.

Method	Params	SST-2	RTE	MRPC	QNLI	Avg.
MLP ∗	577	0.964 $\pm$ 0.002	0.976 $\pm$ 0.002	0.958 $\pm$ 0.002	0.956 $\pm$ 0.003	0.964
Explicit norm. quadratic ∗	409	0.799 $\pm$ 0.006	0.859 $\pm$ 0.002	0.854 $\pm$ 0.004	0.827 $\pm$ 0.005	0.835
Full bilinear ∗	273	0.874 $\pm$ 0.027	0.918 $\pm$ 0.019	0.902 $\pm$ 0.011	0.908 $\pm$ 0.018	0.901
Low-rank bilinear	81	0.971 $\pm$ 0.005	0.967 $\pm$ 0.005	0.942 $\pm$ 0.011	0.962 $\pm$ 0.009	0.961
Factorization machine	71	0.973 $\pm$ 0.002	0.972 $\pm$ 0.002	0.965 $\pm$ 0.005	0.963 $\pm$ 0.003	0.968
SQUARE (Ours)	68	0.967 $\pm$ 0.008	0.980 $\pm$ 0.005	0.972 $\pm$ 0.002	0.958 $\pm$ 0.009	0.969

Appendix

Table 8: Compact-budget comparison on four GLUE-derived controlled-label tasks. Hyperparameters are selected using validation data only, with a target budget of at most

1.5\times

the smallest SQUARE configuration. Values are mean

\pm

s.d. over the matched runs. ∗ The smallest prespecified configuration evaluated for this method exceeds the target; these rows are retained as informative higher-budget references. In particular, the reported MLP result should not be interpreted as a lower bound on the size of an MLP.

Protocol	Method	Params	SST-2	RTE	MRPC	QNLI	Avg.
Encoder-adapted	LoRA	221,953	0.870	0.581	0.832	0.736	0.755
Frozen bottleneck	MLP	5,041	0.822	0.540	0.811	0.571	0.686
Frozen bottleneck	SQUARE (Ours)	49	0.826	0.527	0.815	0.578	0.686
Frozen bottleneck	Explicit quadratic	445	0.824	0.475	0.813	0.580	0.673

Appendix

Table 9: Performance on the original GLUE labels. SST-2, RTE, and QNLI use accuracy, whereas MRPC uses F1; Avg. is their unweighted mean and is not directly comparable to the controlled-label results. Encoder-adapted LoRA updates the backbone, whereas the remaining methods operate on the same frozen bottleneck.

Figure 4: Post-training simulated quantum noise sensitivity of SQUARE with the BERT frozen bottleneck in MNLI. A trained SQUARE model is evaluated under depolarizing, bit-flip, phase-flip, and amplitude-damping noise injected only during evaluation ( Singkanipa and Lidar, 2025 ) .

Method	Reported learned scalars	Test AUC
SQUARE (Ours)	234	0.984 $\pm$ 0.014
SVM (RBF)	—	0.974 $\pm$ 0.030
Classical Fourier	204	0.961 $\pm$ 0.050
Graph-Laplacian	—	0.953 $\pm$ 0.003
Explicit polynomial	334	0.947 $\pm$ 0.030
kNN ( $k=25$ )	—	0.909 $\pm$ 0.011

Appendix

Table 10: Broad-family comparison under a shared frozen-bottleneck protocol. Results are mean test AUC

\pm

sample standard deviation over two interaction rules and six seeds. “—” indicates that a directly comparable learned-scalar count is not reported.

Backbone	MLP	SQUARE	Gain	MLP params	SQUARE params
OPT-350M	0.7960	0.8294	$+0.0334$	776	180
GPT-2	0.7847	0.8258	$+0.0411$	776	180
OpenLLaMA-3B	0.7052	0.7714	$+0.0662$	1,633	157
Mistral-7B	0.7012	0.7481	$+0.0469$	1,633	157

Appendix

Table 11: Validation-selected controlled-interaction accuracy across four frozen LM backbones. Gain denotes SQUARE minus MLP within each backbone and protocol. OPT-350M and GPT-2 use the V20 configuration, whereas OpenLLaMA-3B and Mistral-7B use H16; parameter counts therefore differ across the two groups. These results test whether the qualitative comparison persists across frozen representation geometries and are not directly comparable to the primary disjoint held-out test in Table 3 .

Figure 5: Learning curves on GLUE-derived controlled nonlinear interaction classification with additional frozen backbones. We report validation accuracy averaged across controlled GLUE-derived tasks for OPT-350M and GPT-2. Shaded regions denote variation across tasks. Across both frozen backbones, SQUARE shows stable optimization and reaches the strongest final validation accuracy, while MLP remains the strongest classical baseline. The gap between SQUARE and SQUARE without the PQC suggests that the measured quantum feature map contributes beyond the lightweight downstream head.

Method	Quantum	Params	CoLA	SST-2	STS-B	QQP	MNLI	QNLI	RTE	WNLI	Avg.
BitFit	✗	33	0.4775	0.4825	0.4325	0.4700	0.5125	0.4600	0.5379	0.5070	0.4850
LoRA r=4	✗	104	0.6000	0.6050	0.6450	0.6700	0.7225	0.7675	0.8159	0.7887	0.7018
LoRA r=8	✗	200	0.6500	0.6125	0.6460	0.6700	0.7275	0.7679	0.8231	0.7928	0.7112
AdaLoRA	✗	621	0.6850	0.6025	0.6450	0.6675	0.7200	0.7675	0.8303	0.7324	0.7063
Prefix	✗	289	0.4775	0.4825	0.4325	0.4700	0.5125	0.4600	0.5379	0.5070	0.4850
MLP	✗	776	0.7775	0.8025	0.7925	0.7950	0.7825	0.7775	0.8520	0.7887	0.7960

Appendix

Table 12: Task-wise validation accuracy on GLUE-derived controlled nonlinear interaction classification with OPT-350M. All methods use the same frozen OPT-350M bottleneck representation, and Params counts only trainable adapter parameters. Several baselines approach majority-class performance under this protocol.

Method	Quantum	Params	CoLA	SST-2	STS-B	QQP	MNLI	QNLI	RTE	WNLI	Avg.
BitFit	✗	33	0.4825	0.4900	0.5075	0.4425	0.4375	0.4525	0.4946	0.5070	0.4768
LoRA r=4	✗	104	0.7700	0.7425	0.7800	0.6300	0.7525	0.7325	0.6029	0.7528	0.7204
LoRA r=8	✗	200	0.7775	0.7475	0.7875	0.6675	0.7550	0.7325	0.6282	0.7810	0.7346
AdaLoRA	✗	621	0.7750	0.7575	0.7825	0.6375	0.7550	0.7325	0.6534	0.8028	0.7370
Prefix	✗	289	0.4850	0.4900	0.5075	0.4425	0.4375	0.4525	0.4946	0.5070	0.4771
MLP	✗	776	0.7625	0.7925	0.8300	0.6950	0.8275	0.7675	0.7978	0.8046	0.7847

Appendix

Table 13: Task-wise validation accuracy on GLUE-derived controlled nonlinear interaction classification with GPT-2. All methods use the same frozen GPT-2 bottleneck representation, and Params counts only trainable adapter parameters. Several weak baselines collapse to near-majority-class prediction on this backbone and return identical task-wise accuracies; the rows are retained for completeness.

Method	Quantum	Params	CoLA	SST-2	STS-B	QQP	MNLI	QNLI	RTE	WNLI	Avg.
Linear	✗	17	0.6358	0.5783	0.6083	0.6583	0.6083	0.6392	0.6474	0.6150	0.6238
BitFit	✗	33	0.4808	0.5375	0.5592	0.5158	0.4717	0.4908	0.5535	0.5352	0.5181
LoRA r=8	✗	417	0.6300	0.5825	0.6058	0.6525	0.6075	0.6417	0.6450	0.6103	0.6219
AdaLoRA	✗	621	0.6292	0.5725	0.5892	0.6592	0.6050	0.6400	0.6510	0.6103	0.6195
MLP	✗	1633	0.6667	0.6425	0.7442	0.6858	0.6767	0.6367	0.6967	0.8920	0.7052
SQUARE (Ours)	✔	157	0.7667	0.7358	0.7633	0.7825	0.7833	0.8025	0.8231	0.7136	0.7714

Appendix

Table 14: Task-wise validation accuracy on GLUE-derived controlled nonlinear interaction classification with a frozen OpenLLaMA-3B backbone, averaged over three seeds. All methods operate on the same frozen OpenLLaMA-3B bottleneck representation. Params counts trainable adapter and task-head parameters only and therefore differs from the main-text budget because the head width and adapter ranks are configured for this backbone. Best performance in each column is shown in bold . SQUARE achieves the highest macro-average accuracy while using substantially fewer trainable parameters than LoRA, AdaLoRA, and the larger MLP head.

Method	Quantum	Params	CoLA	SST-2	STS-B	QQP	MNLI	QNLI	RTE	WNLI	Avg.
Linear	✗	17	0.6017	0.5692	0.6167	0.6225	0.6083	0.5758	0.5860	0.6244	0.6006
BitFit	✗	33	0.4950	0.4883	0.5200	0.5517	0.4842	0.4500	0.5090	0.5634	0.5077
LoRA r=8	✗	417	0.5975	0.5675	0.5800	0.6308	0.6133	0.5783	0.5752	0.6291	0.5965
AdaLoRA	✗	621	0.5958	0.5700	0.5808	0.6233	0.6000	0.6000	0.5909	0.5962	0.5946
MLP	✗	1633	0.6225	0.6592	0.7625	0.6650	0.6567	0.6475	0.6811	0.9155	0.7012
SQUARE (Ours)	✔	157	0.7442	0.7442	0.7100	0.7158	0.7850	0.7642	0.7377	0.7840	0.7481

Appendix

Table 15: Task-wise validation accuracy on GLUE-derived controlled nonlinear interaction classification with a frozen Mistral-7B-v0.1 backbone, averaged over three seeds. All methods operate on the same frozen Mistral-7B-v0.1 bottleneck representation. Params counts trainable adapter and task-head parameters only and therefore differs from the main-text budget because the head width and adapter ranks are configured for this backbone. Best performance in each column is shown in bold . SQUARE achieves the highest macro-average accuracy while using substantially fewer trainable parameters than LoRA, AdaLoRA, and the larger MLP head.

Backbone	Reranker	Label Acc.	Label F1	Cand. AUC
OpenLLaMA-3B	Random	0.485	0.490	–
	Linear	0.515	0.679	0.522
	MLP	0.707	0.701	0.736
	SQUARE (Quantum)	0.704	0.712	0.745
	SQUARE (Hybrid)	0.708	0.724	0.745
Mistral-7B-v0.1	Random	0.510	0.491	–

Appendix

Table 16: Controlled candidate reranking on the two larger frozen backbones using an MRPC-derived controlled reranking task (mean over three seeds). Label Acc./F1 measure the correctness of the selected candidate, and Cand. AUC measures candidate-relevance ranking quality. Under a parameter-matched comparison, both SQUARE variants improve candidate-level ranking AUC over the classical MLP on both backbones. SQUARE (Hybrid) further achieves the highest Label Accuracy and F1 on both OpenLLaMA-3B and Mistral-7B-v0.1, indicating that combining the structured quantum representation with the hybrid prediction pathway provides the most consistent reranking performance across the evaluated metrics.

Method	Params	Fwd. ms	Train ms	Train samples/s
BitFit	33	0.0036	0.0403	24812.7
LoRA- $r{=}4$	104	0.0060	0.0467	21430.0
LoRA- $r{=}8$	200	0.0059	0.0462	21627.3
Prefix	289	0.0112	0.0660	15153.3
MLP	776	0.0058	0.0513	19477.8
QAA	140	8.9023	14.7052	68.0

Appendix

Table 17: Adapter-level runtime benchmark with batch size

32

. Latency is measured per sample after freezing the backbone representation.

Implementation	ms / step	ms / sample	relative to MLP
SQUARE, PennyLane default.qubit + backprop	$14.26\pm 1.72$	$0.223$	$22.4\times$
SQUARE, native PyTorch statevector	$3.60\pm 0.11$	$0.056$	$5.7\times$
MLP $16{\to}8{\to}1$ ( $145$ parameters)	$0.64\pm 0.03$	$0.010$	$1.0\times$

Appendix

Table 18: Matched-protocol training cost (CPU, batch

64

, float64, mean

\pm

s.d. over

100

steps).

Selector	BLEU	ROUGE-1	ROUGE-2	ROUGE-L	Score	HitRate	NonlinearScore
Uniform random (exact)	0.0193	0.2078	0.0622	0.1749	0.2066	0.1656	0.4991
MLP	0.0215	0.2256	0.0664	0.1861	0.2149	0.2083	0.4917
SQUARE, pure path	0.0211	0.2172	0.0605	0.1775	0.2128	0.1917	0.5222
SQUARE, residual path	0.0246	0.2365	0.0757	0.1895	0.2224	0.2417	0.5032
Oracle- $q$	0.0288	0.3123	0.1098	0.2622	0.2861	1.0000	0.5662
Oracle- $\rho$	0.0172	0.1935	0.0604	0.1618	0.2261	0.3083	0.6890

Appendix

Table 19: Alpaca candidate reranking results. All methods score the same fixed candidate pool using the same frozen BERT bottleneck; higher is better.

Method	cluster	radial ring	checkerboard	two-moons	islands
SQUARE (depth-1, $\sim$ 93p)	0.999 $\pm$ 0.001	0.825 $\pm$ 0.045	0.849 $\pm$ 0.039	0.964 $\pm$ 0.034	0.986 $\pm$ 0.013
SQUARE untr. ( $\sim$ 81p)	0.999 $\pm$ 0.001	0.841 $\pm$ 0.034	0.838 $\pm$ 0.042	0.981 $\pm$ 0.016	0.964 $\pm$ 0.023
SQUARE ${}_{\text{ru}}$ (depth-2, $\sim$ 121p)	0.999 $\pm$ 0.001	0.999 $\pm$ 0.001	0.962 $\pm$ 0.033	0.999 $\pm$ 0.001	0.994 $\pm$ 0.004
SQUARE ${}_{\text{ru}}$ untr.	0.999 $\pm$ 0.002	0.933 $\pm$ 0.043	0.829 $\pm$ 0.015	0.997 $\pm$ 0.003	0.988 $\pm$ 0.007
MLP ( $\sim$ 129p)	0.999 $\pm$ 0.001	0.804 $\pm$ 0.106	0.484 $\pm$ 0.046	0.920 $\pm$ 0.004	0.992 $\pm$ 0.009
Adapter ( $\sim$ 125p)	0.999 $\pm$ 0.002	0.773 $\pm$ 0.110	0.490 $\pm$ 0.040	0.920 $\pm$ 0.003	0.994 $\pm$ 0.007

Appendix

Table 20: Additional boundary controls on a frozen Qwen2.5-1.5B-Instruct tutoring PCA-2 bottleneck. Mean test AUC

\pm

sample standard deviation over six seeds. SQUARE is the depth-1 learned-projection/magnitude-readout configuration; “untr.” freezes all circuit parameters. SQUARE

{}_{\text{ru}}

adds depth-2 input re-uploading. These end-to-end boundary models are not claimed to be exactly quadratic in the original PCA-2 coordinates.

Method (rule)	exact	100 shots	1,000	10,000
SQUARE (checkerboard)	0.849	0.833	0.849	0.849
SQUARE ${}_{\text{ru}}$ (checkerboard)	0.962	0.954	0.962	0.963
SQUARE ${}_{\text{ru}}$ (radial ring)	0.999	0.997	0.999	0.999
SQUARE ${}_{\text{ru}}$ (two-moons)	0.999	0.999	0.999	0.999

Appendix

Table 21: Finite-shot readout on the tutoring bottleneck: mean test AUC over six seeds when the exact statevector probabilities are replaced by multinomial sampling at

100/1{,}000/10{,}000

shots. AUC differs from the exact readout by at most

0.016

100

shots and by

\leq 0.001

\geq 1{,}000

shots.

Protocol	Head architecture	Angles	Proj.	Head	Total
V20: validation/backbone suites and runtime benchmark	$\mathrm{Lin}(20{\to}8)$ – $\tanh$ – $\mathrm{Lin}(8{\to}1)$ , bias-free, on the 20-dimensional readout	12	–	168	180
H16: primary held-out test and post-entanglement ablation (Apps. B.12 , B.13 )	$\mathrm{Lin}(16{\to}8)$ – $\tanh$ – $\mathrm{Lin}(8{\to}1)$ on the 16 basis probabilities	12	–	145	157
B32: boundary controls (App. B.10 ), nonlinear projection + magnitude readout	$\mathrm{Lin}(32{\to}1)$ on $[v;\sqrt{p(v)}]$	12	48 †	33	93

Appendix

Table 22: Parameter decomposition of the central SQUARE configurations (

d{=}B{=}16

n{=}4

unless noted). “Readout” is the head input dimension. The primary held-out model uses a biased probability-only head; the separate validation/backbone model uses a bias-free probability-plus-

Z

head. † Learned input projection

\mathrm{Lin}(2{\to}16)

–

\tanh

, used only in the boundary suite.

Method	Params	Avg. test	$\Delta$ vs. SQUARE [avg. taskwise interval]
LoRA $r{=}8$	417	0.5897	$-0.167$ [ $-0.181,-0.137$ ]
MLP	1,633	0.6817	$-0.075$ [ $-0.081,-0.053$ ]
SQUARE w/o PQC	145	0.6412	$-0.115$ [ $-0.125,-0.092$ ]
SQUARE, frozen circuit	145	0.6155	$-0.141$ [ $-0.147,-0.120$ ]
SQUARE	157	0.7565	–
Givens-12 + squares + head	157	0.7271	$-0.029$ [ $-0.041,-0.003$ ]

Appendix

Table 23: Disjoint held-out test summary for same-pipeline controls (eight tasks, five shared seeds).

\Delta

is comparator minus SQUARE. Displayed endpoints are descriptive averages of taskwise example-paired bootstrap 95% intervals, not a calibrated confidence interval for the macro-average.

Pre-entangler rotations	Entangler	Angles	Total	Avg. test
$RY$	none	8	153	0.7891
$RY$	linear CNOT	8	153	0.7879
$RY$	ring CNOT	8	153	0.7568
$RYRZ$	none	12	157	0.7889
$RYRZ$	linear CNOT	12	157	0.7875
$RYRZ$	ring CNOT (default)	12	157	0.8462

Appendix

Table 24: Circuit configuration analysis with a trainable post-entanglement

RY

block in every row. Avg. test is mean held-out accuracy over QNLI, MNLI, and RTE with five shared seeds.

Experiment	Data Source	Train	Validation	Test / Eval	Seeds	Evaluation Unit
2D decision-boundary analysis (nominal)	Controlled 2D bottleneck features	1,000	400	–	3	Binary example
GLUE-derived interaction classification	GLUE inputs with controlled nonlinear labels	7,600	2,250	–	3	Sentence / sentence-pair example
Alpaca candidate reranking	Alpaca instruction-following candidates	960	120	120	3	Query–candidate pair
Disjoint held-out control suite	Eight GLUE inputs with controlled nonlinear labels	Disjoint task-specific subsets			5	Sentence / sentence-pair example

Appendix

Table 25: Dataset construction and split information for the main experiments. Controlled tasks use labels generated from nonlinear interaction rules over frozen BERT bottleneck features. Reranking tasks use fixed candidate pools shared across methods.

Item	Value
Train examples	1,000
Validation examples	400
Input dimension	2
Label construction	Controlled nonlinear rule
Evaluation unit	Binary example
Metrics	Accuracy, F1, AUC

Appendix

Table 26: Split information for controlled nonlinear decision-boundary experiments. The same split is used for qualitative boundary analysis and circuit configuration ablation.

Item	Value
Train queries	960
Validation queries	120
Test queries	120
Candidates per query	8
Evaluation unit	Query–candidate pair
Selection rule	$\hat{j}=\arg\max_{j}s_{i,j}$

Appendix

Table 27: Split information for Alpaca instruction-following candidate reranking. All methods rerank the same fixed candidate set for each query.

Method	Hyperparameter	Values
BitFit	Batch Size	$\{16,32\}$
	Optimizer	AdamW
	Scheduler	Linear Scheduler
	Learning Rate	$\{1\mathrm{e}{-4},3\mathrm{e}{-4},1\mathrm{e}{-3}\}$
	Trainable Parameters	Bias terms only
LoRA	Batch Size	$\{16,32\}$

Appendix

Table 28: Hyperparameters used for baseline adapters and SQUARE. Curly brackets indicate values considered during hyperparameter selection. The frozen LM encoder and bottleneck projection are fixed for all methods.

Explore similar work

Sep 27, 2026cs.AI

QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAG

Fidelity-based quantum retrieval ranks candidates by the fidelity between query and archive states. Applying a shared input-independent unitary after fixed state encoding leaves that fidelity unchanged, so training the circuit cannot alter the ranking. Quantum parameter-efficient input-dependent retrieval adaptation (QuPID) repairs this by making the circuit input-dependent through data re-uploading and by comparing measurement readouts, vectors of local Pauli expectations, rather than states. The result is a small readout for adapting frozen image features to a local archive with limited data: training simulates the circuit classically, and inference runs on a GPU with fixed learned parameters. We characterize the class as a structured factorization of input-modulated quadratic feature maps, bound the frequency support of its re-uploading channel, and give a parameter-count generalization bound that motivates its small budget. Under a shared frozen backbone and a label-free protocol, QuPID's 60 parameters give higher precision-at-5 (P@5) on ChestX-ray14 and MURA than frozen medical encoders, and than adapters and low-rank adaptation (LoRA) with up to 5.25 million trainable parameters. On ChestX-ray14, the P@5 gain over the frozen encoder is +0.116, the lead over retuned adapters is widest at 512 adaptation examples (+0.040), and the full-budget margin over an equally compact classical rotation-plane head is +0.023 with a 95% interval excluding zero. Medical imaging is the primary testbed; the pattern recurs on two non-medical benchmarks, in report generation, and under simulated gate noise and finite-shot readout.

Hyojun Ahn, Emily Jimin Roh, Soohyun Park +3

Korea University · Sookmyung Women’s University · Virginia Tech +2

Qubit Frozen Encoder

Ask PDF

May 7, 2026quant-ph

Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.

Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2

Multiverse Computing, Parque Científico y Tecnológico de Gipuzkoa, Paseo May 7 de Miramón 170, 20014, Donostia / San Sebastián, Spain · Department of Basic Sciences, Tecnun – University of Navarra, San Sebastián, Spain · Multiverse Computing, Centre for Social Innovation, 192 Spadina Avenue, ON M5T 2C2, Toronto, Canada +2

Large Language Model Quantization Quantum Processor Phase Transitions

Ask PDF

Aug 5, 2026cs.LG

Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning

Quantized orthogonal fine-tuning (qoft) enables parameter-efficient adaptation of low-bit language models by learning structured activation rotations before frozen quantized weights. However, its task-specific updates remain constrained to linear orthogonal transformations, limiting input-dependent nonlinear corrections. We introduce AuroOFT, which keeps qoft as a stable quantization-compatible branch while attaching a zero-start gated low-rank nonlinear residual to each adapted linear layer. AuroOFT maps activations into an RMS-normalized compact latent space and uses adaptive nonlinear bases with bounded or token-dependent gating. The zero-initialized up projection makes AuroOFT functionally identical to qoft at initialization, while orthogonality remains a branch-level stability property rather than a property of the combined nonlinear layer. Under matched data, optimization, decoding, and parser protocols, AuroOFT improves Macro-6 over matched qoft by 1.30-2.70% on the 1.5B/3B Qwen2.5 settings, exceeds QLoRA by 6.52-10.62%, and saves 32.3-44.7% trainable parameters relative to QLoRA in representative scales. The small exam-style multiple-choice math set is treated only as a protocol-sensitivity diagnostic. Our code is available at the anonymous repository: https://anonymous.4open.science/r/AuroOFT-F3FD.

Yue Han, Dianlin Wang

College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology

Large Language Model Quantization Parameter-Efficient Fine-Tuning Methods Model Fine-Tuning Token-Level Uncertainty

Ask PDF