PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
Authors: Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, Yawei Li
Organizations: Nanyang Technological University, Singapore · Nanjing University of Posts and Telecommunications, Nanjing, China · Nanjing University, Nanjing, China · Qualcomm AI Research, Amsterdam, Netherlands
Smaller activation outliers do not necessarily imply better low-bit quantization: their alignment with the quantizer matters. We introduce PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4. The affine offsets represent the energy in this subspace without widening the range within the group. We formulate rotation design as a Ky Fan trace maximization and derive a closed-form solution that is provably optimal for this alignment objective. Compact Householder transformations and their compact-WY representation enable gradient-free construction and efficient application at both foldable and online sites. A predictive range law further connects unaligned activation energy and group size to quantization-relevant variation. Experiments on Llama, Qwen, and Mistral span dense models up to 70B parameters and a 30B mixture-of-experts model. Under W4A4KV4, PrismQuant sets the state of the art on Llama-3.2-3B among the compared methods in both perplexity and accuracy. On Llama-3.1-70B, it attains 3.85 perplexity and 72.46% average zero-shot accuracy, only 0.22 percentage points below full precision. In the deployment study on Llama-3.1-8B, our optimized implementation achieves 1.51x prefill and 1.22x CUDA Graph decode speedups over matched FP16 baselines, with 56.34% lower decode peak memory and only 2.35% additional Graph decode latency over Hadamard. Code and models are available at https://github.com/ForeverBlue816/PrismQuant and https://huggingface.co/collections/ForeverBlue/prsimquant.
Figures & tables
Figure 1: Aligning activations with quantizer geometry. Llama-3.2-3B layer 27 down-projection inputs, g=128 . (a–d) PrismQuant reduces the mean within-group range from 1.76 to 0.84 relative to Hadamard under matched asymmetric INT4 quantization. (e–g) Token/group traces illustrate the mechanism; the dashed line marks the stored affine offset.
Figure 2: Quantizer-aware alignment predicts INT4 error. All 28 Llama-3.2-3B down-projection inputs, g=128 , evaluated on identical tokens. (a) Energy captured by the group-constant subspace. (b,c) Within-group range and activation NMSE under matched asymmetric INT4; PrismQuant reduces them by 25% and 40% on average relative to Hadamard.
Figure 3: Overview of PrismQuant. Calibration takes the leading eigenvectors Vk of the second moment Σ and builds a rotation that turns them into per-group shared levels, which the affine offset z represents outside the INT4 range. R1 is folded into the embedding, head, and all residual-facing weights; R2 is applied after WV and its transpose folded into WO so the V cache is quantized in the aligned basis; at the down-projection input only the rank- k correction I−W′Y′⊤ and the block Hadamard H128 run online (clock), with R4⊤ folded into WD . The schematic uses right-acting row-vector rotations.
Zero-shot accuracy ↑
Method
PPL ↓
ARC-c
ARC-e
BoolQ
HellaS
OBQA
PIQA
SIQA
WinoG
Avg. ↑
Llama-3.2-3B (bf16)
7.80
46.20
71.70
73.10
73.70
43.40
77.40
47.20
69.10
62.73
QuaRot (NeurIPS ’24)
10.10
38.60
59.00
65.90
66.50
35.80
74.40
43.10
65.20
56.06
SpinQuant (ICLR ’25)
9.20
38.90
64.80
68.00
69.10
39.40
74.90
45.10
62.90
57.89
KurTail (EMNLP-F ’25)
9.00
42.20
66.70
69.80
68.80
39.80
75.60
44.80
64.60
59.04
ResQ (ICML ’25)
8.80
43.10
65.60
68.80
70.50
38.40
75.10
45.60
64.80
58.99
Table 1: W4A4KV4 quantization across the Llama family. WikiText-2 perplexity ( ↓ ) and zero-shot accuracy (%, ↑ ); Avg. averages the eight displayed tasks. Published rows and the 3B/8B bf16 references follow Wang et al. (2026) (3B) and He et al. (2025) (8B/70B); the 70B reference, Hadamard, and PrismQuant rows are measured in our pipeline. Bold marks the best quantized result per column within each model; PrismQuant rows are shaded light for k=8 and darker for k=max .
PPL ↓
Zero-shot accuracy ↑
Method
WT2
C4
ARC-c
ARC-e
BoolQ
HellaS
OBQA
PIQA
SIQA
WinoG
Avg. ↑
Qwen3-30B-A3B (bf16)
6.11
10.70
57.68
79.55
81.41
81.40
44.00
80.96
52.15
72.53
68.71
Hadamard
6.68
11.47
55.38
79.04
81.41
79.24
44.80
79.33
50.26
70.32
67.47
PrismQuant ( k=8 )
6.45
11.19
57.25
79.29
81.90
80.08
45.40
80.30
51.28
72.61
68.52
PrismQuant ( k=max )
6.42
11.18
55.63
79.84
83.55
79.60
44.40
79.60
51.94
71.27
68.23
Table 2: W4A4KV4 quantization of Qwen3-30B-A3B-Base. WikiText-2 and C4 perplexity ( ↓ ) and zero-shot accuracy (%, ↑ ) on the eight-task suite of Table 1 . All rows are evaluated in our pipeline with three seeds; the router is kept in bf16, and every expert receives its own R4 ( k=6 , its full slot count), so the two PrismQuant rows differ only in the rank of R1 . Bold marks the best quantized result per column.
Configuration
Llama - 3.2 - 3B
Llama - 3.1 - 8B
Qwen3 - 8B
Hadamard gap, PHad−Pbf16 (PPL)
+0.154
+0.142
+0.442
PrismQuant ( k=8 )
38.59
35.53
59.12
PrismQuant ( k=16 )
38.30
35.74
51.97
PrismQuant ( k=32 )
41.88
40.23
85.57
PrismQuant ( k=64 )
40.83
41.84
99.85
Table 3: Alignment rank. Activation-only recovery ( % , ↑ ) of the Hadamard-to-bf16 WikiText-2 perplexity gap by PrismQuant at rank k ; the first row lists the gap itself. Weights and KV states remain in bf16; ranks are capped per site by the number of group slots. Bold marks the largest recovery; row shading darkens with k .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Perplexity ↓
Task score ↑
Method
WT2
C4
MMLU
MMLU-R
GSM8K
ARC-e
Qwen3-0.6B (bf16)
12.66
19.38
52.52
55.83
60.96
57.95
Hadamard
16.50
25.36
43.46
43.49
28.96
56.69
PrismQuant ( k=8 )
15.28
23.92
44.22
46.23
31.54
53.62
PrismQuant ( k=max )
15.38
23.92
44.69
45.27
28.66
56.36
Appendix
Table 4: W4A4KV4 quantization across the Qwen3 dense family. All models are Base checkpoints evaluated in our pipeline with three seeds. WT2 and C4 report perplexity ( ↓ ); task scores are percentages ( ↑ ). MMLU and MMLU-R (Redux) are 5-shot; GSM8K is 4-shot CoT; ARC-e is zero-shot ( Raffel et al., 2020 ; Cobbe et al., 2021 ; Wang et al., 2024 ; Gema et al., 2025 ) . Bold marks the best quantized value per model and column, including rounded ties.
PPL ↓
Zero-shot accuracy ↑
Mean ↑
Method
WT2
C4
ARC-c
ARC-e
BoolQ
HellaS
OBQA
PIQA
SIQA
WinoG
Avg. 5
Avg. 8
Mistral-7B-v0.3 (bf16)
5.45
8.32
54.35
80.13
83.64
80.67
47.40
81.99
47.75
74.19
74.27
68.77
QuaRot (NeurIPS ’24)
6.41
10.01
45.22
72.26
—
75.14
—
79.22
—
68.82
68.13
—
SmoothRot (SMC ’25)
6.19
9.78
46.59
73.19
—
75.97
—
79.11
—
68.11
68.59
—
SpinQuant (ICLR ’25)
5.80
—
52.50
77.30
80.20
79.20
58.40
80.30
48.90
72.30
72.32
68.60
PrefixQuant (TPAMI ’26)
5.76
—
50.00
77.74
—
78.39
—
80.41
—
70.88
71.48
—
Appendix
Table 5: W4A4KV4 quantization on Mistral-7B-v0.3. WT2/C4 perplexity ( ↓ ) and zero-shot accuracy (%, ↑ ). Avg. 5 averages ARC-c, ARC-e, HellaS, PIQA, and WinoG; Avg. 8 covers all eight tasks. The bf16 reference is ours; QuaRot and SmoothRot rows are their reported RTN results, and cross-paper protocols differ. A dash denotes an unreported result. Bold marks the best displayed quantized result per column.
Table 6: Ablations of metadata allocation and expert calibration. (a) Activation-only WikiText-2 PPL ( ↓ ) under matched representation budgets; H/PQ denote Hadamard/PrismQuant at k=max , and green parentheses give PQ−H . (b) W4A4KV4 WikiText-2 PPL ( ↓ ) for the MoE calibration variants at k=max ; Δ PPL is relative to shrinkage, from unrounded values. Bold denotes the lower PPL within each H/PQ pair in (a) and the lowest PPL in (b); mint rows mark the configuration adopted in the main tables.
(a) Both sites quantized: WikiText-2 PPL
Subspace estimator
Llama-3.2-3B
Llama-3.1-8B
PPL
Δ [90% CI]
PPL
Δ [90% CI]
All tokens (PrismQuant)
7.705
0
6.285
0
Top-1 token / seq. (OffQ-style)
7.723
+0.017 [ − 0.002, + 0.036]
6.301
+0.016 [ + 0.012, + 0.020]
BOS excluded
7.800
+0.095 [ + 0.092, + 0.098]
6.342
+0.057 [ + 0.049, + 0.065]
Global top 1% of tokens
7.711
+0.006 [ − 0.012, + 0.024]
6.293
+0.008 [ + 0.002, + 0.014]
Appendix
Table 7: Which tokens define the subspace. Activation-only WikiText-2 PPL with bf16 weights and KV states, three paired seeds, k=max . (a) Both sites quantized; Δ is the paired difference to the all-token estimator with its 90% Student- t interval. (b) Δ when only one site is quantized (BOS included); bold marks intervals that exclude zero. (c) Diagnostics at the down-projection input: captured group-constant energy f and per-group range relative to Hadamard. Mint rows mark our estimator.
(a) Calibration set: size and corpus
Calibration
WT2 PPL
Δ [90% CI]
C4 PPL
Δ [90% CI]
Hadamard (no calibration)
7.877
0
13.160
0
WT2, 64 seq.
7.690
−0.187 [ − 0.199, − 0.174]
12.851
−0.309 [ − 0.330, − 0.289]
WT2, 128 seq. default
7.694
−0.183 [ − 0.194, − 0.171]
12.844
−0.316 [ − 0.337, − 0.295]
WT2, 256 seq.
7.703
−0.174 [ − 0.185, − 0.163]
12.852
−0.309 [ − 0.330, − 0.287]
Appendix
Table 8: Calibration and estimation robustness on Qwen3-4B-Base. Activation-only WikiText-2 (WT2) and C4 PPL, three paired seeds, k=max ; Δ is the paired difference to Hadamard with its 90% interval. (a) Calibration set size and corpus. (b) Randomized eigensolver passes versus exact fp64 eigendecomposition; f is the captured energy at the down-projection input, Δ is relative to the exact solver. Mint rows mark the default.
(a) Quantizer format
Quantizer
Qwen3-4B-Base
Llama-3.2-3B
Had
PQ
Δ
Had
PQ
Δ
Asymmetric, g=128 default
7.877
7.697
−0.179
7.767
7.705
−0.061
Symmetric, g=128
7.958
7.764
−0.194
7.819
7.759
−0.060
Asymmetric, g=d ( k=1 )
8.075
7.857
−0.218
7.926
7.886
−0.041
Symmetric, g=d ( k=1 )
8.178
8.041
−0.137
8.008
7.995
−0.013
Appendix
Table 9: Quantizer format and target column. Activation-only WikiText-2 PPL, three paired seeds, both sites quantized. (a) The same PrismQuant rotation ( k=max ; k=1 for g=d ) under four quantizers; Δ is PQ minus Hadamard, and every 90% interval excludes zero. (b) Anchors moved from the constant Walsh column to a nonconstant one at g=128 , k=max ; Δ is relative to the constant column under the asymmetric and symmetric formats, and s is the offset-specific share Δasym/(PPLHad−PPLPQ) . Mint rows mark the configuration of the main tables.
Variant
k=8
k=max
PPL
Δ [90% CI]
PPL
Δ [90% CI]
Balanced Π , low-energy fillers default
7.707
0
7.697
0
Identity Π
7.710
+0.002 [ − 0.004, + 0.009]
7.697
− 0.001 [ − 0.007, + 0.006]
Random Π
7.708
+0.001 [ − 0.005, + 0.007]
7.694
− 0.003 [ − 0.011, + 0.004]
High-energy fillers
7.698
−0.009 [ − 0.015, − 0.003]
7.697
0 (identical)
Fixed signs D=+1
7.724
+0.017 [ + 0.011, + 0.023]
7.697
− 0.001 [ − 0.007, + 0.005]
Appendix
Table 10: Anchor placement, fillers, and signs on Qwen3-4B-Base. Activation-only WikiText-2 PPL, three paired seeds, G identical across rows; Δ is relative to the default at the same rank with its 90% interval. Hadamard: 7.877 .
Qwen3-4B-Base
Llama-3.2-3B
∣z∣/range , median / P99 / max (worst layer)
0.51 / 10.0 / 21.5
0.52 / 5.8 / 10.2
Offset error / step, median / P99 (worst layer)
0.0012 / 0.025
0.0012 / 0.017
Groups with error >0.5 step, overall
8×10−5%
5×10−5%
worst layer/site
0.003%
0.0008%
Median (offset error)/(unaligned cost)
0.0085
0.0093
Appendix
Table 11: fp16 offset rounding at fixed codes. PrismQuant at g=128 , k=max , activation-only, three seeds. Offset error is ∣zj−fp16(zj)∣ in units of the group’s step; the unaligned cost is 2∣c∣/15 for the same level c . Δ PPL rows re-store only the offset at the stated precision.
Figure 4: From null-space geometry to a shared range law. (a) Standardized token projections at the Llama-3.2-3B layer-27 down input; arrows are the group-constant direction pulled back through each rotation. (b) Cumulative energy in the leading directions at seven layers. (c) Group range relative to paired Hadamard versus 1−f for 2,520 activation, 280 V-cache, and 112 multi-slot observations under one pooled fit; dashed is the identity. (d) The same fitted line on 5,342 Qwen3-30B-A3B experts held out of the fit: bin medians (Q1–Q3 bars) follow the dense reference line, and the scatter is widest for experts with fewer than 2,048 tokens.
Model
Site
k=8 median ∣e∣
k=max median ∣e∣
k=max P90 ∣e∣
k=max median e
k=maxκPQ/κH
Llama-3.2-3B
q/k/v
1.7%
1.7%
8.9%
+1.7%
0.98
Llama-3.2-3B
down
11.1%
8.9%
15.0%
+8.4%
0.92
Llama-3.1-8B
q/k/v
2.0%
1.3%
6.4%
+1.3%
0.99
Llama-3.1-8B
down
12.2%
8.8%
15.8%
+8.6%
0.92
Appendix
Table 12: Per-site error of the range law at g=128 . Relative error e=(s−smeas)/smeas of the predicted step, activation-only protocol, three paired seeds; medians and 90th percentiles of ∣e∣ over layers, and the median signed error, at the default operating point k=8 and at k=max . Positive signed error means the law over-predicts the step; the last column is the crest-factor ratio of Equation 30 , 1/(1+mediane) .
Figure 5: Predicted versus measured quantization step. Each point is one layer and site of the activation-only studies at g=128 ; whiskers are 90% intervals over three paired seeds and the dashed line is the identity. Points above the line are over-predicted. The q/k/v input follows the identity closely at every rank; the down-projection input sits below it, more so on Qwen3-4B-Base; the ratio of measured to predicted step there is the crest-factor ratio of Equation 30 .
Figure 6: Activation design and local transform cost. (a) Activation representation budget versus WikiText-2 PPL on Llama-3.2-3B; (g,m) denotes group size and the number of represented directions. (b) Activation-only PPL recovery versus rank. The highlighted k=8 category is the deployment operating point; the actual rank is capped separately at each quantization site. Llama results use three seeds and 64 chunks per seed; Qwen results use three seeds and 146 chunks, with bf16 weights and KV states. (c) Original local cost measurements at T=2048 on an NVIDIA RTX PRO 6000 Blackwell Server Edition: 100ttransform/(tdecoderlayer+ttransform) . These separately benchmarked durations are not whole-model decode overhead; end-to-end deployment is evaluated separately.
Figure 7: Practical efficiency of the compact PrismQuant implementation. The upper strip shows the two-kernel R4 computation followed by the shared INT4 quantizer and GEMM; box widths do not represent time. (a) Eager prefill throughput relative to matched FP16, with 2048 input tokens per sequence. (b,c) Matched CUDA Graph decode latency and peak allocated memory on A40, using batch size 1 and a 2048-token prefix. (d) Local implementation ablations: pre-bound versus generic launches at T=1 , and complete R4 paths using Tensor Core versus shuffle-based Kernel B at T=2048 . Each ablation has its own reference; these are not whole-model speedups. Timing centers are pooled medians; ablation centers are medians of paired-session ratios. Whiskers show session ranges, not confidence intervals. All performance models use random weights.
Prefill ↑
Decode ↓
Graph deployment
Method
B1
B16
Eager
Graph
Peak GB ↓
Speedup ↑
Llama-3.2-3B (FP16)
10,163.48
13,985.13
21.86
15.04
8.74
1.00×
QuaRot–Hadamard
12,819.74
17,894.66
42.83
16.33
4.37
0.92×
PrismQuant ( k=8 )
12,674.40
17,630.27
43.08
16.65
4.39
0.90×
Llama-3.1-8B (FP16)
5,241.70
6,418.43
30.22
28.83
18.77
1.00×
QuaRot–Hadamard
7,607.40
9,556.00
52.31
23.10
8.15
1.25×
Appendix
Table 13: End-to-end performance on a single NVIDIA A40. Prefill reports input tokens/s ( ↑ ), and decode reports ms/token ( ↓ ). Both decode columns use the matched current-stream backend; Graph speedup and peak memory use CUDA Graph execution. Speedup is relative to the FP16 row of the same model and mode. All rows use random weights; bold marks the best quantized entry per model and metric.
Model
T=1
T=2048
T=32768
Llama-3.2-3B
1.25×
0.93×
0.85×
Llama-3.1-8B
1.73×
1.17×
1.12×
Appendix
Table 14: Rotation-plus-quantization frontend speedup over QuaRot. Each value is the reciprocal of the median of three paired-session PrismQuant/Hadamard elapsed-time ratios. Values above 1.00× indicate faster execution and are bold. These are directly measured local frontend chains, not end-to-end model speedups; T is the number of input activation rows.
Figure 8: Local end-to-end activation magnitudes at q_proj on Qwen3-8B-Base. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( k=8 ), and PrismQuant ( k=max ). The window is sample 0, tokens 0–127, and channels 0–511, with g=128 . Heights represent absolute input activations before quantization. Linear heights and square-root colors share limits across all four rows within each column; the pale plane marks z=0 . The rotated rows include upstream quantization effects.
Figure 9: Local end-to-end activation magnitudes at down_proj on Qwen3-8B-Base. The rotated rows are observed after the online R4 transform and before activation quantization. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( k=8 ), and PrismQuant ( k=max ). We retain sample 0, tokens 0–127, and channels 0–511, corresponding to four groups with g=128 . Heights are absolute activations, not centered residuals. Each column shares linear height limits and square-root color mapping across its four rows, with a pale z=0 plane.
Figure 10: Paired-local activation magnitudes at q_proj on Qwen3-8B-Base. All methods transform the same captured input tensor within each block, isolating the local effect of rotation. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( k=8 ), and PrismQuant ( k=max ). The displayed window contains sample 0, tokens 0–127, and channels 0–511, with g=128 and no pooling. Heights are linear and colors use square-root mapping, with identical local limits across the four rows of each column; the pale plane marks z=0 .
Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to 2.3× prefill speedup and 3.5× memory saving over FP16.
Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in-training by introducing a LogSumExp term to the loss that breaks the symmetries, thus selecting a basis that minimizes activation outliers. A stop-gradient operator ensures that only rotation matrices are updated, yielding the language modeling objective completely unaltered. Our requires no specific calibration data, no quantization simulation, and adds negligible training overhead. With the LLaMA-2 7B model under W4A4 quantization with group size 128, perplexity drops from 8.22 to 6.73, competing with post-training methods that require frozen models and calibration datasets. Under W4A16, perplexity drops from 11.16 to 5.45. Code is available at https://github.com/MPedraBento/gauge-quant.
Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an effective and efficient data-aware alternative for post-training quantization. Experiments on Llama and Mistral models show that SpectrumQuant is competitive with state-of-the-art rotation strategies while requiring less than one-third of the GPU memory and significantly less time than learned rotations approaches like SpinQuant and DartQuant.
Dwaipayan Haldar, Indranil Patra, Azhar Yousuf +1
Indian Institute of Science Bengaluru-560012, Karnataka, India