cs.ITSep 4, 2026

A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products

Authors: Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara, Keita Teranishi, Sudip Seal

Organizations: Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA

Abstract

We consider the problem of minimizing error in quantized matrix multiplication C=ABC=AB. Scalar quantization of the factors introduces rounding errors whose scale depends on the maximum absolute entries -- the ranges -- of their rows and columns. These ranges determine the quantization grid steps. To reduce the error, we optimize over product-preserving transformations that alter the factor ranges and grid steps without changing CC. Specifically, we seek the smallest leading expected squared error over invertible inner changes of basis and orthogonal outer rotations. Under independent, zero-mean subtractive dither noise on an unbounded lattice, we prove the output-only bound Elead≥(cA+cB)/K∥AB∥∗2E_{\rm lead} \ge (c_A+c_B)/K \Vert AB\Vert_*^2, where KK is the inner dimension, cAc_A and cBc_B are normalized noise variances, and ∥AB∥∗\Vert AB\Vert_* is the nuclear norm. The bound is tight: an SVD-aligned Hadamard construction attains the infimum whenever a Hadamard matrix of order KK exists, including every power of two, while an SVD-aligned DCT construction is within a factor of two for every KK. Without outer rotations, Gram-matrix balancing minimizes factorization energy, and finite-set flattening achieves the bound within Clog⁡(K(m+n))C\log(K(m+n)). For power-of-two KK, conditional expectations deterministically select the Hadamard signs in O((m+n)K2)O((m+n)K^2) exact-real operations. Synthetic experiments verify both constructions and illustrate the tradeoff between regularization and conditioning. These results characterize the full-gauge optimum and quantify the cost of preserving row and column indices.

Figures & tables

Explore similar work

CardsList
  1. Contraction-Gauge Preconditioning for Quantized Matrix Multiplication

    Jul 21, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Post-Training QuantizationLow-Bit Quantization

  2. Product-Aware Deterministic Rounding for Quantized Matrix Multiplication

    Sep 11, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Neural Network QuantizationActivation Quantization