Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer

Haoran Jin; Kangqi Zhang; Jirong Yang; Barry Lyu; Qiuyi Ding; Ruijie Gao; Nathan Bleier

cs.AISep 26, 2026

Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer

Authors: Haoran Jin, Kangqi Zhang, Jirong Yang, Barry Lyu, Qiuyi Ding, Ruijie Gao, Nathan Bleier

Organizations: Department of Electrical Engineering and Computer Science, University of Michigan

Abstract

Matrix multiplications dominate the inference cost of modern transformer-based vision models, yet existing efficiency techniques such as post-training quantization and mixed-precision inference are largely limited to the small set of fixed-width formats (INT4, INT8, BF16, and FP16) supported by conventional accelerators. We revisit stochastic computing (SC) as a way to lift this constraint: viewed as a dense adaptive quantizer, SC controls precision by bit-stream length L rather than a fixed datapath, while each multiplication reduces to a single AND/XNOR gate. We build a GPU library that emulates SC matrix multiplication at scale, exposes stream lengths as first-class kernel arguments, and evaluates SC end-to-end on image classification, object detection and instance segmentation, class-conditional image generation, and visual world-model planning. On top of this substrate, we develop a dynamic per-row mixed-precision policy that assigns stream length per token or group at matched average budget, requires no retraining, and uses the same SC hardware across schedules. Across tasks, SC remains competitive with fixed-format INT quantization at matched bit budgets, while per-row mixed precision helps maintain accuracy at lower average stream lengths. These results provide software-level feasibility evidence that SC can serve as a dense-precision substrate for fine-grained mixed-precision inference on modern vision transformers.

Figures & tables

Panel of Figure 1 — Figure 1: Circuit primitives of stochastic computing. (a) A SNG implemented as an integer comparator: at each cycle the source value is compared against an RNG draw, emitting a single bit. (b) Stochastic multiplication via a bitwise gate: the unipolar product is realized by an AND gate whose output bit-probability factors as the product of the input marginals. The red circle denotes early termination. (c) Conditional bitstream generation (C-BSG): the second RNG is gated by the first stream so the conditional distribution of $z^{B}$ matches its marginal, enforcing $\mathrm{SCC}=0$ and making the bipolar XNOR multiplier exact at finite $L$ .

Figure 2: Overview of our adaptive fine-grained quantization and scheduling flow.

	SC gate		INT8 mult.		Energy/Op Reduction
Pair	Area	Power	Area	Power	$k=8$	$k=7$	$k=6$	$k=5$
	( $\mu$ m 2 )	( $\mu$ W)	( $\mu$ m 2 )	( $\mu$ W)
And / UMul	1.06	0.37	285.4	168.7	$1.77\times$	$3.54\times$	$7.08\times$	$14.2\times$
Xnor / Mul	1.60	0.63	249.2	249.7	$1.56\times$	$3.13\times$	$6.25\times$	$12.5\times$

Table 1: SC vs. binary multiplier comparison at 400 MHz.

Figure 3: SC sensitivity is highly non-uniform across operator class, block depth, and stream length: replacing a single

(o,b)

matmul with SC perturbs the final ViT-L/14 feature far more for late FC2 blocks than for any other tile, motivating the mixed-precision policy of Section 4.4.

Method	W/A	top-1 (%)	top-5 (%)	$\Delta$ vs FP
FP (DINOv2 ViT-L/14)	$16/16$	86.40	97.65	$\;\;\;0.00$
QwT	$8/8$	84.57	96.56	$-1.83$
SC uniform	$8/8$	85.57	97.40	$-0.83$
SC uniform + compensation	$8/8$	85.70	97.47	$-0.70$
SC uniform	$\log_{2}\!192/\log_{2}\!192$	84.67	97.11	$-1.73$
SC uniform + compensation	$\log_{2}\!192/\log_{2}\!192$	85.42	97.43	$-0.98$

Table 2: ImageNet-1k classification (full val,

N{=}50{,}000

). W/A is weight/activation bits: the underlying INT setting for QwT; for SC,

W{=}A{=}\log_{2}\bar{L}

with

\bar{L}

the average per-row stream length.

Method	W/A	AP b	AP m	$\Delta_{b}/\Delta_{m}$
FP (EVA-01 ViT-g)	$16/16$	62.06	52.39	$\;\;\;0.00/\;\;0.00$
QwT	$8/8$	61.20	51.59	$-0.86/-0.80$
SC uniform	$8/8$	61.98	52.18	$-0.08/-0.21$
SC uniform	$\log_{2}\!192/\log_{2}\!192$	61.69	51.85	$-0.37/-0.54$
SC uniform + compensation	$\log_{2}\!192/\log_{2}\!192$	61.82	52.06	$-0.24/-0.33$
QwT	$7/7$	61.13	51.51	$-0.93/-0.88$

Table 3: COCO val object detection & instance segmentation, EVA-01 ViT-g + cascade Mask R-CNN (

N{=}5000

1024^{2}

). W/A as in Table 2 .

Method	$\bar{L}$	IS $\uparrow$	sFID $\downarrow$	KID $\times 10^{3}\downarrow$	FID $\downarrow$
FP	–	$285.95$	$24.75$	$13.47{\pm}1.24$	$24.81$
SC MP	64	$279.76$	$19.93$	$9.7717{\pm}1.0457$	$17.33$
SC uniform	96	$284.62$	$20.91$	$11.1134{\pm}1.1318$	$18.47$

Table 4: ImageNet class-conditional generation with DiT-XL/2,

256{\times}256

. All rows use the same sampler,

50

denoising timesteps,

\mathrm{CFG}{=}4.0

, and

10{,}000

generated images (

10

per ImageNet class). Schedule records the SC length setting,

\bar{L}

is the realized average stream length over SC-executed matmuls, and KID is reported as

10^{3}{\times}\mathrm{KID}

Method	W/A	SR	Plateau iter	$\Delta$ vs FP
FP (Dino-WM, ViT-S/14 enc. + 6-block pred.)	$16/16$	$0.833$	$5$	$\;\;\;0.000$
SC uniform	$8/8$	$0.833$	$3$	$\;\;\;0.000$
SC uniform	$\log_{2}\!192/\log_{2}\!192$	$0.833$	$3$	$\;\;\;0.000$
SC MP	$\log_{2}\!192/\log_{2}\!192$	$0.933$	$16$	$\;\;\;{+}0.100$
SC uniform	$7/7$	$0.800$	$16$	$-0.033$
SC MP	$7/7$	$0.833$	$13$	$\;\;\;0.000$

Table 5: Visual planning success rate on Dino-WM wall_single , matched CEM budget. SR is the fraction of

30

episodes solved within

20

MPC steps; bit budget follows Table 2 .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Sample images generated by full precision and SC with uniform stream lengths $L\in\{32,48,64,96,128\}$ . The eight prompts are: “golden retriever”, “otter”, “lesser panda”, “geyser”, “macaw”, “valley”, “balloon”, and “arctic fox”.

Adaptive Quantization

See all 19 papers

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator
Sep 20, 2026Mikolaj Walczak, Edward Humes, Chao Fang +2Mixed-Precision Quantization AI Accelerator Inference
Johns Hopkins University Baltimore, Maryland, USA
MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers
Jul 30, 2026Md. Mehrab Hossain Opi, Robiul Islam Ryad, Md. Umar FarukMixed-Precision Quantization Vision Transformer
Department of Computer Science and Engineering Khulna University of Engineering & Technology (KUET) Khulna 9203, Bangladesh
Sustainable Transformer Neural Network Acceleration with Stochastic Photonic Computing
Apr 10, 2026S. Afifi, O. Alo, I. Thakkar +1Efficient Transformer Inference Transformer Inference
Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA · Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA

Sep 20, 2026J/K move · Enter open · S save

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

Mikolaj Walczak, Edward Humes, Chao Fang, Marian Verhelst+1

Johns Hopkins University Baltimore, Maryland, USA

Mixed-Precision Quantization AI Accelerator Inference Efficient Transformer Inference Post-Training Quantization Weight-Only Quantization

Open paper AskPDF arXiv

Due to limited support for intra-tensor heterogeneous precision in conventional accelerators, neural network quantization remains largely restricted to per-tensor precision assignment. We present ShatterQuant, a hardware-software co-designed framework enabling mixed-precision quantization within each tensor by assigning independent bit-widths to blocks of a weight projection. ShatterQuant couples precision granularity with PE configuration, such that each precision determines an effective block height. We introduce (1) a hardware-aware post-training method that assigns intra-tensor precision based on block-level standard deviation and weight sensitivity; (2) the ShatterQuant Transformer Accelerator supporting 1/2/4/8-bit weight precision, precision-dependent PE configuration, block rescaling, and integrated softmax and piecewise-linear nonlinearities; and (3) an evaluation of model-hardware tradeoffs using an implementation in the TSMC 16nm PDK operating at 1 GHz, achieving 1.5 TOPS, 760 GOPS/

mm^2

area efficiency, and 2.8 TOPS/W energy efficiency. On DeiT and ImageNet-1K, ShatterQuant achieves accuracy within

3.3\%

of state-of-the-art mixed-precision techniques while using a 2 bit lower effective bitwidth, while for PixelDiT demonstrates comparable generation quality. ShatterQuant demonstrates how fine-grained intra-tensor mixed-precision can be realized through hardware-software co-design.

Abstract

Adaptive Quantization

Explore similar work

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

Sustainable Transformer Neural Network Acceleration with Stochastic Photonic Computing