MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference

Romain Facq; Sami Ben Ali; Olivier Sentieys

cs.ARSep 23, 2026

MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference

Authors: Romain Facq, Sami Ben Ali, Olivier Sentieys

Organizations: Univ Rennes, Inria, CNRS, IRISA, Rennes, France

Abstract

Microscaling quantization techniques are increasingly used to represent neural network parameters with 8 bits or fewer while preserving near-full precision accuracy. However, applying these methods efficiently in convolutional layers is not straightforward. A naive approach transfers full-precision weights and activations to processing units and quantizes each tensor twice, resulting in much more memory movement than expected. Additional overhead comes from the activation tensors, whose sizes grow substantially because of the im2col transformation applied before quantization. We propose MicroQonv, a way to combine microscaling with convolutional layers' forward and backward operations by quantizing each tensor only once and quantizing the activation tensor before applying a modified version of im2col: channel-batch-first im2col. MicroQonv reduces the quantization cost by a factor of $\times2$ for weights and gradients, and by up to $\times9$ for activations, at a negligible accuracy cost. It reduces memory movement and storage by up to $\times7.53$ compared to their full-precision counterparts. This way, MicroQonv reduces microscaling-quantized activation memory movement by $\times3.5$ for state-of-the-art object detection models YOLOV8nano and $\times2.2$ for YOLOV26nano. It also enables 4-bit microscaling in a quantized latent replay strategy for continual learning at the edge, improving accuracy by +5.7% to +11%.

Figures & tables

Fig. 1 : Generic compute architecture model considered.

Fig. 2 : Example of the channel-first layout for a convolution layer with

K_{h}=K_{w}=2

and

H_{in}=H_{out}=3

. The tensor is flattened along the spatial and batch dimensions, producing

NH_{in}W_{in}

contiguous representation across columns, each containing all

C_{in}

channels. These columns are streamed to generate the im2col matrix. Forming the transposed representation requires reading across lines rather than columns.

Fig. 3 : Channel-batch-first layout: activations are stored compactly in off-chip memory for contiguous im2col and transpose access. Highlighted squares show MicroQconv quantization blocks across

N

and

C_{in}

dimensions. This layout also enables activation reuse, reducing redundant memory reads by avoiding repeated accesses to the same data.

Method	Per elt. quantization			Backward Mem. Move.
	A	W	G	A	W
Dacapo [ 24 ]	$\times 18$	$\times 2$	$\times 2$	$\times 7.53$	$\times 7.53$
Cuyckens et al. [ 25 ]	$\times 9$	$\times 1$	$\times 1$	$\times 9$	$\times 1$
MicroQonv	$\times\textbf{1}$	$\times\textbf{1}$	$\times\textbf{1}$	$\times\textbf{1}$	$\times\textbf{1}$

TABLE I : Quantization overhead and memory movement during backpropagation for Weights (W), Activations (A) and Gradients (G) of a convolutional layer with

K_{h}=K_{w}=3

quantized with 4-bit microscaling.

Method	Test acc %		Normalized MSE
	R32-C100	R18-Inet100	A	W	G
FP32 baseline	$71.05\pm.29$	$75.55\pm.22$	$NA$	$NA$	$NA$
Dacapo [ 24 ]	$70.86\pm.24$	$75.19\pm.12$	$0.891$	$0.982$	$0.827$
Cuyck. et al. [ 25 ]	$70.82\pm.11$	$75.20\pm.18$	$0.875$	$0.992$	$0.996$
MicroQonv	$70.62\pm.13$	$75.08\pm.36$	$1.0$	$1.0$	$1.0$
MicroQonv+	$71.10\pm.28$	$75.76\pm.12$	$0.236$	$1.0$	$1.0$

TABLE II : Comparison of test accuracy and quantization MSE per element on ResNet-32 (R32) with CIFAR-100 (C100) and Resnet-18 (R18) with ImageNet (Inet100).

Method	Finetuning		From scratch
	mAP50	mAP50-95	mAP50	mAP50-95
FP32 baseline	$82.06\pm.06$	$60.89\pm.06$	$76.53\pm.12$	$55.56\pm.10$
MicroQonv -8	$81.87\pm.15$	$60.62\pm.12$	$76.52\pm.27$	$55.39\pm.19$
Dacapo-8 [ 24 ]	$81.88\pm.10$	$60.75\pm.10$	-	-
Cuyck. et al. -8	$81.85\pm.06$	$60.75\pm.11$	-	-

TABLE III : Object detection performance on YOLOV8nano model and PascalVOC dataset.

Method	Finetuning		From scratch
	mAP50	mAP50-95	mAP50	mAP50-95
FP32 baseline	$53.91\pm.40$	$33.75\pm.24$	$74.26\pm 1.2$	$49.44\pm.73$
MicroQonv -8	$55.10\pm 1.3$	$34.20\pm.78$	$74.11\pm 1.2$	$48.74\pm.55$
Dacapo-8 [ 24 ]	$54.44\pm.47$	$33.56\pm.89$	-	-
Cuyck. et al. -8	$54.88\pm 1.3$	$33.90\pm.79$	-	-

TABLE IV : Object detection performance on YOLOV26nano model and KITTI dataset.

Fig. 4 : Memory transfers considered for DRAM simulation.

Method	Activations			Weights
R32-Cifar100	S(MB)	E(mJ)	L(ms)	S(MB)	E(mJ)	L(ms)
Dacapo	38.67	92.56	46.63	1.87	4.402	1.95
Cuyck. et al.	42.57	100.40	50.89	0.244	0.529	0.164
MicroQonv	5.04	11.68	6.09	0.244	0.529	0.164
R18-Imagenet	S(MB)	E(mJ)	L(ms)	S(MB)	E(mJ)	L(ms)
Dacapo	279.44	669.68	337.91	46.72	111.9	56.37

TABLE V : Energy (E) in mJ and Latency (L) in ms for all transfers in Fig. 4 . Tensor Size (S) is in MegaBytes (MB).

Technique	Elements Format			Test acc %
	A	W	G	R32-C100	R18-Inet100
Baseline	FP32	FP32	FP32	$71.05\pm.29$	$75.55\pm.22$
Dacapo [ 24 ]	FP4	FP4	FP4	$70.86\pm.24$	$75.19\pm.12$
Cuyck. et al. [ 25 ]	FP4	FP4	FP4	$70.82\pm.11$	$75.20\pm.18$
MicroQonv	FP4	FP4	FP4	$70.62\pm.13$	$75.08\pm.36$
MicroQonv+	FP6	FP4	FP4	$71.10\pm.28$	$75.76\pm.12$

TABLE VI : Performance comparison of quantization methods on ResNet-32 (R32) with CIFAR-100 (C100) and Resnet-18 (R18) with ImageNet (Inet100). FP4E corresponds to the E3M0 format.

Panel of Figure 11 — Fig. 5 : Test of accuracy along training of MicroQonv -6 (in blue) and FP32 baseline (in green).

Method	Finetuning		From scratch
	mAP50	mAP50-95	mAP50	mAP50-95
FP32 baseline	$82.33\pm.02$	$61.33\pm.09$	$77.11\pm.29$	$56.25\pm.22$
MicroQonv -6	$82.12\pm.20$	$61.08\pm.13$	$76.95\pm.26$	$55.98\pm.26$
Per Channel-6	$81.31\pm.14$	$60.15\pm.12$	$76.58\pm.20$	$55.55\pm.15$
Per Tensor-6	$65.88\pm.42$	$43.89\pm.35$	-	-
Dacapo-6 [ 24 ]	$82.19\pm.10$	$61.14\pm.06$	-	-

TABLE VII : 6 bit Object detection performance on YOLOV8nano-PascalVOC.

Method	Finetuning		From scratch
	mAP50	mAP50-95	mAP50	mAP50-95
FP32 baseline	$53.91\pm.73$	$33.86\pm.61$	$74.26\pm 1.2$	$49.44\pm.73$
MicroQonv -6	$54.11\pm.93$	$33.51\pm.51$	$73.02\pm.62$	$48.18\pm.54$
Per Channel-6	$51.22\pm.64$	$31.49\pm.39$	$71.96\pm 1.1$	$46.99\pm.81$
Per Tensor-6	$9.1\pm.76$	$4.09\pm.42$	-	-
Dacapo-6 [ 24 ]	$54.08\pm.11$	$33.36\pm.63$	-	-

TABLE VIII : 6 bit Object detection performance on YOLOV26nano-KITTI.

Fig. 6 : Quantized Latent Replay (QLR) strategy overview. Feature extractor layers are also quantized.

Fig. 7 : End-test accuracy vs replay buffer memory size.

CIFAR-100	25MB	50MB	75MB	125MB
QLR [ 31 ]	$39.3\pm.7$	$49.1\pm.7$	$54.2\pm.5$	$60.1\pm.7$
MicroQonv	$50.3\pm.5$	$59.0\pm.5$	$62.4\pm.7$	$65.8\pm.5$
CORE50	15MB	25MB	50MB	75MB
QLR	$59.1\pm.5$	$64.0\pm.5$	$70.7\pm.8$	$73.2\pm 1.1$
MicroQonv	$68.5\pm 1.0$	$72.2\pm.4$	$75.5\pm.5$	$78.0\pm.8$
CUB200	10MB	20MB	30MB	40MB

TABLE IX : LR end test accuracy of ResNet18 for various LR memory sizes in MB and datasets.

Explore similar work

VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models
Oct 2, 2026Elad Dror Cohen, Ofir Gordon, Lior Dikstein +2Neural Network Quantization Post-Training Quantization
Arm AI Research (AAIR), Arm Ltd
MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration
Sep 17, 2026Yuan Liao, Jae-sun SeoAI Accelerator Inference VLM Quantization
Cornell Tech, Cornell University New York, NY, USA
16-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costs
Oct 2, 2026Rui Liu, Benjamin PaaßenNeural Network Quantization Efficient Neural Network Inference
Faculty of Technology, Bielefeld University

Oct 2, 2026J/K move · Enter open · S save

VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models

Elad Dror Cohen, Ofir Gordon, Lior Dikstein, Idan Achituve+1

Arm AI Research (AAIR), Arm Ltd

Neural Network Quantization Post-Training Quantization Activation Quantization

Open paper AskPDF arXiv

Microscaling (MX) formats are emerging as a hardware-supported approach to efficient training and inference. They combine low-precision elements with shared block scales, but their impact on vision models remains underexplored. We systematically investigate post-training MX quantization across vision models and tasks. An analysis of direct conversion identifies three sources of error: block-scale representation, the poor alignment of some small convolutional weight tensors with nonuniform element grids, and the underuse of signed codes by nonnegative activations. These findings motivate VisionMX, a post-training MX quantization method that optimizes bounded weight rounding and applies a foldable affine correction to activations. We evaluate VisionMX across image classification, object detection, semantic segmentation, and low-light image enhancement using several MX-style formats. It improves on direct conversion and the evaluated post-training quantization baselines, with the largest performance recoveries in architectures most sensitive to MX conversion

Abstract

Explore similar work

VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models

MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

16-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costs