VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models

Elad Dror Cohen; Ofir Gordon; Lior Dikstein; Idan Achituve; Hai Victor Habi

cs.CVOct 2, 2026

VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models

Authors: Elad Dror Cohen, Ofir Gordon, Lior Dikstein, Idan Achituve, Hai Victor Habi

Organizations: Arm AI Research (AAIR), Arm Ltd

Abstract

Microscaling (MX) formats are emerging as a hardware-supported approach to efficient training and inference. They combine low-precision elements with shared block scales, but their impact on vision models remains underexplored. We systematically investigate post-training MX quantization across vision models and tasks. An analysis of direct conversion identifies three sources of error: block-scale representation, the poor alignment of some small convolutional weight tensors with nonuniform element grids, and the underuse of signed codes by nonnegative activations. These findings motivate VisionMX, a post-training MX quantization method that optimizes bounded weight rounding and applies a foldable affine correction to activations. We evaluate VisionMX across image classification, object detection, semantic segmentation, and low-light image enhancement using several MX-style formats. It improves on direct conversion and the evaluated post-training quantization baselines, with the largest performance recoveries in architectures most sensitive to MX conversion

Figures & tables

Panel of Figure 1 — Figure 1: Three sources of sensitivity under block-scaled quantization. (a) Relative loss from the floating-point reference under separate weight-only and activation-only MXFP4-UE8M0 RTN quantization; each point is a model–task pair, and the inset enlarges the low-loss region. (b) Example quadric loss approximation, the contour plot compares the rounding up–down set (dashed rectangle) with the 21-point RangeRound set $\mathcal{M}_{\mathrm{FP4}}(\bm{w})$ (teal points). The star denotes the float weights, B the coincident nearest and best assignment in the rounding up–down set, and C the full-grid and RangeRound optimum. (c) MXFP4 element-code occupancy of ReLU output with and without AAC.

Figure 2: RangeRound per-weight penalty

1-|2u_{i}-1|^{\beta}

in normalized MX element-code space, for

\beta\in\{20,2\}

. The penalty vanishes at legal FP4 codes

q_{k}

(open circles) and reaches one at cell midpoints. The axis break omits

(1.5,3)

. Colored markers illustrate relaxed values

\widetilde{z}=0.10

and

4.10

on the

\beta=2

curve.

Format	Method	Tiny	Small		Medium		Large
		MobileNetV2	ResNet-18	CaiT-XXS24	ConvNeXt-Tiny	EfficientFormer-L3	ResNeXt-101 32x8d	EfficientViT-L2
		14.30 MB	46.80 MB	47.90 MB	114.40 MB	125.60 MB	355.20 MB	255.00 MB
Float	–	72.15	69.76	78.39	82.52	82.55	82.83	85.06
MXFP4 UE8M0	RTN	0.40	57.72	69.48	76.27	63.07	64.89	0.10
	AdaRound	41.20 $\pm$ 0.45	65.73 $\pm$ 0.10	65.24 $\pm$ 0.18	78.67 $\pm$ 0.05	79.33 $\pm$ 0.09	55.24 $\pm$ 0.46	2.54 $\pm$ 0.37
	QDrop	57.99 $\pm$ 0.14	66.27 $\pm$ 0.13	73.39 $\pm$ 0.06	79.75 $\pm$ 0.06	80.17 $\pm$ 0.05	78.80 $\pm$ 0.06	4.80 $\pm$ 0.13

Table 1: ImageNet-1K Top-1 classification accuracy (%). We report the mean and sample standard deviation over five seeds. Within each format and network, darker blue marks the best result and light blue marks the second-best result.

Format	Method	Faster R-CNN R50-FPN	RetinaNet R50-FPN	FCOS R50-FPN
Float	–	36.90	36.40	39.10
MXFP4 UE8M0	RTN	24.50	23.80	25.10
	AdaRound	5.45 $\pm$ 0.18	30.83 $\pm$ 0.08	31.96 $\pm$ 0.04
	QDrop	32.45 $\pm$ 0.08	31.69 $\pm$ 0.10	33.90 $\pm$ 0.10
	VisionMX-LW (Ours)	2.11 $\pm$ 0.12	32.62 $\pm$ 0.10	34.81 $\pm$ 0.03
	VisionMX-BW (Ours)	33.15 $\pm$ 0.10	32.30 $\pm$ 0.08	34.69 $\pm$ 0.08

Table 2: W4A4 post-training quantization results on COCO val2017 object detection using MXFP4 with UE8M0 and UE5M3 block scales. Higher AP is better. Within each format and network, darker blue marks the best result and light blue marks the second-best result.

Format	Method	DeepLabV3 R50	DeepLabV3 MobileNetV3
Float	–	66.20	60.00
MXFP4 UE8M0	RTN	56.90	4.80
	AdaRound	59.72 $\pm$ 0.20	20.08 $\pm$ 0.61
	QDrop	62.61 $\pm$ 0.19	47.21 $\pm$ 0.46
	VisionMX-LW (Ours)	62.24 $\pm$ 0.09	52.96 $\pm$ 0.21
	VisionMX-BW (Ours)	63.13 $\pm$ 0.08	53.31 $\pm$ 0.28

Table 3: W4A4 post-training quantization results on COCO val2017 semantic segmentation. Higher mIoU is better. Within each format and network, darker blue marks the best result and light blue marks the second-best result.

RR	Bias	AAC	MobileNetV2	CaiT-XXS24
✗	✗	✗	52.73	73.15
✗	✗	✓	64.54	73.27
✗	✓	✗	54.91	73.16
✗	✓	✓	64.84	73.55
✓	✗	✗	56.12	73.35
✓	✗	✓	65.24	73.50

Table 4: VisionMX-BW ablation with MXFP4-UE8M0. ImageNet Top-1 accuracy is reported in percent.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: ResNet-18 MXFP4 block-scale diagnostic on ImageNet top-1 accuracy. Both runs use group size 32, RTN, no bias correction, and no AdaRound. In the weight-only setting, activations remain in floating point; in the activation-only setting, weights remain in floating point. The plots show UE8M0 through UE1M7.

Panel of Figure 4 — Figure 4: ConvNeXt-Tiny MXFP4 block-scale diagnostic on ImageNet top-1 accuracy. Both runs use group size 32, RTN, no bias correction, and no AdaRound. In the weight-only setting, activations remain in floating point; in the activation-only setting, weights remain in floating point. The plots show UE8M0 through UE1M7.

Model	Mode	Float	UE8	Best
ResNet-18	W	69.76	62.43	65.87 (UE5M3)
ResNet-18	A	69.76	63.14	$65.8$ (UE4M4)
ConvNeXt-Tiny	W	82.51	81.32	81.82 (UE5M3)
ConvNeXt-Tiny	A	82.51	78.69	80.29 (UE4M4)

Appendix

Table 5: ImageNet top-1 accuracy for the block-scale diagnostic. W and A denote weight-only and activation-only settings. Best results are selected among the plotted formats UE8M0 through UE1M7; the ResNet-18 activation-only UE4M4 accuracy is estimated from the plotted point. UE8 is the MX-spec scale format. UE5 denotes UE5M3, with five exponent bits and three mantissa bits.

Format	E/M bits	Bias	Positive scale range
UE8M0	8/0	0	$2^{-127}$ – $2^{127}$
UE7M1	7/1	63	$2^{-62}$ – $1.5\cdot 2^{64}$
UE6M2	6/2	31	$2^{-30}$ – $1.75\cdot 2^{32}$
UE5M3	5/3	15	$2^{-14}$ – $1.875\cdot 2^{16}$
UE4M4	4/4	7	$2^{-6}$ – $1.9375\cdot 2^{8}$
UE3M5	3/5	3	$2^{-2}$ – $1.96875\cdot 2^{4}$

Appendix

Table 6: Unsigned eight-bit block-scale formats in the diagnostic sweep. The ranges list positive scale values before any optional global multiplier.

Format	Assignment rule	Weight pair	Loss
FP4	Best up–down	$(1.5,4)$	0.365
FP4	Full grid	$(2,4)$	0.315
INT4	Nearest	$(12/7,36/7)$	0.117041
INT4	Best up–down	$(6/7,36/7)$	0.110918
INT4	Full grid	$(6/7,36/7)$	0.110918

Appendix

Table 7: Enumeration of all legal pairs and restricted candidate sets for

\bm{w}=(1.4,4.9)^{\top}

and coupling

0.5

. INT4 losses are rounded to six decimal places.

Figure 5: Correlation sweep over the admissible interval

[0,0.5]

for the fixed example, with one logarithmic loss axis. FP4 full-grid and window minima coincide, while INT4 full-grid and best up–down minima coincide. The vertical reference line marks the FP4 improvement threshold

7/18

. Minima are computed by exhaustive enumeration.

Figure 6: INT4 and FP4 reconstruction for the same weights and matrix satisfying the factor-two row condition. The dashed blue rectangles enclose the rounding up–down candidates. In the FP4 panel, teal points mark all 21 candidates in

\mathcal{M}_{\mathrm{FP4}}(\bm{w})

, including points beneath solution markers. Nested open markers indicate coincident assignments. The full-grid FP4 optimum is outside the rounding up–down set and inside the window; the INT4 full-grid optimum remains within the rounding up–down set. Both panels use the same axis limits and loss contours.

Format	Method	Tiny	Middle		Large
		DeiT-T	ViT-S/16	Swin-T	DeiT-B
		22.90 MB	88.20 MB	114.30 MB	346.30 MB
Float	TBD	72.17	81.38	81.38	81.98
MXFP4 UE8M0	RTN	63.54	75.12	76.97	79.71
	AdaRound	61.69 $\pm$ 0.06	73.07 $\pm$ 0.09	77.96 $\pm$ 0.07	78.99 $\pm$ 0.02
	QDrop	66.50 $\pm$ 0.11	75.88 $\pm$ 0.08	79.40 $\pm$ 0.07	80.34 $\pm$ 0.03

Appendix

Table 8: ImageNet-1K Top-1 classification accuracy (%) for ViT-family models. Values with uncertainty report the mean and sample standard deviation over seeds 0–4; RTN is deterministic and uses seed 0 only. MR-GPTQ (FP4 block32) and SpinQuant (FP4 full) are joint weight-and-activation results without reported seed variation. VisionMX results are retained from the original table. Within each format and network, darker blue marks the best result and light blue marks the second-best result.

Format	Method	RetinexFormer			HVI-CIDNet
		PSNR $\uparrow$	SSIM $\times 100$ $\uparrow$	LPIPS $\times 100$ $\downarrow$	PSNR $\uparrow$	SSIM $\times 100$ $\uparrow$	LPIPS $\times 100$ $\downarrow$
Float	–	22.80	85.43	17.08	23.89	87.92	12.24
MXFP4 UE8M0	RTN	22.36	76.39	22.51	20.48	73.02	26.76
	AdaRound	22.59	77.52	22.70	23.09 $\pm$ 0.06	77.94 $\pm$ 0.21	24.92 $\pm$ 0.33
	QDrop	22.55 $\pm$ 0.05	77.08 $\pm$ 0.12	22.67 $\pm$ 0.19	22.93 $\pm$ 0.06	78.12 $\pm$ 0.05	24.11 $\pm$ 0.13
	VisionMX-LW	22.45	79.98	21.38	23.33 $\pm$ 0.03	82.39 $\pm$ 0.06	18.24 $\pm$ 0.19

Appendix

Table 9: W4A4 post-training quantization results on all 100 LOLv2-Real test pairs for RetinexFormer and HVI-CIDNet. Higher RGB PSNR and SSIM and lower LPIPS-Alex are better. SSIM and LPIPS-Alex values are multiplied by 100. Within each format, network, and metric, darker blue marks the best result and light blue marks the second-best result.

Task	Model	Metric	Float	Weight-only	Activation-only
Classification	MobileNetV2	Top-1 (%)	72.15	1.86	34.43
Classification	DeiT-T	Top-1 (%)	72.17	69.00	67.08
Classification	ResNet-18	Top-1 (%)	69.76	63.14	63.29
Classification	CaiT-XXS24	Top-1 (%)	78.39	74.85	74.04
Classification	ConvNeXt-Tiny	Top-1 (%)	82.52	81.30	78.63
Classification	EfficientFormer-L3	Top-1 (%)	82.55	75.46	76.85

Appendix

Table 10: RTN single-side ablation using MXFP4-UE8M0. Higher is better for all metrics.

			UE5M3 scale		UE8M0 scale
Task	Model	Float	RTN w/o BC	RTN w/ BC	RTN w/o BC	RTN w/ BC
Classification	MobileNetV2	72.15	69.44	70.95	64.60	69.81
Classification	DeiT-T	72.17	71.87	71.84	71.79	71.78
Classification	ResNet-18	69.76	69.16	69.24	69.17	69.35
Classification	CaiT-XXS24	78.39	78.18	78.17	77.97	77.95
Classification	ConvNeXt-Tiny	82.52	82.23	82.28	82.17	82.20

Appendix

Table 11: MXFP8 E4M3 RTN results using UE5M3 and UE8M0 scales. The metrics are top-1 accuracy (%) for classification, bounding-box mAP (%) for detection, mIoU (%) for segmentation, and PSNR (dB) for low-light enhancement.

RR	Bias	AAC	MobileNetV2	CaiT-XXS24
✗	✗	✗	41.80	53.86
✗	✗	✓	64.07	60.90
✗	✓	✗	49.47	55.45
✗	✓	✓	65.46	63.89
✓	✗	✗	47.34	59.18
✓	✗	✓	64.25	62.49

Appendix

Table 12: AdaRound

\rightarrow

VisionMX-LW ablation with MXFP4-UE8M0. All rows use layerwise optimization without QDrop, 20,000 steps, 2,048 calibration samples, warmup

0.20

, MSE loss, and

\epsilon=10^{-8}

. MobileNetV2 uses rounding LR

3\times 10^{-3}

, rounding regularization

1.00

, bias LR

10^{-3}

, bias regularization

0

, and RangeRound

\delta=3.00

. CaiT-XXS24 uses rounding LR

10^{-3}

, rounding regularization

0.01

, bias LR

10^{-3}

, bias regularization

0

, and RangeRound

\delta=2.00

. Top-1 accuracy is reported in percent.

Explore similar work

May 23, 2026

MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference. In 2022, the Open Compute Project (OCP) consortium standardized narrow precision formats for deep learning, called the microscaling (MX) format. The MX format is a hardware-friendly dynamic quantization scheme that effectively reduces the data size by sharing an 8-bit exponent across multiple operands. The MX format can be categorized into two types with their own strengths: (i) MXINT which focuses on a high precision consisting only of mantissa bits and (ii) MXFP which focuses on a wider dynamic range by allowing local exponent bits. In this work, we present a versatile MXFP format, called MX-SAFE (MXSF in short), that adaptively uses two modes, i.e., a wider mantissa mode (FP8 E2M5) and a subnormal FP mode (FP5 E3M2), to support both training and direct-cast inference. Furthermore, we propose a tile-based block design to increase hardware efficiency by reducing the burden of re-quantization process during the training with the MXSF format. Owing to the use of the proposed MXSF format, 0.05%/11.1% and 3.55%/3.57% improvements in accuracy, on average, for inference/full-training compared to MXFP8 E2M5 and MXFP8 E4M3 are observed, respectively. Moreover, we present a training-inference accelerator that supports the MXSF format and it achieves similar accuracy to the BF16 baseline while using 24.9% less total energy consumption.

Dahoon Park, Jahyun Koo, Sangwoo Hwang +1

† School of Electrical Engineering, Korea University, South Korea · ‡ Department of Electrical Engineering and Computer Science, DGIST, South Korea

Mixed-Precision Quantization AI Accelerator Inference Energy-Efficient ML Neural Network Quantization

Ask PDF

Sep 17, 2026

MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

The deployment of Vision-Language Models (VLMs) on edge devices is severely bottlenecked by memory bandwidth, necessitating aggressive sub-8-bit quantization. Since edge accelerators are strictly constrained by area and power, they require end-to-end quantized models. However, the extreme dynamic range gap between multi-modal tokens causes standard block formats to suffer "microscaling collapse," where a single massive outlier hijacks the shared exponent, underflowing surrounding elements and destroying attention maps. To break this bottleneck, we propose Micro-Inverted-Scaling (MiX), a novel format that mathematically inverts the microscaling paradigm: rather than grouping multiple mantissas under one shared exponent, MiX groups private, per-element exponents under a single shared mantissa. To handle asymmetric VLM outlier topologies, we introduce an adaptive dual-format (MiX-MX) inference framework. By algebraically factoring out the shared MiX mantissa, this framework maps to a custom accelerator, replacing multipliers with efficient shifters. Evaluated end-to-end on multiple VLMs, our 4.5-bit MiX formulation exhibits equivalent or superior accuracy on multi-modal benchmarks compared to NVFP4. Simultaneously, the MiX accelerator delivers a 25% improvement in area efficiency over the NVFP4 baseline and a 2.3-4.5x speedup with 1.4-2.9x energy reduction across models compared to the state-of-the-art accelerator Focus, proving the inverted-scaling datapath is physically superior for efficient VLM deployment.

Yuan Liao, Jae-sun Seo

Cornell Tech, Cornell University New York, NY, USA

AI Accelerator Inference VLM Quantization Efficient VLM Inference 4-Bit Quantization Low-Bit Quantization

Ask PDF

Apr 1, 2026

MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, where a single model is trained to be robust across multiple quantization formats. We find that multi-format QAT can match single-format QAT at each target precision, yielding one model that performs well overall across different formats, even formats that were not seen during training. To enable practical deployment, we propose the Slice-and-Scale conversion procedure for both MXINT and MXFP that converts a high-precision representation into lower-precision formats without re-training. Building on this, we introduce a pipeline that (i) trains a model with multi-format QAT, (ii) stores a single anchor format checkpoint (MXINT8/MXFP8), and (iii) allows on-the-fly conversion to lower MXINT or MXFP formats at runtime with negligible-or no-additional accuracy degradation. Together, these components provide a practical path to elastic precision scaling and allow selecting the runtime format at inference time across diverse deployment targets.

Zifei Xu, Sayeh Sharify, Hesham Mostafa

d-Matrix, Santa Clara, CA 95054

Quantization-Aware Training Neural Network Quantization Mixed-Precision Quantization

Ask PDF