Low-bit quantization offers a promising avenue for reducing the computational and memory demands of Multimodal Large Language Models (MLLMs). Recent hardware support for low-precision formats, ranging from MXFP8 to ultra-low-bit formats such as MXFP4 and HiF4, has accelerated research into efficient MLLM training and deployment. In this work, we present a systematic study of these quantization schemes in representative MLLMs that span both video generation and reasoning tasks. Our analysis shows that MXFP8 achieves near-lossless performance, whereas aggressive 4-bit quantization leads to significant degradation. Through extensive ablations, we identify activation quantization as the primary source of this performance loss, contributing substantially more than weight quantization. Motivated by this observation, we propose Residual Fallback Quantization (RFQ), a lightweight activation reconstruction framework that supplements the primary ulta-low-bit activation representation with an auxiliary quantized residual pathway. By explicitly modeling and compensating for quantization errors, RFQ improves activation fidelity while preserving the efficiency advantages of ultra-low-bit computation. RFQ requires no architectural modifications and incurs negligible computational overhead. Extensive experiments on Wan2.2 and Qwen3-VL demonstrate that RFQ consistently recovers a substantial portion of the performance lost under the quantization of MXFP4 and HiF4, significantly narrowing the gap to BF16 baselines across both generation and 4 reasoning benchmarks. Our findings establish activation quantization as the dominant bottleneck in ultra-low-bit MLLMs and highlight residual-based activation reconstruction as an effective and practical strategy for robust 4-bit deployment.
Figures & tables
Figure 1: Overview of the impact of activation outliers and the proposed RFQ framework. Activation outliers degrade MLLM performance under FP4 quantization. RFQ reduces outlier-induced quantization errors through residual fallback correction, recovering generation quality close to the BF16 baseline.
Model
MXFP8
W4A8
MXFP4
HiF4
Wan2.2
0.30
0.80
7.10
2.70
Qwen3-VL
0.20
0.70
7.23
3.80
Table 1: Relative training loss increase (%) for the BF16 baseline after QAT on models trained with over 5B tokens. MXFP8 and W4A8 show minimal degradation, while MXFP4 and HiF4 incur larger errors.
Figure 2: Cross-modal sensitivity analysis under ultra-low-bit quantization.
Figure 3: Comparative analysis of activation tail tracking and fallback interception across model modalities. Top Row highlights the unmitigated quantization noise and overflow elements in standard MXFP4 compared against the BF16 baseline. Bottom Row demonstrates how Residual Fallback Quantization (RFQ) restores fidelity by snapping extreme outliers.
Model
Module
Parameter Distribution (%)
Quant.(%)
Atten. Proj.
FFN/MLP
MoE
Other
Wan2.2
Visual
39.86
46.51
–
11.62
97.99
Textual
28.35
53.16
–
–
81.50
Qwen3-VL
Visual
26.61
49.71
–
22.77
99.09
Textual
2.96
–
94.95
–
97.91
Table 2: Parameter breakdown (%) by functional component with respect to the total paramter size. Quant. (%) for the visual and textual modules denotes the proportion of parameters quantized within each module. Projection layers and mergers are included in the “Other” category.
Method
Subject Consistency
Background Consistency
Imaging Quality
Temporal Flickering
Motion Smoothness
Dynamic Degree
Overall Consistency
Aesthetic Quality
Baseline (BF16)
95.74
96.77
65.32
98.42
99.25
45.00
7.03
59.43
MXFP4
95.65
96.60
66.69
98.10
99.15
43.00
7.24
58.93
(-0.09)
(-0.17)
(+1.37)
(-0.32)
(-0.10)
(-2.00)
(+0.21)
(-0.5)
MXFP4 + RFQ (ours)
95.43
96.54
66.48
98.29
99.23
50.00
6.90
59.26
(-0.31)
(-0.23)
(+1.16)
(-0.13)
(-0.02)
(+5.00)
(-0.13)
(-0.16)
HiF4
95.23
96.67
67.36
98.33
99.24
53.00
6.98
59.44
Table 3: For Wan2.2, quantitative comparison on VBench. Values in parentheses indicate the relative change (%) with respect to the BF16 baseline. Blue denotes improvement and red denotes degradation.
Method
RealWorldQA
MMStar
MMBenchEN
SimpleVQA
BF16
72.68
70.80
90.77
16.83
MXFP4
70.98
69.67
90.72
15.16
(-1.70)
(-1.13)
(-0.05)
(-1.67)
MXFP4 + RFQ (ours)
72.16
70.73
90.40
16.44
(-0.52)
(-0.07)
(-0.37)
(-0.39)
HiF4
72.42
71.27
90.50
15.35
Table 4: Accuracy comparison across different datasets and quantization formats on Qwen3-VL. Values in parentheses indicate the absolute change with respect to the BF16 baseline. Blue denotes improvement, while red indicates degradation.
Figure 4: Qualitative results for video generation. While 4-bit quantization methods ( i.e. , MXFP4 and HiF4) degrade generation quality, our proposed RFQ recovers visual fidelity and improves temporal consistency.
Figure 5: Training loss curves comparing low-bit FP4 strategies against the BF16 baseline across Qwen3-VL-30B and Wan2.2-5B.
Model
MXFP4
MXFP4 + RFQ (ours)
HiF4
HiF4 + RFQ (ours)
Wan2.2
7.50
1.14
2.89
0.61
Qwen3VL-30B
7.23
1.43
2.32
0.66
Table 5: Relative training loss increase (%) over the BF16 baseline after QAT with 4-bit quantization. RFQ substantially mitigates the degradation caused by MXFP4 and HiF4.