Foundation models have achieved remarkable results in medical image analysis. However, its large network architecture and high computational complexity significantly impact inference speed, limiting its application on terminal medical devices. Quantization, a technique that compresses models into low-bit versions, is a solution to this challenge. In this paper, we propose a post-training quantization algorithm, Permutation-COMQ. It eliminates the need for backpropagation by using simple dot products and rounding operations, thereby removing hyperparameter tuning and simplifying the process. Additionally, we introduce a weight-aware strategy that reorders the weight within each layer to address the accuracy degradation induced by channel-wise scaling during quantization, while preserving channel structure. Experiments demonstrate that our method achieves the best results in 2-bit, 4-bit, and 8-bit quantization.
Figures & tables
Figure 1: Conceptual illustration of COMQ and Permutation-COMQ Top: COMQ has quantization scale factors dominated by outliers due to heterogeneous weight magnitudes within each quantization unit, leading to coarse quantization of small weights. Bottom: Permutation-COMQ reorders weights by magnitude to group similar values, reducing intra-unit magnitude heterogeneity resulting in finer quantization resolution. The quantized weights are then mapped back to the original order via inverse permutation.
Figure 2: Distribution of magnitude of Simulated Weight.
Figure 3: Relative quantization error for COMQ and permutation-COMQ under different bit-widths
Wbit
Method
DSC
NSD
2
COMQ [ 27 ]
71.8
53.733
RTN [ 12 ]
29.79
30.221
Ours
86.939
78.935
4
COMQ [ 27 ]
91.874
90.011
RTN [ 12 ]
90.526
86.755
Ours
93.434
93.089
Table 1: Comparative Results on the AbdomenCT-1K Dataset. Bold indicates the best
Wbit
Method
DSC
NSD
2
Per-Layer
74.46
54.825
Ours
86.939
78.935
4
Per-Layer
55.439
43.685
Ours
93.434
93.089
8
Per-Layer
93.282
92.461
Ours
93.615
93.204
Table 2: Results of ablation experiments for Per-Channel Weight-Aware
Figure 4: Conceptual illustration of COMQ and Permutation-COMQ. Top: COMQ has quantization scale factors dominated by outliers due to heterogeneous weight magnitudes within each quantization unit, leading to coarse quantization of small weights. Bottom: Permutation-COMQ reorders weights by magnitude to group similar values, reducing intra-unit magnitude heterogeneity resulting in finer quantization resolution. The quantized weights are then mapped back to the original order via inverse permutation.