Compressing neural networks by quantizing model parameters offers useful trade-off between performance and efficiency. Methods like quantization-aware training and post-training quantization strive to maintain the downstream performance of compressed models compared to the full precision models. However, these techniques do not explicitly consider the impact on algorithmic fairness. In this work, we study fairness-aware mixed-precision quantization schemes for medical image classification under explicit bit budgets. We introduce FairQuant, a framework that combines group-aware importance analysis, budgeted mixed-precision allocation, and a learnable Bit-Aware Quantization (BAQ) mode that jointly optimizes weights and per-unit bit allocations under bitrate and fairness regularization. We evaluate the method on Fitzpatrick17k and ISIC2019 across ResNet18/50, DeiT-Tiny, and TinyViT. Results show that FairQuant configurations with average precision near 4-6 bits recover much of the Uniform 8-bit accuracy while improving worst-group performance relative to Uniform 4- and 8-bit baselines, with comparable fairness metrics under shared budgets.
Figures & tables
Figure 1: High-level overview of the proposed FairQuant framework (top pathway) illustrating the group importance calculation based on a calibration set, which is then used to allocate appropriate bit widths to the weights. Regular post-training quantization (bottom pathway).
Dataset
Model
Method
AvgBits ↓
GBOPs ↓
AvgAcc ↑
WorstAcc ↑
Gap ↓
EOpp0 ↓
EOdd ↓
FitzP.
DeiT-Tiny
FP32
32.00
0.07
53.00
47.00
12.10
0.003
0.225
U8
8.00
0.02
53.00
47.00
12.00
0.003
0.224
U4
4.00
0.01
45.50
40.20
10.60
0.003
0.222
Lin et al. [ 16 ]
4.80
0.01
32.60
28.70
7.80
0.004
0.141
FQ-QAT
5.03
0.01
54.10 ±0.46
47.90 ±0.56
12.30 ±0.72
0.003
0.229
FQ-BAQ
4.30
0.01
51.83 ±0.64
46.00 ±0.10
11.67 ±1.31
0.003
0.220
Table 1: Main results across two datasets and different models. All QAT experiments use the following bit allocation {20%: 2, 40%: 4, 40%: 8}, and BAQ is reported for range 4-8. Significant results for average bits, average accuracy, and worst group accuracy compared to the baseline methods are shown in bold.
Figure 2: Ablations and stability for BAQ. Left: sweep of the bitrate regularizer λbaq,b showing test AvgAcc and EOpp on Fitzpatrick17k and ISIC2019. Middle: sweep of the fairness-loss scale λfair on the ResNet-18 backbone. Right: learning-rate sweep for BAQ bit proxies. All panels use a fixed BAQ bit interval [bmin,bmax]=[4,16] . Shaded regions indicate variability across repeated runs.
Mixed-precision post-training quantization is a network compression method that assigns bits layer by layer, under a global memory budget using a small calibration set. The main difficulties are to overcome the combinatorial nature of the allocation problem and to manage the sensitivity to small, potentially corrupted databases. Hence, an efficient allocation method should be fast to compute and preserve model quality when calibration data are corrupted. To design such a method, we derive a layerwise probabilistic analysis of the quantization error that separates propagated error from the local perturbation introduced at a given layer. We use this local term to build a separable score for a simple allocation algorithm, that requires no external solver. The probabilistic nature of our approach brings robustness to corrupted data. On denoising tasks with DRUNet, with an average budget of 4 bits per weight, our method matches or improves state-of-the-art mixed-precision baselines under clean calibration, and is more robust to corrupted calibration, with PSNR gains of up to 7.5 dB under the tested corruptions. Experiments show bit-allocation speed-ups from 28x to 2,570x over the studied baselines. For quantized diffusion models, our experiments show that a direct application of our framework also improves the state-of-the-art.
Foundation models have achieved remarkable results in medical image analysis. However, its large network architecture and high computational complexity significantly impact inference speed, limiting its application on terminal medical devices. Quantization, a technique that compresses models into low-bit versions, is a solution to this challenge. In this paper, we propose a post-training quantization algorithm, Permutation-COMQ. It eliminates the need for backpropagation by using simple dot products and rounding operations, thereby removing hyperparameter tuning and simplifying the process. Additionally, we introduce a weight-aware strategy that reorders the weight within each layer to address the accuracy degradation induced by channel-wise scaling during quantization, while preserving channel structure. Experiments demonstrate that our method achieves the best results in 2-bit, 4-bit, and 8-bit quantization.
Yineng Chen, Peng Huang, Aozhong Zhang +9
University at Albany, SUNY · Southwest Jiaotong University · Purdue University +1
Mixed-precision quantization improves the accuracy of post-training quantization by allocating higher bitwidths to sensitive layers, but existing methods solve the allocation for a single fixed memory budget. In practice the budget varies across deployments and is unknown at calibration time. Adaptive quantization addresses this with one offline calibration that serves any budget, yet current methods score layer sensitivity in a manner that does not consider its dependency on quantization levels of other layers. We show that a layer's sensitivity depends strongly on the bitwidths of its upstream layers and that this dependence shifts the resulting preferred bit allocation. We propose MixQuant, a technique-agnostic adaptive framework that wraps any base quantizer. MixQuant marginalizes each layer's distortion over random quantized upstream configurations to obtain budget-agnostic scores, calibrates the quantizer's parameters on plans the allocator itself produces, and penalizes allocations that leave layers at the lowest bitwidths. A single greedy pass then serves any budget at deployment. Across Llama-3.2-3B, Llama-2-7B, and Mistral-7B under AWQ and GPTQ, MixQuant outperforms adaptive and mixed-precision baselines in every setting, improving average accuracy by up to 8 points and reducing perplexity from 12.43 to 10.70 at the tightest budget, while matching an ILP solver at negligible deployment cost.