cs.CVApr 13, 2026
SaveEvaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance
Organizations: Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA · AI in Medicine, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Department of Computing, Imperial College London, London, UK · Harvard Medical School, Boston, MA, USA
Abstract
AI-based image reconstruction models are increasingly deployed in clinical workflows to improve image quality from noisy data, such as low-dose X-rays or accelerated MRI scans. However, these models are typically evaluated using pixel-level metrics like PSNR, leaving their impact on downstream diagnostic performance and fairness unclear. We introduce a scalable evaluation framework that applies reconstruction and diagnostic AI models in tandem, which we apply to two tasks (classification, segmentation), three reconstruction approaches (U-Net, GAN, diffusion), and two data types (X-ray, MRI) to assess the potential downstream implications of reconstruction. We find that conventional reconstruction metrics poorly track task performance, where diagnostic accuracy remains largely stable even as reconstruction PSNR declines with increasing image noise. Fairness metrics exhibit greater variability, with reconstruction sometimes amplifying demographic biases, particularly regarding patient sex. However, the overall magnitude of this additional bias is modest compared to the inherent biases already present in diagnostic models. To explore potential bias mitigation, we adapt two strategies from classification literature to the reconstruction setting, but observe limited efficacy. Overall, our findings emphasize the importance of holistic performance and fairness assessments throughout the entire medical imaging workflow, especially as generative reconstruction models are increasingly deployed.
Figures & tables
Figure 1: Combined pipeline for downstream bias evaluation and mitigation in medical image reconstruction. MRI and X-ray images undergo realistic simulated degradation and are subsequently reconstructed with three approaches before serving as input to downstream prediction models. Reconstruction quality, downstream performance, and fairness are evaluated. Subsequently, two bias mitigation strategies are applied exclusively during reconstruction fine-tuning.
Figure 2: Downstream performance and PSNR at varying noise levels. Axes for PSNR and task performance are scaled to comparable percentage ranges. Although PSNR declines as noise increases, task performance remains stable. Baseline indicates performance on original images.
Figure 3: Distribution of bias changes (percent change compared to original images) across all reconstruction models, datasets, and tasks, stratified by sensitive attribute. The vertical lines mark the medians. Most shifts cluster near zero, but sex shows a broader positive tail. Separate plots by sensitive attribute are contained in Fig. 20 .
Figure 4: Equalized odds bias change pre- and post-mitigation compared to predictions on original images for CheXpert. Pre-mitigation (“Reconstruction”), bias tends to increase slightly for sex; race exhibits high variance. Bias tends to decline slightly post-mitigation. Error bars represent standard deviation.
| Sex | Age | Race | |
|---|---|---|---|
| Classification | |||
| Segmentation | – |
Table 1: Baseline fairness of the classifiers (EODD) and the segmentation model (SER) for different sensitive attributes. For classification, mean and s.d. are reported across all classification tasks. Segmentation corresponds to UCSF-PDGM performance. Sex exhibits the lowest baseline bias.
| U-Net | GAN | SDE | |
|---|---|---|---|
| Median | 2.28 | -0.21 | 1.59 |
| Absolute Median | 14.6 | 11.8 | 11.5 |
Table 2: Median of bias change (% change in EODD/SER) by reconstruction approach across all datasets, tasks, and attributes by model. SDE and GAN show a slightly smaller bias shift than U-Net.
| Sex | Age | Race | |
|---|---|---|---|
| Standard | 24.1 | -1.88 | 3.30 |
| Reweighted | 10.6 | 0.03 | 1.05 |
| EODD | 7.56 | -2.01 | 0.52 |
Table 3: Median bias change (% change in EODD/SER) by mitigation strategy across all datasets, tasks, and models. Standard corresponds to the original results without mitigation applied.
| Reweighted | EODD | |||
| Chex | UCSF | Chex | UCSF | |
| PSNR | 0.54 | -0.75 | -0.64 | -7.28 |
| Down. | 0.07 | -1.97 | 0.02 | -2.94 |
Table 4: Mean change (%) in PSNR and downstream performance (AUROC/Dice) per dataset after each mitigation averaged over reconstruction models and tasks. Performance drops are modest, except for PSNR in UCSF-PDGM for EODD.
Figure 5: EODD and SER bias change pre- and post-mitigation compared to predictions on original images for UCSF-PDGM tasks. No consistent trends emerge for the classification tasks. Error bars represent standard deviation.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: X-Ray images with photon count 100,000, 10,000, 3,000.
Figure 7: MRI images with acceleration 4, 8, 16.
Figure 8: Reconstruction example from photon count 10,000 for the different models. Grad-CAM Selvaraju et al. (2017) and logit score correspond to the lung lesion prediction of the pre-trained classifier, indicating similar predictions on the reconstructed images.
Figure 9: Reconstruction with corresponding segmentation and Dice score of an MRI image with acceleration 8 for the different models.
| AI/AN | Asian | Black | NH/PI | Other | White | ||
|---|---|---|---|---|---|---|---|
| Female, | 54 | 1539 | 923 | 314 | 2518 | 6456 | 11804 |
| Female, | 39 | 1739 | 608 | 136 | 1710 | 9500 | 13732 |
| Male, | 56 | 1734 | 1023 | 240 | 3553 | 8984 | 15590 |
| Male, | 27 | 1924 | 539 | 171 | 1853 | 11170 | 15684 |
| 176 | 6936 | 3093 | 861 | 9634 | 36110 | 56810 |
Table 5: Patient-wise groups used for analysis based on sex, age, and race for the CheXpert dataset. Unequally distributed with very few samples for American Indian or Alaska Native (AI/AN) and Native Hawaiian or Other Pacific Islander (NH/PI).
| Male | Female | ||
|---|---|---|---|
| 155 | 92 | 147 | |
| 144 | 110 | 254 | |
| 299 | 202 | 501 |
Table 6: Patient distribution by sex and age for the UCSF-PDGM dataset. Patients under 58 and females represent minority groups.
Figure 10: Tumor Type and Tumor Grade and PSNR values for different noise levels on UCSF-PDGM. The image quality and diagnostic performance axes are on a similar percentage scale. Task performance metrics show high stability across models and noise conditions, while PSNR drops with increasing noise.
| Photon Count | Metrics | Baseline | U-Net | GAN | SDE | |
|---|---|---|---|---|---|---|
| 100,000 | AUROC | Atalectasis | 0.87 | 0.87 | 0.86 | 0.87 |
| Cardiomegaly | 0.91 | 0.91 | 0.91 | 0.91 | ||
| Consolidation | 0.91 | 0.91 | 0.91 | 0.91 | ||
| Edema | 0.90 | 0.90 | 0.90 | 0.90 | ||
| EC | 0.79 | 0.78 | 0.78 | 0.79 | ||
| Fracture | 0.76 | 0.75 | 0.75 | 0.76 | ||
Table 7: CheXpert performance across reconstruction models and photon counts. Pathologies with lower baseline AUROC (e.g., fracture, pneumothorax, lung lesion) experience greater performance drops under noise compared to more easily detectable conditions (e.g., effusion, cardiomegaly). Baseline corresponds to original images.
| Acceleration | Metrics | Baseline | U-Net | GAN | SDE | |
|---|---|---|---|---|---|---|
| 4 | AUROC | Tumor Type | 0.79 | 0.79 | 0.77 | 0.78 |
| Tumor Grade | 0.73 | 0.76 | 0.71 | 0.72 | ||
| Dice | 0.72 | 0.72 | 0.71 | 0.72 | ||
| PSNR | 42.94 | 37.71 | 40.23 | |||
| LPIPS | 0.01 | 0.02 | 0.00 | |||
| 8 | AUROC | Tumor Type | 0.79 | 0.77 | 0.83 | 0.79 |
Table 8: Performance metrics for UCSF-PDGM across reconstruction models and noise levels. While PSNR varies with noise and model, downstream segmentation and classification metrics remain relatively stable, indicating robust task performance across conditions.
Figure 11: Influence of fairness weighting parameter ( ) on classifier AUROC performance and fairness metrics for the Equalized Odds (EODD) mitigation constraint, evaluated with U-Net on the CheXpert dataset. There is minor sensitivity of AUROC to lambda; fairness metrics show greater variance but minimal substantial improvement with increased .
Figure 12: Impact of on reconstruction quality (PSNR) compared to fairness for the EODD constraint mitigation. PSNR remains stable across lambda variations, while fairness shows slight variation without substantial improvement.
Figure 13: Change in prediction performance after applying bias mitigation techniques. Each row compares two datasets for a given method: (a) Reweighted sampling, (b) Equalized odds constraint. UCSF-PDGM experiences more performance degradation. However, all techniques show good stability in task performance, with few outliers in the UCSF-PDGM dataset.
Figure 14: Equality of opportunity (EOP) bias change pre- and post-mitigation compared to predictions on original images for CheXpert classification. Pre-mitigation, bias tends to increase slightly for sex; race exhibits high variance. Bias tends to decline slightly post-mitigation.
Figure 15: Equality of opportunity (EOP) and Dice bias change compared to predictions on original images pre- and post-mitigation for UCSF-PDGM classification and segmentation.
Figure 16: Distribution of bias changes when using alternative race subgroups for CheXpert calculations.
Figure 17: Equalized odds bias change pre- and post-mitigation compared to predictions on original images for CheXpert when using alternative race subgroups.
Figure 18: Downstream performance and SSIM at varying noise levels. Axes for SSIM and task performance are scaled to comparable percentage ranges. Baseline indicates performance on original images.
Figure 19: Downstream performance and SSIM at varying noise levels on classification tasks in UCSF-PDGM. Axes for SSIM and task performance are scaled to comparable percentage ranges. Baseline indicates performance on original images.
Figure 20: Distribution of bias changes separated by sensitive attribute.