Fairboard: a quantitative framework for equity assessment of healthcare models

Fairness audits of medical imaging models commonly report the largest performance difference between demographic subgroups and treat any positive value as evidence of bias. Yet a perfectly fair model also produces a positive difference, which grows as subgroups shrink. Some mitigation methods remove demographic information from models because they assume that it produces the difference. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME). Its first step compares each reported difference with the difference expected from a perfectly fair model at the same subgroup counts. Its second step tests whether candidate causes of any excess over this fair-model reference change the difference when they are injected into model features. We evaluated 36 encoders on nine datasets in three modalities and audited 89 published differences across six modalities. For 10 frozen encoders and 13 chest radiograph findings, the reference is a median 41% of the race difference and 22% of the age difference. It is exceeded by 40 of 53 published differences in sensitivity or false positive rate and by one of 36 in the area under the receiver operating characteristic curve (AUROC). Adding race information to the features of two encoders does not change the race difference significantly. At matched disease performance, mitigation reduces the race difference by less than its spread across three pretraining seeds (medians 0.005 and 0.012) and the age difference by more (0.008 and 0.006). Image-text pretraining raises the worst-group AUROC for race by a median 0.049 over self-supervised pretraining. Reporting the fair-model reference beside every new and published subgroup difference could direct mitigation to the differences above it.

Medical imaging is central to modern diagnostics, and artificial intelligence (AI) systems are increasingly used to support image-based analysis by improving efficiency, accuracy, and access to care. However, inequities in healthcare access and differential disease prevalence create severe demographic imbalances in clinical image data. Such imbalances are compounded by the fact that diseases can manifest with distinct features across demographic groups, rendering certain phenotypic presentations naturally rare. AI models trained on such imbalanced data risk perpetuating diagnostic bias and widening healthcare disparities. Here we introduce FairGen, a fairness-aware diffusion framework that synthesizes demographically balanced medical images while preserving pathology-relevant visual features. By embedding physician-aligned preferences into the generation process, FairGen improves subgroup coverage during synthesis and downstream classification. Applied to dermatology, radiology, and neuroimaging benchmark tasks, FairGen achieves fairness improvements of 95.9% for skin images, 80.0% for chest radiography, and 35.2% for brain MRI, while maintaining competitive diagnostic accuracy relative to models trained on original clinical data. Clinician-facing expert review and external validation on independent cohorts further support that these gains extend beyond standard fidelity metrics and are not confined to the original in-distribution datasets.

Medical image segmentation models can perform unevenly across subgroups. Most existing fairness methods focus on improving average subgroup performance, implicitly treating each subgroup as internally homogeneous. However, this can hide difficult cases within a subgroup, where high-loss samples are obscured by the subgroup mean. We call this problem intra-group hidden failure. To solve this, we propose DuetFair mechanism, a dual-axis fairness framework that jointly considers inter-subgroup adaptation and intra-subgroup robustness. Based on DuetFair, we introduce FairDRO, which combines distribution-aware mixture-of-experts (dMoE) with subgroup-conditioned distributionally robust optimization (DRO) loss aggregation. This design allows the model to adapt across subgroups while also reducing hidden failures within each subgroup. We evaluate FairDRO on three medical image segmentation benchmarks with varying degrees of within-group heterogeneity. FairDRO achieves the best equity-scaled performance on Harvard-FairSeg and improves worst-case subgroup performance on HAM10000 under both age- and race-based grouping schemes. On the 3D radiotherapy target cohort, FairDRO further improves worst-group Dice by 3.5 points ($\uparrow 6.0\%$) under the tumor-stage grouping and by 4.1 points ($\uparrow 7.4\%$) under the institution grouping over the strongest baseline.