Who Pays for Byzantine Robustness? Validity, Data-Share Dictatorships, and Fairness When Only a Few Hospitals Collaborate
Organizations: MSc Data Science, Sapienza University of Rome
Abstract
Clinical federated learning (FL) collaborations typically connect only three to six hospitals, yet Byzantine-robust aggregation rules are designed and benchmarked on synthetic partitions across dozens of equal-sized clients. We conduct an empirical audit of eight standard aggregation rules: FedAvg, coordinate-wise median, trimmed mean, Krum, Multi-Krum, Bulyan, geometric median (GeoMed), and Centered Clipping (CC). We evaluate both unweighted and sample-size-weighted variants across three cross-silo medical datasets from the FLamby benchmark: Fed-Heart-Disease ( hospitals), Fed-TCGA-BRCA ( regions), and Fed-ISIC2019 ( centres). We execute 54,214 training runs across enumerated attacker coalitions () paired with exact-twin leave-out controls that decompose performance drops into honest data loss versus adversarial attack damage. We report three findings. First, client-count validity checks fail to capture breakdown in skewed medical consortia: while satisfies every count condition except Bulyan's, assigning sample-size weights turns median and GeoMed into a clean-model dictatorship whenever a single hospital holds more than half the training data, and causes weighted median, GeoMed, and CC to collapse when that dominant centre attacks. Second, exact-twin leave-out controls reveal that when the largest hospital attacks, the apparent failure of count-based robust rules is predominantly caused by the loss of honest training data (7.5 to 10.4 points on ISIC) rather than adversarial perturbation (2.3 to 3.8 points on ISIC). Conversely, weight-dependent rules suffer large true attack damage (16.1 to 25.1 points on ISIC). Third, unweighted robust rules consistently discount whichever institution generates the most atypical parameter updates, penalizing the smallest hospital on Heart but penalizing larger institutions on TCGA and ISIC.
Figures & tables
| Dataset | Client: training share (train/test size), largest first | Model | Main metric | lr | Rounds | (CC) | Seeds | |
|---|---|---|---|---|---|---|---|---|
| Fed-Heart-Disease | 4 | Cleveland 40.9% (199/104), Hungary 35.4% (172/89), VA 17.5% (85/45), Switzerland 6.2% (30/16) | logistic regression | accuracy (test-size weighted) | 0.001 | 106 | 0.03 | 20 |
| Fed-TCGA-BRCA | 6 | Northeast 28.6% (248/63), West 18.9% (164/42), South 18.0% (156/40), Midwest 14.9% (129/33), Europe 14.9% (129/33), Canada 4.6% (40/11) | linear Cox model | C-index (stratified) | 0.003 | 243 | 0.03 | 20 ( : 5) |
| Fed-ISIC2019 | 6 | Centre 0 53.4% (9930/2483), Centre 1 17.0% (3163/791), Centre 2 14.5% (2691/672), Centre 3 9.7% (1807/452), Centre 4 3.5% (655/164), Centre 5 1.9% (351/88) | linear head on frozen EfficientNet-B0 features | balanced accuracy (pooled) | 0.01 | 101 | 1 | 20 |
| Test set | Ours | FLamby (published) | Difference | Within 1 s.d. |
| Fed-Heart-Disease, accuracy, 5 seeds | ||||
| Pooled | 0.748 0.027 | 0.748 0.027 | +0.000 | yes |
| Cleveland | 0.727 0.019 | 0.731 0.022 | 0.004 | yes |
| Hungary | 0.717 0.037 | 0.719 0.033 | 0.002 | yes |
| VA | 0.640 0.051 | 0.640 0.051 | +0.000 | yes |
| Switzerland | 0.650 0.114 | 0.650 0.114 | +0.000 | yes |
| By client count | Weighted, by data share | ||||||
| (Heart) | (TCGA, ISIC) | Heart | TCGA | ISIC | |||
| Rule | (40.9%) | (28.6%) | (53.4%) | ||||
| FedAvg | – | – | – | – | – | – | – |
| Median | D, B1, B2 | ||||||
| Trimmed mean | = median | = median | |||||
| Krum | ignores | ignores | ignores | ||||
| Unweighted | Weighted (w) | |||||||
| Rule | Avg | Worst | Least influence | Avg | Worst | Least influence | ||
| Fed-Heart-Disease (accuracy) | ||||||||
| FedAvg | 70.9 0.4 | 66.6 0.7 | – | – | 74.3 0.3 | 63.0 0.7 | – | – |
| Median | 72.8 0.3 | 69.8 0.6 | Switzerland (0.49) | 0.20 | 73.8 0.3 | 58.8 0.6 | Switzerland (0.00) | 0.20 |
| Trimmed mean | 72.8 0.3 | 69.8 0.6 | Switzerland (0.49) | 0.20 | 74.3 0.3 | 65.4 0.9 | Switzerland (0.53) | 0.20 |
| Krum | invalid | = Krum (ignores weights) | ||||||
| Heart: Cleveland | Heart: Hungary | TCGA: Northeast | ISIC: centre 0 | |||||
|---|---|---|---|---|---|---|---|---|
| Rule | Data loss | Attack | Data loss | Attack | Data loss | Attack | Data loss | Attack |
| FedAvg | 8.2 0.5 | 7.2 1.0 | 12.8 0.4 | 3.0 1.0 | 3.8 0.5 | 18.8 2.1 | 10.6 0.2 | 8.5 0.2 |
| FedAvg (w) | 1.9 0.4 | 15.7 1.4 | 1.9 0.4 | 12.6 1.2 | 3.5 0.4 | 25.8 1.9 | 15.9 0.3 | 17.6 0.2 |
| Median | 19.1 0.4 | +0.3 0.5 | 17.6 0.4 | 1.6 0.3 | 4.4 1.0 | 3.6 0.6 | 7.5 0.2 | 3.8 0.2 |
| Median (w) | 4.5 0.5 | 4.3 0.5 | +0.0 0.4 | 9.8 0.6 | 4.7 0.9 | 7.9 1.7 | 11.4 0.4 | 25.1 0.3 |
| Trimmed mean | 19.1 0.4 | +0.3 0.5 | 17.6 0.4 | 1.6 0.3 | 2.6 0.6 | 4.7 0.7 | 9.1 0.2 | 2.3 0.1 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Rule | Clean | Attacked | Clean | Attacked | Clean | Attacked | Clean | Attacked |
|---|---|---|---|---|---|---|---|---|
| FedAvg | 45.5 0.2 | 38.5 0.2 | 45.5 0.2 | 38.2 0.2 | 47.5 0.2 | 37.5 0.1 | 47.5 0.2 | 37.2 0.1 |
| FedAvg (w) | 49.3 0.3 | 39.5 0.3 | 49.3 0.3 | 41.0 0.3 | 49.9 0.3 | 37.0 0.2 | 49.9 0.3 | 37.7 0.2 |
| Median | 39.0 0.2 | 39.2 0.1 | 39.0 0.2 | 39.2 0.1 | 41.2 0.2 | 40.1 0.1 | 41.2 0.2 | 40.1 0.1 |
| Median (w) | 34.3 0.8 | 30.4 0.6 | 34.1 0.8 | 30.2 0.7 | 35.5 0.7 | 29.5 0.4 | 34.9 0.8 | 29.1 0.5 |
Federated Learning Aggregation
Federated learning aggregation combines client updates into shared or personalized models while balancing data heterogeneity, robustness, privacy, and communication.
See all 293 papers