Federated learning (FL) data corruption can affect either inputs or labels, but it remains unclear whether input-conditional uncertainty and prediction-label loss expose these corruption modes equally. This paper compares two corruption-detection signals in FL: input-conditional uncertainty and prediction-label loss. The uncertainty signal is characterised using a learned aleatoric variance estimate together with Monte Carlo (MC) dropout variance and entropy measures, while the loss is computed against the supplied label. We test these signals against additive image noise and persistent random label flips. On ResNet-20 with CIFAR-10 and SVHN under Dirichlet partitions with data that are not independent and identically distributed (non-IID), the two corruption types behave differently. For persistent random label flips, the within-client per-sample area under the receiver operating characteristic curve (AUC) is 0.85 on CIFAR-10 and 0.95 on SVHN for prediction-label loss, while every uncertainty estimator stays at chance (0.49--0.50). This pattern is consistent with the model remaining confident in the underlying image despite the supplied label being wrong. For image noise, expected-entropy uncertainty rises above chance (0.67 on CIFAR-10 and 0.66 on SVHN), while loss responds comparably (0.64 on both). Each signal is therefore the stronger detector for a different corruption: the prediction-label loss for persistent label flips, and expected-entropy uncertainty for image noise, with its advantage becoming apparent as federation-wide corruption prevalence increases. Robust FL data-quality assessment should match the signal to the corruption rather than rely on uncertainty alone across corruption types.
Figures & tables
Fig. 1: Measurement pipeline. A subset of the 10 clients is corrupt (count M swept): a fraction q of a corrupt client’s samples are corrupted (flips or noise; separate conditions), the rest clean. The global model probes every client each round; uncertainty and loss are recorded as diagnostic signals and scored per-client and within-client per-sample.
Fig. 2: CIFAR-10 examples: a clean sample, a persistent label flip (unchanged image, incorrect label), and additive Gaussian image noise (SD 0.3 ). Flip and noise are applied as separate conditions.
Dataset
Corrupt
AV
AE
EV
EE
1− Acc
CIFAR-10
Flip
0.41
0.63
0.55
0.58
0.96
CIFAR-10
Noise
0.45
0.85
0.70
0.75
0.73
SVHN
Flip
0.45
0.73
0.61
0.63
0.99
SVHN
Noise
0.22
0.90
0.82
0.83
0.90
TABLE I: Per-client corruption-detection AUC, averaged over the full M×q×α× seed grid. All columns are detection AUCs. AV/EV: aleatoric/epistemic variance; AE/EE: aleatoric/epistemic entropy; 1−Acc scores clients by inverse global-probe accuracy.
Dataset
Corrupt
AV
AE
EV
EE
L
CIFAR-10
Flip
0.49
0.49
0.50
0.50
0.85
CIFAR-10
Noise
0.52
0.67
0.60
0.61
0.64
SVHN
Flip
0.49
0.50
0.50
0.50
0.95
SVHN
Noise
0.28
0.66
0.62
0.62
0.64
TABLE II: Within-client per-sample corruption-detection AUC, averaged over the full M×q×α× seed grid. AV/EV: aleatoric/epistemic variance; AE/EE: aleatoric/epistemic entropy; L is prediction-label loss.
CIFAR-10
SVHN
Corruption
M
q=0.2
q=0.4
q=0.6
q=0.2
q=0.4
q=0.6
Clean
0
69.7
93.2
Flip
2
66.8
65.9
64.6
92.4
92.0
91.6
4
63.1
58.5
55.2
90.1
88.6
86.8
6
57.4
50.3
43.4
86.2
81.1
75.4
8
52.1
40.7
31.1
82.4
73.1
59.5
TABLE III: Best clean-test accuracy (%) over rounds 41 – 50 , averaged over α and seeds. M : corrupt clients (of 10 ); q : corrupted fraction within a corrupt client.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
CIFAR-10
SVHN
Corrupt
M
Signal
q=0.2
q=0.4
q=0.6
q=0.2
q=0.4
q=0.6
Flip
2
AV
0.47
0.48
0.47
0.50
0.48
0.50
AE
0.48
0.48
0.49
0.53
0.49
0.47
EV
0.52
0.48
0.48
0.52
0.49
0.46
EE
0.52
0.48
0.48
0.52
0.49
0.46
L
0.93
0.90
0.89
0.98
0.98
0.97
Appendix
TABLE IV: Within-client per-sample AUC by corruption type, M , and q ; each cell averages α× seed runs. Abbreviations as in Table II .
Dataset
Corrupt
α
AV
AE
EV
EE
1− Acc
CIFAR-10
Flip
0.1
0.37
0.69
0.55
0.60
0.91
0.3
0.34
0.60
0.52
0.55
0.98
1.0
0.53
0.61
0.57
0.61
1.00
CIFAR-10
Noise
0.1
0.45
0.76
0.65
0.71
0.67
0.3
0.33
0.87
0.66
0.71
0.76
1.0
0.59
0.92
0.78
0.84
0.77
Appendix
TABLE V: Per-client AUC by Dirichlet α , averaged over the M×q× seed grid. Abbreviations as in Table I .
CIFAR-10
SVHN
Corrupt
M
Signal
q=0.2
q=0.4
q=0.6
q=0.2
q=0.4
q=0.6
Flip
2
AV
0.46
0.41
0.41
0.52
0.57
0.54
AE
0.59
0.58
0.61
0.75
0.74
0.73
EV
0.53
0.53
0.58
0.66
0.67
0.67
EE
0.54
0.58
0.61
0.67
0.68
0.68
1− Acc
0.92
0.98
1.00
0.99
1.00
1.00
Appendix
TABLE VI: Per-client AUC by corruption type, M , and q ; each cell averages α× seed runs. Abbreviations as in Table I .
Dataset
Seed
AE
Loss
AE − Loss
CIFAR-10
42
0.697
0.656
+0.041
CIFAR-10
123
0.650
0.633
+0.016
CIFAR-10
777
0.675
0.646
+0.029
SVHN
42
0.662
0.648
+0.014
SVHN
123
0.647
0.636
+0.011
SVHN
777
0.659
0.643
+0.016
Appendix
TABLE VII: Seed-specific grid averages of within-client image-noise AUC; each row averages the 36 M×q×α configurations. Differences are computed before rounding.
Institute of Computing Technology, Chinese Academy of Sciences · TMLR Group, Hong Kong Baptist University · The Hong Kong University of Science and Technology +1
Department of Information Engineering, The Chinese University of Hong Kong · National Key Laboratory on Wireless Communications, University of Electronic Science and Technology of China