Once historical clients become unavailable, a federation must serve newcomers through the artifact it leaves behind. How much of its knowledge remains useful in an architecture chosen afterward? We introduce FEDNBO, a seven-dataset benchmark that separates knowledge supply, policy identification, and private-model compilation under disjoint data roles. Controlled studies expose inflated oracle gains, architecture-dependent transfer reversals, and a gap between teacher accuracy and private-model utility. Ordinary public-response distillation provides a competitive reference without public task labels or newcomer samples: gains over strengthened local training are 23.52 and 16.62 percentage points on 300 CelebA identities and 300 FedEMNIST writers, respectively. In a complete-pipeline FedEMNIST comparison, this reference reaches 97.13% accuracy with 12.4 seconds of compilation per architecture, versus 97.03% and 67.5 seconds for a HeteroFedTTA reconstruction at the same epoch budget. Equal-count public-input interventions extend support sensitivity to trainable visual backbones. Together, these results establish an auditable benchmark and a strong reference for evaluating the utility, cost and reliability of post-federation knowledge transfer.
Figures & tables
Figure 1: The FedNBO protocol. (a) A frozen bank exports public inputs/responses. (b) Private architectures learn from unlabeled responses; disjoint response validation selects checkpoints. Ridge is a separate control. (c) Shadow policies and newcomer selection-label methods are alternative routes; local wins ties and query labels are evaluation-only. (d) Gain separates supply, identification and compilation. Target utilities are hidden at deployment; glyphs are schematic.
raw FedSGD ResNet-18 → MobileNetV3/EfficientNet-B0
Table 1: Benchmark tasks. TCGA-BRCA uses balanced accuracy; the others use accuracy. UCF51-20 clients are group-preserving simulations; all other client units are natural.
Figure 2: Task and supervision controls. A: Attribute gains on 300 decorrelated CelebA identities include the negative Necklace result. B: Response-minus-public-label-control gains in separate historical studies. These post-confirmation supervised controls are not compiler inputs. Bars: 95% client-bootstrap intervals; cohorts and local references differ across conditions.
Figure 3: Four onboarding audits. A: Same-query versus label-disjoint selection (95% newcomer intervals). B: Active Testing across label budgets. C: Mean selection and LCB power/harm under declared or shifted populations (50,000 trials). D: Fixed KD-1 gains on 12 HAR subjects, averaged over seeds within each architecture.
Fixed-policy gain
PCV deployment
Dataset
1
2
3
4
Choice
Gain [95% CI]
Harm
CREMA-D
−7.08
−6.78
−8.80
−6.87
local
0.00 [0.00,0.00]
0.0
MELD
+11.16
+16.05
+17.66
+17.89
pure KD
+17.89 [15.30,22.03]
0.0
UCF51-20
+1.90
+1.57
−1.18
−1.45
local
0.00 [0.00,0.00]
0.0
TCGA-BRCA
+13.33
+17.62
+17.18
+14.21
joint KD 0.1
+13.33 [ −1.31 ,30.00]
33.3
UCI HAR
+3.23
+3.88
+1.98
+1.69
local
0.00 [0.00,0.00]
0.0
Table 3: Fixed policies and shadow-selected deployment. Gain/CI: pp; harm: %. Policies 1–4 are global/cluster 0/cluster 1/uniform on CREMA-D and KD 0.1/1/10/pure KD elsewhere. PCV uses only shadow labels; all LCB rules abstain. Test cohorts: MELD/TCGA, 3 clients; UCF, 4 simulated silos.
Policy
Gain [95% CI]
Harm
Linear gain
CNN gain
Fixed joint KD 1
+3.88 [2.19,5.51]
16.7
−1.73
+9.49
Disagreement audit, 6 labels
+3.22 [1.83,4.64]
8.3
−1.08
+7.52
Disagreement audit, 24 labels
+4.47 [2.71,6.34]
8.3
−0.93
+9.87
Same-query oracle (diagnostic)
+8.03
0.0
+2.89
+13.18
Table 4: HAR architectures, selection labels and compatible baselines. A: sealed 12-subject audit; gain/CI: pp, harm: %. Oracle is diagnostic. B: separate supported slices with different references; PeFLL uses 12 labeled descriptors, ATP an unlabeled batch of 20. Accuracy: %.
Figure 4: Public inputs and private transfer. On the same 270 shadow identities: A: anchor-count frontier; B: blur applied only to public inputs; C: broad-minus-restricted neural support with equal counts, 336 updates and 512 extra validation responses. Bars in C are paired 95% client intervals conditional on three draws; EfficientNet at 128 is inconclusive.
Private backbone
Fit inputs
Broad acc.
Restr. acc.
Difference [95% CI]
Δ KL
Harm
MobileNet blocks
64
72.07
69.01
3.06 [1.84,4.32]
−0.0490
27.4/32.2
128
78.46
75.42
3.04 [1.93,4.20]
−0.0426
17.8/24.8
512
85.32
80.94
4.38 [3.26,5.54]
−0.0861
10.4/13.7
EfficientNet blocks
64
75.96
70.12
5.83 [4.51,7.21]
−0.0242
25.6/35.2
128
82.19
82.04
0.15 [ −0.76 ,1.10]
+0.0248
14.1/14.4
512
87.82
84.94
2.89 [1.84,3.94]
−0.0667
8.5/13.0
Table 5: Equal-count public-support ablation. Broad/restricted pools share teacher, initialization, 336 updates, 512 validation responses and 270 identities. Accuracy/harm: %; differences: pp. CIs resample clients after averaging three draws. Δ KL is broad minus restricted; harm is broad/restricted.
Architecture
Pipeline
Accuracy
Harm
Compile (s)
Target (ms)
MLP
Original + KD
95.87/95.82
12.0/12.0
11.8/23.1
0.71/0.66
Meta + KD
95.74/95.97
12.7/11.0
11.0/22.4
0.66/0.66
Meta + dual
95.76/95.82
12.3/12.0
67.1/132.2
6.11/6.23
CNN
Original + KD
98.39/98.33
2.0/2.3
12.9/25.7
0.79/0.77
Meta + KD
98.44/98.49
2.3/2.7
12.8/25.3
0.69/0.69
Meta + dual
98.29/98.42
2.7/2.0
67.9/136.5
42.88/10.94
Table 6: FedEMNIST compiler-budget sensitivity. Entries are 100/200 epochs with matched inputs, initialization, optimizer and batch order. Accuracy/harm: %; harm averages three seeds by architecture. Meta + dual reconstructs HeteroFedTTA. Timings exclude I/O/loading; source effort differs and is reported separately.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Study
roles
frozen choice
policy/result time
policy SHA
CelebA sparse handoff
878/310/291 identities
fit pseudo-label prototype; 8 labels
09-26 23:53/09-26 23:55
9f2392c1
CelebA ridge
new 300 identities
4,096 anchors; ridge alpha 1
09-27 01:26/09-27 01:27
59f616ab
CelebA trainable blocks
second new 300
late blocks; 8 epochs; response selection
09-27 02:07/09-27 02:09
561559fa
CelebA 3-label
third new 300
3 incidence labels; late blocks
09-27 10:39/09-27 10:40
a0b9c7e6
CelebA decorrelated 3-label
fourth new 300
correlation rule; inherited blocks
09-27 12:24/09-27 12:26
9a2de6ac
FedEMNIST
1000/400/300 writers
4,096 anchors; 2 architectures; 20-label local
09-27 13:53/09-27 14:02
6f66f2a4
Appendix
Table S1: Original confirmation chronology. Hash prefixes identify full policy hashes in the receipts. Revised baselines reuse these cohorts rather than create new confirmations.
Study (clients)
Local
Private
Teacher
Gain [95% CI]
Harm (%)
CelebA sparse (291)
63.49
65.23
92.40
1.74 [0.15,3.22]
37.5
CelebA ridge (300)
64.23
80.85
88.13
16.63 [14.63,18.63]
15.3
CelebA blocks (300)
63.15
86.46
88.17
23.31 [21.33,25.27]
6.0
CelebA correlated (300)
55.95
75.48
76.44
19.53 [17.92,21.19]
7.7
CelebA decorrelated (300)
56.40
66.07
67.19
9.66 [8.09,11.20]
19.7
FedEMNIST original (300)
77.90
96.75
99.34
18.85 [17.49,20.20]
1.3
Appendix
Table S2: Historical cohort endpoints, distinct from the strengthened references in main Table 2. Each row retains its own local baseline; rows are not paired interventions. Accuracy is %, gain is pp; the two attribute studies use balanced accuracy. Harm averages architectures; decorrelated ties use the recorded correction. The original FedEMNIST teacher was centralized.
Dataset
Four fixed gains (pp)
PCV choice
PCV gain [95% CI]
Harm
CREMA-D
-7.08 / -6.78 / -8.80 / -6.87
local
0.00 [0.00,0.00]
0.0%
MELD
+11.16 / +16.05 / +17.66 / +17.89
pure KD
+17.89 [15.30,22.03]
0.0%
UCF51-20
+1.90 / +1.57 / -1.18 / -1.45
local
0.00 [0.00,0.00]
0.0%
TCGA-BRCA
+13.33 / +17.62 / +17.18 / +14.21
joint KD 0.1
+13.33 [ −1.31 ,30.00]
33.3%
UCI HAR
+3.23 / +3.88 / +1.98 / +1.69
local
0.00 [0.00,0.00]
0.0%
Appendix
Table S3: Fixed-policy opportunity and deployable population selection. The four gains are global/cluster 0/cluster 1/uniform for CREMA-D and joint KD 0.1/1/10/pure KD elsewhere. PCV uses shadow labels only; local has zero gain. All LCB decisions abstain. TCGA and MELD have only three test clients.
Study/model
Fit
Broad acc.
Restr. acc.
Difference [95% CI]
Δ KL
Harm (%)
MobileNet blocks
64
72.07
69.01
3.06 [1.84,4.32]
-0.0490
27.4/32.2
MobileNet blocks
128
78.46
75.42
3.04 [1.93,4.20]
-0.0426
17.8/24.8
MobileNet blocks
512
85.32
80.94
4.38 [3.26,5.54]
-0.0861
10.4/13.7
EfficientNet blocks
64
75.96
70.12
5.83 [4.51,7.21]
-0.0242
25.6/35.2
EfficientNet blocks
128
82.19
82.04
0.15 [-0.76,1.10]
+0.0248
14.1/14.4
EfficientNet blocks
512
87.82
84.94
2.89 [1.84,3.94]
-0.0667
8.5/13.0
Appendix
Table S5: Equal-count support interventions. Accuracy is %; differences and intervals are pp. Broad/restricted conditions share teacher, clients and training budget within each study. Intervals resample clients after averaging three draws. Restricted means pose-tail on CelebA and four scripts on CREMA-D. Harm columns are broad/restricted; KL difference is broad minus restricted; lower KL is better. Studies use different representations and local references.
Study
Supply
Compile loss
Gain
Inherited
Compiler
Total
CelebA ridge
+23.90
7.27
+16.63
4.0%
11.3%
15.3%
CelebA blocks
+25.02
1.71
+23.31
4.0%
2.0%
6.0%
CelebA decorrelated
+10.78
1.12
+9.66
10.3%
9.3%
19.7%
FedEMNIST
+21.44
2.59
+18.85
0.3%
1.0%
1.3%
Appendix
Table S7: Historical harm decomposition using each cohort’s original local reference. Supply, loss and gain are pp; harm is a percentage after architecture averaging. All quantities use query labels and are diagnostic.
Figure S1 : Utility, candidate opportunity and private architecture. A: Strengthened local controls versus compilation on separate CelebA and FedEMNIST cohorts. B: Four fixed policies on each dataset. Policies 1–4 denote global, cluster 0, cluster 1 and uniform on CREMA-D; elsewhere they denote KD weights 0.1, 1, 10 and pure KD. These are dataset-specific candidate menus. C: UCI HAR gains by private architecture under fixed KD, 6- and 24-label disagreement audits, and a same-query diagnostic oracle. The oracle is not an allowed deployment method. Each comparison retains its original local reference.
Figure S2 : Complete-pipeline budget sensitivity. Accuracy, client harm, compilation time and target inference time at 100 and 200 compilation epochs. M/C denote pixel-MLP/CNN; Orig, Meta and Dual denote original+KD, meta+KD and meta+dual pipelines. Points are the reported three-seed means. Timing is sequential simulator time and excludes loading and I/O; it is not a parallel deployment latency estimate.
Figure S3 : Equal-count neural public-support comparison. Broad versus pose-restricted public inputs at 64, 128 and 512 fit images. M/E denote MobileNetV3-Small/EfficientNet-B0. Each setting uses 336 neural updates, 512 additional shared validation responses, three anchor draws and the same 270 shadow identities. Panels show accuracy and architecture-specific client harm; the main support figure reports paired accuracy differences with client-bootstrap intervals.
Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.
Polycarpo Souza Neto, José Mairton Barros da Silva Júnior, Charles Casimiro Cavalcante
Universidade Federal do Cear´a, Fortaleza, CE, Brazil · Uppsala University, Uppsala, Sweden
Federated pre-training offers a way to train foundation models on private or distributed data without centralizing the underlying datasets. However, evaluating federated pre-training remains challenging because differences in client participation and local data availability can make directly comparable evaluation difficult. Moreover, pre-training test perplexity is tied to the pre-training distribution, while downstream benchmarks introduce task-specific adaptation that may not faithfully reflect the test perplexity established during pre-training. In this work, we study which evaluation protocol more reliably reflects federated pre-training quality. Using a controlled set of centralized and federated-trained models of a 16M parameter transformer model trained on identical client data, we assess evaluation protocols by whether they preserve a reference ranking established on the same pre-training testset. We compare downstream fine-tuning on GLUE, including full, head-only, and reduced-data variants, with next-token prediction on GLUE text as an intrinsic evaluation signal. Our results show that downstream fine-tuning does not reliably preserve the pre-training ranking, whereas direct next-token prediction exhibits a strong correspondence with the pre-training test perplexity. These findings suggest that downstream fine-tuning alone can be misleading when comparing federated pre-trained models, and that evaluation signals closer to the original pre-training objective deserve greater attention.
Federated knowledge must remain usable by recipients with different modalities, private architectures, and tasks. We present FedSocket, which makes recipient execution a design requirement of the exchanged model. A shared Q combines recipient-computable inputs, task-owned outputs, and ownership-aware aggregation, connecting heterogeneous private models through a common prediction interface. Private models teach local Q copies; the returned Q supports local learning and Joint inference, with only Q parameters and counts exchanged. Across six datasets, FedSocket improves missing-modality recipient accuracy over Local by 14.44 and 15.51 percentage points on MELD and UCF-51. Under matched inference capacity, Joint exceeds independent ensembles by 11.06 points in UCF-51 accuracy and 4.87 points in mean bidirectional Flickr30k R@1. Joint also improves over Q alone on all four heterogeneous endpoints, demonstrating the value of combining local and exchanged predictions. Teacher controls, sharing-path interventions, and component factorials identify the roles of supervision, sharing, and deployment. FedSocket makes exchanged knowledge directly usable from federated training to recipient inference.