Shallow Probes Are Sufficient for Near-Oracle Personalization Depth Allocation in Federated Learning

Beyazit Bestami Yuksel; Emrah Dikbiyik; Ayse Yilmazer Metin

cs.LGOct 18, 2026

Shallow Probes Are Sufficient for Near-Oracle Personalization Depth Allocation in Federated Learning

Authors: Beyazit Bestami Yuksel, Emrah Dikbiyik, Ayse Yilmazer Metin

Organizations: Department of Computer Engineering, Istanbul Technical University, Istanbul, Turkey · Department of Computer Engineering, Marmara University, Istanbul, Turkey

Abstract

Rather than proposing a new federated optimizer, we study whether a small client-specific measurement budget can support cost-aware personalization-depth decisions under a utility that trades Macro-F1 against deployment cost. Exhaustive evaluation of six depths defines an offline oracle that is not deployable. We audit the Probe Sufficiency Hypothesis: three shallow probes (P0--P2) may contain most decision-relevant information for approaching that oracle. FedPBA is a secondary learned instantiation for deep-action extrapolation. On CIFAR-10/100 and patient-disjoint PTB-XL (45 sessions; 900 clients; 2700 evaluations; GroupKFold), training-free greedy probing achieves regret 0.0071 and 98.3 percent oracle recovery, beating metadata-only policies and FedPBA (0.0110 / 97.3 percent). Aggregate shallow-best versus oracle correlation is 0.9818 but falls to 0.578 on deep-oracle rows; PTB-XL deep-oracle cases remain a high-regret boundary without a validated escalation gate.

Figures & tables

Action	Locally trainable components
$P_{0}$	None (global-only)
$P_{1}$	Classifier head
$P_{2}$	Adapter and head
$P_{3}$	Last encoder block, adapter, head
$P_{4}$	Last two encoder blocks, adapter, head
$P_{5}$	Full model

TABLE I: Personalization ladder (action space).

Fig. 1: Probe-first personalization-depth allocation. Left: offline oracle exhaustively evaluates $P_{0}$ – $P_{5}$ (empirical upper bound; not deployable). Right: three shallow probes $P_{0}$ – $P_{2}$ are measured and a policy selects a depth without evaluating $P_{3}$ – $P_{5}$ . Greedy is the direct shallow-sufficiency test; FedPBA is the learned extrapolation policy; context-only and best uniform are information-control baselines.

Class	Policy	Probes	Regret	Rec. %
C	Greedy	3	$\mathbf{0.0071}$	$\mathbf{98.3}$
D	FedPBA	3+pred.	$0.0110$	$97.3$
A	Best uniform †	0 ‡	$0.0483$	$88.4$
B	Best context-only	0	$0.0476$	$88.5$
A	Majority	0	$0.0894$	$78.4$
A	Personalize-all	0	$0.5185$	$-25.0$

TABLE II: Policy comparison (

n{=}2700

; GroupKFold).

Fig. 2: Mean regret by policy class ( $n{=}2700$ ). Probe policies sit well below fixed and context-only classes.

Fig. 3: Oracle utility recovery across policies (oracle mean $U{=}0.4147$ ).

Probe signal	$r$
$P_{0}$ utility	$0.8353$
$P_{1}$ utility	$0.8605$
$P_{2}$ utility	$0.8323$
$\max(P_{0},P_{1},P_{2})$ utility	$\mathbf{0.9818}$

TABLE III: Probe utility versus oracle utility (Pearson

r

Fig. 4: Shallow-best vs. oracle utility ( $r{=}0.9818$ overall, $n{=}2700$ ; deep-oracle-only $r{=}0.578$ , $n{=}133$ ).

Comparison	$\Delta\bar{U}$	95% CI	Holm $p$	$d_{z}$
Greedy vs. unif.	$+0.0411$	$[0.026,0.059]$	$1.7{\times}10^{-13}$	$0.72$
FedPBA vs. unif.	$+0.0372$	$[0.022,0.054]$	$1.7{\times}10^{-13}$	$0.67$
Greedy vs. FedPBA	$+0.0039$	$[0.003,0.005]$	$1.7{\times}10^{-13}$	$1.05$

TABLE IV: Session-level paired tests (

n{=}45

; Holm-adjusted).

Fig. 5: Mean regret with session-cluster 95% bootstrap CI.

Fig. 6: Probe-to-exhaustive wall-time ratios across sessions (median ratio $\approx 0.30$ ).

Fig. 7: Per-dataset mean regret (session-cluster 95% CI).

Fig. 8: Mean regret by cost weight $\lambda$ (session-cluster 95% CI).

Fig. 9: Conditional rate $P(\mathrm{greedy}{=}P_{0}\mid\textrm{oracle deep})$ by dataset and $\lambda$ (N/A if no deep-oracle rows).

	Greedy	FedPBA
Count	37 (all PTB-XL)	39 (38 PTB-XL; 1 CIFAR-10)
Mean regret	$0.3665$	$0.3512$
Regret ${>}0.05$	34	35

TABLE V:

P_{0}

selected while oracle prefers

\{P_{3},P_{4},P_{5}\}

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 961 papers

FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling
Apr 17, 2026Xingyan Chen, Tian Du, Changqiao Xu +4Federated Learning Heterogeneous Federated Learning
Beijing Key Laboratory of MEMS Technology and Device Reliability for Industrial Internet, the School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing 100876, P. R. China · School of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, P. R. China · Institute of Artificial Intelligence and the School of Computer Science, Beihang University, Beijing 100191, China, and with the SKLSDE +3
ORDERS: An Empirical Study of Norm-Rank Aggregation for Personalized Federated Learning
Oct 7, 2026Koffka KhanFederated Learning Federated Learning Aggregation
Department of Computing and Information Technology, Faculty of Science and Technology, The University of the West Indies, St. Augustine, Trinidad and Tobago
Is High Local Accuracy Enough? Geometric Consensus with Support-Adaptive Statistical Personalization for Federated Learning
Oct 22, 2026Junyu Lin, Bosong Qi, Pengcheng Shi +1Federated Learning Heterogeneous Federated Learning
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430072, China

Apr 17, 2026J/K move · Enter open · S save

FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling

Xingyan Chen, Tian Du, Changqiao Xu, Fuzhen Zhuang+3

Beijing Key Laboratory of MEMS Technology and Device Reliability for Industrial Internet, the School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing 100876, P. R. China · School of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, P. R. China · Institute of Artificial Intelligence and the School of Computer Science, Beihang University, Beijing 100191, China, and with the SKLSDE+3

Federated Learning Heterogeneous Federated Learning

Open paper AskPDF arXiv

Federated Learning (FL) faces challenges from client data heterogeneity and resource-constrained mobile devices, which can degrade model accuracy. Personalized Federated Learning (PFL) addresses this issue by adapting shared global knowledge to local data distributions. A promising approach in PFL is model decoupling, which separates the model into global and personalized parameters, raising the key question of which parameters should be personalized to balance global knowledge sharing and local adaptation. In this paper, we propose a Federated Optimal Brain Personalization (FedOBP) algorithm with a quantile-based thresholding mechanism and introduce an element-wise importance score. This score extends Optimal Brain Damage (OBD) pruning theory by incorporating a federated approximation of the first-order derivative in the Taylor expansion to evaluate the importance of each parameter for personalization. Moreover, we move the metric computation originally performed on clients to the server side, to alleviate the burden on resource-constrained mobile devices. To the best of our knowledge, this is the first work to bridge classical saliency-based pruning theory with federated parameter decoupling, providing a rigorous theoretical justification for selecting personalized parameters based on their sensitivity to local loss landscapes. Extensive experiments demonstrate that FedOBP outperforms state-of-the-art methods across diverse datasets and heterogeneity scenarios, while requiring personalization of only a very small number of personalized parameters.

Abstract

Federated Learning

Explore similar work

FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling

ORDERS: An Empirical Study of Norm-Rank Aggregation for Personalized Federated Learning

Is High Local Accuracy Enough? Geometric Consensus with Support-Adaptive Statistical Personalization for Federated Learning