FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning

Konstantinos Ziliaskopoulos; Alexander Vinel; Jiaqi Wang

cs.LGOct 6, 2026

FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning

Authors: Konstantinos Ziliaskopoulos, Alexander Vinel, Jiaqi Wang

Organizations: Auburn University

Abstract

Decision-focused learning (DFL) trains predictive models for downstream optimization, but existing methods largely assume centralized data. In cross-silo settings, federated learning offers a natural alternative, yet standard federated methods optimize prediction over decision quality and do not address heterogeneity in downstream objectives or feasible sets. This heterogeneity is especially challenging for DFL because small perturbations in polyhedral problems can cause discontinuous changes in optimal decisions, destabilizing client updates and aggregation. We propose FedRSPO+, a heterogeneity-aware framework for decision-focused federated learning, built on RSPO+, a regularized predict-then-optimize surrogate that smooths the decision map through projection. We show that RSPO+ upper bounds decision error and regret for the regularized decision and, under exact regularization and consistent LP solution selection, for the original LP decision. We further derive cross-client heterogeneity bounds that depend on both objective and feasible-set heterogeneity, vanish at homogeneity, and require no strong convexity. FedRSPO+ uses an annealed, modular training procedure compatible with standard federated personalization and aggregation methods. Experiments on synthetic knapsack, shortest-path, and real-world energy pricing tasks compare against prediction-only federated learning and DFL baselines under varying heterogeneity and communication budgets. Results suggest that smoothing is a useful ingredient for stable collaborative decision learning and provide a heterogeneity-aware foundation for federated DFL.

Figures & tables

Figure 1 : DFFL setting: a central server broadcasts a global model to heterogeneous clients, each with its own objective function and feasible region.

Figure 2 : FedRSPO+: A heterogeneity-aware federated decision-focused learning pipeline. The server broadcasts a global model, clients perform personalized calibration, local prediction, and projection-based smoothing (RSPO+), and updates are aggregated. During personalization, co-annealed regularization enables stable training under heterogeneous objectives and feasible sets.

Component	Execution	Overview	Communication
(1) Calibration	client, once	set $\lambda_{j},\eta_{j}$ (Eq. ( 11 ))	–
(2) Federated training	server, $R_{\mathrm{fed}}$ rounds	SGD on $F_{j}(\theta;\lambda_{j})$ , FedAvg ( $\propto n_{j}$ )	model weights $\theta$
(3) Personalization	client, $R_{\mathrm{per}}$ rounds	SGD on $F_{j}(\theta;\lambda_{j,t})$ , co-anneal $\lambda,\eta$	–
(4) Freezing	client	stop annealing when $\tilde{U}_{j,t}\leq\tau$ (Eq. ( 6 ))	–

Table 1 : Overview of the four stages of FedRSPO+, along with the frequency and setting of each stage. We elaborate on each component throughout Section 3 .

	SPO+	RSPO+ (Ours)
Q1	$O\!\big(c_{d}+\delta_{N}+D_{\min}\big)$	$O\!\big(c_{d}/\lambda+\sqrt{\delta/\lambda}+\delta\big)$
Q2	No ( $D_{\min}$ persists)	Yes
Q3	Yes, for basic stability	No

Table 2 : Heterogeneity bound comparison.

c_{d}=\|c_{\mathrm{ref}}-c_{\mathrm{cl}}\|

is the objective shift,

\delta

is the Hausdorff set distance,

\delta_{N}

is the translation-invariant Hausdorff set distance, and

D_{\min}

is the minimum set diameter. Q1 : Rate?; Q2 : Vanishes at homogeneity?; Q3 : Requires strong convexity?

	MSE		PFYL		DPO		SPO+		RSPO+ (Ours)
$(\eta_{\mathrm{o}},\eta_{\mathrm{c}},\eta_{\mathrm{d}})$	Fed	+P	Fed	+P	Fed	+P	Fed	+P	Fed	+P
$(0,0,0)$	15.9 $\pm$ 3.3	10.5 $\pm$ 2.8	8.5 $\pm$ 1.5	6.1 $\pm$ 1.1	16.3 $\pm$ 2.7	12.6 $\pm$ 2.2	9.7 $\pm$ 2.1	5.6 $\pm$ 1.4	2.3 $\pm$ 0.4	1.9 $\pm$ 0.3
$(0.5,0,0)$	16.0 $\pm$ 3.4	10.6 $\pm$ 2.8	8.8 $\pm$ 1.5	6.3 $\pm$ 1.2	16.7 $\pm$ 2.5	12.8 $\pm$ 2.3	10.0 $\pm$ 2.1	5.8 $\pm$ 1.5	2.5 $\pm$ 0.3	1.9 $\pm$ 0.3
$(1,0,0)$	16.4 $\pm$ 3.4	10.8 $\pm$ 2.9	9.9 $\pm$ 1.6	6.9 $\pm$ 1.3	17.4 $\pm$ 2.5	13.4 $\pm$ 2.2	10.9 $\pm$ 2.2	6.3 $\pm$ 1.6	3.0 $\pm$ 0.3	2.0 $\pm$ 0.3
$(0,0.5,0)$	13.2 $\pm$ 3.5	9.8 $\pm$ 2.7	13.3 $\pm$ 2.4	8.1 $\pm$ 1.9	19.6 $\pm$ 3.6	13.1 $\pm$ 3.1	11.9 $\pm$ 2.5	7.3 $\pm$ 2.1	3.9 $\pm$ 0.6	2.3 $\pm$ 0.4
$(0,1,0)$	13.3 $\pm$ 2.3	10.5 $\pm$ 2.0	15.0 $\pm$ 2.3	9.1 $\pm$ 1.5	20.8 $\pm$ 3.3	14.1 $\pm$ 2.4	12.9 $\pm$ 2.0	8.1 $\pm$ 1.5	4.6 $\pm$ 0.5	2.6 $\pm$ 0.4

Table 3 : Mean client relative regret (%)

\downarrow

across heterogeneity configurations. Each axis (

\eta_{\mathrm{o}}

:objective

,\eta_{\mathrm{c}}

:constraint

,\eta_{\mathrm{d}}

:data) is varied independently while others are fixed at 0. “Fed”: global federated model; “+P”: after local personalization. Results: mean

\pm

std over 5 seeds.

Setting	Interp-DFFL	OptNet-style	FedRSPO+
$(0,0,0)$	$2.42\pm 0.30$	$9.4\pm 1.9$	$\mathbf{1.86\pm 0.25}$
$(1,0,0)$	$2.42\pm 0.33$	$9.7\pm 2.0$	$\mathbf{2.02\pm 0.26}$
$(0,1,0)$	$3.51\pm 0.77$	$11.3\pm 2.0$	$\mathbf{2.58\pm 0.38}$
$(0,0,1)$	$\mathbf{2.47\pm 0.39}$	$11.3\pm 1.9$	$3.37\pm 0.22$
PJM ($)	$69.6\pm 1.7$	$69.8\pm 1.3$	$\mathbf{60.7\pm 0.8}$

Table 4 : Comparison of FedRSPO+ against Interp-DFFL and OptNet across fractional knapsack heterogeneity settings and PJM cases. Values are mean

\pm

standard deviation across training seeds of mean absolute regret.

Panel of Figure 3 — Figure 3 : Relative regret (%) of the best baseline SPO+ and our method RSPO+ across different values of the learning rate scale $\alpha$ from Eq. 11 , number of clients, and communication budgets. Lower is better ( $\downarrow$ ) . Numbers above the bars show the improvement in relative regret of RSPO+ relative to SPO+.

Method	Mean absolute regret $\downarrow$
MSE	$175.453\pm 7.260$
SPO+	$81.676\pm 1.768$
RSPO+	$72.890\pm 4.231$
RSPO+ w/ P	$68.136\pm 1.402$
RSPO+ w/ P + $\lambda_{j}/\eta_{j}$	$63.006\pm 1.059$
RSPO+(Full)	$\mathbf{60.654}\pm 0.796$

Table 5 : Ablation study for FedRSPO+ on PJM data. Values are mean

\pm

standard deviation across training seeds of mean absolute regret. “w/ P” denotes the personalization phase,

\lambda_{j}/\eta_{j}

denotes per-client parameters, and Full additionally includes time-inverse scheduling.

Method	Federated (%)	+P (%)
DPO	$11.309\pm 2.244$	$11.018\pm 1.718$
MSE	$14.784\pm 2.491$	$7.241\pm 1.336$
PFYL	$8.959\pm 1.688$	$11.449\pm 1.048$
SPO+	$5.978\pm 1.842$	$1.850\pm 0.377$
Ours	$7.504\pm 1.929$	$2.571\pm 0.901$

Table 6 : Mean optimality gap (

\downarrow

) on the Warcraft Shortest Path benchmark. Results are mean

\pm

std over 5 seeds.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Warm-start	Personalization
Communication rounds	10	10
Local epochs per round	3	1
Batch size	64	64
Client fraction	0.4	1.0
$\kappa^{\lambda}$ (RSPO+ only)	0.0	1.0
$\eta_{0}$ (baseline)	$10^{-3}$	$10^{-3}$

Appendix

Table 7 : Synthetic knapsack hyperparameters. RSPO+-only entries are marked; SPO+ and MSE use the same configuration with constant schedules (

\kappa=0

Hyperparameter	Warm-start	Personalization
Communication rounds	10	10
Local epochs per round	3	2
Batch size	64	64
Client fraction	0.5	1.0
$\lambda_{0}$ (RSPO+ only)	2.0	2.0
$\kappa^{\lambda}$ (RSPO+ only)	0.0	1.0

Appendix

Table 8 : Warcraft shortest path hyperparameters.

Method	$(0,0,0)$ $\downarrow$	$(1,0,0)$ $\downarrow$	$(0,1,0)$ $\downarrow$
Local-only RSPO+	$9.74\pm 2.73$	$9.80\pm 2.78$	$6.99\pm 1.89$
FedSPO+	$9.69\pm 2.08$	$10.93\pm 2.19$	$12.89\pm 1.99$
RSPO+ (surrogate only)	$2.26\pm 0.40$	$3.05\pm 0.25$	$4.71\pm 0.49$
RSPO+ w/ $\lambda_{j}/\eta_{j}$	$2.26\pm 0.40$	$3.05\pm 0.25$	$4.63\pm 0.46$
RSPO+ w/ $\lambda_{j}/\eta_{j}$ + P	$2.16\pm 0.36$	$2.41\pm 0.38$	$4.08\pm 0.41$
RSPO+ (Full)	$\mathbf{1.86}\pm 0.26$	$\mathbf{2.02}\pm 0.26$	$\mathbf{2.58}\pm 0.40$

Appendix

Table 9: Fractional-knapsack component ablation. Values are mean test relative regret (%)

\pm

sample standard deviation across five seeds, after averaging clients within each seed; lower is better.

Hyperparameter	Warm-start	Personalization
Communication rounds	10	10
Local epochs per round	3	1
Batch size	64	64
Client fraction	0.4	1.0
$\kappa^{\lambda}$ (RSPO+ only)	0.0	1.0
$\eta_{0}$ (baseline)	$10^{-3}$	$10^{-3}$

Appendix

Table 10 : PJM hyperparameters.

Target training days	Local RSPO +	Federated only	FedRSPO + personalized	Gap vs. local (Fed+P)	Relative reduction
					Fed-only	Fed+P
3	$168.48\pm 10.29$	$73.58\pm 1.86$	$73.06\pm 2.19$	$95.43\pm 9.30$	$56.3\%$	$56.6\%$
7	$167.49\pm 9.49$	$73.58\pm 1.86$	$73.64\pm 2.14$	$93.85\pm 8.71$	$56.1\%$	$56.0\%$
14	$164.44\pm 10.91$	$73.58\pm 1.86$	$73.31\pm 2.15$	$91.13\pm 10.46$	$55.3\%$	$55.4\%$
30	$164.37\pm 7.91$	$73.58\pm 1.86$	$73.12\pm 2.11$	$91.26\pm 7.62$	$55.2\%$	$55.5\%$
60	$153.86\pm 8.17$	$73.58\pm 1.86$	$72.20\pm 1.84$	$81.67\pm 6.51$	$52.2\%$	$53.1\%$

Appendix

Table 11 : Leave-one-client-out experiment for PJM. Results are reported in absolute decision regret; lower is better.

Target training days	Federated-only beats local	Median client reduction [IQR]
3	$19/19$	$51.1\%\;[37.2\%,\,66.3\%]$
7	$19/19$	$47.9\%\;[33.9\%,\,65.4\%]$
14	$19/19$	$48.6\%\;[27.4\%,\,65.6\%]$
30	$19/19$	$45.1\%\;[28.1\%,\,64.8\%]$
60	$18/19$	$41.0\%\;[27.4\%,\,57.2\%]$

Appendix

Table 12 : Client-level benefit of federated-only training over the original local RSPO + baseline with the PJM experiment.

Common demand	Percentage change in gap
4	$-18.50\%\pm 9.73\%$
6	$-8.37\%\pm 10.94\%$
8	$-4.97\%\pm 10.03\%$
10	$-24.33\%\pm 8.71\%$
12	$-20.13\%\pm 8.16\%$

Appendix

Table 13 : Effect of equal client demands on the regret gap between federated-only SPO + and personalized FedRSPO + on the PJM-derived constructed-cost benchmark.

$(\eta_{o},\eta_{c},\eta_{d})$	Centralized	Local-only	Fed	Fed + P
SPO+
$(0,0,0)$	$0.32\pm 0.06$	$2.42\pm 0.30$	$9.7\pm 2.1$	$5.6\pm 1.4$
$(0,0,0.5)$	$3.49\pm 0.62$	$2.93\pm 0.54$	$14.1\pm 2.0$	$6.9\pm 1.3$
$(0,0,1)$	$6.97\pm 0.89$	$2.47\pm 0.39$	$17.7\pm 2.1$	$6.9\pm 0.8$
RSPO+ (Ours)
$(0,0,0)$	$0.75\pm 0.08$	$9.74\pm 2.73$	$2.3\pm 0.4$	$1.9\pm 0.3$

Appendix

Table 14: Synthetic knapsack: relative regret (%) of centralized and local-only baselines on representative heterogeneity configurations, alongside the federated results from Table 3 for reference. Mean

\pm

std over 5 seeds. Lower is better (

\downarrow

Regime	MSE	SPO+	RSPO+ (Ours)
Centralized	$70.26\pm 0.61$	$69.37\pm 0.44$	$69.32\pm 1.18$
Local-only	$163.76\pm 10.12$	$69.57\pm 1.72$	$61.53\pm 0.66$
Federated (naive)	$175.45\pm 7.26$	$81.68\pm 1.77$	$72.89\pm 4.23$
FedRSPO+ (Full)	—	—	$\mathbf{60.65\pm 0.80}$

Appendix

Table 15: PJM energy pricing: mean absolute regret across training regimes. Mean

\pm

std over 5 seeds. Federated rows are reproduced from Table 5 for reference. Lower is better (

\downarrow

Dataset	Decision gap (%)	Objective gap (%)
Knapsack	21.44	3.26
PJM	42.57	10.43

Appendix

Table 16 : QP–LP discrepancies after personalization. Values are mean normalized gaps (%; lower is better), averaged over clients and five seeds, and additionally over 27 heterogeneity configurations for knapsack. The PJM diagnostic uses 10 personalization rounds.

Metric	Mean	Median	Range
Global QP–LP gap	3.715	3.654	3.080–4.456
Personalized QP–LP gap	1.166	1.118	0.922–1.521
Personalized / global ratio	0.314	0.316	0.258–0.354
Final freeze statistic	0.01067	0.00959	0.00822–0.01761
Frozen-client fraction	0.761	0.800	0.300–1.000

Appendix

Table 17 : QP–LP decision alignment on synthetic knapsack. Personalization substantially reduces the predicted decision discrepancy between the regularized RSPO+ decision and the LP decision used at evaluation.

	MSE		PFYL		DPO		SPO+		RSPO+
Heterogeneity	Fed	+P	Fed	+P	Fed	+P	Fed	+P	Fed	+P
$(0,0,0)$	$37.76\pm 2.44$	$34.89\pm 2.16$	$41.08\pm 3.14$	$38.93\pm 3.22$	$44.99\pm 4.03$	$44.43\pm 4.32$	$39.57\pm 3.16$	$36.79\pm 3.14$	$\mathbf{12.64\pm 0.91}$	$\mathbf{11.04\pm 0.70}$
$(0.5,0,0)$	$37.88\pm 2.45$	$34.97\pm 2.20$	$41.17\pm 3.10$	$39.01\pm 3.19$	$45.02\pm 4.00$	$44.45\pm 4.27$	$39.68\pm 3.10$	$36.90\pm 3.16$	$\mathbf{13.30\pm 1.04}$	$\mathbf{11.54\pm 0.82}$
$(1,0,0)$	$38.42\pm 2.43$	$35.42\pm 2.04$	$41.57\pm 2.95$	$39.38\pm 3.06$	$45.08\pm 3.78$	$44.49\pm 4.05$	$40.20\pm 3.00$	$37.35\pm 3.02$	$\mathbf{15.09\pm 1.14}$	$\mathbf{13.00\pm 0.97}$
$(0,0.5,0)$	$42.26\pm 2.69$	$39.38\pm 2.68$	$47.22\pm 2.96$	$45.43\pm 2.89$	$49.87\pm 3.27$	$49.38\pm 3.31$	$45.00\pm 3.06$	$42.46\pm 2.95$	$\mathbf{14.83\pm 1.19}$	$\mathbf{12.98\pm 0.93}$
$(0,1,0)$	$44.03\pm 2.24$	$41.08\pm 2.22$	$49.45\pm 2.82$	$47.93\pm 2.75$	$51.97\pm 2.87$	$51.57\pm 2.97$	$47.11\pm 2.76$	$44.68\pm 2.87$	$\mathbf{15.71\pm 0.58}$	$\mathbf{13.73\pm 0.52}$

Appendix

Table 18 : Bias-free linear predictor results on synthetic knapsack. Entries report mean client relative regret (%) as mean

\pm

sample standard deviation across five seeds. The heterogeneity tuple represents objective, constraint, and data-distribution heterogeneity, respectively. “Fed” denotes the federated endpoint; “+P” includes subsequent personalization. Bold indicates the lowest mean regret within each training stage.

Panel of Figure 4 — Figure 4 : (a) Relative regret (%) of SPO+ and RSPO+ across different (a) values of the freezing threshold $\tau$ , (b) values of $\kappa^{\lambda}$ , and (c) values of $\kappa^{\mathrm{LR}}$ . Lower is better ( $\downarrow$ ).

Explore similar work

Apr 21, 2026math.OC

Decision-Focused Federated Learning Under Heterogeneous Objectives and Constraints

We consider Decision-Focused Federated Learning (DFFL), a predict-then-optimize setting in which multiple clients collaboratively train predictive models for downstream linear optimization problems without exchanging raw data. Besides the data heterogeneity typical of standard federated learning, clients may also have different objective functions and feasible regions. Building on the SPO+ surrogate loss, we derive heterogeneity bounds that separate objective shift, measured through cost-vector distances, from feasible-set shift, measured through support-function and shape-distance terms. We show that, for general compact feasible sets, small objective perturbations can still induce nonvanishing decision-focused loss discrepancies, while strongly convex feasible regions yield sharper stability-based bounds. We then lift these pointwise bounds to a local-versus-federated excess-risk comparison, showing that federation is beneficial when the statistical advantage of pooling exceeds a client-specific heterogeneity penalty. Computational experiments on polyhedral and strongly convex problems confirm that federation is substantially more robust under strongly convex feasible regions. Finally, we evaluate a simple validation-based interpolation between local and federated DFFL models. This interpolation mitigates the theoretical tradeoff and reduces aggregate regret and worst-client harm in both synthetic experiments and a PJM energy-pricing case study.

Konstantinos Ziliaskopoulos, Alexander Vinel

Auburn University

Federated Learning Heterogeneity Convex Sets

Ask PDF

Jul 5, 2026cs.LG

FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture

Routing-prediction federated learning has emerged as a new paradigm that reframes inter-client heterogeneity as a resource for system-level intelligence: at inference time, the server routes each external query to the best-matched client for prediction. Existing approaches, however, typically treat each client as internally homogeneous, overlooking latent subpopulations within local data. For example, patients with the same diagnosis at one hospital may exhibit morphologically distinct disease subtypes. The coexistence of inter-client and intra-client heterogeneity, which we call dual heterogeneity, can impair both routing and prediction. To address this challenge, we propose FedSPM, a routing-enabled semiparametric mixture framework that represents each client using client-specific latent components. Each component combines a predictive distribution for classification with a feature distribution for routing. To flexibly model feature distributions while effectively sharing information across clients, FedSPM models their density ratios relative to a common nonparametric measure estimated via empirical likelihood. We develop a federated expectation-maximization algorithm that optimizes a tractable surrogate and prove convergence of the exact profiled objective at the standard

\mathcal{O}(1/\sqrt{T})

rate when the surrogate errors are properly controlled. Experiments on controlled benchmarks and real-world medical data demonstrate consistent improvements in routing and prediction under dual heterogeneity. Code is available at https://github.com/zijianwang0510/FedSPM.

Zijian Wang, Pengfei Li, Guangyu Yang +1

Institute of Statistics and Big Data, Renmin University of China · Department of Statistics and Actuarial Science, University of Waterloo

Heterogeneous Federated Learning Federated Learning Heterogeneity Fedavg Mixture Models

Ask PDF

Sep 7, 2026cs.LG

Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration

This paper proposes a robust decentralized personalized federated learning method R-DPFL, that enables clients to reduce the impact of Byzantine attacks via robust neighborhood direction estimation and history-based update trend prediction, rather than purely aggregating client models as in the existing work. In R-DPFL, each client first computes the current-round model update by aggregating the received neighborhood update vectors. It then predicts what this update should be based on its historical values and local model changes. Finally, R-DPFL computes the difference between these two quantities, adaptively clips this difference, and adds it to the local update. We prove convergence of the learning process through rigorous analysis and show that honest clients maintain stable personalized descent dynamics under Byzantine neighbor perturbations without requiring consensus among neighboring models. Extensive experiments on CIFAR-10 demonstrate that RDPFL consistently outperforms state-of-the-art decentralized and personalized federated learning baselines under heterogeneous and adversarial settings.

Xiao Ma, Hong Shen, Hui Tian +2

Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China

Federated Learning Byzantine Attacks Approximate Nearest-Neighbor Search Adversarial Robustness

Ask PDF