FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization

Amirardalan Dehghanpour; Seyed Mohammad Azimi-Abarghouyi; Christopher G. Brinton

cs.LGSep 26, 2026

FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization

Authors: Amirardalan Dehghanpour, Seyed Mohammad Azimi-Abarghouyi, Christopher G. Brinton

Organizations: Independent Researcher · Chalmers University of Technology · Purdue University

Abstract

Task-wise federated multi-objective optimization (FedMOO) trains a shared model for competing prediction objectives under heterogeneous data, partial participation, and communication constraints. Existing methods commonly derive task weights from gradient or update geometry. This requires task-specific information or iterative server-side optimization. We introduce FedHV, which maps reference-relative objective slacks to closed-form inverse-slack weights. Each client optimizes one weighted loss and returns objective estimates with its model update. The protocol adds exactly 2m auxiliary scalars per participating client, yielding Theta(d + m) total per-client communication, compared with the Theta(md) task-specific communication of FSMGDA, and requires no additional synchronization stage. We analyze the resulting one-round-delayed weights under client heterogeneity, multi-step local updates, partial participation, and finite-sample objective reports. Under a fixed-horizon positive-slack reference condition, with the prescribed horizon-dependent step size and vanishing report error, FedHV achieves an O(T^(-1/2)) rate for the average squared log-hypervolume gradient norm; persistent report error determines the resulting stationarity neighborhood. The same bound controls the squared Pareto-stationarity residual. Across six Dirichlet-partitioned non-IID settings from four vision benchmark families and three training seeds, FedHV exceeds FSMGDA and FedCMOO in mean accuracy in five settings. Among these methods and uniform scalarization, it achieves the highest worst-task accuracy in four settings and improves the difficult CIFAR-10 objective in both CIFAR10-MNIST settings.

Figures & tables

Panel of Figure 1 — Figure 1: One FedHV communication round. Reports from round $t-1$ determine the weights for round $t$ and accompany the ordinary model exchange. Right: dominant per-round operations and per-client communication; Appendix D gives exact counts.

	Uniform		FSMGDA		FedCMOO		FedHV
Setting	Mean $\uparrow$	Worst $\uparrow$	Mean $\uparrow$	Worst $\uparrow$	Mean $\uparrow$	Worst $\uparrow$	Mean $\uparrow$	Worst $\uparrow$
MultiMNIST	$90.17\pm 0.18$	$88.14\pm 0.34$	$88.78\pm 1.38$	$86.15\pm 2.53$	$89.89\pm 0.16$	$88.79\pm 0.57$	$90.72\pm 0.65$	$88.94\pm 0.61$
MNIST–FMNIST	$86.41\pm 0.51$	$76.98\pm 0.77$	$84.84\pm 0.87$	$75.58\pm 1.05$	$86.69\pm 0.18$	$78.58\pm 0.66$	$86.35\pm 0.45$	$77.03\pm 0.70$
CelebA5 -A	$88.08\pm 0.11$	$86.30\pm 0.19$	$86.98\pm 0.06$	$85.77\pm 0.08$	$86.60\pm 0.31$	$85.00\pm 0.29$	$88.06\pm 0.21$	$86.44\pm 0.16$
CelebA5 -B	$87.94\pm 0.07$	$86.32\pm 0.05$	$85.88\pm 0.07$	$84.70\pm 0.05$	$84.68\pm 1.94$	$82.11\pm 3.55$	$87.88\pm 0.05$	$86.32\pm 0.11$
CIFAR10–MNIST -A	$72.72\pm 0.11$	$51.33\pm 0.18$	$72.76\pm 0.22$	$51.96\pm 0.55$	$71.54\pm 0.48$	$47.98\pm 1.15$	$73.18\pm 0.14$	$53.03\pm 0.35$

Table 1: Final MeanAcc and WorstAcc (%), reported as mean

\pm

sample standard deviation over three training runs on one fixed client partition. Bold denotes the best result and underlining denotes the second-best result among the four methods. Rankings are computed from the unrounded means.

Figure 2: Task-level accuracy, averaged over three runs. (a) Evaluation trajectories on CIFAR10–MNIST -B (

\alpha=0.30

); the terminal cross shows one sample standard deviation per task. (b) Mean accuracy for the five CelebA5 -B objectives (

\alpha=0.30

); the radial axis spans

80

–

91\%

. Appendix C.2 gives numerical values and uncertainty.

Figure 3: FedHV ablations on CIFAR10–MNIST -B. Mean

\pm

std. dev. over three runs for (a) report budget

B_{\mathrm{rep}}

, (b) participating clients

M

, and (c) local steps

K

. Panel (c) includes MeanLoss on the right axis.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Dataset and benchmark examples. The first row illustrates the two shifted digits in MultiMNIST , the digit/clothing pair in MNIST–FMNIST , and the overlaid digit/object pair in CIFAR10–MNIST . The second row shows two positive CelebA examples for one representative attribute from each objective group: Eyeglasses ( $T_{1}$ ), Wavy Hair ( $T_{2}$ ), Wearing Hat ( $T_{3}$ ), Smiling ( $T_{4}$ ), and Bangs ( $T_{5}$ ). Source datasets are described by LeCun et al. (1998) ; Xiao et al. (2017) ; Krizhevsky (2009) ; Liu et al. (2015) .

Objective	Attributes
$T_{1}$	5-o’Clock Shadow, Blond Hair, Bags Under Eyes, Eyeglasses, Mustache, Wavy Hair, Oval Face, Rosy Cheeks
$T_{2}$	Arched Eyebrows, Black Hair, Big Nose, Blurry, Goatee, Straight Hair, Pale Skin, Wearing Earrings
$T_{3}$	Attractive, Brown Hair, Bushy Eyebrows, Double Chin, Mouth Slightly Open, Receding Hairline, Wearing Hat, Young
$T_{4}$	Bald, Big Lips, Chubby, Heavy Makeup, High Cheekbones, Male, Sideburns, Smiling
$T_{5}$	Bangs, Gray Hair, Narrow Eyes, No Beard, Pointy Nose, Wearing Lipstick, Wearing Necklace, Wearing Necktie

Appendix

Table 2: The five multi-label objectives used in CelebA5 .

Setting	$N$	$M$	Samples/client	$\alpha$	Partition rule
MultiMNIST	30	20	600	0.15	all-label Dirichlet
MNIST–FMNIST	30	10	500	0.30	all-label Dirichlet
CelebA5 -A	30	10	500	0.15	all-label Dirichlet
CelebA5 -B	30	10	500	0.30	first-label Dirichlet
CIFAR10–MNIST -A	30	10	500	0.10	all-label Dirichlet
CIFAR10–MNIST -B	30	10	500	0.30	all-label Dirichlet

Appendix

Table 3: Federated partitioning and default participation in the six main settings. The all-label strategy forms Dirichlet allocation strata from all available task-label coordinates, whereas the first-label strategy forms strata using only the first label coordinate. Smaller

\alpha

yields more concentrated client label distributions ( Hsu et al., 2019 ) .

Setting	Adaptive rounds	Batch size	$K$	Momentum	Test period
MultiMNIST $\alpha=0.15$	200	32	10	0	3
MNIST–FMNIST $\alpha=0.30$	500	128	10	0	3
CelebA5 $\alpha=0.15$	300	64	5	0.9	5
CelebA5 $\alpha=0.30$	300	64	5	0.9	5
CIFAR10–MNIST $\alpha=0.10$	500	128	10	0	3
CIFAR10–MNIST $\alpha=0.30$	500	128	10	0	3

Appendix

Table 4: Shared training and evaluation settings.

K

is the number of local SGD steps per selected client.

Setting	Uniform	FSMGDA	FedCMOO	FedHV
MultiMNIST $\alpha=0.15$	$1.0/.3$	$2.0/.1$	$1.2/.5$	$1.2/.3$
MNIST–FMNIST $\alpha=0.30$	$1.2/.4$	$2.0/.1$	$1.2/.5$	$1.6/.3$
CelebA5 $\alpha=0.15$	$.8/.3$	$1.6/.3$	$1.6/.3$	$1.0/.2$
CelebA5 $\alpha=0.30$	$.8/.3$	$1.6/.3$	$1.6/.3$	$1.0/.2$
CIFAR10–MNIST $\alpha=0.10$	$1.2/.4$	$2.0/.1$	$1.6/.3$	$1.6/.3$
CIFAR10–MNIST $\alpha=0.30$	$1.2/.4$	$1.6/.3$	$1.6/.3$	$1.6/.3$

Appendix

Table 5: Server/local learning rates

\eta_{g}/\eta_{\ell}

used in the reported runs. Each pair is fixed across training seeds.

Benchmark family	$\bm{r}^{\rm train}$
MultiMNIST	$(2.458940,\;2.453542)$
MNIST–FMNIST	$(2.454883,\;2.464302)$
CIFAR10–MNIST	$(2.452658,\;2.466771)$
CelebA5	$(1.297312,\;1.023714,\;1.060341,\;1.108676,\;1.085493)$

Appendix

Table 6: Frozen training references used by FedHV. These are optimization references

\bm{r}^{\rm train}

and are distinct from the common evaluation reference

\bm{r}^{\rm eval}=3\,\mathbf{1}_{m}

Setting	Minimum report slack	Floor activation
MultiMNIST	$0.1463$	$0/1200$
MNIST–FMNIST	$0.1413$	$0/3000$
CelebA5 -A	$0.2981$	$0/4500$
CelebA5 -B	$0.3000$	$0/4500$
CIFAR10–MNIST -A	$0.0282$	$0/3000$
CIFAR10–MNIST -B	$0.1412$	$0/3000$

Appendix

Table 7: Reference diagnostics over all adaptive rounds and three training seeds. “Minimum report slack” is

\min_{t,i}(r_{i}-\widehat{f}_{i}^{,t})

for the server-aggregated report; it is a practical diagnostic rather than the true-slack condition in Assumption A.4 .

Setting	Method	MeanLoss $\downarrow$	HV $\uparrow$
MultiMNIST	Uniform	$0.328\pm 0.008$	$7.14\pm 0.04$
	FSMGDA	$0.359\pm 0.032$	$6.97\pm 0.17$
	FedCMOO	$0.354\pm 0.009$	$7.00\pm 0.05$
	FedHV	$0.312\pm 0.019$	$7.23\pm 0.10$
MNIST–FMNIST	Uniform	$0.379\pm 0.006$	$6.82\pm 0.03$
	FSMGDA	$0.427\pm 0.020$	$6.57\pm 0.10$

Appendix

Table 8: Final loss-based results for Uniform and the three adaptive FedMOO methods, mean

\pm

standard deviation over three seeds. Lower MeanLoss is better; higher HV is better. HV uses the common evaluation reference

\bm{r}^{\rm eval}=3\,\mathbf{1}_{m}

, not FedHV’s training reference. Bold marks the best result and underline the second-best result within each setting.

Setting	Method	Task 1 Acc.	Task 2 Acc.	Task 1 Loss	Task 2 Loss
MultiMNIST $\alpha=0.15$	FSMGDA	$91.41\pm 0.31$	$86.15\pm 2.53$	$0.292\pm 0.012$	$0.425\pm 0.056$
	FedCMOO	$90.99\pm 0.41$	$88.79\pm 0.57$	$0.339\pm 0.025$	$0.369\pm 0.008$
	FedHV	$92.50\pm 0.68$	$88.94\pm 0.61$	$0.257\pm 0.018$	$0.366\pm 0.021$
MNIST–FMNIST $\alpha=0.30$	FSMGDA	$94.11\pm 0.81$	$75.58\pm 1.05$	$0.204\pm 0.021$	$0.650\pm 0.020$
	FedCMOO	$94.81\pm 0.43$	$78.58\pm 0.66$	$0.186\pm 0.019$	$0.558\pm 0.017$
	FedHV	$95.67\pm 0.31$	$77.03\pm 0.70$	$0.161\pm 0.018$	$0.602\pm 0.012$

Appendix

Table 9: Final task-level test performance for the four two-objective settings, mean

\pm

standard deviation over three seeds. Accuracies are percentages. Bold is best and underline is second best in each metric among FSMGDA, FedCMOO, and FedHV within a setting.

Figure 5: Alternate task-level views using the same visual encoding as Figure 2 . (a) Round-by-round training trajectories for CIFAR10–MNIST -A (all-label Dirichlet,

\alpha=0.10

), with MNIST accuracy on the horizontal axis and CIFAR-10 accuracy on the vertical axis. Each path follows scheduled evaluation checkpoints from early to late training; the terminal cross shows

\pm 1

standard deviation on both objectives. (b) Mean five-objective radar profile for CelebA5 -A (all-label Dirichlet,

\alpha=0.15

). The radial scale is truncated to

83

–

91\%

for legibility; task-wise uncertainty is reported in Table 9 . The main paper reports the corresponding B settings.