CovFedAvg: Local Low-Loss Curves and Federated Model Selection

Federico Lucchetti; Marcus Völp

cs.LGOct 29, 2026

CovFedAvg: Local Low-Loss Curves and Federated Model Selection

Authors: Federico Lucchetti, Marcus Völp

Organizations: Center for Psychiatric Neuroscience, Department of Psychiatry, Lausanne University Hospital and the University of Lausanne, Lausanne, Switzerland · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg

Abstract

Clients observing different parts of a prediction task may learn locally equivalent models with incompatible behavior outside their observed support. We study whether redundant models permit local low-loss paths toward a compatible shared model. CovFedAvg retains local models and curve controls on clients, fits a common endpoint through aggregated gradient contributions, and selects an actual candidate model using distributed training losses. No centralized calibration examples or client-to-client sample exchange are required. Exact selection guarantees no higher evaluated training loss than the previous model and the current-round FedAvg candidate; this is not a global-optimality, stationarity, generalization, or differential-privacy guarantee. We state conditional client-compatibility and approximate-selection bounds and retain corrected transformation identities and a separate full-gradient safeguard theorem for reference algorithms. A complementary input-support example recovers the shared absolute-value regression, while matched direct fine-tuning also solves it. Across a synthetic support task, recorded activity-recognition subjects, and participating and future gas-sensor batches, we evaluate five methods, five seeds, two network widths, and 20 rounds without calibration data. At width 16, CovFedAvg raises participating-Gas macro balanced accuracy from FedAvg's 88.88% to 89.84%; matched distributed fine-tuning reaches 91.65%. Results support feasibility and modest gains over FedAvg, but do not establish a curve-specific advantage over equally costly optimization. Exact-sum numerical experiments establish data flow; end-to-end privacy requires additional mechanisms and analysis.

Figures & tables

Figure 1: Original schematic motivating transformed client domains. Top: uniform intensity scaling; middle: spatially varying pixel changes; bottom: a parameterized visual deformation. The inverse arrows require an invertible map, which the finite examples do not establish. The digit interpolation may change class; it illustrates deformation rather than the label-preserving assumption of the formal model.

Figure 2: Original geometric rationale for curve aggregation (idealized schematic). Models

w_{1},w_{2}

can lie in a curved low-loss region even when their Euclidean average

w_{F}

has higher loss; the bottom profile motivates seeking a low-loss path

\gamma_{\theta}(u)

. The original labels

\mathcal{M}

and

\mathcal{M}_{\mathrm{SYM}}

illustrate this picture and do not assert an inferred manifold or connection. Neither low centroid risk nor a mapping from input deformations to parameter paths follows from the sketch. The digit strip is a deformation analogy and may change class.

Figure 3: Complementary input support and selection of a shared model. The selected endpoint recovers both branches; averaging the same fitted paths does not. Matched direct fine-tuning also recovers the solution. The right panel projects parameters onto

(u,v)

; its analytic zero-loss lines assume

b=0.3

, whereas fitted methods can have slightly different intercepts. The dotted segment connects the analytic local solutions and does not depict a learned low-loss curve.

Domain / width	Method	Macro BAcc. (%)	Worst BAcc. (%)
support shift / 16	FedAvg	$73.13\pm 1.90$	$59.63\pm 1.29$
support shift / 16	FedAvg + selection	$73.13\pm 1.90$	$59.63\pm 1.29$
support shift / 16	Matched distributed fine-tuning	$74.11\pm 1.70$	$60.41\pm 1.37$
support shift / 16	CovFedAvg	$73.65\pm 2.08$	$59.76\pm 1.18$
support shift / 16	Same-fit curve centroid	$73.49\pm 1.96$	$59.67\pm 1.24$
support shift / 64	FedAvg	$73.29\pm 1.85$	$60.03\pm 0.83$

Table 1: Local-data study: mean

\pm

sample standard deviation over five seeds. BAcc. averages recalls over classes present in each test domain; macro averages domains and worst takes the smallest domain score. No calibration records are used. All methods use the same training data within each task.

Figure 4: CovFedAvg and controls without centralized calibration. Left: mean source training objective at width 16. Right: final macro domain balanced accuracy versus width, with sample standard deviation over five seeds. Each outer round includes method-specific additional queries, whose costs are reported separately. Source-objective selection does not guarantee improved held-out accuracy or preservation of each client.

	HAR, unseen people		Gas, known batches		Gas, future batches
Method	Macro BA	Worst BA	Macro BA	Worst BA	Macro BA	Worst BA
FedAvg	$91.6\pm 0.6$	$80.2\pm 2.0$	$88.9\pm 0.6$	$56.6\pm 2.1$	$76.5\pm 1.7$	$71.6\pm 3.4$
FedProx	$91.6\pm 0.6$	$80.3\pm 2.0$	$88.8\pm 0.6$	$56.2\pm 1.6$	$76.5\pm 1.7$	$71.6\pm 3.4$
SCAFFOLD	$91.8\pm 0.7$	$80.8\pm 3.3$	$94.7\pm 0.3$	$81.0\pm 4.8$	$76.4\pm 1.4$	$71.1\pm 2.9$
SafeAvg	$91.5\pm 0.6$	$80.4\pm 1.8$	$88.8\pm 0.5$	$56.6\pm 2.1$	$76.6\pm 1.5$	$71.8\pm 3.1$
SafeCal	$91.8\pm 0.6$	$80.2\pm 2.4$	$90.8\pm 1.4$	$66.1\pm 10.5$	$76.5\pm 1.8$	$71.8\pm 3.7$

Table 2: Recorded native-domain tasks at

E=5

, round 20. Macro BA averages domain balanced accuracies; worst BA is the poorest domain. Entries are mean

\pm

sample standard deviation across five seeds. All eleven methods are shown, including degraded adapter results on future domains.

Figure 5: At

E=5

, FedAvg under native client grouping and an IID-training reshuffle of the identical source records and client sizes; native SCAFFOLD and pooled Centralized are references. Bars show macro balanced accuracy with sample standard deviations across five seeds. Held-out native targets and the source-only global standardizer remain fixed. The much larger Gas gap shows that native heterogeneity can affect optimization; HAR retains strong FedAvg performance despite measured subject differences.

Figure 6: Recorded native-domain experiments. CovFedAvg (calibration)is safeguarded CovFedAvg ( SafeCurve ); CovFedAvg (calibration + adapters)additionally uses client affine adapters ( AdapterSafeCurve ). Left: regularized source-training objective across rounds at

E=5

. Right: final held-out macro domain balanced accuracy against local epoch count, with sample standard deviations across five seeds. The future-Gas task has only

E=5

. The safeguard certifies the source objective; held-out accuracy is a separate outcome.

Figure 7: An explicit limit of curve aggregation. Every point of the upper unit semicircle has zero loss for

F(w)=(\|w\|^{2}-1)^{2}

, but the path centroid

(0,2/\pi)

has loss

(1-4/\pi^{2})^{2}\approx 0.354

. An improvement over the endpoint average does not make the centroid a zero-loss solution.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Method	Macro acc. (%)	Worst acc. (%)	Test CE
Digits, IID	FedAvg	$94.1\pm 1.5$	$91.1\pm 4.0$	$0.330\pm 0.032$
Digits, IID	FedProx	$94.1\pm 1.5$	$91.1\pm 4.0$	$0.331\pm 0.032$
Digits, IID	FedAvg+cal	$94.2\pm 0.7$	$91.7\pm 1.4$	$0.228\pm 0.018$
Digits, IID	CovFedAvg	$94.6\pm 0.7$	$91.7\pm 2.2$	$0.311\pm 0.029$
Digits, shifted	FedAvg	$93.2\pm 0.9$	$90.0\pm 1.8$	$0.422\pm 0.039$
Digits, shifted	FedProx	$93.2\pm 0.9$	$90.0\pm 1.8$	$0.424\pm 0.039$

Appendix

Table 3: Feasibility pilot at round 20: mean

\pm

sample standard deviation over three seeds. Domains in the shifted settings are constructed synthetic measurement domains.

Figure 8: Held-out macro client accuracy and cross entropy. Lines are three-seed means; bands show one sample standard deviation. Public calibration updates have the same gradient-evaluation budget for FedAvg+cal and CovFedAvg.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 961 papers

Sep 17, 2026

FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift

Federated learning (FL) enables collaborative model training without sharing raw data, but its performance degrades under non-IID data and stochastic client participation. Remedies built on classical Federated Averaging (FedAvg) typically presuppose that client participation probabilities are known to the server, which is rarely the case in deployed systems. We first discuss and then characterize the optimization problem that distributionally agnostic FedAvg actually solves when participation is entirely unknown, possibly highly skewed, and of variable size across rounds: uniform aggregation is shown to minimize a well-defined stochastic objective, weighted by the participation-induced marginal, at a standard

\mathcal{O}(1/\sqrt{T})

rate for convex and possibly nonsmooth losses. Building on this characterization, we propose Federated Risk-Averse Averaging (FedeRage), a risk-averse extension of FedAvg that embeds the Conditional Value-at-Risk (CVaR) into the local objective within a natural distributionally robust optimization (DRO) framework. FedeRage implicitly upweights high-loss and infrequently participating clients while adding only a single scalar per-client, and admits an

\mathcal{O}(κ/\sqrt{T})

rate in which the factor

κ

is the upper bound on the ``price" of risk aversion. In contrast with aggregation-alignment schemes based on optimal transport, which require the availability distribution as an input, FedeRage remains agnostic to it. Several experiments on three heterogeneous benchmarks indicate consistent improvements over state-of-the-art methods in accuracy, fairness, and convergence speed.

Herlock Rahimi, Dionysis Kalogerias

Federated Learning Federated Learning Aggregation Non-IID Federated Learning Distributionally Robust Optimization

Ask PDF

May 20, 2026

Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning

Federated Learning enables collaborative model training across decentralized data sources without data transfer. Averaging-based FL is limited by the presence of non-IID data, which negatively impacts convergence speed and final model accuracy. Conventional alternatives suffer from significant inefficiency. Clients with noisy or highly heterogeneous data contribute expensive gradient computations that are either discarded or heavily down-weighted before aggregation. These reactive approaches waste computational resources, require more communication rounds and result in unnecessary privacy exposure. In this paper, we propose a proactive client selection framework that aims to find an optimal federation of clients whose combined data match utility and fairness requirements before training begins. Our method relies on mutual information computed from differentially private contingency tables to quantify the relevance of cross-feature correlations in the union dataset. We introduce a Potential Federation Loss (PFL) over the set of fixed-size federations, which balances two objectives. Maximizing collective data utility while ensuring fair cross-features correlations to prevent group unfairness. Client selection is expressed as an optimal subset search problem over the PFL objective, which we solve using simulated annealing under strong differential privacy guarantees for clients' local statistics. Experimental results on four benchmarks show faster, fairer, and more accurate models trained on optimally found federations, compared to uniform sampling, even when state-of-the-art adaptive aggregation or sampling strategies are employed.

Adda Akram Bendoukha, Heber Hwang Arcolezi, Nesrine Kaaniche +1

Samovar, T´el´ecom SudParis, Institut Polytechnique de Paris, France · ´ETS Montr´eal · CEA-LIST

Client Selection in Federated Learning Non-IID Federated Learning Federated Learning

Ask PDF

Jul 3, 2026

FedAvg for HAR: Exploring the Tradeoff Between Personalized and Generalization Accuracy

The federated learning (FL) paradigm fosters distributed pervasive computing combined with artificial intelligence techniques, allowing for optimized data usage and improved mitigation of privacy concerns. Indeed, model training occurs on the client's local devices, and model parameters are subsequently shared with a centralized server. However, there is a need to find a tradeoff between models' personalization and generalization capabilities. In this paper, we design and implement several testing scenarios devoted to evaluating and comparing the centralized, local, and federated paradigm performances. We also design and implement a scenario that emulates a change in clients' data. We then present experimental results of the FedAvg algorithm applied to the Human Activity Recognition (HAR) domain to understand the trade-off between personalized and generalized accuracy. Results show that, although FedAvg confirms a higher degree of personalization capabilities while keeping a high degree of generalization with respect to the traditional centralized learning, this result is not so obvious under stressful conditions, such as when varying class distribution over clients.

Andrea De Luna, Susanna Peretti, Chiara Contoli +1

DiSPeA University of Urbino

Federated Learning Aggregation Human Activity Recognition Non-IID Federated Learning Federated Learning

Ask PDF

Abstract

Federated Learning

Explore similar work

FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift

Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning

FedAvg for HAR: Exploring the Tradeoff Between Personalized and Generalization Accuracy