Private and Robust Contribution Evaluation in Federated Learning

Delio Jaramillo Velez; Gergely Biczok; Alexandre Graell i Amat; Johan Ostman; Balazs Pejo

cs.CRFeb 25, 2026

Private and Robust Contribution Evaluation in Federated Learning

Authors: Delio Jaramillo Velez, Gergely Biczok, Alexandre Graell i Amat, Johan Ostman, Balazs Pejo

Organizations: University of La Laguna Tenerife, Spain · HUN-REN Hungarian Research Network Budapest, Hungary · Chalmers University Gotheburg, Sweden · AI Sweden Gotheborg, Sweden ·

Abstract

Cross-silo federated learning allows multiple organizations to collaboratively train machine learning models without sharing raw data, but client updates can still leak sensitive information through inference attacks. Secure aggregation protects privacy by hiding individual updates, yet it complicates contribution evaluation, which is critical for fair rewards and detecting low-quality or malicious participants. Existing marginal-contribution methods, such as the Shapley value, are incompatible with secure aggregation, and practical alternatives, such as Leave-One-Out, are crude and rely on self-evaluation. We introduce two marginal-difference contribution scores compatible with secure aggregation. Fair-Private satisfies standard fairness axioms, while Everybody-Else eliminates self-evaluation and provides resistance to manipulation, addressing a largely overlooked vulnerability. We provide theoretical guarantees for fairness, privacy, robustness, and computational efficiency, and evaluate our methods on multiple medical image datasets and CIFAR10 in cross-silo settings. Our scores consistently outperform existing baselines, better approximate Shapley-induced client rankings, and improve downstream model performance as well as misbehavior detection. These results demonstrate that fairness, privacy, robustness, and practical utility can be achieved jointly in federated contribution evaluation, offering a principled solution for real-world cross-silo deployments.

Figures & tables

Symbol	Meaning
$N$	Number of clients
$M_{0}$	Initial model before the training round
$M$	Aggregated model after the training round
$U_{i}$	Client $i$ local update within the round
$v(\cdot)$	Evaluation function (loss, accuracy, etc.)

Table 1. Notation

Entity	Assumptions	Capabilities	Goals
Server	Honest-but-curious; follows protocol; not malicious	Receives masked local updates; aggregates local update; aggregates local scores	Coordinate training; learn aggregated model
Client (training)	Honest-but-curious; follows training protocol; uses true local data	Holds local dataset; sends masked local update to server	Achieve best possible global model
Client (post training)	Selfish-but-non-malicious; does not disrupt training	Knows local and aggregated update; estimates marginal differences	Maximize own contribution score or minimize others’

Table 2. Threat model summary

Cont.	$M_{0}$	$+U_{i}$	$+U_{j}$	$-U_{j}$	$-U_{i}$	$M$	Property
Eval.	$M_{0}$	$+U_{i}$	$+U_{j}$	$-U_{j}$	$-U_{i}$	$M$	1	2	3	4	5
$\mathtt{LOO}(i)$					$\medbullet$	$\medbullet$		$\checkmark$	$\checkmark$	$\checkmark$
$\mathtt{FP}(i)$	$\medbullet$	$\medbullet$			$\medbullet$	$\medbullet$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
$\mathtt{EE}(i)$	$\medbullet$		$\medbullet$	$\medbullet$		$\medbullet$	$\checkmark$		$\checkmark$	$\checkmark$	$\checkmark$

Table 3. The building blocks for the scores and the properties they satisfy, e.g.,

\mathtt{LOO}(i)=v(M)-v(M\setminus U_{i})

, which depends only on

M

and

U_{i}

Panel of Figure 1 — Figure 1. Various CE scores for a scenario with 4 clients ( $\mu=0.5$ ) at the 10th communication round using the ISIC2019 dataset.

		$\mathtt{LOO}$	$\mathtt{EE}$	$\mathtt{FP}$	$\mathtt{COS}$
ISIC	$L_{2}$	$1.057\pm.432$	$0.339\pm.159$	$0.112\pm.168$	$2.028\pm.807$
	$\phi$	$0.821\pm.042$	$0.942\pm.054$	$0.933\pm.045$	$0.002\pm.667$
	$\rho$	$0.833\pm.051$	$0.964\pm.019$	$0.961\pm.034$	$0.349\pm.398$
	$\kappa$	$0.783\pm.082$	$0.901\pm.024$	$0.912\pm.022$	$0.098\pm.235$
Brain	$L_{2}$	$0.612\pm.261$	$0.163\pm.074$	$0.032\pm.021$	$0.964\pm.363$
	$\phi$	$0.793\pm.111$	$0.891\pm.063$	$0.891\pm.063$	$0.083\pm.065$

Table 4. The various metrics of the privacy-preserving marginal difference based contribution scores, along with

\mathtt{COS}

for a setting with 9 clients (

\mu=0.5

) at the 10th communication round.

Panel of Table 5 — Table 5. The $L_{2}$ error and coefficient $\phi$ of our privacy-preserving marginal difference based scores compared to the true Shapley value for a setting with $9$ clients ( $\mu=0.5$ ) at the 10th communication round.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Feb 25, 2026

Beyond performance-wise Contribution Evaluation in Federated Learning

Federated learning offers a privacy-friendly collaborative learning framework, yet its success, like any joint venture, hinges on the contributions of its participants. Existing client evaluation methods predominantly focus on model performance, such as accuracy or loss, which represents only one dimension of a machine learning model's overall utility. In contrast, this work investigates the critical, yet overlooked, issue of client contributions towards a model's trustworthiness -- specifically, its reliability (tolerance to noisy data), resilience (resistance to adversarial examples), and fairness (measured via demographic parity). To quantify these multifaceted contributions, we employ the state-of-the-art approximation of the Shapley value, a principled method for value attribution. Our results reveal that no single client excels across all dimensions, which are largely independent from each other, highlighting a critical flaw in current evaluation scheme: no single metric is adequate for comprehensive evaluation and equitable rewarding allocation.

Balazs Pejo

CrySyS Lab, Hungary

Federated Learning

Ask PDF

Mar 11, 2026

Repurposing Backdoors for Good: Ephemeral Intrinsic Proofs for Verifiable Aggregation in Cross-silo Federated Learning

While Secure Aggregation (SA) protects update confidentiality in Cross-silo Federated Learning, it fails to guarantee aggregation integrity, allowing malicious servers to silently omit or tamper with updates. Existing verifiable aggregation schemes rely on heavyweight cryptography (e.g., ZKPs, HE), incurring computational costs that scale poorly with model size. In this paper, we propose a lightweight architecture that shifts from extrinsic cryptographic proofs to Intrinsic Proofs. We repurpose backdoor injection to embed verification signals directly into model parameters. By harnessing Catastrophic Forgetting, these signals are robust for immediate verification yet ephemeral, naturally decaying to preserve final model utility. We design a randomized, single-verifier auditing framework compatible with SA, ensuring client anonymity and preventing signal collision without trusted third parties. Experiments on SVHN, CIFAR-10, and CIFAR-100 demonstrate high detection probabilities against malicious servers. Notably, our approach achieves over

1000\times

speedup on ResNet-18 compared to cryptographic baselines, effectively scaling to large models.

Xian Qin, Xue Yang, Xiaohu Tang

Southwest Jiaotong University

Federated Learning Aggregation

Ask PDF

Sep 2, 2026

Differentially private federated learning with Byzantine-robust aggregation: A cross-domain framework for secure model training in banking and healthcare systems

Federated learning allows banks, hospitals, and other regulated organizations to train a shared model without moving raw records off their own servers, which is attractive wherever data protection law or competitive sensitivity rules out pooling data centrally. Two problems limit how far this promise can be trusted in practice. First, the parameter updates that clients exchange still leak information about local records through gradient inversion and membership inference attacks. Second, an honest averaging rule such as FedAvg has no defense against a subset of clients that submit corrupted or adversarial updates, so a small number of malicious or compromised participants can quietly steer the shared model off course. This paper presents a federated learning framework, DP-BR-FedAvg, that combines a Gaussian-mechanism differential privacy layer with a coordinate-wise trimmed-mean Byzantine-robust aggregation rule, evaluated on a simulated cross-institutional classification task resembling fraud and clinical-risk scoring. Across sixty communication rounds with twenty clients, a quarter of them Byzantine, plain FedAvg collapses on the minority class (F1-score 0.030) while the proposed framework recovers substantially more of the signal (F1-score 0.119) while bounding the privacy loss of any single client's contribution. A Byzantine-robust aggregator with no privacy layer performs best in raw accuracy, quantifying the cost privacy imposes on robustness. The results show that privacy and robustness mechanisms interact rather than simply add, and that system design for regulated, adversarial, cross-institutional settings needs to budget for that interaction.

Srikumar Nayak

Advanced Artificial Intelligence (AI) Engineering and Cybersecurity Threat Research, LTIMindtree Research, New York City, NY, USA

Federated Learning Aggregation Byzantine-Robust Federated Learning Differentially Private Federated Learning Federated Learning

Ask PDF

Abstract

Federated Learning

Explore similar work

Beyond performance-wise Contribution Evaluation in Federated Learning

Repurposing Backdoors for Good: Ephemeral Intrinsic Proofs for Verifiable Aggregation in Cross-silo Federated Learning

Differentially private federated learning with Byzantine-robust aggregation: A cross-domain framework for secure model training in banking and healthcare systems