Trustworthy Distributed Machine Learning in Cloud-Edge-IoT Systems: A Heterogeneity-Aware Survey of Security, Privacy, and Robustness

Long Dang; Ali Alfatemi; Kaiqi Xiong; Mohammed Aledhari; Abdellah Chehri; Shawqi Al-Maliki; Mohamed Rahouti

cs.CROct 28, 2026

Trustworthy Distributed Machine Learning in Cloud-Edge-IoT Systems: A Heterogeneity-Aware Survey of Security, Privacy, and Robustness

Authors: Long Dang, Ali Alfatemi, Kaiqi Xiong, Mohammed Aledhari, Abdellah Chehri, Shawqi Al-Maliki, Mohamed Rahouti

Organizations: Cyber Florida, University of South Florida, Tampa, FL, USA · Department of Computer and Information Science, Fordham University, New York, NY, USA · Data Science Department, University of North Texas, Denton, 76207, USA · Department of Mathematics and Computer Science at the Royal Military College of Canada (RMC) in Kingston, Ontario, Canada ·

Abstract

Distributed machine learning (DML), including conventional distributed training and federated learning (FL), enables collaborative model development across cloud-edge-IoT systems without fully centralizing raw data. However, distributing data, computation, updates, and orchestration increases the attack surface and introduces coupled challenges across privacy, robustness, reliability, fairness, and governance. This survey presents a security-centered, heterogeneity-aware roadmap for trustworthy DML, with FL treated as a prominent high-risk case. We first characterize the assets, adversaries, trust assumptions, and coordination patterns that define DML/FL security boundaries. We then analyze how statistical, system, and trust heterogeneity, including non-IID data, resource asymmetry, intermittent participation, and cross-silo/cross-device trust differences, affect convergence, fairness, attack detectability, and defense reliability. Next, we synthesize attack-defense interactions across poisoning, backdoors, Byzantine manipulation, privacy leakage, model extraction, network/orchestration attacks, Sybil behavior, and strategic participation. We further compare layered defenses, including robust aggregation, drift-aware optimization, differential privacy, secure aggregation, homomorphic encryption, secure multiparty computation, trusted execution environments, watermarking, provenance, authenticated communication, and monitoring. Finally, we discuss deployment, evaluation, governance, and open research guidelines, emphasizing reproducible, non-independent and identically distributed (non-IID) partitions, attack-aware benchmarks, privacy budgets, attack success rate, client-level disparity, latency, communication, memory, energy/carbon overhead, and auditable lifecycle artifacts.

Figures & tables

Survey	Year	DML / FL	Threat / Trust Model	Heterogeneity	Attack-Defense	Privacy / Protected Comp.	Deploy. / Eval. / Gov.	Main Emphasis
Verbraaken et al. [ 1 ]	2020	✓	✗	$\triangle$	✗	✗	$\triangle$	Broad distributed deep learning systems survey focused mainly on scalability, orchestration, and infrastructure rather than security.
Li et al. [ 3 ]	2020	✓	$\triangle$	✓	$\triangle$	$\triangle$	$\triangle$	Foundational FL survey covering architectures, statistical heterogeneity, communication efficiency, optimization, and privacy-related challenges.
Yang et al. [ 6 ]	2020	✓	$\triangle$	✓	$\triangle$	$\triangle$	$\triangle$	Early FL survey addressing concepts, communication settings, application domains, and adversarial considerations.
Kairouz et al. [ 4 ]	2021	✓	$\triangle$	✓	$\triangle$	$\triangle$	$\triangle$	Landmark FL survey spanning optimization, personalization, privacy, fairness, robustness, and open problems, but not centered on a security-systems roadmap.
Rodríguez-Barroso et al. [ 7 ]	2023	✓	✓	$\triangle$	✓	$\triangle$	$\triangle$	Comprehensive FL threats survey with attack/defense taxonomies and empirical analysis across privacy and integrity threats.
Zhang et al. [ 8 ]	2024	✓	$\triangle$	$\triangle$	✓	$\triangle$	$\triangle$	Trustworthy FL survey covering privacy, security, fairness, explainability, and accountability under a broad trustworthiness lens.

TABLE I : Comparison of representative surveys related to secure distributed and federated machine learning. ✓ indicates substantial coverage,

\triangle

indicates partial or secondary coverage, and ✗ indicates limited or no meaningful coverage. The comparison reflects relative emphasis based on survey scope rather than an exhaustive claim about all topics mentioned in each paper

Panel of Figure 2 — Fig. 1 : Roadmap for heterogeneity-aware trustworthy DML in cloud-edge-IoT systems. Statistical non-IID heterogeneity can exacerbate poisoning, Byzantine manipulation, privacy leakage, model extraction, network tampering, and Sybil/free-rider behavior. These risks motivate a layered defense portfolio spanning heterogeneity-aware optimization, robust and secure aggregation, differential privacy, protected computation, intellectual property (IP) protection, and client vetting. Evaluation should report utility, attack success rate, client-level fairness, privacy budgets, communication/energy overhead, standardized non-IID partitions, public attack suites, and governance artifacts.

Panel of Figure 3 — Fig. 2 : Effect of heterogeneity on Byzantine-robust aggregation. Robust rules such as median, trimmed mean, Krum, and geometric median are more reliable when benign updates are concentrated; under non-IID data, honest dispersion can reduce attack detectability. Preprocessing methods such as nearest-neighbor mixing and bucketing can partially mitigate this effect. Data points are from [ 6 ] , implemented using [ 46 ] .

Attack class	Threat assumptions and adversary capability	Goal and affected stage	Primary defenses	Defense limitations and tensions
Data/model poisoning and backdoors	Malicious clients or data owners can corrupt local data, labels, loss functions, or submitted updates; attackers may be single-client, colluding, or adaptive.	Availability degradation or targeted integrity failure during local training, update submission, and aggregation.	Robust aggregation, clipping, anomaly scoring, backdoor validation, client vetting, provenance.	Robust rules often assume concentrated benign updates; severe non-IID data can make honest minority updates appear anomalous. Secure aggregation can hide update-level signals needed for anomaly detection. DP noise and clipping may mask weak malicious signals or reduce the utility of rare classes.
Byzantine manipulation	A bounded or unbounded fraction of clients may submit arbitrary, coordinated, or optimization-aware updates independent of any local dataset.	Aggregation-time integrity and convergence disruption.	Median/trimmed mean, Krum, geometric median/RFA, sign-based aggregation, norm bounding.	Guarantees weaken under collusion, high dimensionality, adaptive attacks, client drift, and heterogeneous benign updates. Cryptographic confidentiality does not ensure the semantic correctness of encrypted updates.
Gradient leakage, membership inference, and property inference An honest-but-curious server, compromised aggregator, eavesdropper, or inference API user observes gradients, updates, model snapshots, or outputs.	Confidentiality breach during training, aggregation, or inference.	DP-SGD/PATE, secure aggregation, HE/MPC, TEEs, output control, privacy accounting.	Secure aggregation hides individual updates but not necessarily leakage from the final model. DP provides formal privacy but may reduce accuracy, fairness, and robustness, especially for small or non-IID clients. HE/MPC/TEEs add latency, memory, communication, energy, or hardware-trust costs.
Model extraction and IP theft	Querying adversary, malicious tenant, or insider can repeatedly access prediction APIs or deployed artifacts.	Model/IP theft during inference and deployment.	Rate limiting, confidence masking, query monitoring, watermarking, fingerprinting, and provenance.	Output restriction can reduce usability and calibration. Watermarking/fingerprinting supports ownership verification but does not prevent poisoning or privacy leakage and may be weakened by fine-tuning, pruning, or model distillation.
Network and orchestration attacks	On-path adversaries, compromised gateways, or orchestration faults can replay, delay, downgrade, substitute, or tamper with updates and control messages.	Confidentiality, integrity, and availability failures during communication, scheduling, and synchronization.	Authenticated transport, update signing, replay protection, attestation, secure aggregation, monitoring.	Channel security cannot stop malicious authenticated clients. Attestation and signing increase key management complexity and may not address denial-of-service attacks, stragglers, or stale-update attacks.
Sybil, free-rider, and game-theoretic attacks	Open or weakly vetted clients create multiple identities, submit low-effort updates, or strategically adapt to rewards, sampling, and detection rules.	Bias aggregation, manipulate incentives, or obtain benefits without useful contribution during admission, sampling, and training.	Identity management, capped sampling, reputation, contribution auditing, incentive design, anomaly monitoring.	Privacy-preserving aggregation limits direct auditing of contributions. Reputation systems may be gamed, and strong identity controls can be costly or impractical in cross-device IoT settings.

TABLE II : Analytical comparison of major DML/FL attack classes and defense-composition limits in cloud-edge-IoT systems

Phase	Mechanism	Reference
Local training	Drift-aware optimization	[ 3 , 38 , 70 , 71 , 35 ] , [ 72 , 73 , 74 ]
	Differential privacy	[ 75 , 76 ]
	Cryptographic protection	[ 77 , 78 , 79 ] , [ 80 , 81 ] , [ 82 ]
Before aggregation	Secure aggregation	[ 61 ]
During the aggregation	Robust aggregation	[ 53 , 54 , 40 , 55 ]
After aggregation	Watermarking	[ 83 , 84 ]

TABLE III : Defense mechanisms in IoT-oriented DML

Attack	Test accuracy reported in [ 41 ] : IID and non-IID ( $\alpha=0.1$ ), 25% malicious clients	Test error reported in [ 88 ] : non-IID ( $\alpha=0.5$ ), varying attacker counts
Inner Product Manipulation (IPM)	Under IPM with $\epsilon=0.5$ , CC achieves 77.77% test accuracy with IID data and 56.97% with non-IID data. Under the stronger IPM attack with $\epsilon=100$ , its test accuracy falls to 10.00% in both settings.	CC maintains a test error of approximately 60% as the number of IPM attackers increases from 3 to 18.
Sign Flipping (SF)	Under SF, CC achieves only 10.00% test accuracy with both IID and non-IID data.	CC reaches a test error of approximately 90% when the number of SF attackers exceeds 9.
Label Flipping (LF)	Under LF, CC achieves 69.80% test accuracy with IID data and 52.41% with non-IID data.	CC reaches a test error of approximately 90% when the number of LF attackers exceeds 12.
A Little Is Enough (ALIE)	Under ALIE, CC achieves 78.53% test accuracy with IID data and 59.29% with non-IID data.	CC reaches a test error of approximately 90% when the number of ALIE attackers exceeds 12.

TABLE IV : Centered Clipping (CC) results reported by two independent studies on CIFAR-10 with FedAvg. The left column summarizes test accuracy from [ 41 ] using a fixed malicious-client fraction of 25% under IID and non-IID data (Dirichlet

\alpha=0.1

for the non-IID partition). The right column summarizes test error from [ 88 ] for non-IID data (Dirichlet,

\alpha=0.5

) as the number of attackers varies. Because the studies use different data partitions, attacker configurations, and evaluation metrics, their numerical results are not directly comparable

IoT domain / setting	Use cases	DML pattern / topology	IoT-specific constraints and threats	Recommended defenses / practices	Metrics to report
Industrial sensing / IIoT	Predictive maintenance; quality inspection; adaptive scheduling	Cross-silo FL; plant gateway-edge/cloud hybrid; periodic or event-driven synchronization	Plant/site skew, heterogeneous sensors, legacy devices, intermittent links, constrained gateways, physical tampering, gateway compromise, data locality, and operational safety	Authenticated transport, device identity management, gateway-level anomaly screening, non-IID-aware aggregation, robust aggregation, provenance, drift monitoring, and fail-safe rollback policies [ 93 , 94 , 95 , 96 , 97 ]	OEE/throughput, defect rate, downtime, false alarms, ASR, communication, latency, memory, energy, and fairness across lines/sites
Vehicular IoT	Cooperative perception; fleet learning; controller/model updates	Cross-device FL; hierarchical vehicle-RSU-edge/cloud learning	High mobility, spatiotemporal skew, intermittent V2X links, stale updates, strict tail latency, physical/environmental interference, and safety-critical poisoning/backdoors	Hierarchical aggregation, stale-update control, signed updates, distribution-aware robust aggregation, poisoning/backdoor stress tests, edge placement for latency-sensitive tasks, and local fallback policies [ 98 , 99 , 3 , 6 ]	mAP or task utility, tail/edge latency, stale-update rate, link dropout, ASR, backdoor persistence, client-drift metrics, and communication/energy cost
Smart healthcare and wearables	Wearable monitoring; collaborative diagnostics; outcome prediction; imaging	Cross-device wearable FL; cross-silo hospital FL; edge/cloud orchestration	PHI sensitivity, patient/site-level non-IID data, cohort shift, label imbalance, battery limits, small memory, intermittent participation, sensor drift, missing measurements, and device loss or compromise	Lightweight local training, dropout-tolerant secure aggregation, DP - SGD/PATE with explicit privacy accounting, selected HE/MPC or TEE support for sensitive operations, gateway buffering, attestation for managed devices, and fairness auditing across patients/sites [ 100 , 101 , 3 , 99 ]	AUROC/AUPRC, DP $\epsilon/\delta$ , per-site or per-patient disparity, dropout rate, ASR, latency, memory, communication, and battery/energy overhead
Smart cities / CPS	Traffic control; energy response; environmental monitoring; anomaly detection	Cross-domain FL across agencies/utilities; edge federation across cameras, meters, roadside units, and gateways	Device heterogeneity, district-level domain shift, heterogeneous ownership, PII exposure, public-service reliability, physical-layer disruption, and unreliable clients/gateways	Cross-domain governance, secure aggregation, DP when PII is present, personalization/FedBN or clustered FL for spatial skew, authenticated communication, audit logs, and resilience testing under device, gateway, and link failures [ 102 , 103 , 104 , 105 , 99 ]	Service reliability, availability, travel-time or response utility, privacy budget, ASR, fairness by district/site, communication overhead, latency, and energy
IoT cybersecurity	DDoS/IDS; malware detection; log and telemetry analytics	Cross-tenant or cross-organization FL; streaming edge telemetry; gateway/cloud aggregation	Highly non-IID traffic, adaptive adversaries, poisoned telemetry, compromised clients/gateways, tenant isolation, model/IP exposure, and evolving attacks	Byzantine-robust aggregation, clipping/norm bounds, secure aggregation where compatible, attack-aware benchmarks, provenance, authenticated telemetry, query/interface monitoring, and model watermarking/fingerprinting when IP risk is high [ 6 , 7 , 106 , 107 ]	TPR@FPR, false positives, ASR/backdoor persistence, detection latency, communication, memory, energy, and model-extraction risk
Cloud-edge IoT services	Managed IoT training/inference; federated MLOps; edge analytics	Device/gateway streams with cloud-hosted training, inference, monitoring, and rollback	Edge-cloud placement, multi-tenancy, access control, model extraction, orchestration faults, data residency, cost, and energy/carbon constraints	Access control, authenticated APIs, rate limiting, output control, provenance, deployment gates, model rollback, explicit placement of filtering/aggregation/privacy accounting, and energy/carbon-aware scheduling [ 108 , 106 , 109 , 110 ]	Cost/hour, time-to-accuracy, SLO adherence, tail latency, extraction risk, memory, energy, kgCO 2 e, and rollback frequency

TABLE V : IoT-centered DML deployment settings, constraints, design guidelines, and reporting metrics. Abbrev.: DP = differential privacy; DP-SGD = differentially private stochastic gradient descent; HE = homomorphic encryption; MPC = secure multiparty computation; TEE = trusted execution environment; RSU = roadside unit; ASR = attack success rate; SLO = service-level objective; OEE = overall equipment effectiveness; PHI = protected health information; AUROC = area under the receiver operating characteristic curve; AUPRC = area under the precision-recall curve; DDoS = distributed denial of service; IDS = intrusion detection system; TPR = true-positive rate; FPR = false-positive rate

Ref.	Non-IID data partitioning	Supported attacks and defenses	Reported metrics
[ 42 ]	Federated image datasets (FEMNIST and CelebA), text datasets (Sentiment140, Shakespeare, and Reddit), and a synthetic tabular dataset.	No support.	Statistical metrics: device-level test accuracy, training loss, and number of samples per user. System metrics: total computation across devices, measured in FLOPs, and client-server network traffic, measured in bytes written.
[ 135 ]	Label-distribution skew based on fixed-sample and Dirichlet partitioning; feature-distribution skew based on Gaussian and cube-based sampling; quantity skew based on a Dirichlet distribution; and FEMNIST.	No support.	Test accuracy, computation time (seconds), and communication volume (MB).
[ 121 ]	Twenty real-world FL datasets spanning different scales and task categories, together with a client system-behavior dataset.	Limited support.	Accuracy or loss, rounds to target accuracy, time to target accuracy, power (W), energy (J), latency (seconds), wall-clock time, communication cost, and device-availability duration.
[ 46 ]	Dirichlet- and gamma-similarity-based distributions.	Attacks: six Byzantine attacks. Defenses: four pre-aggregation methods and ten Byzantine-robust aggregation methods.	Test accuracy, training loss, and worst-case accuracy.
[ 134 ] , [ 138 ]	Dirichlet partitioning; LEAF datasets (FEMNIST and Shakespeare); and PubMedQA.	Attacks: four model-poisoning, two data-poisoning, and three data-reconstruction methods. Defenses: twelve pre-aggregation, three aggregation-stage, and two post-aggregation methods.	Test accuracy and test loss.
[ 133 ]	Dirichlet partitioning and FEMNIST.	Attacks: eight model-poisoning, two data-poisoning, and five hybrid-poisoning methods. Defenses: six methods against data poisoning and fifteen methods against model poisoning.	Test accuracy, attack success rate (ASR), targeted attack impact, and targeted defense robustness.

TABLE VI : Comparison of open-source federated learning benchmark tools

Privacy-Preserving Federated Learning

See all 45 papers

May 10, 2026

Privacy-Preserving Distributed Learning in IoT Systems: A Unified Threat Model and Evaluation Framework

The increasing deployment of Internet-of-Things (IoT) devices has accelerated the use of distributed learning frameworks, where data remains local while model updates are shared across decentralized systems. Although this reduces centralized data collection, it introduces privacy risks through the exchange of gradients, model parameters, and intermediate representations. A variety of privacy-preserving techniques have been proposed to address these risks, including differential privacy, cryptographic methods, and lightweight system-level approaches. However, existing surveys often evaluate these methods in isolation and lack a unified framework for comparing their effectiveness under realistic attack models and IoT resource constraints. This paper presents a structured analysis of privacy-preserving techniques for distributed learning in IoT environments. A unified threat model is introduced that captures model inversion, membership inference, gradient leakage, and communication-based attacks. Building on this model, an evaluation framework is developed to compare methods in terms of both privacy robustness and system-level efficiency, including computational, memory, and communication overhead. Using this framework, representative approaches including differential privacy, homomorphic encryption, secure multi-party computation, distributed selective stochastic gradient descent, and Bloom Filter-based methods are analyzed. The results highlight a fundamental trade-off between privacy strength and system efficiency. In particular, Bloom Filter-based encodings are shown to provide lightweight privacy through collision-induced ambiguity while maintaining low computational and communication overhead. The paper provides a unified perspective on privacy-preserving design choices for distributed learning in IoT systems.

John Cartmell, Alexander Williams

Department of Electrical Engineering and Computer Science

Internet of Things Privacy-Preserving ML Differential Privacy Distributed Training

Ask PDF

Mar 13, 2026

Privacy-Preserving Machine Learning for IoT: A Cross-Paradigm Survey and Future Roadmap

The rapid proliferation of the Internet of Things has intensified demand for robust privacy-preserving machine learning mechanisms to safeguard sensitive data generated by large-scale, heterogeneous, and resource-constrained devices. Unlike centralized environments, IoT ecosystems are inherently decentralized, bandwidth-limited, and latency-sensitive, exposing privacy risks across sensing, communication, and distributed training pipelines. These characteristics render conventional anonymization and centralized protection strategies insufficient for practical deployments. This survey presents a comprehensive IoT-centric, cross-paradigm analysis of privacy-preserving machine learning. We introduce a structured taxonomy spanning perturbation-based mechanisms such as differential privacy, distributed paradigms such as federated learning, cryptographic approaches including homomorphic encryption and secure multiparty computation, and generative synthesis techniques based on generative adversarial networks. For each paradigm, we examine formal privacy guarantees, computational and communication complexity, scalability under heterogeneous device participation, and resilience against threats including membership inference, model inversion, gradient leakage, and adversarial manipulation. We further analyze deployment constraints in wireless IoT environments, highlighting trade-offs between privacy, communication overhead, model convergence, and system efficiency within next-generation mobile architectures. We also consolidate evaluation methodologies, summarize representative datasets and open-source frameworks, and identify open challenges including hybrid privacy integration, energy-aware learning, privacy-preserving large language models, and quantum-resilient machine learning.

Zakia Zaman, Praveen Gauravaram, Mahbub Hassan +2

School of CSE, University of NSW, AU · TCS Research & Innovation, Tata Consultancy Services (Australia & New Zealand), Australia

Privacy-Preserving ML Internet of Things Federated Learning Differential Privacy Secure Multi-Party Computation

Ask PDF

Apr 19, 2026

Decentralised Trust and Security Mechanisms for IoT Networks at the Edge: A Comprehensive Review

INTRODUCTION: The proliferation of the amalgamation of IoT and edge computing has increased the demand for decentralised trust and security mechanisms capable of operating across heterogeneous and resource-limited devices. Approaches such as federated learning, Zero Trust architectures, lightweight blockchain and distributed neural models offer alternatives to centralised control. OBJECTIVES: This review examines various state-of-the-art decentralised mechanisms and evaluates their effectiveness in terms of securing IoT networks at the edge. METHODS: Thirty recent studies were analysed to compare how decentralised architectures establish trust, support secure communication and enable intrusion and anomaly detection. Frameworks, such as DFGL-LZTA, SecFedDNN and COSIER were assessed. RESULTS: Decentralised designs enhance privacy, reduce single points of failure and improve adaptive threat response, though challenges remain in scalability, efficiency and interoperability. CONCLUSION: The study identifies key considerations and future research needs for building secure and resilient trust-aware IoT edge ecosystems.

Khandoker Ashik Uz Zaman, Mahdi H. Miraz, Mohammed N. M. Ali

Xiamen University Malaysia, Malaysia · Wrexham University, UK · University of South Wales, UK

Edge Computing Cybersecurity

Ask PDF

Abstract

Privacy-Preserving Federated Learning

Explore similar work

Privacy-Preserving Distributed Learning in IoT Systems: A Unified Threat Model and Evaluation Framework

Privacy-Preserving Machine Learning for IoT: A Cross-Paradigm Survey and Future Roadmap

Decentralised Trust and Security Mechanisms for IoT Networks at the Edge: A Comprehensive Review