Aggregated Gradient

Aggregated gradient methods are central to distributed machine learning, aiming to efficiently combine gradient updates from multiple sources (e.g., devices in federated learning) to train a shared model. Current research focuses on improving the robustness and privacy of aggregation techniques, addressing challenges like stragglers, data heterogeneity, and adversarial attacks through adaptive weighting schemes, coded computation, and secure aggregation protocols. These advancements are crucial for scaling machine learning to larger datasets and diverse environments while mitigating privacy risks and improving model accuracy and efficiency in various applications.

Papers

November 12, 2024

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
Satish Kumar Keshri, Nazreen Shah, Ranjitha Prasad
Catastrophic Forgetting Early Stage Convergence End to End Learning Federated Continual Learning Aggregated Gradient

November 6, 2024

Adaptive Consensus Gradients Aggregation for Scaled Distributed Training
Yoni Choukroun, Shlomi Azoulay, Pavel Kisilev
Deep Subspace Subspace Learning Aggregated Gradient Scalable Training

October 21, 2024

Federated Communication-Efficient Multi-Objective Optimization
Baris Askin, Pranay Sharma, Gauri Joshi, Carlee Joe-Wong
Multi Objective Optimization Objective Function Aggregated Gradient

October 11, 2024

Gradients Stand-in for Defending Deep Leakage in Federated Learning
H. Yi, H. Ren, C. Hu, Y. Li, J. Deng, X. Xie
Sensitive Data Gradient Leakage Model Gradient Aggregated Gradient Deep Leakage

June 22, 2024

Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning
Zhibo Wang, Zhiwei Chang, Jiahui Hu, Xiaoyi Pang, Jiacheng Du, Yongle Chen, Kui Ren
Secure Aggregation Gradient Inversion Attack Aggregated Gradient Label Inference Attack Private Label Label Leakage

March 22, 2024

Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation
Chengxi Li, Ming Xiao, Mikael Skoglund
Differential Privacy Mutual Information Federated Learning Framework Straggler Mitigation Aggregated Gradient Adaptive Federated Learning

March 3, 2024

Quantized Hierarchical Federated Learning: A Robust Approach to Statistical Heterogeneity
Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor
Heterogeneous Data Model Aggregation Robust Framework Hierarchical Federated Learning Statistical Heterogeneity Aggregated Gradient

February 6, 2024

Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning
Idan Achituve, Idit Diamant, Arnon Netzer, Gal Chechik, Ethan Fetaya
Multi Task Learning Natural Gradient Bayesian Inference Inference Task Bayesian Uncertainty Aggregated Gradient Gradient Aggregation

October 4, 2023

zkFL: Zero-Knowledge Proof-based Gradient Aggregation for Federated Learning
Zhipeng Wang, Nanqing Dong, Jiahao Sun, William Knottenbelt, Yike Guo
Zero Knowledge Aggregated Gradient Malicious Content Decentralized Framework

October 2, 2023

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training
Daegun Yoon, Sangyoon Oh
Multiplicative Size Scaling Random Sparsification Deep Neural Network Training Gradient Sparsification Aggregated Gradient Micro Level

September 21, 2023

Enabling Quartile-based Estimated-Mean Gradient Aggregation As Baseline for Federated Image Classifications
Yusen Wu, Jamie Deng, Hao Chen, Phuong Nguyen, Yelena Yesha
Baseline Result Structural Outlier Federated Image Classification Aggregated Gradient Mean Aggregation Decentralized Deep Learning

September 10, 2023

Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Data
Xiaolu Wang, Cheng Jin, Hoi-To Wai, Yuantao Gu
Optimization Purpose Convex Optimization Linear Speedup Streaming Data Aggregated Gradient Deterministic Gradient

August 17, 2023

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient
Rongfei Fan, Xuming An, Shiyuan Zuo, Han Hu
Data Aggregation Air Federated Learning Air Computation Convex Loss Function Aggregated Gradient Private Convex Optimization Gradient Normalization

August 11, 2023

Enhancing Generalization of Universal Adversarial Perturbation through Gradient Aggregation
Xuannan Liu, Yaoyao Zhong, Yuhang Zhang, Lixiong Qin, Weihong Deng
Aggregated Gradient Adversarial Instance Enhancing Generalization Gradient Aggregation

June 7, 2023

Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removal
Anastasiia Sedova, Lena Zellinger, Benjamin Roth
LeArning Abstract Adaptive Importance Noisy Label Aggregated Gradient Label Error Outlier Removal Label Denoising

March 28, 2023

FedAgg: Adaptive Federated Learning with Aggregated Gradients
Wenhao Yuan, Xuehe Wang
Gradient Descent Learning Rate Decentralized Learning Aggregated Gradient Adaptive Federated Learning Training Epoch

February 9, 2023

Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation
Peiyao Xiao, Kaiyi Ji
Federated Learning Bilevel Optimization Aggregated Gradient Federated Bilevel Optimization Hypergradient Estimation

January 25, 2023

When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning
Wenkai Yang, Yankai Lin, Guangxiang Zhao, Peng Li, Jie Zhou, Xu Sun
Learning Rate Client Sampling Aggregated Gradient Server Learning Rate

January 17, 2023

Negative Flux Aggregation to Estimate Feature Attributions
Xin Li, Deng Pan, Chengyin Li, Yao Qiang, Dongxiao Zhu
Deep Neural Network Deep Learning Architecture Feature Attribution Attribution Map Aggregated Gradient Gradient Aggregation DNN Prediction

November 8, 2022

Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design
Yuchang Sun, Jiawei Shao, Yuyi Mao, Songze Li, Jun Zhang
Stochastic Way Theoretical Analysis Stackelberg Game Incentive Mechanism Design Aggregated Gradient

Aggregated Gradient

Papers

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients

Adaptive Consensus Gradients Aggregation for Scaled Distributed Training

Federated Communication-Efficient Multi-Objective Optimization

Gradients Stand-in for Defending Deep Leakage in Federated Learning

Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning

Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation

Quantized Hierarchical Federated Learning: A Robust Approach to Statistical Heterogeneity

Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning

zkFL: Zero-Knowledge Proof-based Gradient Aggregation for Federated Learning

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training

Enabling Quartile-based Estimated-Mean Gradient Aggregation As Baseline for Federated Image Classifications

Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Data

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient

Enhancing Generalization of Universal Adversarial Perturbation through Gradient Aggregation

Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removal

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation

When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning

Negative Flux Aggregation to Estimate Feature Attributions

Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design