Gradient Sharing

Gradient sharing, a core component of federated learning and other distributed training paradigms, aims to collaboratively train machine learning models while preserving data privacy by exchanging model updates instead of raw data. Current research focuses on mitigating privacy risks associated with gradient leakage through techniques like differential privacy and obfuscation, as well as improving efficiency and fairness in gradient aggregation across diverse model architectures (e.g., convolutional neural networks, graph neural networks). These advancements are crucial for enabling secure and equitable deployment of machine learning in sensitive applications like healthcare and finance, while addressing inherent trade-offs between privacy, security, and model accuracy.

Papers

December 17, 2024

Building Gradient Bridges: Label Leakage from Restricted Gradient Sharing in Federated Learning
Rui Zhang, Ka-Ho Chow, Ping Li
Natural Gradient Gradient Information State of the Art Defense Sharing Matter Gradient Based Attack Gradient Sharing Label Leakage Synthetic Gradient

July 12, 2024

TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor Computing
Husheng Han, Xinyao Zheng, Yuanbo Wen, Yifan Hao, Erhu Feng, Ling Liang, Jianan Mu, Xiaqing Li, Tianyun Ma, Pengwei Jin, Xinkai Song, Zidong Du, Qi Guo, Xing Hu
Trusted Execution Environment Data Prefetching Gradient Sharing Tensor Model

July 3, 2024

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
Xianliang Xu, Ting Du, Wang Kong, Ye Li, Zhongyi Huang
Neural Network Physic Informed Neural Network Early Stage Convergence Two Layer Neural Network Gradient Sharing Smooth Activation Function

June 16, 2024

Linkage on Security, Privacy and Fairness in Federated Learning: New Balances and New Perspectives
Linlin Wang, Tianqing Zhu, Wanlei Zhou, Philip S. Yu
Procedural Fairness Privacy Policy Security Related Event Linking Fairness Issue Gradient Sharing

June 6, 2024

R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradients
Tamer Ahmed Eltaras, Qutaibah Malluhi, Alessandro Savino, Stefano Di Carlo, Adnan Qayyum, Junaid Qadir
Convolutional Layer Convolution Layer Analytical Approach Gradient Attack Data Reconstruction Gradient Sharing

January 10, 2024

AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
Shayan Mohajer Hamidi, En-Hui Yang
Loss Function Local Gradient FAIR Federated Learning Gradient Sharing

December 13, 2023

Accelerating Meta-Learning by Sharing Gradients
Oscar Chang, Hod Lipson
Multi Task Learning Meta Learning Gradient Based Meta Learning Gradient Sharing Learning Loop Encode Task

August 15, 2023

Decentralized Graph Neural Network for Privacy-Preserving Recommendation
Xiaolin Zheng, Zhongyu Wang, Chaochao Chen, Jiashu Qian, Yao Yang
Privacy Preserving Effective Recommendation Local Differential Privacy Gradient Sharing Privacy Framework Decentralized Graph Neural Network

May 15, 2023

Quadratic Functional Encryption for Secure Training in Vertical Federated Learning
Shuangyi Chen, Anuja Modi, Shweta Agrawal, Ashish Khisti
Vertical Federated Learning Data Leakage Secure Collaborative Learning Gradient Sharing Functional Encryption

February 21, 2023

Speech Privacy Leakage from Shared Gradients in Distributed Learning
Zhuohang Li, Jiaxin Zhang, Jian Liu
Speech Signal Speaker Identity Distributed Learning Gradient Sharing Speech Privacy

February 2, 2023

FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations
Hui-Po Wang, Dingfan Chen, Raouf Kerkouche, Mario Fritz
Differential Privacy Gradient Sharing Private Convex

October 25, 2022

SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication
Marco Bornstein, Tahseen Rabbani, Evan Wang, Amrit Singh Bedi, Furong Huang
Decentralized Federated Learning SWIFT DynGFN Asynchronous Decentralized Gradient Sharing Communication Free

January 21, 2022

TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data
Isha Garg, Manish Nagaraj, Kaushik Roy
Federated Averaging Data Leakage Weight Update Optimization Proxy Gradient Sharing Federated Data Sanitization Defense

December 6, 2021

When the Curious Abandon Honesty: Federated Learning Is Not Private
Franziska Boenisch, Adam Dziedzic, Roei Schuster, Ali Shahin Shamsabadi, Ilia Shumailov, Nicolas Papernot
Deep Convolutional Neural Network Data Leakage Curiosity Inducing Situation Data Reconstruction Attack Field Trap Gradient Sharing

Gradient Sharing

Papers

Building Gradient Bridges: Label Leakage from Restricted Gradient Sharing in Federated Learning

TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor Computing

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks

Linkage on Security, Privacy and Fairness in Federated Learning: New Balances and New Perspectives

R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradients

AdaFed: Fair Federated Learning via Adaptive Common Descent Direction

Accelerating Meta-Learning by Sharing Gradients

Decentralized Graph Neural Network for Privacy-Preserving Recommendation

Quadratic Functional Encryption for Secure Training in Vertical Federated Learning

Speech Privacy Leakage from Shared Gradients in Distributed Learning

FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations

SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication

TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data

When the Curious Abandon Honesty: Federated Learning Is Not Private