Local Update

Local update methods in distributed machine learning aim to improve efficiency and robustness by performing multiple model updates on individual devices before aggregating results. Current research focuses on optimizing update frequency and incorporating techniques like gradient compression, layer-wise updates, and cache management to address challenges such as stragglers, communication bottlenecks, and data heterogeneity in federated learning and other distributed settings. These advancements enhance the scalability and privacy of machine learning models, particularly in resource-constrained environments and applications requiring collaborative training across multiple parties. The impact is seen in improved training speed, reduced communication costs, and enhanced resilience to adversarial attacks.

Papers

October 8, 2024

Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates
Cabrel Teguemne Fokam, Khaleelulla Khan Nazeer, Lukas König, David Kappel, Anand Subramoney
Back Propagation Feature Decoupling Local Update Asynchronous SGD

July 6, 2024

A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
Jiajun Song, Jiajun Luo, Rongwei Lu, Shuzhao Xie, Bin Chen, Zhi Wang
Asynchronous Federated Learning Gradient Compression Local Update

April 8, 2024

LGSDF: Continual Global Learning of Signed Distance Fields Aided by Local Updating
Yufeng Yue, Yinan Deng, Jiahui Wang, Yi Yang
Signed Distance Field Fusion Feature Global Deep Local Update Implicit Reconstruction

March 27, 2024

Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates
Natalie Lang, Alejandro Cohen, Nir Shlezinger
Federated Learning Low Latency Convergence Guarantee Local Update Edge Cloud Collaborative Model Update Latency Training

March 23, 2024

Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD update
Fnu Hairi, Zifan Zhang, Jia Liu
Multi Agent Reinforcement Learning Multi Agent Novel Approach External Sample Communication Efficiency Communication Complexity Local Update

November 13, 2023

AGRAMPLIFIER: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification
Zirui Gong, Liyue Shen, Yanjun Zhang, Leo Yu Zhang, Jingwei Wang, Guangdong Bai, Yong Xiang
Poisoning Attack Byzantine Robust Byzantine Attack Byzantine Robust Aggregation Local Update

September 3, 2023

FedFwd: Federated Learning without Backpropagation
Seonghwan Park, Dahun Shin, Jinseok Chung, Namhoon Lee
Federated Learning Back Propagation Forward Forward Algorithm Local Update

April 9, 2023

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
Kfir Y. Levy
Stochastic Gradient Descent Joint Optimization Stochastic Convex Optimization Local SGD Local Update Efficient Query Minibatch Stochastic

March 28, 2023

On the Local Cache Update Rules in Streaming Federated Learning
Heqiang Wang, Jieming Bian, Jie Xu
Federated Learning Local Update Future Data

March 20, 2023

Recursive Euclidean Distance Based Robust Aggregation Technique For Federated Learning
Charuka Herath, Yogachandran Rahulamathavan, Xiaolan Liu
Local Model Robust Aggregation Euclidean Distance Local Update

January 3, 2023

Through-life Monitoring of Resource-constrained Systems and Fleets
Felipe Montana, Adam Hartwell, Will Jacobs, Visakan Kadirkamanathan, Andrew R Mills, Tom Clark
Digital Twin Time Varying Resource Constraint Condition Monitoring Local Update Fleet Data

July 29, 2022

Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates
Fangcheng Fu, Xupeng Miao, Jiawei Jiang, Huanran Xue, Bin Cui
Vertical Federated Learning Model Gradient Local Update Instance Weighting Linear Convergence Rate

May 27, 2022

FedAvg with Fine Tuning: Local Updates Lead to Representation Learning
Liam Collins, Hamed Hassani, Aryan Mokhtari, Sanjay Shakkottai
Fine Tuning Representation Learning Federated Averaging FedAvg Converges Multi Task Representation Model Averaging Local Update

March 16, 2022

MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients
Xiaoyu Cao, Neil Zhenqiang Gong
Model Poisoning Attack Fake User Local Update Attacker Model

March 10, 2022

Crowd Source Scene Change Detection and Local Map Update
Itzik Wilf, Nati Daniel, Lin Manqing, Firas Shama, Omri Asraf, Feng Wensen, Ofer Kruzel
Change Detection 3D Point Cloud Spatio tempOral Local Update Image Descriptor Scene Change Detection Point Cloud Descriptor

February 18, 2022

Differentially Private Federated Learning via Inexact ADMM with Multiple Local Updates
Minseok Ryu, Kibaek Kim
Differential Privacy Privacy Protection Private Federated Learning ADMM Algorithm Local Update

Local Update

Papers

Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates

A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning

LGSDF: Continual Global Learning of Signed Distance Fields Aided by Local Updating

Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates

Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD update

AGRAMPLIFIER: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification

FedFwd: Federated Learning without Backpropagation

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization

On the Local Cache Update Rules in Streaming Federated Learning

Recursive Euclidean Distance Based Robust Aggregation Technique For Federated Learning

Through-life Monitoring of Resource-constrained Systems and Fleets

Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates

FedAvg with Fine Tuning: Local Updates Lead to Representation Learning

MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients

Crowd Source Scene Change Detection and Local Map Update

Differentially Private Federated Learning via Inexact ADMM with Multiple Local Updates