Asynchronous Learning

Asynchronous learning focuses on training machine learning models where updates from different parts of the system (e.g., agents, devices, or data streams) occur independently and at varying times, unlike synchronous methods requiring global synchronization. Current research emphasizes developing robust algorithms and architectures, such as asynchronous federated learning, actor-critic methods for multi-agent systems, and event-driven approaches for real-time data processing, that address challenges like data staleness, communication overhead, and device heterogeneity. This research is significant because it enables efficient training of large-scale models, improves scalability in distributed settings, and facilitates real-time applications in areas like robotics and resource allocation, where synchronous approaches are impractical.

Papers

April 28, 2024

A Note on Asynchronous Challenges: Unveiling Formulaic Bias and Data Loss in the Hayashi-Yoshida Estimator
Evangelos Georgiadis
Short Note Algorithmic Bias Linear Estimator Intrinsic Bias Asynchronous Learning Publication Bias Poisson Process Novel Estimator Data Loss

March 16, 2024

Federated Learning based on Pruning and Recovery
Chengjie Ma
Edge Pruning Prompt Recovery Heterogeneous Client Asynchronous Learning Training Framework Asynchronous Algorithm

February 8, 2024

Asynchronous Diffusion Learning with Agent Subsampling and Local Updates
Elsa Rizk, Kun Yuan, Ali H. Sayed
Practical Algorithm Global Model Local Context Client Sampling Asynchronous Learning Local Datasets

February 2, 2024

ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data
Carmen Martin-Turrero, Maxence Bouvier, Manuel Breitenstein, Pietro Zanuttigh, Vincent Parret
Temporal Data PointNet Model Asynchronous Architecture Asynchronous Learning Event Based Sensor

January 9, 2024

SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning
Hector A. Gonzalez, Jiaxin Huang, Florian Kelber, Khaleelulla Khan Nazeer, Tim Langer, Chen Liu, Matthias Lohrmann, Amirhossein Rostami, Mark Schöne, Bernhard Vogginger, Timo C. Wunderlich, Yexin Yan, Mahmoud Akl, Christian Mayr
Neural Network Neuromorphic Computing Neuromorphic Chip Asynchronous Architecture Asynchronous Learning

November 8, 2023

Zeroth-order Asynchronous Learning with Bounded Delays with a Use-case in Resource Allocation in Communication Networks
Pourya Behmandpoor, Marc Moonen, Panagiotis Patrinos
Optimization Purpose Reward Function Resource Allocation Convergence Rate Use Case Computer Network Asynchronous Learning Learning Based Resource Allocation Consensus Based Approach

September 21, 2023

Bridging the Gap: Learning Pace Synchronization for Open-World Semi-Supervised Learning
Bo Ye, Kai Gan, Tong Wei, Min-Ling Zhang
Pseudo Label Self Supervised Pre Trained Model Contrastive Clustering Asynchronous Learning Open World Semi Supervised

August 30, 2023

AsyncET: Asynchronous Learning for Knowledge Graph Entity Typing with Auxiliary Relations
Yun-Cheng Wang, Xiou Ge, Bin Wang, C. -C. Jay Kuo
Asynchronous Learning Knowledge Graph Entity Entity Type Prediction Auxiliary Item Relationship

June 16, 2023

Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80
Vincent Berenz, Felix Widmaier, Simon Guist, Bernhard Schölkopf, Dieter Büchler
Robotics Domain Real Time Real Robot Simulation Environment Robotic Application Complex Robotic System Asynchronous Learning Robotics Software

November 16, 2022

Asynchronous Bayesian Learning over a Network
Kinjal Bhar, He Bai, Jemin George, Carl Busart
Network Programming Langevin Dynamic Bayesian Learning Gossip Algorithm Asynchronous Learning

October 28, 2022

Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout
Chen Dun, Mirian Hipolito, Chris Jermaine, Dimitrios Dimitriadis, Anastasios Kyrillidis
High Efficiency Structured Dropout Asynchronous Federated Learning Asynchronous Learning Asynchronous Approach

September 20, 2022

Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning
Yuchen Xiao, Weihao Tan, Christopher Amato
Multi Agent Reinforcement Learning Actor Critic Reinforcement Learning Multi Agent Actor Critic Asynchronous Learning Asynchronous Approach

September 1, 2022

Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis
Shigehiko Schamoni, Michael Hagmann, Stefan Riezler
Privacy Policy Ensemble Method Model Ensembling Patient Specific Early Diagnosis Asynchronous Learning Standard Sepsis Better Prediction

May 31, 2022

March 23, 2022

Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots
Yufeng Yuan, A. Rupam Mahmood
Reinforcement Learning Real Time Sequential Learning Physical Robot Real World Reinforcement Learning Asynchronous Learning

February 17, 2022

Delay-adaptive step-sizes for asynchronous learning
Xuyang Wu, Sindri Magnusson, Hamid Reza Feyzmahdavian, Mikael Johansson
Machine Learning Step Size Slow Convergence Asynchronous Learning Asynchronous Algorithm Asynchronous Iteration

January 10, 2022

Desynchronous Learning in a Physics-Driven Learning Network
Jacob F Wycoff, Sam Dillavou, Menachem Stern, Andrea J Liu, Douglas J Durian
Decentralized Learning Physic Informed Asynchronous Learning

Asynchronous Learning

Papers

A Note on Asynchronous Challenges: Unveiling Formulaic Bias and Data Loss in the Hayashi-Yoshida Estimator

Federated Learning based on Pruning and Recovery

Asynchronous Diffusion Learning with Agent Subsampling and Local Updates

ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data

SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning

Zeroth-order Asynchronous Learning with Bounded Delays with a Use-case in Resource Allocation in Communication Networks

Bridging the Gap: Learning Pace Synchronization for Open-World Semi-Supervised Learning

AsyncET: Asynchronous Learning for Knowledge Graph Entity Typing with Auxiliary Relations

Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80

Asynchronous Bayesian Learning over a Network

Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout

Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning

Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis

Decentralized Competing Bandits in Non-Stationary Matching Markets

Asynchronous Hierarchical Federated Learning

Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots

Delay-adaptive step-sizes for asynchronous learning

Desynchronous Learning in a Physics-Driven Learning Network