Decentralized Learning

Ask about this topic

Overview

Decentralized learning studies how multiple devices or organizations can learn together while keeping data and computation local, which is useful when privacy, connectivity, or centralized infrastructure is limited. Methods include gossip averaging, decentralized stochastic gradient descent, knowledge distillation, random-walk updates, and communication-efficient model sharding, with applications from edge devices to large language model pretraining. Work evaluates these approaches on image, text, graph, and time-series benchmarks, as well as dynamic and wireless networks, and develops convergence and generalization guarantees. A central tension is balancing communication and computation costs with learning quality under non-IID data, changing network topologies, privacy requirements, and malicious participants.

Latest papers

CardsList

101 papers11 papers in the last four weeks, down 39% on the four weeks before. 0.1% of all new papers.

  1. Fast and Efficient Asynchronous Gossip Algorithm for Robust and Non-Smooth Convex Decentralized Learning

    Jan 28, 2026Anna van Elst, Olivier Fercoq, Igor Colin +1Decentralized LearningNonsmooth Convex Optimization

    LTCI, Télécom Paris, Institut Polytechnique de Paris

  2. Self-Creating Random Walks for Decentralized Learning under Pac-Man Attacks

    Jan 12, 2026Xingran Chen, Parimal Parag, Rohit Bhagat +1Adversarial AttacksDecentralized Learning

    Engineering Systems and Design Pillar, Singapore University of Technology and Design, 8 Somapah Road, Singapore 487372 · Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, Karnataka 560012, India · Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA

  3. On the Escaping Efficiency of Distributed Adversarial Training Algorithms

    Sep 14, 2025Ying Cao, Kun Yuan, Ali H. SayedAdversarial TrainingAdversarial Robustness

    Institute of Electrical and Micro Engineering in EPFL, Lausanne · Center for Machine Learning Research (CMLR) in Peking University

  4. Random Walk Learning and the Pac-Man Attack

    Aug 1, 2025Xingran Chen, Parimal Parag, Rohit Bhagat +2CybersecurityDecentralized Learning

    Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA · Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, Karnataka 560012, India

  5. NoLoCo: No-all-reduce Low Communication Training Method for Large Models

    Jun 12, 2025Jari Kolehmainen, Nikolay Blagoev, Semih Kara +3Communication-Efficient Distributed TrainingDecentralized Learning

    1Gensyn

  6. Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

    Jun 2, 2025Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham +2Communication-Efficient Distributed TrainingDecentralized Learning

    Pluralis Research

  7. Differential Privacy of Decentralized Learning: Centralized Rate Conditions and Nearly Tight Guarantees

    May 26, 2025Antti Koskela, Tejas KulkarniDifferential PrivacyDecentralized Learning

    Nokia Bell Labs

  8. Chisme: Heterogeneity-Aware Gossip Learning

    May 14, 2025Harikrishna Kuttivelil, Katia ObraczkaEdge ComputingDecentralized Learning

    Computer Science & Engineering University of California Santa Cruz, California, USA

  9. Trust-free Personalized Decentralized Learning

    Oct 15, 2024Yawen Li, Yan Li, Junping Du +3Decentralized LearningBlockchain-Based Federated Learning

    Beijing University of Posts and Telecommunications, Beijing, China

  10. Collaborative Knowledge Distillation via a Learning-by-Education Node Community

    Sep 30, 2024Anestis Kaimakamidis, Ioannis Mademlis, Ioannis PitasOnline Knowledge DistillationContinual Learning

    Aristotle University of Thessaloniki, Thessaloniki, 54124, Greece

  11. Byzantine-Robust Aggregation for Securing Decentralized Federated Learning

    Sep 26, 2024Diego Cajaraville-Aboy, Ana Fernández-Vilas, Rebeca P. Díaz-Redondo +1Federated Learning AggregationByzantine-Robust Federated Learning

    atlanTTic Research Center – ICLAB – Universidade de Vigo, Vigo, 36310, Spain