Decentralized Learning

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 73

All topics

Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Data
Jul 2, 2026Xuanyu Chen, Nan Yang, Shuai Wang +1Self-Supervised Learning Non-IID Federated Learning
School of Electrical and Computer Engineering, The University of Sydney · 2Northwestern Polytechnical University
Privacy-Preserving and Verifiable Approximate Distributed Coded Computing
Jul 2, 2026Xavier Martínez-Luaña, Alba Gude-Santos, Manuel Fernández-Veiga +1Byzantine-Robust Federated Learning Decentralized Learning
Galician Research and Development Center in Advanced Telecommunications (GRADIANT) Estrada do Vilar, 56-58, Vigo, 36214, Spain · atlanTTic, Information & Computing Lab (ICLAB), Telecommunication Engineering School, Universidade de Vigo Vigo, 36310, Spain
Decentralised AI Training and Inference with BlockTrain
Jun 23, 2026Peter Toth, Dan OprisaEfficient Transformer Inference Decentralized Learning
Spheroid Labs∗
Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning
Jun 22, 2026Nuocheng Yang, Yechen He, Sihua Wang +3Federated Unlearning Communication-Efficient Distributed Training
Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore
Giskard : Byzantine Robust and Confidential Aggregation for Large-Scale Decentralized Learning
Jun 17, 2026Ousmane Touat, César Sabater, Mohamed Maouche +1Secure Multi-Party Computation Communication-Efficient Distributed Training
INSA Lyon, LIRIS, CNRS · Villeurbanne, France · INRIA, INSA Lyon
Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training
Jun 15, 2026Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi +5Communication-Efficient Distributed Training Long-Context Language Modeling
Pluralis Research
Unifying Local Communications and Local Updates for LLM Pretraining
Jun 9, 2026Pietro Cagnasso, Eugene Belilovsky, Edouard OyallonLanguage Model Pretraining Communication-Efficient Distributed Training
Concordia University · Mila · CNRS, Sorbonne University
Compositional Generative Modeling from Decentralized Data
Jun 8, 2026Mashrur M. Morshed, Vishnu Naresh BoddetiFlow Matching Decentralized Learning
Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA.
Performance Evaluation of Social Learning
Jun 8, 2026Felice Scala, Marco Carpentiero, Vincenzo Matta +1Bayesian Inference Decentralized Learning
DIEM, University of Salerno, I-84084 Fisciano (SA), Italy · CNIT, Italy · EPFL, CH-1015 Lausanne, Switzerland
Improved Convergence Analysis of Topology Dependence in Decentralized SGD
Jun 8, 2026Yuki Takezawa, Anastasia Koloskova, Sebastian U. StichOptimization Convergence Analysis Decentralized Learning
Toyota Motor Corporation · University of Zurich · CISPA Helmholtz Center for Information Security
DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Data
Jun 3, 2026Yunsheng Yuan, Xue Xiao, Lina Wang +1Decentralized Learning Privacy-Preserving ML
School of Computer Science and Technology, Shandong University, Qingdao 266237, China · Inspur Cloud Information Technology Co, Ltd.
DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data
Jun 2, 2026Yunsheng Yuan, Shaowei Li, Kai Wang +5Fine-Tuning LLM Fine-Tuning
School of Computer Science and Technology, Shandong University, Qingdao China · School of Mathematical Science, Peking University, China · IEIT SYSTEM, China +2
Operating Regimes of Decentralized Learning Under Mobility and Bandwidth Constraints
Jun 1, 2026Samuele Sabella, Chiara Boldrini, Lorenzo Valerio +2Communication-Efficient Distributed Training Decentralized Learning
Institute for Informatics and Telematics, National Research Council, Italy
Decentralized Parameter-Free Online Learning with Compressed Gossip
May 27, 2026Tomas Ortega, Hamid JafarkhaniOnline Convex Optimization Decentralized Learning
Center for Pervasive Communications & Computing and EECS Department, University of California, Irvine, Irvine, CA 92697 USA
HEAL: Resilient and Self-* Hub-based Learning
May 26, 2026Mohamed Amine Legheraba, Stefan Galkiewicz, Maria Gradinariu Potop-Butucaru +1Decentralized Learning Federated Learning
NPA · Sorbonne University, CNRS, LIP6, F-75005 Paris, France · NPA, IUF, LINCS +1
Paris 2.0: A Decentralized Diffusion Model for Video Generation
May 25, 2026Ali Rouzbayani, Bidhan Roy, Marcos Villagra +1Video Diffusion Models Text-to-Video Generation
Bagel Labs (bagel.com)
Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization
May 22, 2026Alexander Long, Chamin Hewa Koneputugodage, Thalaiyasingam Ajanthan +5LLM Inference Decentralized Learning
Pluralis Research
F-TIS: Harnessing Diverse Models in Collaborative GRPO
May 21, 2026Nikolay Blagoev, Oğuzhan Ersoy, Wendelin Boehmer +1RL for Language Models Communication-Efficient Distributed Training
Gensyn · University of Neuchatel · TU Delft
Automated Byzantine-Resilient Clustered Decentralized Federated Learning for Battery Intelligence in Connected EVs
May 20, 2026Mouhamed Amine Bouchiha, Abdelaziz Amara Korba, Yacine Ghamri-DoudaneByzantine-Robust Federated Learning Decentralized Learning
SAMOVAR, T´el´ecom SudParis, Paris, France · Department of Computer Science, German University of Technology in Oman (GUtech), Muscat, Oman · L3i, La Rochelle University, La Rochelle, France
Your Neighbors Know: Leveraging Local Neighborhoods for Backdoor Detection in Decentralized Learning
May 19, 2026Sayan Biswas, Antoine Boutet, Davide Frey +7Backdoor Detection Decentralized Learning
1EPFL · 2Inria, INSA Lyon, CITI · 3Univ. Rennes, Inria, CNRS, IRISA
Centralized vs Decentralized Federated Learning: A trade-off performance analysis
May 15, 2026Chaimaa Medjadji, Guilain Leduc, Sylvain Kubler +1Decentralized Learning Federated Learning
University of Luxembourg
Unveiling High-Probability Generalization in Decentralized SGD
May 11, 2026Jiahuan Wang, Ping Luo, Ziqing Wen +2Algorithmic Stability Decentralized Learning
College of Computer Science and Technology National University of Defense Technology Changsha, Hunan 410073, China
\mathsf{VISTA}: Decentralized Machine Learning in Adversary Dominated Environments
May 8, 2026Hanzaleh Akbari Nodehi, Parsa Moradi, Soheil Mohajer +1Mechanism Design Byzantine-Robust Federated Learning
University of Minnesota
Decentralized Conformal Novelty Detection via Quantized Model Exchange
May 7, 2026Kyle Loh, Yu XiangFDR Control Conformal Prediction
Department of Mathematics and Statistics · Florida Atlantic University · Boca Raton, Florida
Learned Neighbor Trust for Collaborative Deployment in Model-Agnostic Decentralized Learning
May 6, 2026Michael Lanier, Luise Ge, Sastry Kompella +1Ensemble Learning Decentralized Learning
Department of Computer Science Washington University in St. Louis and Nexcepta
DeRelayL: Sustainable Decentralized Relay Learning
Apr 30, 2026Haihan Duan, Tengfei Ma, Yuyang Qin +4Decentralized Learning
Artificial Intelligence Research Institute, Shenzhen MSU-BIT University, Shenzhen 518172, China · Guangdong-Hong Kong-Macao Joint Laboratory for Emotion Intelligence and Pervasive Computing, Shenzhen 518172, China · The Chinese University of Hong Kong, Shenzhen, Shenzhen 518172, China +3
Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
Apr 22, 2026Yaiza Bermudez, Samir M. Perlaza, Iñaki EsnaolaDecentralized Learning Statistical Learning Theory
Centre Inria d'Université Côte d'Azur, INRIA, Sophia Antipolis, France. · Laboratoire GAATI, Université de la Polynésie française, Fa'a'ā, French Polynesia. · ECE Dept. Princeton University, Princeton, 08544 NJ, USA. +1
Accelerating Optimization and Machine Learning through Decentralization
Apr 21, 2026Ziqin Chen, Zuang Wang, Yongqiang WangDistributed Optimization Decentralized Learning
Clemson University
Towards Energy Impact on AI-Powered 6G IoT Networks: Centralized vs. Decentralized
Apr 21, 2026Anjie Qiu, Donglin Wang, Sanket Partani +2Energy-Efficient ML Internet of Things
*Institute for Wireless Communication and Navigation (WiCoN), University of Kaiserslautern-Landau (RPTU) · +Intelligent Networks Research Group (IN), German Research Center for Artificial Intelligence (DFKI)
Continuous Limits of Coupled Flows in Representation Learning
Apr 18, 2026Zilin Li, Weiwei Xu, Xuchun Tong +2Disentangled Representation Learning Representation Learning
SIIS, Donghua University, Shanghai

Jul 2, 2026, cs.LGJ/K move · Enter open · S save

Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Data

Xuanyu Chen, Nan Yang, Shuai Wang, Dong Yuan

School of Electrical and Computer Engineering, The University of Sydney · 2Northwestern Polytechnical University

Self-Supervised Learning Non-IID Federated Learning Decentralized Learning Masked Image Modeling

Open paper AskPDF arXiv

Recent research has introduced distributed self-supervised learning (D-SSL) approaches to leverage vast amounts of unlabeled decentralized data. However, D-SSL faces the critical challenge of data heterogeneity, and there is limited theoretical understanding of how different D-SSL frameworks respond to this challenge. To fill this gap, we present a rigorous theoretical analysis of the robustness of D-SSL frameworks under non-IID (non-independent and identically distributed) settings. Our results show that pre-training with Masked Image Modeling (MIM) is inherently more robust to heterogeneous data than Contrastive Learning (CL), and that the robustness of decentralized SSL increases with average network connectivity, implying that federated learning (FL) is no less robust than decentralized learning (DecL). These findings provide a solid theoretical foundation for guiding the design of future D-SSL algorithms. To further illustrate the practical implications of our theory, we introduce MAR loss, a refinement of the MIM objective with local-to-global alignment regularization. Extensive experiments across model architectures and distributed settings validate our theoretical insights, and additionally confirm the effectiveness of MAR loss as an application of our analysis.

Momentum

Related topics

Latest papers 73