Communication-Efficient Distributed Training

Ask about this topic

Momentum

20 papers in the last four weeks, up 300% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 151

All topics
CardsList

  1. Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token Shuffling

    Oct 7, 2026Radha Gulhane, Quentin Anthony, Beren MillidgeMixture-of-Experts Language ModelsCommunication-Efficient Distributed Training

    Zyphra

  2. Tram-FL: Reducing Communication and Computation Costs through Sequential Model Circulation in Decentralized Federated Learning

    Oct 6, 2026Kota Maejima, Takayuki Nishio, Asato Yamazaki +1Communication-Efficient Distributed TrainingFederated Learning

    School of Engineering, Institute of Science Tokyo, Tokyo 152-8550, Japan

  3. Distributed Subliminal Learning: Replacing Model Updates with Random-Carrier Outputs

    Oct 4, 2026Dario Fenoglio, Gabriele Dominici, Martin Gjoreski +1Communication-Efficient Distributed TrainingFederated Knowledge Distillation

    Università della Svizzera italiana (USI), Lugano, Switzerland · MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

  4. FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection

    Oct 1, 2026Junkang LiuFederated Learning AggregationMemory-Efficient Fine-Tuning

    Tianjin University

  5. Beyond Uniform Compression: Budgeted Transmission Allocation for Extreme Federated Learning

    Sep 30, 2026Pengfei Li, Mohammad KhalilCommunication-Efficient Distributed TrainingLow-Rank Compression

    Centre for the Science of Learning & Technology (SLATE), University of Bergen, Bergen, Norway

  6. Importance-Aware Feature Sparsification for Wireless Split Learning

    Sep 30, 2026Bumjun Kim, Yoon Huh, Wan ChoiCommunication-Efficient Distributed TrainingSplit Federated Learning

    Department of Electrical and Computer Engineering, Seoul National University (SNU), and the Institute of New Media and Communications, SNU, Seoul 08826, Korea

  7. AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

    Sep 29, 2026Pengyu He, Yan Zhang, Ruien Li +1Communication-Efficient Distributed TrainingDistributed Training

    Department of Electrical Engineering, California Institute of Technology · Department of Computer Science and Technology, Tsinghua University · Department of Computer Sciences, University of Wisconsin - Madison

  8. LionMuon: Alternating Spectral and Sign Descent for Efficient Training

    Sep 28, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Stochastic OptimizationLanguage Model Pretraining

    Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · Applied Artificial Intelligence Institute +1

  9. CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency

    Sep 28, 2026Junwoo Bae, Jin-Hyun AhnCommunication-Efficient Distributed TrainingSplit Federated Learning

    Myongji University

  10. SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

    Sep 24, 2026Mengli Wei, Mengkai Zhu, Jiawen Chen +2Communication-Efficient Distributed TrainingBlack-Box Optimization

    School of Mathematics, Southeast University, Nanjing 210096, China

  11. ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization

    Sep 23, 2026Shengjun Zhang, Tingyi Liu, Heng Zhang +1Communication-Efficient Distributed TrainingMulti-Agent Consensus

    School of Artificial Intelligence, Hubei University, Wuhan, China · School of Economics and Management, Wuhan University, Wuhan, China · School of Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China +1

  12. Accelerating Sharded Data Parallelism at Scale with Federated Learning

    Sep 17, 2026Gianluca Mittone, Marco AldinucciCommunication-Efficient Distributed TrainingHigh-Performance Computing

    University of Turin, Turin, Italy

  13. The Life of a Token: from Words to Bits on the Wire

    Sep 17, 2026Davide Avesani, Pengwenlong Gu, Sotiris Skaperas +1Communication-Efficient Distributed TrainingHigh-Performance Computing

    CEDRIC - ROC · cnam · Cnam

  14. Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training

    Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Communication-Efficient Distributed TrainingLong-Context Language Modeling

    Stanford University

  15. Revisiting Distributed Sign-Based Variance Reduction

    Sep 16, 2026Wei Jiang, Zechao Li, Lijun ZhangSign-Based OptimizationCommunication-Efficient Distributed Training

    School of Computer Science and Engineering, Nanjing University of Science and Technology, China · National Key Laboratory for Novel Software Technology,Nanjing University, China · School of Artificial Intelligence, Nanjing University, China

  16. FedPGT: Progressive Gradient Transmission for Vehicular Federated Learning over Time-Varying Channels

    Sep 16, 2026Jintao Yan, Tan Chen, Yuxuan Sun +2Communication-Efficient Distributed TrainingOnline Resource Allocation

    Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. · School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China.

  17. Adaptive Bayesian Partner Selection for Federated Clinical Centers

    Sep 15, 2026Navid Seidi, Satyaki Roy, Sajal K. DasHealthcareCommunication-Efficient Distributed Training

    Department of Computer Science Missouri University of Science and Technology Rolla, MO, USA · Department of Mathematical Sciences University of Alabama in Huntsville Huntsville, AL, USA

  18. Privacy-Aligned Personalized Federated Learning with Compact Adaptation and Variable-Length Gaussian Communication

    Sep 14, 2026Yilin Xu, Chun Hei Michael Shiu, Chih Wei Ling +1Communication-Efficient Distributed TrainingDifferentially Private Federated Learning

    Department of Computer Science, City University of Hong Kong · Department of Electrical and Computer Engineering, University of British Columbia · School of Computer Science and Engineering, Hebrew University of Jerusalem

  19. High-Probability Convergence of SGD via Batched Updates

    Sep 14, 2026Feng Zhu, Robert W. Heath Jr., Aritra MitraCommunication-Efficient Distributed TrainingStochastic Optimization Convergence

  20. Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms

    Sep 10, 2026Jun-Yi Meng, Zheng-Chu Guo, Yuan MaoKernel RegressionCommunication-Efficient Distributed Training

    School of Mathematical Sciences, Zhejiang University, Hangzhou 310058, P. R. China · College of Informatics, Huazhong Agricultural University, Wuhan 430070, P. R. China

  21. NEXUS-MI: Communication-Aware Federated Personalization for Gateway-Coordinated Motor-Imagery Brain-Computer Interfaces

    Sep 9, 2026Daniel Adu Worae, Aarthy NagarajanElectroencephalographyBrain-Computer Interfaces

    Department of Computer Science and Engineering, University of Notre Dame, 307 Fitzpatrick Hall, Notre Dame, IN 46556, USA · Department of Computer Science and Engineering and the Lucy Family Institute for Data and Society, University of Notre Dame, Notre Dame, IN 46556, USA

  22. Non-Coherent Over-the-Air Federated Learning: Protocol, Convergence, and Device Scheduling

    Sep 8, 2026Haifeng Wen, Nicolò Michelusi, Osvaldo Simeone +2Federated Learning AggregationCommunication-Efficient Distributed Training

    IoT Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ 85281 USA · Institute for Intelligent Networked Systems, Northeastern University London, London, E1 8PH, UK +2

  23. BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

    Sep 2, 2026Bigyan Ghimire, Jon C. CalhounCommunication-Efficient Distributed TrainingLong-Context Language Modeling

    Holcombe Department of Electrical and Computer Engineering Clemson University Clemson, USA

  24. FractalNet-Based Heterogeneous Federated Learning for Orbital Edge Intelligence in Satellite Mega-Constellations: A Wildfire Case Study

    Sep 1, 2026Sai Puppala, Koushik SinhaHierarchical Federated LearningWildfire Detection

    School of Computing

  25. RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks

    Aug 31, 2026Xingran Chen, Rohit Bhagat, Ghadir Ayache +3Fine-TuningCommunication-Efficient Distributed Training

    Engineering Systems and Design Pillar, Singapore University of Technology and Design, Singapore 487372 · Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA · LinkedIn, New York, NY 10118, USA +2

  26. Dion3: Full-Stack Orthogonal Updates

    Aug 12, 2026Noah Amsel, Jack Zhang, Kwangjun Ahn +5Newton-Schulz IterationGPU Kernel Optimization

    New York University · Microsoft Research · Princeton University +3

  27. FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients

    Aug 10, 2026Bostan Khan, Masoud DaneshtalabDynamic Neural NetworksCommunication-Efficient Distributed Training

    Mälardalen University, Västerås, Sweden

  28. Hierarchical Multi-Task Federated Learning in VANETs

    Aug 8, 2026M. Saeid HaghighiFard, Sinem ColeriHierarchical Federated LearningCommunication-Efficient Distributed Training

    Department of Electrical and Electronics Engineering, Koc University, Istanbul, Turkey

  29. LGNNIC: Acceleration of Large-Scale GNN Training using SmartNICs

    Aug 7, 2026Liad Gerstman, Aditya Dhakal, Dejan Milojicic +1Graph Neural NetworksCommunication-Efficient Distributed Training

    Technion Israel Institute of Technology Haifa, Israel · Hewlett Packard Enterprise Labs Fort Collins, CO, USA · Hewlett Packard Enterprise Labs Milpitas, CA, USA

  30. Stream Learning: Partition-Fair Gossip Learning Without Tokens

    Aug 7, 2026Fabien Mathieu, Alexandre Pham, Maria Gradinariu Potop-Butucaru +1Communication-Efficient Distributed TrainingDecentralized Learning

    NPA · Swapcard, Paris, France · Sorbonne Université, CNRS, LIP6, F-75005 Paris, France +2

  31. Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization

    Aug 6, 2026Grigory MalinovskyByzantine-Robust Federated LearningCommunication-Efficient Distributed Training

    King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia

  32. ML-for-ML

    Aug 6, 2026Yutong Zhao, Noga H. Rotman, Gianni Antichi +1Communication-Efficient Distributed TrainingDistributed Training

    UCL and BUPT · United Kingdom · UCL +3

  33. SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction

    Aug 6, 2026Guofan Yu, Sitian Chen, Zhenheng Tang +2Graph Neural NetworksCommunication-Efficient Distributed Training

    Hong Kong Baptist University · Hong Kong University of Science and Technology · Hong Kong University of Science and Technology (Guangzhou)

  34. DG-FedReuse: Proxy-Gradient-Gated Cached-Update Reuse with Matched Sparse Uplink Accounting

    Aug 5, 2026Rahil Aftab, Vineet Kumar Rakesh, Soumya Mazumdar +1Federated Learning AggregationCommunication-Efficient Distributed Training

    Department of Computer Science and Engineering Jamia Hamdard New Delhi 110062, India · Engineering Science Homi Bhabha National Institute Anushaktinagar, Mumbai 400094, Maharashtra, India · Computer and Informatics Group Variable Energy Cyclotron Centre 1/AF, Bidhannagar, Kolkata 700064, West Bengal, India +1

  35. FedCritic-MIMO: Communication-Efficient Serverless Federated Critic Learning for Massive-MIMO Resource Control in Open and Disaggregated 6G RANs

    Aug 4, 2026Amin Farajzadeh, Melike Erol-KantarciFederated RL6G Networks

    School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada · with the NETCORE Lab

  36. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning AggregationCommunication-Efficient Distributed Training

    Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

  37. FedRings: A Scalable and Topology-Aware Federated Learning Framework for LEO Satellite Constellations

    Aug 4, 2026Ziwu Liu, Inês Pinto Gouveia, Rehana Yasmin +2Communication-Efficient Distributed TrainingDecentralized Learning

    CEMSE Division, King Abdullah University of Science and Technology (KAUST) Thuwal 23955-6900, Kingdom of Saudi Arabia

  38. Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback

    Jul 31, 2026Maria Smirnova, Alexey KravatskiySign-Based OptimizationCommunication-Efficient Distributed Training

    MIRIAI Moscow, Russia

  39. Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

    Jul 23, 2026Hariharan Ramesh, Jyotikrishna DassFederated Learning AggregationCommunication-Efficient Distributed Training

    Department of Electrical and Computer Engineering University of Arizona, Tucson, AZ 85719

  40. Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries

    Jul 23, 2026Hyeong-Gun Joo, Songnam Hong, Dong-Joon ShinSecure Multi-Party ComputationFederated Learning Aggregation

    Department of Electronic Engineering, Hanyang University, Seoul, South Korea

  41. SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    Jul 22, 2026Dongfang Li, Xiaodong Luo, Ruoyu Sun +62Communication-Efficient Distributed TrainingLLM Post-Training

  42. Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts

    Jul 21, 2026Minyu Cui, Anna Wingkvist, Morgan EricssonGPU Kernel OptimizationCommunication-Efficient Distributed Training

    Linnaeus University · Växjö, Sweden

  43. AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

    Jul 20, 2026Bas Meuwissen, Vasileios Tsouvalas, Nirvana MeratniaFine-TuningCommunication-Efficient Distributed Training

    Eindhoven University of Technology, Eindhoven, The Netherlands

  44. Agora: Collective and Permissionless Internet-Scale Pretraining of Large Language Models

    Jul 14, 2026Gil Avraham, Violetta Shevchenko, Hadi Mohaghegh Dolatabadi +9Communication-Efficient Distributed TrainingFault-Tolerant Distributed Training

    Pluralis Research

  45. Mixed-Timescale Differential Coding for Downlink Model Broadcast in Wireless Federated Learning

    Jul 14, 2026Chung-Hsuan Hu, Zheng Chen, Erik G. LarssonCommunication-Efficient Distributed TrainingFederated Learning

    Department of Electrical Engineering (ISY), Linköping University, Linköping, SE-58183 Sweden

  46. PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs

    Jul 13, 2026Jing Liu, Kun Yang, Yan Wang +5Multimodal Federated LearningFederated Learning Aggregation

    College of Future Information Technology, Fudan University, Shanghai 200433, China · Division of Natural and Applied Sciences, Duke Kunshan University, Suzhou 215316, China · Department of Electrical and Computer Engineering, The University of British Columbia, BC V6T 1Z4, Canada +8

  47. GeoFP8: Geometry-Aware FP8 Gradient Compression for Distributed LLM Training

    Jul 8, 2026Jieying Wang, Zizhong Wang, Fangru Linghu +3Language Model PretrainingCommunication-Efficient Distributed Training

    Department of Computer Science, Rutgers University, Piscataway, NJ, USA · Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA

  48. Robust Federated Learning Under Real-World Client Churn

    Jul 8, 2026Dhruv Garg, Neha Lakhani, Debopam Sanyal +3Federated Learning AggregationCommunication-Efficient Distributed Training

    Georgia Tech USA · Cisco Research USA

  49. Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

    Jul 7, 2026Quoc Bao Phan, Tuy Tan NguyenTensor NetworksMultimodal Federated Learning

    Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU-FSU College of Engineering, Florida State University, Tallahassee, FL 32310, USA

  50. Can Model Merging Improve Aggregation in DiLoCo?

    Jul 3, 2026Stefan Horoi, Benjamin Thérien, Guy Wolf +1Communication-Efficient Distributed TrainingDistributed Optimization

    Université de Montréal · Mila – Quebec AI Institute · Concordia University

  51. Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models

    Jul 2, 2026Xuan-Phi Nguyen, Shrey Pandit, Yiran Zhao +3Communication-Efficient Distributed TrainingHigh-Performance Computing

    Salesforce AI Research

  52. SCAPE: Accurate and Efficient LLM Training with Extreme Sparse Communication

    Jul 2, 2026Mingkai Zheng, Junlin Chen, Haotian Xie +1Language Model PretrainingCommunication-Efficient Distributed Training

    Rutgers University

  53. TallyTrain: Communication-Efficient Federated Distillation

    Jun 30, 2026Radhakrishna Achanta, Will ReedFederated Learning AggregationCommunication-Efficient Distributed Training

    Cisco Systems Inc.

  54. HSAP: A Hierarchical Sequence-aware Parallelism for Hybrid-Context Generative Models

    Jun 29, 2026Songxin Zhang, Zejian Xie, Zhuoyang Song +4Communication-Efficient Distributed TrainingLLM Training

    Department of Statistics and Data Science, Southern University of Science and Technology · International Digital Economy Academy

  55. Federated Hash Projected Latent Factor Learning

    Jun 24, 2026Jialan HeCommunication-Efficient Distributed TrainingPrivacy-Preserving ML

    College of Computer and Information Science Southwest University Chongqing 400715, China

  56. TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

    Jun 24, 2026Erdenebileg Batbaatar, Young YoonCommunication-Efficient Distributed TrainingPrivacy-Preserving ML

    Neouly, Republic of Korea · Department of Computer Engineering, Hongik University, Republic of Korea

  57. Not Every Sync Is Safe: Calibrated DiLoCo Scheduling for Shared AI Infrastructure

    Jun 24, 2026Maxwell Twelftree, David Lemphers, An-chi He +1Communication-Efficient Distributed TrainingDistributed Optimization

    Maincode, Melbourne, Australia

  58. Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning

    Jun 22, 2026Nuocheng Yang, Yechen He, Sihua Wang +3Federated UnlearningCommunication-Efficient Distributed Training

    Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

  59. Factored Gossip DiLoCo: Reducing Blocking Communication in DiLoCo

    Jun 22, 2026Chamin Hewa Koneputugodage, Thalaiyasingam Ajanthan, Sameera Ramasinghe +7Communication-Efficient Distributed TrainingDistributed Optimization

    1Pluralis Research

  60. FeLoG: Scalable and Efficient Distributed Graph Embedding with Feedback Loop Mechanism

    Jun 20, 2026Peng Fang, Arijit Khan, Ziqiang Wu +4Communication-Efficient Distributed TrainingGraph Embedding

    Huazhong University of Science and Technology · Bowling Green State University