Distributed Optimization

Follow this topic Ask about this topic

Distributed optimization studies how multiple machines, clients, or robots can solve a shared problem using local data and limited communication. Common approaches include consensus and decentralized gradient methods, local or federated SGD, ADMM and primal-dual algorithms, and asynchronous updates; communication can be reduced through compression, quantization, or less frequent synchronization. These methods support settings from large-model training and statistical learning to robot coordination, energy systems, and networked control. A central challenge is balancing communication and computation against convergence when data and worker speeds differ, messages are delayed, or participants may be unreliable or adversarial.

Latest papers

92 papers11 papers in the last four weeks, up 267% on the four weeks before. 0.1% of all new papers.

Learning to Act and Cooperate for Distributed Black-Box Consensus Optimization
May 1, 2026Zi-Bo Qin, Feng-Feng Wei, Tai-You Chen +1Multi-Agent System Optimization Multi-Agent Coordination
School of Computer Science and Engineering, South China University of Technology, Guangzhou, China.
Hierarchical Federated Learning for Networked AI: From Communication Saving to Architecture-Aware Design
May 1, 2026Seyed Mohammad Azimi-Abarghouyi, Mehdi Bennis, Leandros TassiulasHierarchical Federated Learning Communication-Efficient Distributed Training
Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden · Centre for Wireless Communications, University of Oulu, Oulu, Finland · Department of Electrical Engineering, Yale University, New Haven, USA
FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost
Apr 27, 2026Chenhao Feng, Haoli Zhang, Shakhzod Ali-Zade +17Sequential Recommendation Communication-Efficient Distributed Training
1Meta Inc.
CommFuse: Hiding Tail Latency via Communication Decomposition and Fusion for Distributed LLM Training
Apr 27, 2026Rezaul Karim, Austin Wen, Wang Zongzuo +3Communication-Efficient Distributed Training Distributed Optimization
Toronto Ascend Team
Decoupled DiLoCo for Resilient Distributed Pre-training
Apr 23, 2026Arthur Douillard, Keith Rush, Yani Donchev +14Language Model Pretraining Distributed Optimization
1Google DeepMind · 2Google Research
A Network-Aware Evaluation of Distributed Energy Resource Control in Smart Distribution Systems
Apr 21, 2026Houchao GanDistributed Optimization Feedback Control
Department of Computer Science City University of New York
Accelerating Optimization and Machine Learning through Decentralization
Apr 21, 2026Ziqin Chen, Zuang Wang, Yongqiang WangDistributed Optimization Decentralized Learning
Clemson University
Semantic-based Distributed Learning for Diverse and Discriminative Representations
Apr 20, 2026Zhuojun Tian, Chaouki Ben Issaid, Mehdi BennisRepresentation Learning Distributed Optimization
Center for Wireless Communications, University of Oulu, Oulu, Finland · Division of Information Science and Engineering, KTH Royal of Technology, Stockholm, Sweden · Center for Wireless Communications, University of Oulu, Oulu 90014, Finland
Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
Apr 11, 2026Boyuan Ruan, Xiaoyu Wang, Ya-Feng LiuDistributed Optimization Fault-Tolerant Distributed Training
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Ministry of Education Key Laboratory of Mathematics and Information Networks, School of Mathematical Sciences, Beijing University of Posts and Telecommunications, Beijing 102206, China
The Principle of Maximum Heterogeneity Optimises Productivity in Distributed Production Systems Across Biology, Economics, and Computing
Apr 8, 2026Guillhem Artis, Danyal Akarca, Jascha AchterbergMulti-Agent System Optimization Distributed Optimization
Callosum, London, United Kingdom (www.callosum.com) · Imperial College London, London, United Kingdom · University of Cambridge, Cambridge, United Kingdom +1
Asynchronous Distributed Bandit Submodular Maximization under Heterogeneous Communication Delays
Apr 7, 2026Pranjal Sharma, Zirui Xu, Vasileios TzoumasSubmodular Optimization Distributed Optimization
Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109, USA
Tight Convergence Rates for Online Distributed Linear Estimation with Adversarial Measurements
Apr 7, 2026Nibedita Roy, Vishal Halder, Gugan Thoppe +4Distributed Inference Robust Estimation
Dept. of Computer Science and Automation, Indian Institute of Science, Bengaluru 560012, India · Department of Computer Science, IMT Atlantique, Plouzané 29280, France
Communication-free Sampling and 4D Hybrid Parallelism for Scalable Mini-batch GNN Training
Apr 3, 2026Cunyang Wei, Siddharth Singh, Aishwarya Sarkar +7Graph Neural Networks Distributed Training
University of Maryland, College Park, USA · NVIDIA, Inc. · Iowa State University, USA +3
Gradient Manipulation in Distributed Stochastic Gradient Descent with Strategic Agents: Truthful Incentives with Convergence Guarantees
Mar 30, 2026Ziqin Chen, Yongqiang WangDistributed Training Decentralized Learning
Department of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634 USA
High dimensional theory of two-phase optimizers
Mar 27, 2026Atish AgarwalaMomentum Methods Distributed Optimization
Google DeepMind
Distributed Real-Time Vehicle Control for Emergency Vehicle Transit: A Scalable Cooperative Method
Mar 26, 2026WenXi Wang, JunQi ZhangIntelligent Transportation Systems Multi-Agent Control
School of Computer Science and Technology, Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University, Shanghai 200092, China
Distributed Gradient Clustering: Convergence and the Effect of Initialization
Mar 20, 2026Aleksandar Armacki, Himkant Sharma, Dragana Bajović +3Clustering Distributed Optimization
Carnegie Mellon University, Pittsburgh, Pennsylvania, USA · Indian Institute of Technology Kharagpur, Kharagpur, India · Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia +1
Distributed Equilibrium-Seeking in Target Coverage Games via Self-Configurable Networks under Limited Communication
Mar 18, 2026Jayanth Bhargav, Zirui Xu, Vasileios Tzoumas +2Multi-Agent Coordination Nash Equilibrium Computation
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907 USA · Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109 USA
Accelerating Byzantine-Robust Distributed Learning with Compressed Communication via Double Momentum and Variance Reduction
Mar 16, 2026Yanghao Li, Changxin Liu, Yuhao YiDistributed Optimization Communication-Efficient Distributed Training
Sichuan University · East China University of Science and Technology
Byzantine-Robust Optimization under $(L_0, L_1)$ -Smoothness
Mar 12, 2026Arman Bolatov, Samuel Horváth, Martin Takáč +1Distributed Optimization Byzantine-Robust Federated Learning
Mohamed bin Zayed University of Artificial Intelligence, UAE
Overlapping Domain Decomposition for Distributed Pose Graph Optimization
Mar 3, 2026Aneesa Sonawalla, Yulun Tian, Jonathan P. HowPose Graph Optimization Robotic Optimization
MIT Department of Aeronautics and Astronautics. · Draper Scholar, The Charles Stark Draper Laboratory, Cambridge, MA. · University of Michigan Robotics Department.
riMESA: Consensus ADMM for Real-World Collaborative SLAM
Mar 1, 2026Daniel McGann, Michael KaessSimultaneous Localization and Mapping Pose Graph Optimization
The authors are with the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
A distributed semismooth Newton based augmented Lagrangian method for distributed optimization
Feb 27, 2026Qihao Ma, Chengjing Wang, Peipei Tang +2Distributed Optimization Augmented Lagrangian Method
School of Mathematics, Southwest Jiaotong University, No.999, Xian Road, West Park, High-tech Zone, Chengdu 611756, China · School of Computer and Computing Science, Hangzhou City University, Zhejiang Key Laboratory of Big Data Intelligent Computing, Hangzhou, 310015, China · Department of Control Science and Engineering, College of Electronics and Information Engineering, National Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University, Shanghai 201804, China +1
Event-Triggered Gossip for Distributed Learning
Feb 22, 2026Zhiyuan Zhai, Xiaojun Yuan, Wei Ni +3Communication-Efficient Distributed Training Decentralized Learning
Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration
Feb 18, 2026Luyao Guo, Xinli Shi, Wenying Xu +1Distributed Optimization Nonsmooth Convex Optimization
School of Computer Science and Engineering, Suzhou University of Technology, Suzhou 215500, China · School of Mathematics, Southeast University, Nanjing 210096, China · School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China
Distributed Quantum Gaussian Processes for Multi-Agent Systems
Feb 16, 2026Meet Gandhi, George P. KontoudisGaussian Processes Quantum Machine Learning
Colorado School of Mines, Golden, USA
Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
Feb 3, 2026Grigory Begunov, Alexander TyurinDistributed Optimization Stochastic Gradient Descent
AXXX, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia
DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers
Feb 2, 2026Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis +2Deep Learning Optimization GPU Acceleration
Institute of Science and Technology Austria · Lancaster University · Red Hat AI
Census-Based Population Autonomy For Distributed Robotic Teaming
Nov 4, 2025Tyler M. Paine, Anastasia Bizyaeva, Michael R. BenjaminMulti-Robot Coordination Distributed Optimization
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Woods Hole Oceanographic Institution Woods Hole, MA 02543, USA · Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY 14853, USA
Douglas-Rachford Splitting for Group-Sparse Feedback Linear-Quadratic Control
Jul 26, 2025Lechen Feng, Xun Li, Yuan-Hua NiDistributed Optimization Distributed Control
Department of Applied Mathematics, The Hong Kong Polytechnic University, China · College of Artificial Intelligence, Nankai University, Tianjin, China

May 1, 2026J/K move · Enter open · S save

Learning to Act and Cooperate for Distributed Black-Box Consensus Optimization

Zi-Bo Qin, Feng-Feng Wei, Tai-You Chen, Wei-Neng Chen

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China.

Multi-Agent System Optimization Multi-Agent Coordination Black-Box Optimization Distributed Optimization Large Language Model-Guided Optimization Multi-Agent Consensus

Open paper AskPDF arXiv

Distributed blackbox consensus optimization is a fundamental problem in multi-agent systems, where agents must improve a global objective using only local objective queries and limited neighbor communication. Existing methods largely rely on handcrafted update rules and static cooperation patterns, which often struggle to balance local adaptation, global coordination, and communication efficiency in heterogeneous nonconvex environments. In this paper, we take an initial step toward trajectory-driven self-design for distributed black-box consensus optimization. We first redesign the agent-level swarm dynamics with an adaptive internal mechanism tailored to decentralized consensus settings, improving the balance between exploration, convergence, and local escape. Built on top of this adaptive execution layer, we propose Learning to Act and Cooperate (LACMAS), a trajectorydriven framework in which large language models provide sparse highlevel guidance for shaping both agentinternal action behaviors and agentexternal cooperation patterns from historical optimization trajectories. We further introduce a phased cognitive scheduling strategy to activate different forms of adaptation in a resource-aware manner. Experiments on standard distributed black-box benchmarks and real-world distributed tasks show that LAC-MAS consistently improves solution quality, convergence efficiency, and communication efficiency over strong baselines, suggesting a practical route from handcrafted distributed coordination toward self-designing multi-agent optimization systems.