Alternating Direction Method of Multipliers

Also known as ADMM

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 20

Sep 24, 2026cs.LG

SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of

\mathcal{O}(T^{-1/3})

using only

Θ(T^{2/3})

communication rounds, where

T

is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.

Mengli Wei, Mengkai Zhu, Jiawen Chen +2

School of Mathematics, Southeast University, Nanjing 210096, China

Communication-Efficient Distributed Training Black-Box Optimization Alternating Direction Method of Multipliers Decentralized Learning Differentially Private Federated Learning Derivative-Free Optimization+1

Jul 14, 2026math.OC

Learning-enabled Acceleration of Scenario-based Model Predictive Control

Scenario-based model predictive control (SBMPC) is a variant of model predictive control (MPC) that explicitly accounts for uncertainty by optimizing control actions over multiple predicted scenarios. However, its computational complexity increases rapidly with the number of scenarios and prediction horizon, limiting is applicability to real-time planning and control. This paper presents a learning-accelerated Alternating Direction Method of Multipliers (ADMM) algorithm for efficiently solving SBMPC problems by leveraging parallel computing and Moreau envelope learning, while maintaining high solution accuracy. We reformulate the SBMPC problems into consensus forms that can be decomposed via ADMM, separating the scenario-dependent dynamics from non-anticipativity constraints and enabling parallel updates across scenarios and time steps. Building on this decomposition, we utilize existing learning-to-optimize schemes, which leverages Moreau envelope learning of the cost function to accelerate the primal update in ADMM, thereby reducing computation time. The proposed framework is evaluated on a microgrid energy management problem subject to load and renewable generation uncertainties. Comparisons with IPOPT and MadNLP, popular and modern nonlinear programming solvers, demonstrate substantial computational speedups while maintaining reliable closed-loop control performance.

Trinh Tran, Binh Nguyen, Truong X. Nghiem

Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA

Alternating Direction Method of Multipliers Model Predictive Control

Jul 13, 2026cs.RO

WarpMPC: Large-Batch MPC on GPU via ADMM with Unrolled $LDL^\top$ Factorization

This paper introduces numerical optimizations for maximizing throughput on GPU when solving large batches (10,000 to over 100,000) of sequential quadratic programming (SQP) iterations, where all problems have the same structure. The optimizations are implemented in a toolbox WarpMPC for model-predictive control (MPC) in JAX and Warp. Based on the insight that all MPC problem instances in a batch share the same sparsity in time, cost, and constraints, we propose unrolling sparse linear factorizations and solves, which dominate alternating direction method of multipliers (ADMM) solver runtime. We avoid memory access bottlenecks and wasting computations via optimized memory layout, padding-reducing segmentation of the unrolled factorization, and dependency level scheduled backsolves, additionally accelerating sensitivity computation. We achieve throughputs of 8,000 to 250,000 SQP iterations per second on nonlinear cartpole, quadrotor, and humanoid robot benchmarks, outperforming baselines by 3

\times

to 25

\times

. We illustrate practical usefulness by synthesizing a dataset and training a neural network approximation of an MPC in under 4 minutes that stabilizes a nano quadrotor in hardware experiments.

Henrik Hose, Se Hwan Jeon, Charles Khazoom +2

Institute for Data Science in Mechanical Engineering (DSME), RWTH Aachen University, Germany · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge MA, USA.

GPU Kernel Optimization GPU Acceleration Nonlinear MPC Alternating Direction Method of Multipliers Quadratic Programming

Jun 26, 2026math.OC

Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization

We analyze Bregman ADMM for nonconvex linearly constrained problems under two-sided relative smoothness, a condition that replaces the standard Lipschitz gradient assumption with a Hessian comparison relative to a Bregman kernel. This setting covers polynomial objectives arising in matrix and tensor models for which a global Lipschitz-gradient constant need not exist. We show that on an invariant open state-space domain, one iteration of Bregman ADMM defines a smooth primal--dual fixed-point map whose strict-saddle KKT points are unstable fixed points; consequently, from random initialization the iterates converge to a strict saddle with probability zero. Combined with existing first-order convergence results, this yields almost-sure second-order stationarity of limiting KKT points. We extend the analysis to a multi-block star consensus formulation for distributed optimization. The technical novelty lies in a determinant reduction with a Bregman-specific symmetrization and scaling step in the two block spectral argument, together with a null space cancellation exploiting the star graph structure in the consensus case. Numerical experiments on distributed matrix factorization illustrate the theory, and a symmetric tensor factorization example demonstrates the broader Bregman proximal splitting idea beyond the separable consensus setting.

Shuang Li, Zhihui Zhu, Qiuwei Li

Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa 50014, USA · Department of Computer Science and Engineering, Ohio State University, Columbus, Ohio 43210, USA · DAMO Academy, Alibaba Group US, Bellevue, Washington 98004, USA

Optimization Convergence Analysis Distributed Optimization Alternating Direction Method of Multipliers Nonconvex Optimization

Jun 18, 2026cs.RO

Deep-Unfolded Coordination

Distributed optimization is a highly scalable and structurally transparent technique to solve multi-agent robotics problems; however, such methods often suffer from the need for highly-specialized, problem-specific hyperparameter tunings. In this work, we propose Deep Coordinator, a deep-unfolding framework that learns to dynamically adjust the hyperparameters of ADMM-DDP, a popular distributed solver for robotics tasks, at solve-time in response to optimizer performance. Our architecture consists of unrolling a fixed number of ADMM-DDP iterations into a neural network with learnable functions between layers mapping the optimizer state to the next hyperparameters. To the best of our knowledge, Deep Coordinator is the first deep-unfolding framework to adapt the penalty parameters of a non-convex optimizer at solve-time; we show that the mainstream supervised approach can yield degenerate solutions when training such models, and propose an unsupervised learning scheme. On simulations with fleets of cars and quadrotors, Deep Coordinator produces trajectories of comparable quality 6.18-9.44x faster than conventional solvers. Furthermore, Deep Coordinator retains its performance benefits when deployed to systems up to 8x larger than trained on.

Hunter Kuperman, Minchan Jung, Rahul V. Ghosh +2

Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology United States

Trajectory Optimization Multi-Agent System Optimization Distributed Optimization Alternating Direction Method of Multipliers Multi-Robot Coordination Deep Unfolding

Jun 8, 2026cs.LG

LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization

We propose LEAF, a learning-enabled ADMM framework for accelerated convex optimization. The key idea is to approximate the Moreau envelope of the objective function using an Input Convex Neural Network (ICNN), resulting in a learned model that preserves convexity and smoothness. This leads to the proposed Moreau Envelope Learning ADMM (MEL-ADMM) and its splitting variant sMEL-ADMM. Unlike existing approaches that learn high-dimensional operators directly, LEAF learns a scalar-valued Moreau envelope, significantly reducing model complexity and improving data efficiency. The framework accommodates a broad class of convex problems with smooth and non-smooth objectives. By embedding convexity explicitly through the ICNN architecture, the proposed approach maintains high approximation accuracy while preserving key structural properties of the optimization problem. Both MEL-ADMM and sMEL-ADMM are developed with theoretical guarantees of convergence and feasibility under the learned model. Rigorous analysis shows that the proposed methods achieve convergence rates comparable to classical ADMM while reducing per-iteration computational cost. Numerical experiments demonstrate up to an order-of-magnitude speedup over state-of-the-art solvers while maintaining low optimality gaps

Binh Nguyen, Trinh Tran, Truong X. Nghiem

Department of Electrical and Computer Engineering, College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, USA

Learning to Optimize Alternating Direction Method of Multipliers Input Convex Neural Networks

Jun 1, 2026eess.SP

Physics-Aware Linearized ADMM and Its Unrolling

Recently, partial differential equations (PDEs) have been used to directly model the measurement process in signal processing, although their evaluation is costly. In this paper, we propose a novel alternating direction method of multipliers (ADMM)-based algorithm called physics-aware linearized ADMM (PA-LADMM) for inverse problems from PDE-based measurement processes. The key idea is the linearization of the subproblem with PDEs, leading to a cost-efficient update rule that calls only a PDE solver and its gradient evaluation per iteration. The algorithm has a theoretical convergence guarantee under certain conditions. In addition, we combine it with deep unfolding to unroll the PA-LADMM and train its internal parameters using supervised data. Two distinct experiments, compressed sensing with optical fiber communication and image restoration from noisy anisotropic diffusion, demonstrated the effectiveness of the proposed algorithms.

Satoshi Takabe, Shunta Arai, Tadashi Wadayama

Department of Computer Science Institute of Science Tokyo Tokyo, Japan · Center for Advanced Intelligence Project RIKEN Tokyo, Japan · Department of Engineering Nagoya Institute of Technology Nagoya, Japan

Alternating Direction Method of Multipliers Algorithm Unrolling Compressed Sensing PDE Inverse Problems Image Inverse Problems

May 29, 2026cs.LG

Learning Multi-Agent Coordination via Sheaf-ADMM

We present a differentiable optimization framework for multi-agent coordination. An input is decomposed into overlapping local views, each processed by an agent that solves a convex subproblem parameterized by a neural encoder. Agents coordinate through the Alternating Direction Method of Multipliers (ADMM) with inter-agent constraints specified by a cellular sheaf. The sheaf specifies which aspects of neighboring solutions must agree, allowing for heterogeneous notions of global consensus. Backpropagating through the unrolled optimization jointly trains all components of the multi-agent system. We evaluate on maze pathfinding, image classification, and Sudoku, where agents with individually insufficient local views learn to coordinate to produce correct global outputs. On MNIST, the local-view decomposition yields improved robustness to distribution shifts relative to a standard CNN. On Sudoku, the optimization-derived structure yields markedly higher solve rates than parameter-matched MPNN baselines. Finally, the ADMM structure exposes distinct primal, consensus, and dual state variables, opening the coordination dynamics to direct analysis and intervention -- a property unavailable in standard message-passing architectures.

Jeffrey Seely, Bartłomiej Cupiał, Llion Jones

Sakana AI, Tokyo, Japan · University of Warsaw · AKCES NCBR

Multi-Agent System Optimization Multi-Agent Coordination Alternating Direction Method of Multipliers Multi-Agent Consensus Differentiable Optimization

May 19, 2026stat.ML

A Unified Framework for Structure-Aware Clustering and Heterogeneous Causal Graph Learning

In complex multivariate systems, interactions among variables are defined by dependency structures, often encoded as directed acyclic graphs (

\text{DAGs}

). However, dependency structures can vary across subjects, and ignoring this structural heterogeneity introduces bias and obscures subpopulation-specific dependencies. To address this, we propose Directed Acyclic Graph-based Dependency Clustering via Alternating Direction Method of Multipliers (DAG-DC-ADMM), a unified framework built upon Structural Equation Modeling (SEM) that jointly learns cluster assignments and cluster-specific dependency structures. We encode acyclicity via a smooth constraint and integrate a groupwise truncated Lasso fusion penalty (gTLP) to cluster subjects based on their structural similarity. This yields a nonconvex optimization problem that incorporates sparsity, acyclicity, and structural consensus constraints. We address the nonconvexity by using the augmented Lagrangian method and solve it with an adapted version of the Alternating Direction Method of Multipliers (ADMM) for difference-of-convex programs. For certain graph structures, such as upper triangular adjacency matrices, our algorithm is guaranteed to converge to a Karush-Kuhn-Tucker (KKT) point. Experiments demonstrate that our method recovers cluster-specific causal dependency structures with a high true positive rate and a low false discovery rate. This capability enables the robust discovery of heterogeneous dependencies across subjects where the subpopulation label is unknown.

Honglin Du, Muxuan Liang, Xiang Zhong

Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL 32611, USA · Department of Biostatistics, MD Anderson Cancer Center, Houston, TX 77030, USA

Structural Causal Models Clustering Causal Discovery Causal Structure Learning Alternating Direction Method of Multipliers

May 11, 2026cs.LG

ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models

Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (PTQ) is a leading approach for compressing LLMs. Popular weight quantization procedures, including GPTQ and RTN, suffer in model utility, especially at aggressive quantization levels (sub-4-bit). We propose ADMM-Q, a novel weight quantization algorithm that considers the layer-wise quantization problem. Our algorithm is based on a combinatorial variant of the Alternating Direction Method of Multipliers (ADMM). Our operator-splitting procedure updates weights continuously to minimize the layer-wise reconstruction error, while gradually enforcing the quantization constraints with convergence guarantees. We propose additional algorithmic enhancements (e.g., penalty scheduling, preconditioning, and a local search post-processing step) to make ADMM-Q efficient at LLM scale. ADMM-Q is modular and can be used as a drop-in replacement for any weight quantizer within existing quantization pipelines: ADMM-Q is fully composable with existing techniques including range clipping, learned or random rotations, and activation scaling. Using ADMM-Q in place of GPTQ on Qwen3-8B, we decrease WikiText-2 perplexity in: (i) the W3A16 weight-only setting (12.85

\rightarrow

10.06); (ii) the W4A8 SmoothQuant procedure (9.29

\rightarrow

8.68); and (iii) the W2A4KV4 SpinQuant procedure (66.11

\rightarrow

19.42).

Ryan Lucas, Mehdi Makni, Xiang Meng +2

MIT Operations Research Center · MIT Sloan School of Management · MIT Center for Statistics

LLM Quantization Alternating Direction Method of Multipliers Constrained Optimization Post-Training Quantization Low-Bit Quantization Neural Network Quantization

May 10, 2026cs.LG

Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective

Decentralized federated learning (FL) is a promising approach for training machine learning models on sensor networks, Internet of Things (IoT) devices, and other edge systems where no central server exists. While federated learning offers advantages such as preserving data privacy, it often suffers from non-independent and identically distributed (IID) data distributions across devices, which cause significant performance degradation. This issue is particularly severe when directly optimizing model parameters, because neural network training is inherently non-convex and standard convergence guarantees for convex optimization do not apply. Unlike existing decentralized FL methods that primarily operate in parameter space, we propose federated function-space alternating direction method of multipliers (FedF-ADMM). FedF-ADMM exploits the convexity of loss functionals within function space to derive alternating direction method of multipliers (ADMM)-based update directions, which are subsequently projected onto the parameter space via knowledge distillation. We further introduce a stabilization coefficient to enhance robustness under severe non-IID settings and analyze its behavior from a control-theoretic perspective by interpreting it as a proportional-integral (PI) term. Experiments under challenging non-IID scenarios, including settings where each device has data from only a single label, demonstrate that FedF-ADMM achieves faster and more stable convergence than existing decentralized FL methods, while attaining higher accuracy and better consensus among devices.

Akihito Taya, Yuuki Nishiyama, Kaoru Sezaki

Institute of Industrial Science, The University of Tokyo, Meguro-ku, Tokyo 153-8505, Japan · Center for Spatial Information Science, The University of Tokyo, Kashiwa-shi, Chiba 277-8568, Japan

Alternating Direction Method of Multipliers Non-IID Federated Learning Federated Learning

May 9, 2026cs.CV

FlowADMM: Plug-and-play ADMM with Flow-based Renoise-Denoise Priors

Plug-and-play (PnP) methods for solving inverse problems have recently achieved strong performance by leveraging denoising priors based on powerful generative diffusion and flow models. However, existing diffusion- and flow-based PnP methods typically rely on stochastic renoise-denoise operations, which complicate the analysis of their convergence behavior. In this work, we identify and formalize the deterministic renoise-denoise operator underlying flow-based plug-and-play methods. This perspective reveals that these methods implicitly define a deterministic operator given by the expectation of a denoiser over the latent noise distribution. Building on this insight, we propose FlowADMM, a PnP algorithm that integrates the renoise-denoise operator into the classical alternating direction method of multiplier (ADMM) framework. We establish convergence guarantees for FlowADMM under weak Lipschitz conditions on the underlying flow network, and extend the analysis to non-stationary time schedules. Empirically, FlowADMM achieves state-of-the-art performance among flow-based PnP methods on a range of inverse problems, including denoising, deblurring, super-resolution, and inpainting, while requiring fewer data consistency evaluations than prior approaches.

Hendrik Sommerhoff, Michael Moeller

Computer Vision Group, University of Siegen Hölderlinstraße 3, 57076 Siegen, Germany

Alternating Direction Method of Multipliers Image Restoration Constrained Optimization Inverse Problems

May 5, 2026stat.ML

Low Rank Tensor Completion via Adaptive ADMM

We consider a novel algorithm, for the completion of partially observed low-rank tensors, as a generalization of matrix completion. The proposed low-rank tensor completion (TC) method builds on the conventional nuclear norm (NN) minimization-based low-rank TC paradigm, by leveraging the alternating direction method of multipliers (ADMM) optimization framework. To that extend the original NN minimization problem is reformulated into multiple subproblems, which are then solved iteratively via closed-form proximal operators, making use of over-relaxation and an adaptive penalty parameter update scheme, to further speed up convergence and improve the overall performance of the method. Simulation results demonstrate the superior performance of the new method in terms of normalized mean square error (NMSE), compared to the conventional state-of-the-art (SotA) techniques, including NN minimization approaches, as well as a mixture of the latter with a matrix factorization approach, while its convergence can be significantly improved by initializing the algorithm with the solution of the SotA.

Niclas Führling, Getuar Rexhepi, Giuseppe Thadeu Freitas de Abreu

School of Computer Science and Engineering, Constructor University, Bremen, Germany

Tensor Completion Nuclear Norm Minimization Alternating Direction Method of Multipliers Low-Rank Approximation

Apr 29, 2026math.OC

Learning Over-Relaxation Policies for ADMM with Convergence Guarantees

The Alternating Direction Method of Multipliers (ADMM) is a widely used method for structured convex optimization, and its practical performance depends strongly on the choice of penalty and relaxation parameters. Motivated by settings such as Model Predictive Control (MPC), where one repeatedly solves related optimization problems with fixed structure and changing parameter values, we propose learning online updates of the relaxation parameter to improve average performance on problem classes of interest, while guaranteeing that asymptotic convergence is not compromised for the worst-case realization of such problems. This choice is computationally attractive in the Operator Splitting Quadratic Program (OSQP)-like architectures, since adapting relaxation does not trigger the matrix refactorizations associated with penalty updates. We establish convergence guarantees for ADMM with time-varying penalty and relaxation parameters under mild assumptions, and show on benchmark quadratic programs that the resulting learned policies improve both iteration count and wall-clock time on average over baseline OSQP.

Junan Lin, Paul J. Goulart, Luca Furieri

Department of Engineering Science, University of Oxford, OX1 3PJ Oxford, UK

Learning to Optimize Alternating Direction Method of Multipliers Constrained Optimization

Apr 24, 2026cs.RO

ATRS: Adaptive Trajectory Re-splitting via a Shared Neural Policy for Parallel Optimization

Parallel trajectory optimization via the Alternating Direction Method of Multipliers (ADMM) has emerged as a scalable approach to long-horizon motion planning. However, existing frameworks typically decompose the problem into parallel subproblems based on a predefined fixed structure. Such structural rigidity often causes optimization stagnation in highly constrained regions, where a few lagging subproblems delay global convergence. A natural remedy is to adaptively re-split these stagnating segments online. Yet, deciding when, where, and how to split exceeds the capability of rule-based heuristics. To this end, we propose ATRS, a novel framework that embeds a shared Deep Reinforcement Learning policy into the parallel ADMM loop. We formulate this adaptive adjustment as a Multi-Agent Shared-Policy Markov Decision Process, where all trajectory segments act as homogeneous agents and share a unified neural policy network. This parameter-sharing architecture endows the system with size invariance, enabling it to handle dynamically changing segment counts during re-splitting and generalize to arbitrary trajectory lengths. Furthermore, our formulation inherently supports zero-shot generalization to unseen environments, as our network relies solely on the internal states of the numerical solver rather than on the geometric features of the environment. To ensure solver stability, a Confidence-Based Election mechanism selects only the most stagnating segment for re-splitting at each step. Extensive simulations demonstrate that ATRS accelerates convergence, reducing the number of iterations by up to 26.0% and the computation time by up to 19.1%. Real-world experiments further confirm its applicability to both large-scale offline global planning and real-time onboard replanning within 35 ms per cycle, with no sim-to-real degradation.

Jiajun Yu, Guodong Liu, Li Wang +6

Trajectory Optimization Alternating Direction Method of Multipliers Constrained Optimization

Apr 14, 2026math.OC

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization

This paper presents HUANet, a constrained deep neural network architecture that unrolls the Alternating Direction Method of Multipliers (ADMM) into a trainable neural network for accelerating parametric constrained convex optimization. Existing end-to-end learning methods operate as black-box mappings from parameters to solutions, often without explicitly incorporating optimality principles or guaranteeing constraint satisfaction. To address these limitations, HUANet embeds a hard-constrained neural network within each unrolled ADMM iteration, where a differentiable correction stage enforces the affine equalities of the primal subproblem. Furthermore, we incorporate first-order optimality conditions into a self-supervised training loss to promote the convergence of the proposed unrolled algorithm. Extensive numerical experiments for benchmark optimization problems and a control application demonstrate and validate the effectiveness of HUANet in accelerating constrained convex optimization solving.

Trinh Tran, Binh Nguyen, Truong X. Nghiem

Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA

Learning to Optimize Alternating Direction Method of Multipliers Algorithm Unrolling Constrained Optimization

Mar 11, 2026cs.RO

ADMM-based Continuous Trajectory Optimization in Graphs of Convex Sets

This paper presents a numerical solver for computing continuous trajectories in non-convex environments. Our approach relies on a customized implementation of the Alternating Direction Method of Multipliers (ADMM) built upon two key components: First, we parameterize trajectories as polynomials, allowing the primal update to be computed in closed form as a minimum-control-effort problem. Second, we introduce the concept of a spatio-temporal allocation graph based on a mixed-integer formulation and pose the slack update as a shortest-path search. The combination of these ingredients results in a solver with several distinct advantages over the state of the art. By jointly optimizing over both discrete and continuous domains, our method accesses a larger search space than existing decoupled approaches, enabling the discovery of superior trajectories. Additionally, the solver's structural robustness ensures reliable convergence from naive initializations, removing the bottleneck of complex warm starting in non-convex environments.

Lukas Pries, Jon Arrizabalaga, Zachary Manchester +1

Autonomous Aerial Systems Lab, Dep. of Aerospace and Geodesy, TU Munich, Germany · Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, USA

Trajectory Optimization Alternating Direction Method of Multipliers Nonconvex Optimization Constrained Optimization Motion Planning

Jan 23, 2026cs.MS

Learning to Optimize by Differentiable Programming

Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost. This tutorial highlights a shift in optimization: using differentiable programming not only to execute algorithms but to learn how to design them. Modern frameworks such as PyTorch, TensorFlow, and JAX enable this paradigm through efficient automatic differentiation. Embedding first-order methods within these systems allows end-to-end training that improves convergence and solution quality. Guided by Fenchel-Rockafellar duality, the tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.

Liping Tao, Xindi Tong, Chee Wei Tan

Nanyang Technological University, Singapore · Nanyang Technological University, Singapore, Singapore

Primal-Dual Optimization Learning to Optimize Alternating Direction Method of Multipliers Differentiable Programming Constrained Optimization Differentiable Optimization

Jun 2, 2025cs.LG

Efficient Learning of Balanced Signed Graphs via Sparse Linear Programming

Signed graphs are equipped with both positive and negative edge weights, encoding pairwise correlations as well as anti-correlations in data. A balanced signed graph is a signed graph with no cycles containing an odd number of negative edges. Laplacian of a balanced signed graph has eigenvectors that map via a simple linear transform to ones in a corresponding positive graph Laplacian, thus enabling reuse of spectral filtering tools designed for positive graphs. We propose an efficient computation method to learn a balanced signed graph Laplacian directly from data. Specifically, extending a previous linear programming (LP) based sparse inverse covariance estimation method called CLIME, we formulate a new LP problem for each Laplacian column

i

, where the linear constraints restrict weight signs of edges stemming from node

i

, so that nodes of same / different polarities are connected by positive / negative edges. We derive a feasible CLIME parameter

ρ_i

for each sign-constrained column problem. We solve the LP problem efficiently by tailoring a sparse LP method based on ADMM. We theoretically prove that the row / column updates produce a non-increasing objective sequence, and show that the iterations are terminated in a finite number of steps. Extensive experimental results on synthetic and real-world datasets show that our balanced graph learning method outperforms competing methods and enables reuse of spectral filters, wavelets, and graph neural nets (GNN) constructed for positive graphs.

Haruki Yokota, Hiroshi Higashi, Yuichi Tanaka +1

Graph Structure Learning Graph Representation Learning Alternating Direction Method of Multipliers Linear Programming

Feb 1, 2025math.OC

On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization

Distributed empirical risk minimization (ERM) is often studied through two influential yet seemingly separate families of methods: CoCoA-type algorithms, derived from distributed dual coordinate ascent, and ADMM-type algorithms, derived from consensus and proximal splitting. In this paper, we investigate the connection of the two types of algorithms from a unified primal-dual perspective. We show that consensus ADMM, linearized consensus ADMM, two distributed proximal ADMM variants, and ridge-regularized CoCoA can all be written in a common update form involving a global primal variable and block dual variables. This reformulation makes several previously hidden connections explicit: For ridge-regularized ERM, CoCoA coincides with a particular proximal ADMM scheme at the level of the dual update. Moreover, consensus ADMM on the primal problem is equivalent to proximal ADMM on the dual problem under an explicit parameter mapping together with a sign reversal of the saddle objective; similar correspondences also hold for the linearized variants. These results indicates that the ADMM-type algorithms, when fine tuned, performs at least as good as CoCoA, under ridge regularized ERM problems. The unified view also yields a natural primal-dual gap stopping criterion for consensus ADMM and a unified

O(1/T)

ergodic convergence analysis for the ADMM-type methods. Experiments on synthetic regression problems and real SVM datasets support the predicted relationships, clarify the role of tuning parameters, and show that suitably tuned ADMM variants can outperform CoCoA in the ridge-regularized setting.

Runxiong Wu, Andi Wang

Department of Industrial & Systems Engineering University of Wisconsin–Madison

Primal-Dual Optimization Distributed Optimization Alternating Direction Method of Multipliers Empirical Risk Minimization