Unrolled Neural Networks for Constrained Optimization

Samar Hadou; Alejandro Ribeiro

cs.LGJan 24, 2026

Unrolled Neural Networks for Constrained Optimization

Authors: Samar Hadou, Alejandro Ribeiro

Organizations: Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA 19104 USA

Abstract

We develop unrolled neural networks to solve constrained optimization problems, offering accelerated, learnable counterparts to dual ascent (DA) algorithms. Our framework, termed constrained dual unrolling (CDU), comprises two coupled neural networks that jointly approximate the saddle point of the Lagrangian. The primal network emulates an iterative optimizer that finds a stationary point of the Lagrangian for a given dual multiplier, sampled from an unknown distribution. The dual network generates trajectories towards the optimal multipliers across its layers while querying the primal network at each layer. Unlike standard unrolling, which maps the updates of a prescribed iterative solver directly to network layers, CDU preserves the underlying optimization dynamics while learning the update mappings. The desired primal-dual dynamics are therefore induced through constrained learning by enforcing primal-descent and dual-ascent constraints. We formulate training the two networks as a nested optimization problem and propose an alternating procedure that updates the primal and dual networks in turn, mitigating uncertainty in the multiplier distribution required for primal network training. We establish theoretical guarantees on the feasibility and optimality of the coupled networks. We numerically evaluate the framework on quadratic programs (QPs) with linear constraints and power allocation in wireless networks.

Figures & tables

Fig. 1 : Trajectories generated by (left) DA algorithm, (middle) constrained dual unrolling and (right) its unconstrained counterpart for a QP instance: (Top) primal trajectories toward the stationary point of the Lagrangian

{\mathcal{L}}(\cdot,\bm{\lambda};{\mathbf{z}})

, and (bottom) dual trajectories maximizing the dual function

g(\cdot\,;{\mathbf{x}}^{*}(\bm{\lambda}),{\mathbf{z}})

. The constrained networks generate descent trajectories emulating the DA algorithm, while the unconstrained models generate random trajectories and fail to hit the optimum of the dual function (bottom right).

Panel of Figure 2 — Fig. 2 : Constrained-optimization unrolling in ( 8 )–( 10 ). (Left) The primal network with $K=3$ unrolled layers approximates the stationary point ${\mathbf{x}}_{l}$ for a given $\bm{\lambda}_{l}$ . (Middle) The dual network with $L=3$ layers (red) calls the primal network (blue) at each layer, steering the iterates toward $\bm{\lambda}_{3}\approx\bm{\lambda}^{*}$ . (Right) The solution ${\mathbf{x}}^{*}$ is recovered by feeding $\bm{\lambda}_{3}$ into the primal network.

		constraint violation %			complementary slackness %
Method	obj. gap %	mean	p99	max	all	active	inactive
CDU (ours)	$4.28{\scriptstyle\,\pm 0.61}$	$\textbf{0.024}{\scriptstyle\,\pm 0.005}$	$\textbf{0.64}{\scriptstyle\,\pm 0.11}$	$\textbf{1.40}{\scriptstyle\,\pm 0.43}$	$4.08{\scriptstyle\,\pm 0.66}$	$\textbf{0.51}{\scriptstyle\,\pm 0.20}$	$3.57{\scriptstyle\,\pm 0.53}$
OSQP	$\textbf{0.39}{\scriptstyle\,\pm 0.31}$	$0.059{\scriptstyle\,\pm 0.021}$	$1.43{\scriptstyle\,\pm 0.43}$	$2.49{\scriptstyle\,\pm 0.78}$	$\textbf{0.87}{\scriptstyle\,\pm 0.29}$	$0.66{\scriptstyle\,\pm 0.30}$	$\textbf{0.21}{\scriptstyle\,\pm 0.08}$
SCS	$43.08{\scriptstyle\,\pm 9.50}$	$2.802{\scriptstyle\,\pm 0.240}$	$27.31{\scriptstyle\,\pm 2.36}$	$41.45{\scriptstyle\,\pm 7.87}$	–	–	–
DA	$71.69{\scriptstyle\,\pm 14.20}$	$3.010{\scriptstyle\,\pm 0.348}$	$30.26{\scriptstyle\,\pm 3.46}$	$46.41{\scriptstyle\,\pm 7.89}$	$49.29{\scriptstyle\,\pm 9.16}$	$46.57{\scriptstyle\,\pm 8.77}$	$2.72{\scriptstyle\,\pm 0.51}$
unconstrained ablation	$3.30{\scriptstyle\,\pm 0.70}$	$0.553{\scriptstyle\,\pm 0.072}$	$7.00{\scriptstyle\,\pm 0.94}$	$11.97{\scriptstyle\,\pm 2.60}$	$12.75{\scriptstyle\,\pm 2.03}$	$5.89{\scriptstyle\,\pm 1.11}$	$6.86{\scriptstyle\,\pm 1.03}$

TABLE I: Baseline Comparisons, all quantities are averaged over 100 test instances. The objective gap is

|{\mathbf{f}}({\mathbf{x}}_{L})-{\mathbf{f}}({\mathbf{x}}^{\star})|/(1+|{\mathbf{f}}({\mathbf{x}}^{\star})|)

; the violation is

\max\{\mathbf{0},{\mathbf{A}}{\mathbf{x}}_{L}-{\mathbf{b}}\}/(1+\max_{i}|b_{i}|)

by its mean, its 99th percentile and its maximum; and the complementary-slackness is

\sum_{i}|\lambda_{L,i}({\mathbf{A}}{\mathbf{x}}_{L}-{\mathbf{b}})_{i}|/|{\mathbf{f}}({\mathbf{x}}^{\star})|

over all constraints, over the constraints the method leaves active, and over the constraints it satisfies strictly and is reported for dual methods only. The reference

({\mathbf{x}}^{\star},\bm{\lambda}^{\star})

is ADMM in double precision to

10^{-9}

. Classical solvers are stopped at our network’s forward time on one RTX 3090.

Fig. 3 : Evolution of the maximum constraint violation and complementary slackness across the unrolled dual layers. The constrained model exhibits a consistent decrease in both quantities, whereas the unconstrained model shows larger oscillations and higher residuals at the final layers.

	$n=1000$	$n=2000$	$n=5000$	$n=10000$
CDU	34	37	144	498
OSQP	38	124	1166	8132
SCS	287	864	4368	25357
DA	418	867	$>$ 30s	$>$ 30s
CPLEX	1349	–	–	–

TABLE II: Mean wall-clock time (ms) to reach target accuracy. The target is set per instance to the error our network attains.

m/n=0.81

held constant across all problems.

Fig. 4 : OOD generalization. (Top) The objective gap, mean constraint violation, and complementary slackness as the problem size

n

increases while the constraint-to-variable ratio is fixed at

m/n=0.81

. (Bottom) The same metrics as the constraint ratio

m/n

increases at fixed problem size

n=5000

. OSQP and SCS are given

6\times

the inference-time budget of CDU. While OSQP attains the optimum in some regimes, its performance degrades more sharply as the problem size or number of constraints increases. CDU maintains more stable performance and degrades more gracefully overall.

Fig. 5 : Comparison of the mean power, rates and objective of three iterative algorithms: Constrained Dual Unrolling (ours), its unconstrained counterpart, and state-augmented dual ascent. The 6-layer outputs of dual unrolling methods are evenly distributed across the 600 iterations for clearer visual comparison. Our constrained dual unrolled networks achieve comparable performance to dual ascent in much fewer steps.

Fig. 6 : Descent Guarantees. (Left) The Lagrangian function across the unrolled primal layers, evaluated over test dataset of problem instances and multiplier samples. (Middle) The constraint violation across the unrolled dual layers. (Right) The complementary slackness

\bm{\lambda}_{L}^{\top}\max\{\mathbf{0},{\mathbf{f}}({\mathbf{x}}_{l})\}

across the unrolled dual layers. Our constrained model preserves a descent behavior over the unrolled layers, in contrast to the unconstrained model.

Fig. 7 : Robustness under OOD problems, varying (left) the constraint level

r_{\text{min}}

, (middle) the network size

n

, and (right) the percentage of constrained agents. The red dotted line represents the in-distribution scenario:

r_{\text{min}}=1.5,n=100,

and

50\%

constrained agents. The iterative methods are given

6\times

the inference-time budget of CDU. Across all three OOD shifts, CDU degrades more gradually than the unconstrained and supervised learning baselines and remains competitive with the iterative optimization methods.

Method	Violation $\leq 0.06$	Rate $\geq 2.98$	Both
Constrained Dual Unrolling	0.97	0.97	0.97
SA + Dual Ascent	5.67	–	–
SA + Dual Ascent (warm)	1.20	1.20	1.20
WMMSE-ADMM	5.81	2.91	5.81
WMMSE-ADMM (warm)	6.80	3.88	6.80

TABLE III : Mean wall-clock time (ms) to reach target accuracy. The target is set to the performance of our

6

-layer model. Warm-start runtimes include the

0.97

ms initialization cost. “–” indicates that the target is not reached.

Constrained Optimization

Constrained optimization finds the best decisions while satisfying equality, inequality, resource, safety, or fairness requirements.

See all 335 papers

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization
Apr 14, 2026Trinh Tran, Binh Nguyen, Truong X. NghiemLearning to Optimize Alternating Direction Method of Multipliers
Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA
NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees
Apr 30, 2026Bimol Nath Roy, Rahul Golder, MM Faruque HasanNeural Network Optimization Learning to Optimize
Texas A&M University
lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers
Sep 27, 2026Yuankun Chen, Zifei Nie, Kangyu Lin +2Differentiable Programming Nonconvex Optimization
School of Artificial Intelligence, Jilin University, China · Graduate School of Informatics, Kyoto University, JPN · Department of Civil and Systems Engineering, Johns Hopkins University, USA

Apr 14, 2026J/K move · Enter open · S save

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization

Trinh Tran, Binh Nguyen, Truong X. Nghiem

Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA

Learning to Optimize Alternating Direction Method of Multipliers Algorithm Unrolling Constrained Optimization

Open paper AskPDF arXiv

This paper presents HUANet, a constrained deep neural network architecture that unrolls the Alternating Direction Method of Multipliers (ADMM) into a trainable neural network for accelerating parametric constrained convex optimization. Existing end-to-end learning methods operate as black-box mappings from parameters to solutions, often without explicitly incorporating optimality principles or guaranteeing constraint satisfaction. To address these limitations, HUANet embeds a hard-constrained neural network within each unrolled ADMM iteration, where a differentiable correction stage enforces the affine equalities of the primal subproblem. Furthermore, we incorporate first-order optimality conditions into a self-supervised training loss to promote the convergence of the proposed unrolled algorithm. Extensive numerical experiments for benchmark optimization problems and a control application demonstrate and validate the effectiveness of HUANet in accelerating constrained convex optimization solving.

Abstract

Constrained Optimization

Explore similar work

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization

NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees

lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers