Neural Compression of Long ADMM Trajectory for Multiparametric Quadratic Program

Liang Wu; Honghui Zheng; Bo Yang; Xu Yang; Yilin Mo; Ján Drgoňa

math.OCOct 27, 2026

Neural Compression of Long ADMM Trajectory for Multiparametric Quadratic Program

Authors: Liang Wu, Honghui Zheng, Bo Yang, Xu Yang, Yilin Mo, Ján Drgoňa

Organizations: Johns Hopkins University · Tsinghua University

Abstract

Solving large-scale multiparametric quadratic programs (mpQPs) in real time often requires thousands of iterative optimization updates, creating a major computational bottleneck in model predictive control and learning-enabled systems. This paper proposes TQPNet, a self-supervised neural compression framework that learns to compress long ADMM optimization trajectories into a compact neural warm start followed by a few refinement iterations. We derive a low-dimensional ADMM scheme operating on the primal-dual variables $(x,β)$ and show that its iteration admits an equivalent ReLU-layer representation, termed ADMM(ReLU). Building on this insight, TQPNet combines a multilayer perceptron with ReLU activations (MLP(ReLU)) and a small number of unrolled ADMM(ReLU) refinement layers. The network is trained offline using an optimality-condition-informed residual loss, eliminating the need for solver-generated labels. The proposed residual-based framework enables both self-supervised training and online solution certification through adaptive ADMM(ReLU) refinement. Experiments on large-scale mpQPs and real-time MPC applications demonstrate that TQPNet achieves high accuracy, strong generalization, and substantially faster inference than conventional optimization-based methods.

Figures & tables

\min_{x\in{\mathbb{R}}^{n}}~\frac{1}{2}x^{\top}Qx+q(p)^{\top}x,~\text{s.t.}~Ax\geq c(p)

(1)

Table 1

Figure 1: Architecture of TQPNet based on the hybrid MLP(ReLU) - ADMM(ReLU) neural network.


$\displaystyle x^{k+1}$	$\displaystyle=\Big(Q+\rho A^{\top}A\Big)^{-1}\Big(-q(p)+\rho A^{\top}(z^{k}+u^{k})\Big)$	(3a)
$\displaystyle z^{k+1}$	$\displaystyle=\max\Big(c(p),Ax^{k+1}-u^{k}\Big)$	(3b)
$\displaystyle u^{k+1}$	$\displaystyle=u^{k}+z^{k+1}-Ax^{k+1}$	(3c)

Table 3


$\displaystyle y^{k+1}$	$\displaystyle=\argmin_{y}\frac{1}{2}\\|y-(V^{\top}x^{k}-u^{k})\\|_{2}^{2},~\text{s.t.}~Uy\geq c(p)$	(9a)
$\displaystyle x^{k+1}$	$\displaystyle=(Q+\rho D_{V})^{-1}\left(\rho V(y^{k+1}+u^{k})-q(p)\right)$	(9b)
$\displaystyle u^{k+1}$	$\displaystyle=u^{k}+y^{k+1}-V^{\top}x^{k+1}$	(9c)

Table 4

Panel of Figure 2 — Figure 2: Convergence statistics over $100$ random QPs for the low-dimensional ADMM-ReLU and the full-dimensional ADMM.

Panel of Figure 3 — Figure 3: Ablation of $K_{\text{train}}$ . (a) Test relative error vs. epoch at fixed $K_{\mathrm{eval}}{=}10$ ; (b) post-training KKT residual vs. refinement steps $K$ . $K_{\text{train}}{\leq}10$ fails to compress; $K_{\text{train}}{\geq}25$ stabilizes training and accelerates inference $1.7$ – $1.8\!\times$ over ADMM(ReLU) .

Figure 4: Relative solution error across parameter sets at

n{=}100

m{=}200

. LQPNet is more accurate in distribution, but the TQP–LQP gap shrinks under distribution shift.

Figure 5: (a) Solve latency. (b) Offline time.

Panel of Figure 6 — Figure 6: (a) 4096 robots concurrently controlled by TQPNet -based MPC in Isaac Gym. (b) Solve latency vs. number of parallel environments.

Figure 7: Solution quality vs. refinement steps

K

on the quadruped locomotion MPC. (a) Relative QP cost error vs. refinement steps

K

. (b) Constraint violation vs. refinement steps

K

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

\displaystyle\left(\begin{bmatrix}x_{1}\\ \beta_{1}\end{bmatrix},\cdots,\begin{bmatrix}x_{n_{b}}\\ \beta_{n_{b}}\end{bmatrix}\right)=\left(\mathrm{ADMM\text{-}ReLU}_{K,\rho}\big(\mathcal{N}(p_{1};\theta,M,L);p_{1}\big),\cdots,\mathrm{ADMM\text{-}ReLU}_{K,\rho}\big(\mathcal{N}(p_{n_{b}};\theta,M,L);p_{n_{b}}\big)\right)

Appendix

Algorithm 2 Unified offline training and online inference

$n$	$m$	$n_{p}$	$K{=}0$	$K{=}1$	$K{=}5$	$K{=}10$	$K{=}25$	$K{=}50$
10	20	4	$1.0\!\times\!10^{-2}$	$1.8\!\times\!10^{-1}$	$1.4\!\times\!10^{-2}$	$5.3\!\times\!10^{-3}$	$3.6\!\times\!10^{-3}$	$1.4\!\times\!10^{-2}$
100	200	20	$3.5\!\times\!10^{-1}$	$6.3\!\times\!10^{-1}$	$3.6\!\times\!10^{-1}$	$5.8\!\times\!10^{-1}$	$5.7\!\times\!10^{-2}$	$5.6\!\times\!10^{-2}$

Appendix

Table 1: Median test-set rel error after fixed

K_{\mathrm{eval}}{=}10

refinement at two scales. All entries trained for

2000

epochs.

Panel of Figure 8 — Figure 8: Ablation of $K_{\text{train}}$ at $n{=}10$ , $m{=}20$ , $n_{p}{=}4$ . (a) Test relative error vs. epoch at fixed $K_{\mathrm{eval}}{=}10$ ; (b) post-training KKT residual vs. refinement steps $K$ . $K_{\text{train}}{=}10$ already saturates, in contrast to the phase transition in Fig. 3 .

Method	$K^{\star}$	Time (ms)	Speedup	$\Delta K$
Full-dim ADMM	$>\!10^{4}$	$>\!50.5$	—	—
Low-dim ADMM(ReLU)	4729	10.689	1.0 $\times$	—
MLP only	3578	7.777	1.4 $\times$	1132
MLP + 1-layer ADMM(ReLU)	3692	8.032	1.3 $\times$	974
MLP + 5-layer ADMM(ReLU)	3728	7.751	1.4 $\times$	1000
MLP + 10-layer ADMM(ReLU)	3392	7.305	1.5 $\times$	1344

Appendix

Table 2: Iterations

K^{\star}

and wall-clock to KKT residual

\leq 10^{-7}

, median over

100

test mpQPs at

n{=}100

m{=}200

n_{p}{=}20

. Speedup is relative to cold low-dim ADMM(ReLU);

\Delta K

is the per-QP iteration reduction vs. that baseline. Full-dim ADMM does not reach the target within

K{=}10^{4}

Panel of Figure 9 — Figure 9: Wall-clock view of inference convergence: KKT residual vs. cumulative time per QP. TQPNet ’s wall-clock advantage over ADMM(ReLU) holds at both scales, with a larger proportional speedup at $n{=}10$ .

Set	$K=10$		$K=30$		$K=50$
	TQPNet	LQPNet	TQPNet	LQPNet	TQPNet	LQPNet
Train	5.8e-02	1.5e-02	5.5e-02	1.4e-02	5.2e-02	1.3e-02
Valid	5.9e-02	1.6e-02	5.6e-02	1.5e-02	5.3e-02	1.4e-02
Boundary	2.2e-01	1.0e-01	2.1e-01	9.7e-02	2.0e-01	9.1e-02
OOD	3.5e-01	1.9e-01	3.4e-01	1.9e-01	3.2e-01	1.8e-01

Appendix

Table 3: Generalizability: median relative error (

n=100

m=200

n_{p}=20

). Train on

p\sim\mathcal{U}[-1,1]

$n{+}m$	$n_{p}$	Speedup	RMSE	Cost	Latency	RMSE	Cost	Latency	RMSE	Cost	Latency	TQP	LQP	Latency
			OSQP			TQPNet ( $K{=}10$ )			LQPNet ( $K{=}10$ )			Offline		Explicit
60	3	$0.0\times$	0.221	33	0.03	0.221	33	1.31	0.221	33	1.23	65	24	0.57
140	4	$0.1\times$	0.218	45	0.07	0.218	45	1.30	0.218	45	1.31	65	25	4.11
315	6	$0.2\times$	0.217	63	0.21	0.217	63	1.23	0.217	63	1.20	65	35	—
560	8	$0.4\times$	0.263	124	0.59	0.264	124	1.31	0.263	124	1.33	71	43	—
875	10	$1.1\times$	0.217	99	1.38	0.217	99	1.27	0.217	99	1.23	84	65	—

Appendix

Table 4: Scalability on random MPC problems (

T{=}200

closed-loop steps with shared process noise

w\sim\mathcal{U}[-0.3,0.3]^{n_{s}}

). Online latency in ms, offline training time in s. TQPNet/LQPNet use a fixed network size: hidden width

M{=}512

and depth

L{=}4

across all scales.

Method	RMSE (m)	Latency med/max (ms)	Iter med/max
OSQP ( $\varepsilon{=}10^{-6}$ )	$0.0273$	$0.73\,/\,7.03$	$50\,/\,450$
OSQP ( $\varepsilon{=}10^{-7}$ )	$0.0273$	$0.75\,/\,13.48$	$50\,/\,950$
OSQP ( $\varepsilon{=}10^{-8}$ )	$0.0273$	$0.75\,/\,21.10$	$50\,/\,1500$
ADMM(ReLU) ( $K{=}50$ )	$0.0306$	$5.84\,/\,13.05$	$50\,/\,50$
TQPNet ( $K{=}10$ )	$0.0302$	$1.67\,/\,4.39$	$10\,/\,10$
TQPNet ( $K{=}20$ )	$0.0301$	$2.78\,/\,6.81$	$20\,/\,20$

Appendix

Table 5: Single-UAV lemniscate tracking (

N_{H}{=}25

n{=}100

m{=}300

T{=}600

). OSQP solved via raw C API with warm-started prob.update between steps. RMSE is comparable across methods. Latency and iteration counts report per-step median / maximum: OSQP exhibits worst-case latency

10

–

25\!\times\!

its median, whereas TQPNet at fixed

K

has bounded worst-case execution.

Panel of Figure 10 — Figure 10: Single-UAV lemniscate tracking ( $T{=}600$ ), showing the best TQPNet variant ( $K{=}30$ ) alongside cold-start ADMM(ReLU) and OSQP ( $\varepsilon{=}10^{-8}$ ). All methods overlap at trajectory resolution and tracking errors are within the same magnitude; full method comparison in Table 5 .

Quadratic Programming

Quadratic programming optimizes quadratic objectives under constraints, supporting applications from robotics and control to finance and machine learning.

See all 20 papers

Learning Over-Relaxation Policies for ADMM with Convergence Guarantees
Apr 29, 2026Junan Lin, Paul J. Goulart, Luca FurieriLearning to Optimize Alternating Direction Method of Multipliers
Department of Engineering Science, University of Oxford, OX1 3PJ Oxford, UK
WarpMPC: Large-Batch MPC on GPU via ADMM with Unrolled $LDL^\top$ Factorization
Jul 13, 2026Henrik Hose, Se Hwan Jeon, Charles Khazoom +2GPU Kernel Optimization GPU Acceleration
Institute for Data Science in Mechanical Engineering (DSME), RWTH Aachen University, Germany · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge MA, USA.
NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees
Apr 30, 2026Bimol Nath Roy, Rahul Golder, MM Faruque HasanNeural Network Optimization Learning to Optimize
Texas A&M University

Apr 29, 2026J/K move · Enter open · S save

Learning Over-Relaxation Policies for ADMM with Convergence Guarantees

Junan Lin, Paul J. Goulart, Luca Furieri

Department of Engineering Science, University of Oxford, OX1 3PJ Oxford, UK

Learning to Optimize Alternating Direction Method of Multipliers Constrained Optimization

Open paper AskPDF arXiv

The Alternating Direction Method of Multipliers (ADMM) is a widely used method for structured convex optimization, and its practical performance depends strongly on the choice of penalty and relaxation parameters. Motivated by settings such as Model Predictive Control (MPC), where one repeatedly solves related optimization problems with fixed structure and changing parameter values, we propose learning online updates of the relaxation parameter to improve average performance on problem classes of interest, while guaranteeing that asymptotic convergence is not compromised for the worst-case realization of such problems. This choice is computationally attractive in the Operator Splitting Quadratic Program (OSQP)-like architectures, since adapting relaxation does not trigger the matrix refactorizations associated with penalty updates. We establish convergence guarantees for ADMM with time-varying penalty and relaxation parameters under mild assumptions, and show on benchmark quadratic programs that the resulting learned policies improve both iteration count and wall-clock time on average over baseline OSQP.

Abstract

Quadratic Programming

Explore similar work

Learning Over-Relaxation Policies for ADMM with Convergence Guarantees

WarpMPC: Large-Batch MPC on GPU via ADMM with Unrolled LDL⊤LDL^\topLDL⊤ Factorization

NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees

WarpMPC: Large-Batch MPC on GPU via ADMM with Unrolled $LDL^\top$ Factorization