Learned Preconditioning for a Primal-Dual Interior-Point Method
Organizations: Department of Mathematics, University of California, Los Angeles · School of Data, Mathematical, and Statistical Sciences, University of Central Florida
Abstract
Interior-point methods (IPMs) are among the most widely used algorithms for constrained optimization, yet their Newton-based search directions require costly second-order information and large linear-system solves. Learning to optimize offers cheaper updates learned from data, but the singular behavior of logarithmic barriers near constraint boundaries makes IPMs highly sensitive to perturbations, complicating both warm starting and learning reliable updates. We introduce pdLIP, an IPM for smooth nonlinear programs that integrates learned preconditioning with pdProj, an all-shifted primal-dual projected-search IPM. A shared coordinate-wise recurrent network predicts a positive diagonal preconditioner that scales the right-hand side of the reduced Newton system for the primal step, and the remaining slack and multiplier directions are recovered analytically. The learned iterations avoid Hessian evaluations and Newton-system solves, using only first-order and coordinate-wise operations amenable to GPU parallelization. Training is self-supervised, with a loss based on a penalty-barrier merit function and the residual of perturbed optimality conditions, requiring neither target directions nor precomputed solutions. Primal and dual shifts mitigate the barrier's sensitivity to perturbations near constraint boundaries, enabling effective warm starting. Across four classes of 200-dimensional convex and nonconvex constrained problems, pdLIP warm starts reduce pdProj refinement iterations by 63-67% compared with cold starts at the same KKT residual tolerance of , with negligible warm-start generation cost relative to the subsequent pdProj solve. Improvements persist on box-constrained QPs with 1000 variables and extend to applications including portfolio optimization, support vector machines, and a nonlinear control example.
Figures & tables
| Class | IPOPT | Cold pdProj | pdLIP + pdProj | Reduction (Iter. / Time) | |||
|---|---|---|---|---|---|---|---|
| Iter. | Iter. / Time | WS Cost | Iter. / Time | Total Time | pdLIP | IPM-LSTM | |
| C-RHS | s | ms | s | s | |||
| C-ALL | s | ms | s | s | |||
| NC-RHS | s | ms | s | s | |||
| NC-ALL | s | ms | s | s | |||
| Dim. | IPOPT | Cold pdProj | pdLIP + pdProj | Reduction | ||
|---|---|---|---|---|---|---|
| Iter. | Iter. / Time | WS Cost | Iter. / Time | Total Time | Iter. / Time | |
| ms | ms | ms | ms | |||
| s | ms | s | s | |||
| s | ms | s | s | |||
| Problem | IPOPT | Cold pdProj | pdLIP + pdProj | Reduction | |||
|---|---|---|---|---|---|---|---|
| Iter. | Iter. / Time | WS Cost | Iter. / Time | Total Time | Iter. / Time | ||
| Portfolio | ms | ms | ms | ms | |||
| Portfolio | s | ms | s | s | |||
| SVM | s | ms | s | s | |||
| SVM | s | ms | s | s | |||
| Problem | IPOPT | Cold pdProj | pdLIP + pdProj | Reduction | ||
| Iter. | Iter. / Time | WS Cost | Iter. / Time | Total Time | Iter. / Time | |
| Quadrotor | s | ms | s | s | ||
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | LR | Mean | Mean | ||||
|---|---|---|---|---|---|---|---|
| Rank | LR | Mean | Mean | Combined | |||
|---|---|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| Rank | Mean | Mean | |||
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| 5 |
| Problem Class | Method | Iter. | Grad. Calls | Func. Calls |
|---|---|---|---|---|
| Convex QP RHS | IPOPT | |||
| Cold pdProj | ||||
| Warm-started pdProj | ||||
| Convex QP ALL | IPOPT | |||
| Cold pdProj | ||||
| Warm-started pdProj |
| Problem Class | Method | Iter. | Grad. Calls | Func. Calls |
|---|---|---|---|---|
| Convex QP RHS | IPOPT | |||
| Cold pdProj | ||||
| Warm-started pdProj | ||||
| Iteration reduction | – | – | ||
| Convex QP ALL | IPOPT | |||
| Cold pdProj |
| Problem Class | Cold Time | WS Cost | Total Time | Time Red. |
|---|---|---|---|---|
| Convex QP RHS | s | ms | ms | |
| Convex QP ALL | s | ms | ms | |
| Nonconvex RHS | s | ms | ms | |
| Nonconvex ALL | s | ms | ms |
| Problem | Method | Iter. | Grad. Calls | Func. Calls |
|---|---|---|---|---|
| IPOPT | ||||
| Cold pdProj | ||||
| Warm-started pdProj | ||||
| IPOPT | ||||
| Cold pdProj | ||||
| Warm-started pdProj |
| Problem | Method | Iter. | Grad. Calls | Func. Calls | |
|---|---|---|---|---|---|
| Portfolio | IPOPT | ||||
| Cold pdProj | |||||
| Warm pdProj | |||||
| Portfolio | IPOPT | ||||
| Cold pdProj | |||||
| Warm pdProj |
| Problem | Method | Iter. | Grad. Calls | Func. Calls |
|---|---|---|---|---|
| Quadrotor | IPOPT | |||
| Cold pdProj | ||||
| Warm-started pdProj |
| Problem | Method | Converged | Avg. Runtime |
|---|---|---|---|
| Box QP, | pdLIP | ms | |
| IPOPT | ms | ||
| Constrained QP RHS, | pdLIP | ms | |
| IPOPT | ms | ||
| Constrained QP ALL, | pdLIP | ms | |
| IPOPT | ms |
| Problem | Method | Converged | Avg. Runtime |
|---|---|---|---|
| Convex RHS | pdLIP | ms | |
| IPOPT | ms | ||
| Convex ALL | pdLIP | ms | |
| IPOPT | ms | ||
| Nonconvex RHS | pdLIP | ms | |
| IPOPT | ms |