lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

Yuankun Chen; Zifei Nie; Kangyu Lin; Ján Drgoňa; Liang Wu

math.OCSep 27, 2026

lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

Authors: Yuankun Chen, Zifei Nie, Kangyu Lin, Ján Drgoňa, Liang Wu

Organizations: School of Artificial Intelligence, Jilin University, China · Graduate School of Informatics, Kyoto University, JPN · Department of Civil and Systems Engineering, Johns Hopkins University, USA

Abstract

Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.

Figures & tables

Solver	Target problem	Hard constraints	Matrix-free	Platform
lapanda (Ours)	General NLP	General	✓	C / MATLAB / Python
CasADi (2019)	General NLP	General	✗	C / MATLAB / Python
acados (2025)	General OCP	General	✗	C / MATLAB / Python
TurboMPC (2026)	General OCP	General	✗	Python
SafePDP (2021)	General OCP	General	✗	Python
PANDA (2026)	Composite NLP	Prox.-friendly	✓	MATLAB

Table 1: Comparison of representative differentiable optimization solvers.

Figure 1: Overview of the lapanda framework.

$n$	Forward time (ms)			Backward time (ms)			Memory (MB)
$n$	lapanda	Explicit	CasADi	lapanda	Explicit	CasADi	lapanda	Explicit	CasADi
100	8.23	–	36.50	4.04	4.22	3.04	1.78	5.01	17.55
200	14.37	–	108.19	5.89	10.23	17.05	2.06	7.78	29.39
500	49.75	–	496.03	14.36	51.26	245.99	3.12	25.32	84.70
1000	200.00	–	1968.78	37.36	218.21	2325.16	5.31	79.65	249.66

Table 2: Matched-accuracy comparison on the constrained Rosenbrock benchmark.

Figure 2: Normalized constraint values

\sqrt{x_{i}^{2}+x_{i+1}^{2}}/r_{i}(\theta)

for

n=200

Figure 3: Open-loop imitation learning results on different constrained OCPs.

Figure 4: Closed-loop imitation learning results including training loss and policy evolution.

OCP	Method	Fwd. time (ms)	Bwd. time (ms)	Total time (ms)	Memory (MB)
CartPole	lapanda	5.10	0.09	5.20	1.64
CartPole	SafePDP	22.06	9.65	31.71	7.71
Quadrotor	lapanda	1.20	0.24	1.45	1.79
	SafePDP	29.86	13.41	43.27	9.23
	TurboMPC-CPU	10.46	7.74	18.20	241.30
	TurboMPC-GPU	21.78	2.28	24.06	336.48

Table 3: Detailed performance comparison of different solvers on closed-loop OCP benchmarks.

Figure 5: Mobile robot trajectory alignment.

Obstacle	Method	Success	Fwd. (ms)	Bwd. (ms)	Memory (MB)
Circle	lapanda	✓	1.76	0.16	2.16
Circle	acados	✓	3.27	0.24	4.52
Rectangle	lapanda	✓	55.5	1.13	2.60
Rectangle	acados	✗	–	–	–

Table 4: Embedded deployment performance comparison.

Figure 6: Learning performance for the rectangular obstacle-avoidance imitation task.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Value
CasADi forward solver	IPOPT
CasADi sensitivity pipeline	SQP/qpOASES + sparse QR
Forward ALM/IPOPT tolerance	$10^{-3}$
Krylov tolerance	$10^{-2}$
lapanda/Explicit Krylov method	CG/MINRES
Maximum Krylov iterations	$200$

Appendix

Table 5: Solver settings for the constrained Rosenbrock benchmark.

Penalty $\rho$	Relative error $\searrow$	Cosine similarity	Violation $\searrow$	Backward time (ms) $\nearrow$
$1$	$5.59\times 10^{-1}$	$0.988404$	$2.47\times 10^{-1}$	$0.15$
$3$	$2.91\times 10^{-1}$	$0.998718$	$1.28\times 10^{-1}$	$0.15$
$10$	$1.14\times 10^{-1}$	$0.999793$	$4.67\times 10^{-2}$	$0.18$
$30$	$4.15\times 10^{-2}$	$0.999967$	$1.84\times 10^{-2}$	$0.22$
$100$	$1.34\times 10^{-2}$	$0.999994$	$6.74\times 10^{-3}$	$0.31$
$300$	$4.80\times 10^{-3}$	$0.999999$	$2.32\times 10^{-3}$	$0.53$

Appendix

Table 6: Penalty sweep for the

n=200

constrained Rosenbrock diagnostic.

$n$	Final penalty	Relative error	Cosine similarity	Violation	Backward time (ms)
$100$	$4.67\times 10^{2}$	$7.08\times 10^{-2}$	$0.999925$	$7.99\times 10^{-4}$	$0.15$
$200$	$4.59\times 10^{2}$	$9.42\times 10^{-2}$	$0.999910$	$5.75\times 10^{-4}$	$0.25$
$500$	$3.24\times 10^{2}$	$9.89\times 10^{-2}$	$0.999922$	$6.18\times 10^{-4}$	$0.62$
$1000$	$4.53\times 10^{2}$	$9.87\times 10^{-2}$	$0.999932$	$6.86\times 10^{-4}$	$1.54$

Appendix

Table 7: Balanced sensitivity computation using the forward-converged penalty.

$n$	Forward-converged			Direct backward refinement			Aligned subproblem refinement
$n$	$\rho$	Grad. err.	Bwd. (ms)	$\rho$	Grad. err.	Bwd. (ms)	$\rho$	Grad. err.	Refine solve (ms)	Bwd. (ms)
$100$	$4.12\times 10^{2}$	$7.07\times 10^{-2}$	$0.15$	$4.12\times 10^{3}$	$1.13\times 10^{-2}$	$0.26$	$4.12\times 10^{3}$	$9.29\times 10^{-3}$	$3.71$	$0.27$
$200$	$2.00\times 10^{2}$	$9.54\times 10^{-2}$	$0.23$	$2.00\times 10^{3}$	$2.42\times 10^{-2}$	$0.43$	$2.00\times 10^{3}$	$1.37\times 10^{-2}$	$4.17$	$0.44$
$500$	$2.27\times 10^{2}$	$9.97\times 10^{-2}$	$0.58$	$2.27\times 10^{3}$	$1.31\times 10^{-2}$	$1.16$	$2.27\times 10^{3}$	$1.31\times 10^{-2}$	$11.30$	$1.20$
$1000$	$2.36\times 10^{2}$	$9.96\times 10^{-2}$	$1.50$	$2.36\times 10^{3}$	$1.24\times 10^{-2}$	$3.04$	$2.36\times 10^{3}$	$1.25\times 10^{-2}$	$29.46$	$2.97$

Appendix

Table 8: Comparison of post-forward penalty-refinement strategies.

$n$	lapanda	Explicit KKT	CasADi
$100$	$0.95\;(1.16)$	$1.09\;(2.32)$	$0.02\;(0.03)$
$200$	$1.35\;(1.46)$	$0.68\;(0.74)$	$0.01\;(0.02)$
$500$	$1.30\;(1.35)$	$0.23\;(0.23)$	$0.43\;(0.43)$
$1000$	$1.24\;(1.26)$	$0.33\;(0.33)$	$<0.01\;(<0.01)$

Appendix

Table 9: Mean (maximum) relative gradient error (%) under the matched-accuracy protocol.

Protocol	OCP	Epochs	Learning rate	$\varepsilon_{\mathrm{in}}/\varepsilon_{\mathrm{ALM}}$	$(\rho_{0},\tau_{\rho})$	Max iter. (PANDA/ALM)	Warm start
Open-loop	CartPole	800	$2\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10,5)$	$1500/20$	Previous epoch
	Quadrotor	800	$2\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10,5)$	$1500/20$	Previous epoch
	Robot Arm	800	$2\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10^{3},5)$	$1500/20$	Previous epoch
Closed-loop	CartPole	500	$1\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10,5)$	$1500/20$	Previous MPC step
	Quadrotor	500	$1\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10,5)$	$1500/20$	Previous MPC step
	Robot Arm	500	$1\times 10^{-3}$	$10^{-3}/10^{-3}$	$(10^{3},5)$	$1500/20$	Previous MPC step

Appendix

Table 10: Learning configurations and lapanda hyperparameters for the OCP imitation experiments.

Method	Forward pass	Backward pass	Maximum iterations
SafePDP	IPOPT-based constrained OCP solve	Constrained auxiliary system (COC)	IPOPT: $3000$
TurboMPC-CPU	SQP-ADMM with admm_jax_loop_pcg	admm_jax_loop_pcg	SQP: $50$ ; ADMM: $1000$
TurboMPC-GPU	SQP-ADMM with admm_fused_cudss	direct_cudss_ffi	SQP: $50$ ; ADMM: $1000$

Appendix

Table 11: Method-specific solver configurations of the differentiable OCP baselines.

Figure 7: Imitation loss comparison between SafePDP-barrier and lapanda.

OCP	Metric	Open-loop epoch				Closed-loop epoch
OCP	Metric	$0$	$200$	$400$	$800$	$0$	$125$	$250$	$500$
CartPole	Rel. err. (%)	$0.012$	$0.004$	$0.012$	$0.010$	$0.002$	$0.006$	$0.006$	$0.007$
CartPole	Cos. sim.	$0.99999999$	$1.00000000$	$1.00000000$	$1.00000000$	$1.00000000$	$1.00000000$	$1.00000000$	$1.00000000$
Quadrotor	Rel. err. (%)	$0.048$	$0.286$	$0.363$	$1.306$	$0.058$	$0.137$	$0.134$	$0.076$
Quadrotor	Cos. sim.	$0.99999997$	$0.99999999$	$0.99999998$	$0.99999982$	$0.99999992$	$0.99999999$	$1.00000000$	$1.00000000$
Robot Arm	Rel. err. (%)	$8.459$	$0.076$	$0.688$	$0.027$	$1.178$	$4.572$	$0.084$	$0.136$

Appendix

Table 12: Sensitivity accuracy of lapanda over the OCP imitation-learning experiments.

Figure 8: Computation times at each MPC instant during a representative closed-loop rollout.

Figure 9: Mean solve time versus horizon length.

Figure 10: Planned trajectory for the circular-obstacle avoidance task on the embedded platform.

Task	Method	Hessian treatment	Maximum iterations	Forward tol.	ALM settings
Circle	lapanda	–	PANDA/ALM: $2000/100$	$10^{-1}/(2\times 10^{-3})$	$\rho_{0}=10^{4}$ , $\tau_{\rho}=10$
Circle	acados	EXACT + MIRROR	NLP/QP: $1000/200$	$2\times 10^{-3}$	–
Rectangle	lapanda	–	PANDA/ALM: $2000/100$	$10^{-3}/10^{-4}$	$\rho_{0}=10^{4}$ , $\tau_{\rho}=10$
Rectangle	acados	Gauss-Newton + MIRROR	NLP/QP: $1000/50$	$10^{-4}$	–

Appendix

Table 13: Solver settings for the embedded obstacle-avoidance experiments.

Figure 11: Solution trajectories and computation times for the smoothed rectangular-obstacle task.

Fig. 12: Software architecture and multi-platform workflow of lapanda.

Explore similar work

dOPT: Differentiating Conic Optimization via Geometric Reduction
Sep 27, 2026Fengyu Yang, Connor W. Magoon, Tyler Watts +1Convex Optimization Differentiable Optimization
Department of Mathematics University of North Carolina at Chapel Hill
NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees
Apr 30, 2026Bimol Nath Roy, Rahul Golder, MM Faruque HasanInequality Constraints Neural Network Optimization
Texas A&M University
Scalable Deep Unfolding of Conic Optimizers
Jun 11, 2026Alex Oshin, Rahul Vodeb Ghosh, Evangelos A. TheodorouDeep Unfolding Networks Convex Relaxation
Georgia Institute of Technology

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

dOPT: Differentiating Conic Optimization via Geometric Reduction

Fengyu Yang, Connor W. Magoon, Tyler Watts, Shahar Z. Kovalsky

Department of Mathematics University of North Carolina at Chapel Hill

Convex Optimization Differentiable Optimization Quadratic Programs Implicit Differentiation Natural Language Processing

Open paper AskPDF arXiv

Optimization layers enable the incorporation of structured constraints and decision problems into learning systems. Training such systems requires differentiating through the embedded optimization problem, which can be challenging for general conic programs. We introduce dOPT, a solver-agnostic framework that, rather than differentiating the full conic formulation, reduces it at a computed primal-dual solution to an equality-constrained quadratic program that preserves the reference solution and its first-order sensitivity. The reduction captures the local first- and second-order conic geometry relevant to differentiation and remains well defined at singular configurations. Computing solution derivatives then requires a single symmetric linear solve, independently of the forward solver. We derive explicit reductions for convex NLPs, QPs, SOCPs, and SDPs. Numerical experiments validate the computed gradients and show favorable backward-pass scalability, with substantial speedups over existing differentiable conic optimization methods as problem size increases.

Abstract

Explore similar work

dOPT: Differentiating Conic Optimization via Geometric Reduction

NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees

Scalable Deep Unfolding of Conic Optimizers