cs.LGOct 28, 2026

Convex-SKM-Net: Low-Latency Block Halfspace Corrections for Neural Convex Constraint Satisfaction

Authors: Haoyu Zhu, Xiaochen Zhang, Qingchun Hou

Organizations: Zhejiang University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Fast inference and strict constraint satisfaction are essential for learning-based decision-making. Existing methods often rely on constraint-specific projections or feasible-set parameterizations, while iterative repair can be costly. We introduce Convex-SKM-Net, a convex constraint-satisfaction layer that forms linearization halfspaces from residual and (sub)gradient oracles without constraint-specific inequality projectors. It applies fixed-depth dual projected-gradient descent (dual PGD) to minimum-displacement block problems using GPU-friendly matrix operations. We jointly learn the predictor and a positive-definite mobility, adapting the correction geometry to improve objective quality. We prove convergence of the outer iterates at fixed dual depth and linear convergence in distance under Slater's condition and bounded subgradients. All our test outputs on three second-order-cone model-predictive control (SOC-MPC) horizons and six power-system convex relaxations satisfy a 10−310^{-3} feasibility tolerance. Comparisons with the latest constraint-satisfaction methods show inference speedups of up to 144.3×144.3\times on SOC-MPC and 7.7×7.7\times on a convex quadratically constrained quadratic programming (QCQP) benchmark with joint training. On QCQP, the mean objective gap is 0.012%0.012\%, with full test feasibility at 10−410^{-4}. Learning mobility reduces the five-seed affine-task mean gap by 22.4%22.4\% and improves gaps in matched power-system comparisons.

Figures & tables

Figure 1: Repair in reduced space with fixed (blue) or learned (red) mobility. The inset shows block reselection under one linearization. Trajectories are schematic.
OperationLocal objectiveComputation
Single-row SKMMinimum displacement into one halfspaceClosed-form projection
Equality-LSBoundary fit on all selected rowsPseudoinverse or linear solve
Block-pinvBoundary fit on violated rowsPseudoinverse
Violation-GD12∥[UBΔ+s]+∥2\tfrac{1}{2}\|[U_{B}\Delta+s]_{+}\|^{2}Primal gradient steps
Dual PGD (ours)Minimum displacement under inequalitiesDual projected-gradient steps
Table 1: Local operations on selected linearizations. Boundary fitting selects a minimum-norm least-squares displacement.
MethodGap (%)Feas. (%)Time (ms)
Gurobi0.00010020.809
MOSEK0.0001004.974
DC35.9091000.566
Π\Pi net0.13010075.952
SnareNet2.91310058.567
Ours0.1921000.941
Table 2: SOC-MPC: 1,0001{,}000 tests per horizon; mean signed Gap and mean single-instance time. Bold marks best neural Gap/time. Gurobi/MOSEK use single-thread CPUs. —: SnareNet training exceeded 24 GB on RTX 4090 with default XLA.
Figure 2: Fixed-depth SOC-MPC: columns H=20,40,80H=20,40,80 ; mean signed Gap (top) and maximum original violation (bottom) versus mean latency. Each point covers 1,0001{,}000 tests at tolerance 10−310^{-3} . Other methods: Table 2 settings. Full values: Appendix D.1 .
Method57 buses118 buses300 buses
 Gap (%)Feas. (%)Time (ms)Gap (%)Feas. (%)Time (ms)Gap (%)Feas. (%)Time (ms)
(a) SOCP
MOSEK0.00010031.2340.000100108.9780.000100410.376
Π\Pi net0.059100203.3660.39698.52169.5030.10699.220934.989
FSNet0.05310026.5560.39597.0355.9680.1128.53199.870
SnareNet0.11610048.8620.53088.256452.791———
Table 3: SOCP/SDP: mean Gap and mean single-instance time, including failures. External postprocessors share full FT post’s raw predictions. Bold compares our variants, including rounded ties; —: case-300 SnareNet was too slow to evaluate fully.
 DC3JAXoptSnareNetΠ\Pi net ( K=50K=50 )Ours (identity)Ours (learned)
Gap (%)7.8940.3886.1690.3310.2450.190
Feas. (%)99.824100100100100100
Time (ms)3.0437.2263.0410.6420.4100.523
Table 4: Affine task: five-seed means, tolerance 10−310^{-3} . Single-instance latency averages seed timing medians. Identity/learned refers to mobility. Full statistics: Appendix D.2 .
Figure 3: Affine-task fixed-depth comparison: mean nonnegative Gap (left) and maximum violation over 5,1205{,}120 outputs (right) versus latency; both axes are logarithmic. Tolerance is 10−310^{-3} . Gap and latency average five seeds. Full values: Appendix D.2 .
MethodGap (%)Time (ms)
DC366.9180.600
SnareNet0.15918.963
Π\Pi net ( K=200K=200 )0.0406.126
Ours0.0122.469
Table 5: QCQP: all seven configurations achieve 100%100\% feasibility at 10−410^{-4} . Mean nonnegative Gap and prediction-plus-repair latency, averaged over three complete test passes. Bold marks best Gap/time per mode, including rounded ties.
Figure 4: QCQP fixed-depth sweep: joint/post columns, mean Gap/maximum original violation rows, and mean latency; all 833833 tests per point at tolerance 10−410^{-4} . Axes are logarithmic except joint Gap on [0,0.01%][0,0.01\%] . Full values: Appendix D.7 .
MethodSquared affineLog-sum-expRotated SOC
Single row (frozen)29.745 /107.95634.383 /100.01518.444 /66.106
M-SSP16.916 /41.32223.284 /67.21911.842 /73.798
Violation-GD4.076 /5.0043.438 /4.2572.790 /3.585
BPP-1654.299 /43.17533.791 /28.20429.333 /24.068
Dual PGD (ours)3.470 /4.7593.047 /3.8222.363 /3.201
Table 6: Median time to feasibility (ms), small/medium per cell, five successful instances each. Small/medium (n,neq,m)=(256,32,2048)/(512,64,4096)(n,n_{\rm eq},m)=(256,32,2048)/(512,64,4096) .
Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

TaskTrainingValidationTest
SOC-MPC, each HH8,0001,0001,000
Affine-constrained nonconvex optimization7,9521,0241,024
Convex QCQP8,334833833
ACOPF, 57 buses2,000200200
ACOPF, 118 buses4,000400400
ACOPF, 300 buses10,0001,0001,000
Appendix
Table 7: Training, validation, and test sizes. SOC-MPC data are generated separately for each horizon. Each ACOPF relaxation uses the listed sizes.
HHStepTrain stepsPenaltyTest cap
200.10.150.5500
408.75876×10−48.75876\times 10^{-4}2058000
803.54856×10−63.54856\times 10^{-6}5512800
Appendix
Table 8: SOC-MPC DC3 correction settings.
BusesTrain outerTrain innerTrain dualTest innerTest dual
5724448
118210101020
300220202040
Appendix
Table 9: ACOPF train and test correction budgets.
Primitiveg(y)g(y)Subgradient
Affinea⊤y+ca^{\top}y+caa
Squared affine12(a⊤y+c)2−b\frac{1}{2}(a^{\top}y+c)^{2}-b(a⊤y+c)a(a^{\top}y+c)a
Log-sum-explog⁡∑jepj⊤y+βj−b\log\sum_{j}e^{p_{j}^{\top}y+\beta_{j}}-b∑jπjpj\sum_{j}\pi_{j}p_{j}
Norm minus affine∥My+c∥−t⊤y−b\|My+c\|-t^{\top}y-bM⊤u−tM^{\top}u-t
Squared norm∥Fy+c∥2−b\|Fy+c\|^{2}-b2F⊤(Fy+c)2F^{\top}(Fy+c)
PSD−λmin⁡(W(y))-\lambda_{\min}(W(y))(−v∗Wjv)j(-v^{*}W_{j}v)_{j}
Appendix
Table 10: Decision-space residuals and selected subgradients.
HHMobilityHead rateGap (%)Time (ms)
20Identityfixed0.199390.828
20Learned, common-rate control3×10−43\times 10^{-4}0.339241.308
20Learned, main3×10−53\times 10^{-5}0.192000.941
40Identityfixed0.775890.947
40Learned, main3×10−43\times 10^{-4}0.235611.061
80Identityfixed0.947901.100
Appendix
Table 11: SOC-MPC mobility controls. Gaps retain their signs; time is mean single-instance latency.
HHFixed KKGap (%)Max viol.Feas. (%)Time (ms)
201023.589776.13×10−26.13\times 10^{-2}44.12.281
20255.414823.65×10−23.65\times 10^{-2}14.85.055
20500.549721.29×10−21.29\times 10^{-2}9.59.841
201000.145792.75×10−32.75\times 10^{-3}49.919.278
202000.111071.36×10−31.36\times 10^{-3}97.838.747
204000.130166.21×10−46.21\times 10^{-4}100.075.952
Appendix
Table 12: Π\Pi net fixed test depths on SOC-MPC (1,000 cases per horizon). Gap is the signed mean per-case relative objective difference; Max viol. is the maximum original constraint violation over all cases; time averages all instances. Feasibility uses tolerance 10−310^{-3} . Rows K=400K=400 at all three horizons are the main-table results. Each depth is evaluated separately; the original K=(1000,1000,400)K=(1000,1000,400) rows reuse the archived runs on the same checkpoints.
HHStoppingGap (%)Max viol.Feas. (%)Time (ms)
20Original0.019047.291×10−37.291\times 10^{-3}77.920.534
20Calibrated2.913212.642×10−42.642\times 10^{-4}100.058.567
40Original0.067864.385×10−34.385\times 10^{-3}79.261.758
40Calibrated1.762192.418×10−42.418\times 10^{-4}100.0175.515
Appendix
Table 13: SnareNet SOC-MPC stopping controls, using one checkpoint and all 1,0001{,}000 test cases per horizon. Original settings use internal threshold 10−310^{-3} and cap 100100 ; validation-calibrated settings use 10−410^{-4} and 500500 and appear in the main table. Gap retains its sign; external feasibility always uses 10−310^{-3} . Time is mean single-instance latency.
HHMethodGap (%)Feas. (%)Time (ms)
20Gurobi-4.50e-510020.809
20MOSEK-1.07e-61004.974
20DC35.908961000.566
20Π\Pi net0.1301610075.952
20SnareNet2.9132110058.567
20Ours0.192001000.941
Appendix
Table 14: SOC-MPC main-table results at full precision; settings match Table 2 .
MethodGap (%)Max viol.Feas. (%)Single time (ms)Batch time (ms)
DC37.893930.3093899.8243.0433.809
JAXopt0.388419.999×10−49.999\times 10^{-4}1007.226111.679
SnareNet6.168979.02×10−49.02\times 10^{-4}1003.04161.441
Π\Pi net ( K=50K=50 )0.331171.02×10−41.02\times 10^{-4}1000.6425.370
Ours (identity mobility)0.245189.999×10−49.999\times 10^{-4}1000.4101.864
Ours (learned mobility)0.190329.995×10−49.995\times 10^{-4}1000.5234.384
Appendix
Table 15: Complete affine-task results, including maximum original violation and batch timing. Five seeds each test 1,0241{,}024 instances; violation takes the maximum, other columns average seeds.
Test modeGap (%)Max viol.Feas. (%)Single time (ms)Batch time (ms)
K=1K=129.774612.381×1002.381\times 10^{0}75.840.0790.410
K=2K=24.142834.083×1004.083\times 10^{0}00.1060.554
K=5K=50.025291.738×1001.738\times 10^{0}00.1580.845
K=10K=100.252402.711×10−12.711\times 10^{-1}00.2111.335
K=15K=150.358117.469×10−27.469\times 10^{-2}0.0590.2661.833
K=20K=200.328091.794×10−21.794\times 10^{-2}0.020.3122.317
Appendix
Table 16: Fixed-depth and early-stop Π\Pi net on the affine task. Gap and latency average five seeds; Max viol. is the maximum across all 5,1205{,}120 outputs. Feasibility uses 10−310^{-3} , and Gap includes infeasible outputs.
MobilityTrain tolTest tolGap (%)Max viol.Single time (ms)Batch time (ms)
Identity10−310^{-3}10−310^{-3}0.245189.999×10−49.999\times 10^{-4}0.4101.864
Learned10−310^{-3}10−310^{-3}0.190329.995×10−49.995\times 10^{-4}0.5234.384
Identity10−610^{-6}10−310^{-3}0.246069.99×10−49.99\times 10^{-4}0.3411.678
Identity10−610^{-6}10−610^{-6}0.246369.96×10−79.96\times 10^{-7}0.8332.918
Learned10−610^{-6}10−310^{-3}0.191679.99×10−49.99\times 10^{-4}0.5434.908
Learned10−610^{-6}10−610^{-6}0.192159.999×10−79.999\times 10^{-7}3.6059.267
Appendix
Table 17: Training and test tolerance controls for our method on nonconvex optimization with affine constraints; all rows are 100% feasible at their test tolerance.
EpochsAdaRelGap (%)Max viol.Single time (ms)Batch time (ms)
50Off13.06420±2.3694713.06420\pm 2.369479.61×10−49.61\times 10^{-4}3.52562.326
50On, main6.16897±1.723306.16897\pm 1.723309.02×10−49.02\times 10^{-4}3.04161.441
100Off8.72578±1.798898.72578\pm 1.798898.28×10−48.28\times 10^{-4}4.46565.159
100On4.72325±1.431254.72325\pm 1.431259.91×10−49.91\times 10^{-4}2.94358.483
Appendix
Table 18: SnareNet relaxation and training-duration controls. Each row aggregates five seeds, each testing all 1,0241{,}024 instances. Gap gives mean ±\pm sample standard deviation. Every configuration is 100%100\% feasible at 10−310^{-3} . The 100100 -epoch runs retain the 2525 -epoch relaxation decay.
  Mean gap (%)
  PostJoint
Relax.BusesFullHeadIdentity fullIdentity headLearned head
SOCP570.051150.051420.044300.050130.05010
SOCP1180.418780.183610.405760.179310.16658
SOCP3000.098160.041230.099510.041300.03676
SDP570.074900.064200.070790.064080.06401
Appendix
Table 19: ACOPF control mean gaps (%). All models are 100% feasible at 10−310^{-3} ; sample sizes are 200, 400, and 1,000 for 57, 118, and 300 buses.
TaskMethodFailed / TotalAt capMax eq.Max ineq.
SOCP118Π\Pi net6/40062.408×10−122.408\times 10^{-12}1.008×10−21.008\times 10^{-2}
SOCP118FSNet12/400121.855×10−31.855\times 10^{-3}1.221×10−21.221\times 10^{-2}
SOCP118SnareNet47/400471.398×10−31.398\times 10^{-3}3.522×10−23.522\times 10^{-2}
SDP57SnareNet25/200259.795×10−49.795\times 10^{-4}2.278×10−32.278\times 10^{-3}
SDP118SnareNet295/4002951.501×10−31.501\times 10^{-3}4.317×10−24.317\times 10^{-2}
SOCP300FSNet915/1,0009154.780×10−34.780\times 10^{-3}8.692×10−38.692\times 10^{-3}
Appendix
Table 20: Infeasible ACOPF baseline outputs and iteration limits. “At cap” counts only infeasible outputs; maximum residuals include all test outputs.
CaseStepEpochsFeasible testsMax ineq. violation
SOCP573×10−113\times 10^{-11}20000/2000/2000.19304
SOCP1183×10−143\times 10^{-14}20000/4000/400189.69979
SDP573×10−113\times 10^{-11}20000/2000/2000.13667
SDP1183×10−143\times 10^{-14}20000/4000/40094.37722
Appendix
Table 21: DC3 tests after step calibration and increased penalty; maximum violation uses native residuals.
Relax.BusesMax equalityMax inequalityBuild time (ms)
SOCP571.590×10−71.590\times 10^{-7}2.557×10−72.557\times 10^{-7}5.378
SOCP1187.072×10−47.072\times 10^{-4}7.370×10−47.370\times 10^{-4}12.333
SOCP3002.135×10−42.135\times 10^{-4}1.216×10−41.216\times 10^{-4}27.268
SDP576.393×10−66.393\times 10^{-6}3.777×10−73.777\times 10^{-7}16.649
SDP1185.205×10−65.205\times 10^{-6}4.890×10−74.890\times 10^{-7}53.069
SDP3009.826×10−49.826\times 10^{-4}1.195×10−41.195\times 10^{-4}331.634
Appendix
Table 22: MOSEK native-residual audit and mean model-construction time over 200/400/1,000 tests.
MethodSuccessStop time (ms)Mean outerMax viol.Paired ratio
Squared affine
Single-row SKM5/575.606589.09.952e-0421.79
Single-row frozen5/529.745117.89.516e-049.23
Equality-LS/pinv0/51045.233 †8.01.887e+02–
Equality-LS/lstsq0/51044.895 †8.08.198e+02–
Block-pinv5/5547.1145.48.586e-04152.52
Appendix
Table 23: Small synthetic systems (n,neq,m)=(256,32,2048)(n,n_{\rm eq},m)=(256,32,2048) . Dagger: time includes failed outputs.
MethodSuccessStop time (ms)Mean outerMax viol.Paired ratio
Squared affine
Single-row SKM0/5182.660 †1024.01.092e-02–
Single-row frozen5/5107.956242.29.847e-0421.93
Equality-LS/pinv0/51066.688 †8.02.561e+02–
Equality-LS/lstsq0/51069.191 †8.02.539e+02–
Block-pinv5/5642.8805.66.650e-04136.56
Appendix
Table 24: Medium synthetic systems (n,neq,m)=(512,64,4096)(n,n_{\rm eq},m)=(512,64,4096) . Dagger: time includes failed outputs.
  Time (ms)  
ScaleFamilyM-SSPDual PGDRatioM-SSP outer
SmallSquared affine16.9163.6174.88140.2
 Log-sum-exp23.2843.2577.43160.2
 Rotated SOC11.8422.3155.1195.0
MediumSquared affine41.3224.8989.44297.6
 Log-sum-exp67.2194.30315.90362.2
Appendix
Table 25: M-SSP versus same-device dual PGD. Both succeed on all five seeds per group. Ratios are medians of paired instance ratios.
  Absolute gap  
ModeMethodGeom. meanMaxMax ineq.Max eq.
JointDC38.772524213.86583139.980×10−79.980\times 10^{-7}1.293×10−141.293\times 10^{-14}
JointSnareNet0.02025050.07492862.16×10−52.16\times 10^{-5}3.33×10−53.33\times 10^{-5}
JointΠ\Pi net ( K=200K=200 )0.001940930.06163851.347×10−81.347\times 10^{-8}1.205×10−141.205\times 10^{-14}
JointOurs0.001309750.01659731.00×10−61.00\times 10^{-6}8.95×10−158.95\times 10^{-15}
PostSnareNet0.5006541.0380131.42×10−51.42\times 10^{-5}1.85×10−51.85\times 10^{-5}
Appendix
Table 26: Convex QCQP absolute objective gaps and maximum original residuals. Absolute gap is fx(y)−fxreff_{x}(y)-f_{x}^{\rm ref} ; its geometric mean and maximum cover all test instances. All gaps are positive. All configurations are 100%100\% feasible at 10−410^{-4} .
EpochsTest correctionGap (%)Feas. (%)Max viol.Time (ms)
1,0001{,}000Released78.1994597.7195.080×10−15.080\times 10^{-1}0.372
1,5001{,}500Released66.9093096.2791.752×1001.752\times 10^{0}0.378
1,5001{,}500Calibrated66.918451009.980×10−79.980\times 10^{-7}0.600
Appendix
Table 27: DC3 training-budget and test-correction controls. Each row covers all 833833 test inputs. Feasibility uses 10−410^{-4} original-residual tolerance; Gap and latency include failures. Calibration uses validation data only. The last row is the main-table configuration.
ModeKKGap (%)Max viol.Feas. (%)Time (ms)
Joint50.000001.996×1001.996\times 10^{0}00.330
 100.000325.884×10−15.884\times 10^{-1}00.465
 200.064951.119×10−11.119\times 10^{-1}00.753
 300.087133.960×10−23.960\times 10^{-2}01.057
 500.042748.110×10−38.110\times 10^{-3}01.639
 750.040318.920×10−48.920\times 10^{-4}58.5832.422
Appendix
Table 28: Complete fixed-depth Π\Pi net sweep on convex QCQP. Each row tests all 833833 instances. Max viol. is the largest original residual; feasibility uses 10−410^{-4} . Gap includes infeasible outputs. The joint K=200K=200 and post K=400K=400 rows also appear in the main table.
More in this topic

Constrained Optimization

Constrained optimization finds the best decisions while satisfying equality, inequality, resource, safety, or fairness requirements.

See all 342 papers

Explore similar work

CardsList
Sep 24, 2026

AT-SKM-Net: An Accelerated Trainable Sampling Kaczmarz-Motzkin Framework for Linear Hard-Constraint Feasibility on Dynamic Graphs

Graph-structured optimization with linear constraints is fundamental to critical infrastructure but faces scalability limits due to massive strict hard constraints and high dimensionality. While recent projection-based methods such as Trainable Sampling Kaczmarz-Motzkin Net (T-SKM-Net) guarantee feasibility, they face high computational costs in dynamic environments by processing the entire constraint set and requiring expensive matrix factorizations. To bridge this gap, we propose the Accelerated Trainable-SKM (AT-SKM) Net framework. To concentrate computation on the active constraints and eliminate redundant calculations, we introduce a hybrid sampling strategy guided by a topology-aware heterogeneous GNN model. To efficiently handle topological shifts in graph-based constraints, we employ a Cholesky Update mechanism that theoretically reduces the equality projection complexity from O(N^3) to O(N^2) under low-rank perturbations. Experiments on random geometric graphs, N-1 Security-Constrained DC-OPF, and minimum-cost gas transport problem demonstrate that AT-SKM reduces iteration counts by up to 85% and achieves 2.95x-7.29x SKM layer speedups, while maintaining zero constraint violations.

Xiaochen Zhang, Haoyu Zhu, Yao Zhang +1

Zhejiang University

Constrained Optimization
AskPDF
Oct 3, 2026

Only Project Once: Projection-Adaptive Loss for Exact Constraint Satisfaction

Precise constraint satisfaction is a prerequisite to deploying learned models in many areas, motivating methods that repair raw neural predictions with a repair procedure. Current methods unroll multiple repair steps in training and softly penalize constraint violations that remain after the unroll. This is compute- and memory-intensive, lacks robustness when the repair fails to converge, and surrenders most of the constraint satisfaction work to the repair. Our central finding is that, contrary to common practice, a single detached projection step suffices in training. We accomplish this with a Projection-Adaptive Loss (PAL), which uses the constraint residual after this single step to adaptively weigh constraint penalties on the raw prediction. In experiments, PAL is the only method that retains virtually perfect feasibility on extremely nonlinear constraints, and matches or outperforms current methods on synthetic and engineering benchmarks. Because it only requires a single detached projection step, PAL trains 2.5x faster than the canonical repair-based method (DC3) on its own ACOPF benchmark. PAL can also be trained when constraints are expensive to evaluate (e.g., via neural surrogates), a setting where current unrolled methods are memory-intractable.

Tim Aebersold, Soheyl Massoudi, Mark Fuge

ETH Zürich

Constraint SatisfactionConstrained Optimization
AskPDF
Jun 11, 2026

Neural Slack Variables for Shape Constraints

Enforcing functional inequality constraints such as monotonicity and convexity in neural networks is a fundamental challenge in many industrial and scientific applications. Classical one-sided penalty methods, along with primal-dual methods gated by complementary slackness, provide constraint gradients only at violated locations, resulting in fragile satisfaction. Architectures that guarantee feasibility by construction, on the other hand, remain largely limited to elementary cases and impose additional inductive biases. We introduce neural slack variables, a deep learning native primal-side approach that converts constraint enforcement into a regression problem by coupling the primary network with a jointly learned auxiliary network. The auxiliary network serves as a valid target for the primary network's constraint quantities, inducing feasibility and regularity. Neural slack variables achieve zero measured violations on dense-grid monotonicity and convexity test cases, where penalty and primal-dual baselines leave residual violations, and enable arbitrage-free learning of volatility surfaces, an open industrial challenge in quantitative finance.

Ruben Wiedemann, Antoine Jacquier, Lukas Gonon

Imperial College London London, UK · University of St. Gallen St. Gallen, Switzerland

Monotonicity ConstraintsConstrained Optimization
AskPDF