ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection
Organizations: Process Intelligence Research Group, Department of Chemical Engineering, Delft University of Technology
Abstract
Ensuring neural networks adhere to domain-specific constraints is crucial for addressing safety and trustworthiness while also enhancing inference accuracy. Despite the nonlinear nature of most real-world tasks, the majority of existing methods are limited to affine (equality) or convex (inequality) constraints. We introduce ENFORCE, a neural network architecture that uses an adaptive projection module (AdaNP) to enforce nonlinear equality and inequality constraints in the predictions up to a specified tolerance , and exactly in the affine-in- case. For affine constraint sets, we prove that the associated projection mapping is non-expansive (1-Lipschitz), ensuring stable gradient propagation. For nonlinear constraints, we establish local convergence analysis under standard regularity conditions. We evaluate ENFORCE on multiple tasks, including function fitting, real-world engineering case studies, and learning optimization problems. For the latter, we introduce a class of scalable optimization problems as a benchmark for nonlinear constrained learning. In the benchmarks, the predictions of our architecture satisfy nonlinear equality and inequality constraints up to a prescribed tolerance , while maintaining scalability with tractable computational complexity at training and inference time.
Figures & tables
| Method | MAPE [%] | Mean eq. [%] | Max eq. [%] | Inf. time [s] | |
|---|---|---|---|---|---|
| MLP | 0.339 0.083 | 0.994 0.003 | 1.47 0.33 | 17.13 3.94 | 0.002 0.000 |
| Soft ( ) | 0.944 0.143 | 0.972 0.002 | 1.55 0.16 | 7.77 0.40 | 0.002 0.000 |
| ENFORCE | 0.060 0.028 | 0.999 0.000 | 0.00 0.00 | 0.00 0.00 | 0.008 0.003 |
| Method | NRMSE [%] | Infeas. [%] | Infeas. [%] | Inf. time [s] | |
|---|---|---|---|---|---|
| MLP | 0.994 | 7.84 | 15.33 | 9.33 | 0.0002 |
| Soft ( ) | 0.995 | 6.86 | 4.00 | 0.00 | 0.0003 |
| ENFORCE | 0.995 | 7.16 | 0.00 | 0.00 | 0.020 |
| Method | Obj. value | Max eq. | Mean eq. | Inf. time [s] | Train. time [s] |
|---|---|---|---|---|---|
| IPOPT [ 36 ] | – | ||||
| MLP | |||||
| Soft ( ) | |||||
| DC3 [ 7 ] | |||||
| ENFORCE |
| Method | Obj. value | Max eq. | Mean eq. | Inf. time [s] | Train. time [min] |
|---|---|---|---|---|---|
| IPOPT [ 36 ] | – | ||||
| MLP | |||||
| Soft ( ) | |||||
| ENFORCE |
| Method | MSE | Max eq. | Mean eq. | Inf. time [s] | Train. time [s] |
|---|---|---|---|---|---|
| MLP | |||||
| KKT-Hardnet | |||||
| KKT-Hardnet-v2 | |||||
| ENFORCE |
| Method | MSE | Max eq. | Max ineq. | Inf. time [s] | Train. time [s] |
|---|---|---|---|---|---|
| MLP | |||||
| KKT-Hardnet | |||||
| KKT-Hardnet-v2 | |||||
| ENFORCE |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Constraints | ( ) | |||
|---|---|---|---|---|
| Variables | ( ) | |||
| Method | Metric | |||
| IPOPT | Obj. value | |||
| Max eq. | ||||
| Mean eq. | ||||
| Inf. time [s] |
| Constraints | ( ) | |||||
|---|---|---|---|---|---|---|
| Variables | ( ) | |||||
| Method | Metric | |||||
| IPOPT | Obj. value | |||||
| Max Eq. | ||||||
| Avg. Eq. | ||||||
| Inf. time [s] |
Constrained Optimization
Constrained optimization finds the best decisions while satisfying equality, inequality, resource, safety, or fairness requirements.
See all 363 papers