Ensuring neural networks adhere to domain-specific constraints is crucial for addressing safety and trustworthiness while also enhancing inference accuracy. Despite the nonlinear nature of most real-world tasks, the majority of existing methods are limited to affine (equality) or convex (inequality) constraints. We introduce ENFORCE, a neural network architecture that uses an adaptive projection module (AdaNP) to enforce nonlinear equality and inequality constraints in the predictions up to a specified tolerance ε, and exactly in the affine-in-y case. For affine constraint sets, we prove that the associated projection mapping is non-expansive (1-Lipschitz), ensuring stable gradient propagation. For nonlinear constraints, we establish local convergence analysis under standard regularity conditions. We evaluate ENFORCE on multiple tasks, including function fitting, real-world engineering case studies, and learning optimization problems. For the latter, we introduce a class of scalable optimization problems as a benchmark for nonlinear constrained learning. In the benchmarks, the predictions of our architecture satisfy nonlinear equality and inequality constraints up to a prescribed tolerance ε, while maintaining scalability with tractable computational complexity at training and inference time.
Figures & tables
Figure 1: Neural network predictions may violate known (physical) constraints, leading to infeasible outputs. ENFORCE performs feasibility recovery through an adaptive-depth neural projection module that projects predictions toward the feasible manifold defined by nonlinear equality and inequality constraints, enabling hard-constrained learning for surrogate modeling and parametric optimization.
Figure 2: ENFORCE consists of a backbone neural network and an adaptive neural projection (AdaNP) module. The backbone network can be of every kind, such as fully connected, convolutional, or transformer architecture. AdaNP includes an adaptive number of neural projection (NP) layers, each composed of an auto-differentiation and a local projection step.
Method
MAPE [%]
R2
Mean eq. [%]
Max eq. [%]
Inf. time [s]
MLP
0.339 ± 0.083
0.994 ± 0.003
1.47 ± 0.33
17.13 ± 3.94
0.002 ± 0.000
Soft ( λC=1 )
0.944 ± 0.143
0.972 ± 0.002
1.55 ± 0.16
7.77 ± 0.40
0.002 ± 0.000
ENFORCE
0.060 ± 0.028
0.999 ± 0.000
0.00 ± 0.00
0.00 ± 0.00
0.008 ± 0.003
Table 1: Regression accuracy and constraint satisfaction of ENFORCE on 100,000 test samples when compared with a multilayer perceptron (MLP) and a soft-constrained neural network (Soft). Results for λD=0.5 , εT=10−4 , and εI=10−6 are reported. We report the inference time for a batch of 1,000 samples, with f=5 . (Note that MAPE=N100%∑i=1Nyi∗yi∗−y~i ).
Figure 3: Prediction comparison between ENFORCE ( λD=0.5 , εT=10−4 , εI=10−6 ) and a multilayer perceptron (MLP). ENFORCE enhances the overall accuracy and recover ε -feasibility with respect to nonlinear constraints. ENFORCE also performs better under data-scarcity conditions when the models are trained on uniformly sampled fractions of the dataset (Fig. 3(d) ). This observation suggests that constrained learning may enhance data efficiency.
Figure 4: ENFORCE demonstrates improved convergence, achieving lower loss values compared to an unconstrained MLP. Enhanced training performances are reported for the backbone network of ENFORCE even before the action of AdaNP. This effect is enabled by the simultaneous minimization of the projection displacement (in green) and the action of the AdaNP module (in yellow). Note that we report average values across multiple runs, which explains why the depth of AdaNP appears as a step function with non-integer values.
Figure 5: Influence of constrained learning hyperparameters on the accuracy of ENFORCE on the test set (note that here we plot the inverse of the mean squared error (MSE)). The weighting factor λD favors the learning process if appropriately tuned. Conversely, the training tolerance εT exhibits a small impact on performance, suggesting it can be set based on available resources.
Figure 6: Dynamic evolution of AdaNP during training and inference when different training hyperparameters are chosen. At training time, AdaNP is deeper as a smaller constraint tolerance εT is chosen.
Method
R2
NRMSE [%]
Infeas. g1 [%]
Infeas. g2 [%]
Inf. time [s]
MLP
0.994
7.84
15.33
9.33
0.0002
Soft ( λC=1 )
0.995
6.86
4.00
0.00
0.0003
ENFORCE
0.995
7.16
0.00
0.00
0.020
Table 2: Regression accuracy and constraint satisfaction on 300 test samples for the function fitting with inequality constraints case study. Results for λD=0.5 , εT=10−4 , and εI=10−6 are reported. g1 : upper bound violation; g2 : lower bound violation. NRMSE =RMSE/σy , where σy is the standard deviation of the target ground truth.
Figure 7: Function fitting with nonlinear inequality envelopes.
Method
Obj. value
Max eq.
Mean eq.
Inf. time [s]
Train. time [s]
IPOPT [ 36 ]
−10.64±0.00
0.00±0.00
0.00±0.00
0.379±0.060
–
MLP
−52.99±0.01
45.38±0.56
9.14±0.02
0.001±0.001
540.87±14.77
Soft ( λc=1 )
−8.18±0.18
1.47±0.41
0.09±0.00
0.001±0.001
651.17±29.16
DC3 [ 7 ]
−6.27±0.07
0.00±0.00
0.00±0.00
0.004±0.000
1511.03±517.57
ENFORCE
−10.59±0.00
0.00±0.00
0.00±0.00
0.016±0.002
834.78±4.14
Table 3: Results on a batch of 833 instances of nonconvex optimization problems with linear equality constraints involving 200 variables and 150 equality constraints. ENFORCE achieves feasibility up to the specified tolerance, is 25× faster than IPOPT, and learns a 40% better optimum than the state-of-the-art DC3 method. Baseline deep learning and soft-constrained methods show significant constraint violations and suboptimal predictions.
Method
Obj. value
Max eq.
Mean eq.
Inf. time [s]
Train. time [min]
IPOPT [ 36 ]
−29.45±0.00
0.00±0.00
0.00±0.00
3.40±1.40
–
MLP
−53.07±0.00
497.38±4.64
118.39±0.07
0.002±0.001
10.56±0.32
Soft ( λc=1 )
(6.03±0.04)×104
79.30±3.70
16.68±0.06
0.001±0.000
15.04±0.62
ENFORCE
−27.77±0.02
0.00±0.00
0.00±0.00
0.14±0.08
69.35±23.11
Table 4: Performance comparison on nonconvex problems with nonlinear equality constraints involving 200 variables and 150 constraints. ENFORCE predicts feasible and near-optimal solutions across the entire test set, achieving a 25× speedup over IPOPT. Other deep learning methods exhibit significant constraint violations and suboptimal performance.
Method
MSE
Max eq.
Mean eq.
Inf. time [s]
Train. time [s]
MLP
(7.6±2.1)×10−8
0.003±0.001
(2±0)×10−4
(1.0±0.0)×10−4
105±1
KKT-Hardnet
(1.1±0.0)×10−4
<10−7
<10−7
0.29±0.04
38,631±440
KKT-Hardnet-v2
(2.0±0.0)×10−6
<10−7
<10−7
0.005±0.001
415±4
ENFORCE
(1.2±0.1)×10−7
<10−7
<10−7
0.006±0.001
391±2
Table 5: Engineering case study on a chemical process simulation (extractive distillation) with nonlinear thermodynamics.
Figure 8: Pooling flowsheet representation adapted from [ 32 ] . In blue and orange are denoted the input and output variables, respectively. The system is subject to inequality constraints on product specifications and equality constraints enforcing material balances in the pool.
Method
MSE
Max eq.
Max ineq.
Inf. time [s]
Train. time [s]
MLP
64±3
7.4±2.2
3.4±1.5
(2±0)×10−4
579±1
KKT-Hardnet
54±1
<10−7
<10−7
0.66±0.07
61,768±1,155
KKT-Hardnet-v2
48±10
(0.3±2.7)×10−2
(0.3±2.8)×10−2
0.18±0.42
3,406±66
ENFORCE
61±3
<10−7
<10−7
0.057±0.007
4,041±209
Table 6: Engineering case study on the pooling problem introduced by Floudas et al. [ 56 ] and used as a benchmark by Iftakher et al. [ 32 ] . The problem involves both nonlinear equality and inequality constraints.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Memory footprint of a single projection layer P . NC is the number of constraints, and NO is the dimension of the neural network output. The memory usage scales linearly with the batch size, with the growth rate determined by the number of predicted variables and constraints. For a batch size of 200, the memory requirement is on the order of tens of megabytes.
Figure 10: Memory footprint for a fixed batch size of 200 samples. NC is the number of constraints, and NO is the dimension of the neural network output. For large-scale tasks involving thousands of variables and constraints, the AdaNP module can become memory-intensive, with each projection layer requiring more than 1 GB of memory.
Figure 11: Memory usage during training and inference as a function of the number of projection layers n , for two problem scales. NC is the number of constraints, and NO is the dimension of the neural network output. Results are shown for a fixed batch size BS=200 . Smaller-scale setup: NC=50 , NO=100 ; larger-scale setup: NC=150 , NO=200 . Training memory increases linearly with n , with significantly higher usage in larger-scale problems. Inference memory remains nearly constant across all configurations.
Constraints
( NC )
50
70
150
Variables
( NO )
100
100
200
Method
Metric
IPOPT
Obj. value
−11.11±0.00
−4.84±0.00
−10.64±0.00
Max eq.
0.00±0.00
0.00±0.00
0.00±0.00
Mean eq.
0.00±0.00
0.00±0.00
0.00±0.00
Inf. time [s]
0.095±0.033
0.13±0.04
0.379±0.060
Appendix
Table 7: Scaling experiments on a nonconvex optimization problem with linear equality constraints (Eq.( 18 )) evaluating performance across varying numbers of constraints ( NC ) and variables ( NO ). ENFORCE consistently predicts feasible and near-optimal solutions, outperforming alternative deep learning-based methods. DC3 is trained for a greater number of epochs than the other methods, until convergence is reached.
Constraints
( NC )
10
30
50
70
150
Variables
( NO )
100
100
100
100
200
Method
Metric
IPOPT
Obj. value
−26.27±0.00
−21.81±0.00
−18.05±0.00
−11.69±0.00
−29.45±0.00
Max Eq.
0.00±0.00
0.00±0.00
0.00±0.00
0.00±0.00
0.00±0.00
Avg. Eq.
0.00±0.00
0.00±0.00
0.00±0.00
0.00±0.00
0.00±0.00
Inf. time [s]
0.094±0.032
0.244±0.132
0.268±0.125
0.401±0.167
3.40±1.40
Appendix
Table 8: Scaling experiments on a nonconvex optimization problem with nonlinear equality constraints (Eq.( 19 )) evaluating performance across varying numbers of constraints ( NC ) and variables ( NO ). ENFORCE consistently predicts feasible and near-optimal solutions. In the simplest setting, it outperforms the nonlinear programming solver IPOPT in terms of solution quality.
Enforcing constraint satisfaction in neural network outputs is critical for safety, reliability, and physical fidelity in many control and decision-making applications. While soft-constrained methods penalize constraint violations during training, they do not guarantee constraint adherence during inference. Other approaches guarantee constraint satisfaction via a projection layer, but often rely on the existence of a tractable projection onto the feasible set, limiting their utility in more general problem settings. Many real-world problems of interest are nonlinear and lack the special structure admitting a tractable projection, motivating the development of methods that can enforce general nonlinear constraints. To this end, we introduce HardNet++, a constraint-satisfaction method that enforces linear and nonlinear equality and inequality constraints. Our approach iteratively adjusts the network output via damped local linearizations of the constraints. Each iteration is differentiable, admitting an end-to-end training framework, where the constraint satisfaction layer is active during training. We show that under certain regularity conditions, this procedure enforces nonlinear constraint satisfaction to arbitrary tolerance. Finally, we demonstrate tight constraint adherence without loss of optimality in a learning-for-optimization context, where we apply this method to a nonlinear model predictive control problem.
Enforcing nonlinear inequality constraints in neural networks remains challenging, especially when the output is subject to many coupled constraints. Existing hard constraint methods often impose structural restrictions on the constraint set or introduce substantial computational overhead for large-scale nonlinear problems. Here, we propose DiffSlack, a differentiable projection layer for nonlinear inequality-constrained neural prediction. DiffSlack reformulates inequalities as equalities with learnable slack variables, which are predicted as part of the augmented network output and provide a data-driven warm start for damped Gauss-Newton projection. The projection layer maps raw predictions onto the augmented feasible manifold while preserving end-to-end differentiability. A two-stage curriculum further stabilizes training and improves constraint satisfaction. We evaluate DiffSlack on vehicle path planning with 200 nonlinear inequality constraints from collision avoidance, curvature limits, and waypoint spacing. Compared with existing learning-based baselines, DiffSlack achieves a higher planning success rate and stronger geometric constraint satisfaction under a comparable inference budget. Ablation studies further show that the hard projection layer reduces sensitivity to supervision quality. Closed-loop tracking in CARLA and real-world vehicle experiments confirms the executability of the generated trajectories. These results demonstrate that DiffSlack provides a practical and scalable approach to embedding hard inequality constraints into neural networks for engineering applications.
Ziqian Wang, Chenxi Fang, Zhen Zhang
State Key Laboratory of Tribology in Advanced Equipment, Tsinghua University, Beijing 100084, China · Beijing Key Laboratory of Transformative High-end Manufacturing Equipment and Technology, Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China · Automotive Electronics Business Unit at Hirain Inc., China
We present a novel framework for embedding hard constraint satisfaction into neural network (NN) architectures, specifically feedforward neural networks and transformers, with input-dependent affine constraints of arbitrary cardinality. Traditional constraint enforcement approaches either rely on penalty-based soft constraints, which offer no guarantee of satisfaction, or on post-processing methods that enforce constraints after the NN is trained, which may lead to suboptimality. We introduce a trainable constraint-affine (CAffine) layer into NNs, yielding CAffNet, which goes beyond enforcing affine constraints via fixed orthogonal or parallel projections and enables joint optimization with network parameters. Moreover, we impose no restrictions on the constraint space dimensions and establish that our construction preserves the universal approximation properties of NNs, while providing provable guarantees on constraint adherence for all inputs. Experimental validation demonstrates robust performance across diverse domains requiring guaranteed constraint satisfaction. The code is available at https://github.com/ice-t-lab/CAffNet.git.
Yang Zhao, Jungeun Lee, Jeong hwan Jeon +1
Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA 02115 USA · Department of Electrical Engineering, Ulsan National Institute of Science and Technology, Ulsan 44919, Republic of Korea