Constraint learning is an emerging paradigm of modeling optimization problems with the help of machine learning. In this paradigm, trained machine learning models are embedded into the optimization model. For example, we can learn constraints empirically from available data, or we can avoid using a nonlinear objective function by learning a simpler approximation for it. Neural networks become a compelling option if the machine learning models must capture more elaborate and nuanced relationships. However, optimization models embedding neural networks may become too large or too slow to solve. In this tutorial, we present practical considerations for an effective and efficient use of constraint learning with neural networks and validate it with a case study highlighting the benefit of smaller and sparser networks.
Figures & tables
Figure 1: Left: Structure of a feedforward neural network with input x∈Rn0 , hidden layers ℓ=1 to L with nℓ neurons each, and output y∈R . Right: Formula for output hiℓ of neuron i in layer ℓ from activation function σℓ:R→R and preactivation output giℓ , which uses previous layer output hℓ−1 , neuron weights Wi∗ℓ , and bias biℓ . For details, see Appendix C Neural Network Structure and Mapping (in Appendix).
Figure 2: Top: Visual description of neural network used as example. Bottom left: Lines partitioning the space based on what inputs produce a positive output for each neuron, with the arrow pointing to the positive side, the length of the arrow proportional to the magnitude of the weights, and the arrow label denoting the influence on y . Bottom right: Contour plots of y over the lines associated with neural activations. The example and plots are adapted from Tong et al. (2026) .
Reference
Type of problem(s)
Prediction(s) involved
Huang et al. (2003)
Catalyst composition
Catalytic performance
Fernandes (2006)
Synthesis of a specific oil product
Outcome of a catalytic process
Fahmi and Cremaschi (2012)
Biodiesel production plant design
Unit operation, thermodynamics, mixing
Sant Anna et al. (2017)
PSA operation for N 2 /CH 4 separation
Nitrogen/methane purity and recovery
Lombardi et al. (2017)
Computational workload dispatch
Processor efficiency for assignment
Say et al. (2017)
HVAC, navigation, reservoir control
State transition in sensors
Table 1: Applications from constraint learning studies in which neural networks are used.
Type of predictive model
References
Linear regression
Bertsimas et al. (2016) , Verwer et al. (2017)
Logistic regression
Bergman et al. (2022)
Support vector machine
Hooshmand and Nikoomanesh (2025)
Multivariate adaptive regression spline
Martinez et al. (2017)
Regression tree
Verwer et al. (2017) , Ammari et al. (2023)
Decision tree
Bonfietti et al. (2015) , Lombardi et al. (2017) , Kudła and Pawlak (2018)
Table 2: Constraint learning studies in which other types of predictive models are embedded.
Figure 3: Comparison of predictive quality with number of weights of configurations (as a scatter plot), contrasted by pruning rate (as a color gradient). The configurations that are not dominated by any other in both predictive quality and number of weights define the efficient frontier (as a line chart).
Figure 4: Comparison of predictive quality with pruning rate p (as a line chart for each width w ) for neural networks with width 10, 50, 100, and 1,000. The pruning rate p∗ with smallest validation error for each width w is circled. The configurations at the efficient frontier are diamond-shaped.
Figure 5: Comparison of decision quality with number of weights for configurations at the efficient frontier (as line charts with error bars). Using the solution obtained with the optimization model, we evaluate it as an input of the embedded network and compare it with the ground truth.
Figure 6: Comparison of decision quality with number of weights for configurations at the efficient frontier (as a line chart; the same ones from Figure 5 ) vs. other configurations (as a scatter plot), contrasted by pruning rate (as a color gradient). For ground truth value, 104 configurations are below the curve defined by the efficient frontier, 15 are above, and 37 are outside the range. For optimization bias, 84 of those configurations are above the curve, 35 are below, and 37 are outside the range.
Figure 7: Comparison of solution time with number of weights at the efficient frontier (as a line chart) vs. other configurations (as a scatter plot), contrasted by ground truth values (as a color gradient).
Figure 8: Comparison of ground truth value with predictive accuracy at the efficient frontier (as a line chart) vs. other configurations (as a scatter plot), contrasted on the left by the number of weights and on the right by solution time (as a color gradient in both; the plots are otherwise identical). The plots exclude configurations with mean validation MSE above 3×104 . The correlation across for the points shown at the efficient frontier is -0.9535, dropping to -0.7885 for all points.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Geometric illustration of how defining half-spaces with hyperplanes in more ways than the number of dimensions does not imply that every intersection of half-spaces is feasible. Left: two partitionings in one dimension lead to a maximum of 3 regions instead of 4. Right: three partitionings in two dimensions lead to a maximum of 7 regions instead of 8.
Figure 10: Comparison of predictive quality with pruning rate p (as a line chart for each width w ) in small (left) and large (right) neural networks. The pruning rate p∗ with smallest validation error for each width w is circled. The configurations at the efficient frontier are diamond-shaped.
Machine learning models are increasingly used to model chemical process systems, yet they often lack principled uncertainty quantification and mechanisms to enforce physical constraints. We propose a probabilistic neural network framework that guarantees satisfaction of linear equality constraints within a given tolerance, while capturing aleatoric uncertainty. Compared to state-of-the-art methods, our formulation demonstrates improved predictive accuracy, uncertainty calibration, and adherence to constraints on reduced data. It also demonstrates competitive performance, but with significantly faster training times when evaluated on large data regimes. We evaluated this on two batch reactor case studies, enforcing mass balances.
Matthew Marsh, Benoit Chachuat, Antonio del Rio Chanona
The Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, Imperial College London, UK
Training neural networks to satisfy universal constraints over continuous domains poses unique challenges. Common examples include Lyapunov Neural Networks (Lyapunov NNs) and Physics-Informed Neural Networks (PINNs), where analytical solutions are generally either unavailable or overly restrictive. Sample-based methods are therefore commonly used to enforce these constraints, and the choice of samples has a substantial impact on convergence speed, stability, and solution quality. Most existing methods rely on fixed heuristics or handcrafted rules, and are suboptimal in practice. In this paper, we aim to improve upon them by learning, from data and experience, how to dynamically and iteratively adjust the samples in response to the model's evolving learning performance. Trained by reinforcement learning, the learned policy improves empirical constraint satisfaction on test problems while significantly improving efficiency. We validate the approach on both Lyapunov NNs and PINNs, and demonstrate its broader applicability to domains where adaptive input selection is essential for effective training.
Enforcing constraint satisfaction in neural network outputs is critical for safety, reliability, and physical fidelity in many control and decision-making applications. While soft-constrained methods penalize constraint violations during training, they do not guarantee constraint adherence during inference. Other approaches guarantee constraint satisfaction via a projection layer, but often rely on the existence of a tractable projection onto the feasible set, limiting their utility in more general problem settings. Many real-world problems of interest are nonlinear and lack the special structure admitting a tractable projection, motivating the development of methods that can enforce general nonlinear constraints. To this end, we introduce HardNet++, a constraint-satisfaction method that enforces linear and nonlinear equality and inequality constraints. Our approach iteratively adjusts the network output via damped local linearizations of the constraints. Each iteration is differentiable, admitting an end-to-end training framework, where the constraint satisfaction layer is active during training. We show that under certain regularity conditions, this procedure enforces nonlinear constraint satisfaction to arbitrary tolerance. Finally, we demonstrate tight constraint adherence without loss of optimality in a learning-for-optimization context, where we apply this method to a nonlinear model predictive control problem.