Deep learning models are increasingly subject to explicit requirements, or constraints, such as safety levels, fairness limits, and sparsity budgets. In practice, such requirements are often handled through fixed weighted-sum penalization: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited to deep learning problems with non-negotiable requirements, for three main reasons. First, in non-convex settings, the penalized and constrained problems are not, in general, equivalent, so a minimizer of the penalized objective need not solve the constrained problem. Second, a penalty coefficient sets an objective-constraint trade-off rather than the constraint level, so finding an acceptable coefficient often takes trial and error, each trial a full training run. Third, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. As a result, compliance is no longer built into the problem, and requirements can be quietly relaxed. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, one should solve the constrained problem itself instead of a surrogate defined by fixed penalization.
Figures & tables
Figure 1: Solving a sparsity-constrained classification problem. Starting from coefficients yielding low (25%) and high (85%) model density, five bisection search steps are needed to reach the desired 50% density . Numbers mark iterations (endpoints are “0”). Replicates Fig. 5(b) of Gallego-Posada et al. [21] (Appendix B.2 ).
ϵ
x∗=arcsin(ϵ)
xt at convergence
0.1
1.00×10−1
1.00×10−1
0.2
2.01×10−1
2.01×10−1
0.3
3.05×10−1
3.05×10−1
0.4
4.12×10−1
4.12×10−1
0.5
5.24×10−1
5.24×10−1
0.6
6.44×10−1
6.44×10−1
Table 1: Solutions to Example 1 recovered by the Lagrangian approach with PI multiplier updates. At every constraint level tested, the method recovers the constrained minimizer x∗=arcsin(ϵ) .
Figure 2: Solving a binary classification task under a constraint that the average predicted probability of class “0” be at least 70%. No penalty coefficient in the sweep lands on the 70% target. Each either leaves the constraint violated or over-satisfies it at a cost in accuracy, whereas the Lagrangian approach lands on the target across a broad range of dual step-sizes. Cells are blue within one percentage point of the target and shade to red at eleven or more points away. Data and model from Figure 2 of Cotter et al. [8] (details in Appendix B.3 ).
Experiment
Penalized (median, std.)
Constrained (median, std.)
Overhead
Probability constraint
5.40 s (0.27 s)
5.68 s (0.31 s)
+5.2%
Sparsity constraint
8.17 min (0.19 min)
8.30 min (0.07 min)
+1.6%
Table 2: Wall-clock comparison over five runs on an NVIDIA L40S GPU in two illustrative settings. Constrained training adds only a small per-run overhead.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: The trade-off curve between the objective f and constraint g in Example 1 . As x varies over [0,π/2] , the map x↦(f(x),g(x))=(cosx,sinx) traces the non-dominated frontier from (1,0) to (0,1) . The constrained optimum is the interior point (1−ϵ2,ϵ) , whereas fixed weighted-sum penalization recovers only the two endpoints.
Iteration #
Penalty coef. c
Model density (%)
Train acc. (%)
0
1.00×10−3
84.5
99.97
0
1.00×100
25.3
99.97
1
3.16×10−2
68.1
99.96
2
1.78×10−1
54.2
100.00
3
4.22×10−1
39.0
100.00
4
2.74×10−1
46.9
99.99
Appendix
Table 3: Sparsity-constrained neural network training using the penalized approach, targeting at most 50% model density (at least 50% sparsity), within one percentage point. The penalty coefficient is selected via a log-scale bisection search. This table complements Figure 1 .
Penalized ( c )
Lagrangian (dual step-size)
Value
P(y^=0) (%)
Train acc. (%)
P(y^=0) (%)
Train acc. (%)
λT
0
49.96
99.75
49.96
99.75
0.00
1.00×10−4
49.96
99.75
50.30
99.75
0.20
2.15×10−4
49.96
99.75
50.72
99.75
0.42
4.60×10−4
49.96
99.75
51.89
100.00
0.87
1.00×10−3
49.96
99.75
59.56
96.25
1.63
Appendix
Table 4: Probability-constrained linear classification (Appendix B.3 ), targeting an average predicted probability of class 0, P(y^=0) , of at least 70% . Both sweeps use the same grid of values, as penalty coefficients c for the penalized approach and as dual step-sizes for the Lagrangian approach. The multiplier at the end of training is λT . The Lagrangian approach lands on the target for 15 of the 20 dual step-sizes, whereas only c=2.15 avoids a collapsed penalized solution, overshooting to 75% .
Constrained Machine Learning (CML) enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. In this work, we introduce the Stochastic Penalty-Barrier Method (SPBM) for CML problems. SPBM extends classical penalty and barrier methods by incorporating an exponential averaging of the dual variables, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. We analyze the bias that mini-batching introduces in the barrier function and show that the feasible set of the resulting transformed problem is contained within the original one. We compare SPBM with CML baselines across multiple fairness and physics informed neural networks experiments. We find that SPBM is competitive with state-of-the-art methods. We also observe, on our fairness-based computational benchmark, that the per-epoch runtime of CML methods is largely independent of the number of constraints, and within 1.3× of the per-epoch runtime of regularized Adam, for a number of constraints ranging from 90 to 9900.
Adam Bosák, Andrii Kliachkin, Gilles Bareilles +4
Artificial Intelligence Center, CTU in Prague · CMAP, École Polytechnique, Palaiseau, France
Constraints are essential for stabilizing reinforcement learning fine-tuning (RFT) and preventing degenerate outputs, yet they inherently conflict with the optimization objective because stronger constraints limit the ability of a fine-tuned model to discover better solutions. We propose dynamic constraints that resolve this tension by adapting to the evolving capabilities of the fine-tuned model based on the insight that constraints should only intervene when degenerate outputs occur. We implement this by using a reference model as an online refiner that takes the response from the fine-tuned model and generates a minimally corrected version which preserves correct content verbatim while fixing errors. A supervised fine-tuning loss then trains the fine-tuned model to produce the refined output. This mechanism yields a constraint that automatically strengthens or relaxes based on output quality. Experiments on dialogue and code generation show that dynamic constraints outperform both KL regularization and unconstrained baselines, achieving substantially higher task rewards while maintaining training stability.
Hao Ma, Zhiqiang Pu, Yang Liu +1
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences
Enforcing constraint satisfaction in neural network outputs is critical for safety, reliability, and physical fidelity in many control and decision-making applications. While soft-constrained methods penalize constraint violations during training, they do not guarantee constraint adherence during inference. Other approaches guarantee constraint satisfaction via a projection layer, but often rely on the existence of a tractable projection onto the feasible set, limiting their utility in more general problem settings. Many real-world problems of interest are nonlinear and lack the special structure admitting a tractable projection, motivating the development of methods that can enforce general nonlinear constraints. To this end, we introduce HardNet++, a constraint-satisfaction method that enforces linear and nonlinear equality and inequality constraints. Our approach iteratively adjusts the network output via damped local linearizations of the constraints. Each iteration is differentiable, admitting an end-to-end training framework, where the constraint satisfaction layer is active during training. We show that under certain regularity conditions, this procedure enforces nonlinear constraint satisfaction to arbitrary tolerance. Finally, we demonstrate tight constraint adherence without loss of optimality in a learning-for-optimization context, where we apply this method to a nonlinear model predictive control problem.