Only Project Once: Projection-Adaptive Loss for Exact Constraint Satisfaction

Tim Aebersold; Soheyl Massoudi; Mark Fuge

cs.LGOct 3, 2026

Only Project Once: Projection-Adaptive Loss for Exact Constraint Satisfaction

Authors: Tim Aebersold, Soheyl Massoudi, Mark Fuge

Organizations: ETH Zürich

Abstract

Precise constraint satisfaction is a prerequisite to deploying learned models in many areas, motivating methods that repair raw neural predictions with a repair procedure. Current methods unroll multiple repair steps in training and softly penalize constraint violations that remain after the unroll. This is compute- and memory-intensive, lacks robustness when the repair fails to converge, and surrenders most of the constraint satisfaction work to the repair. Our central finding is that, contrary to common practice, a single detached projection step suffices in training. We accomplish this with a Projection-Adaptive Loss (PAL), which uses the constraint residual after this single step to adaptively weigh constraint penalties on the raw prediction. In experiments, PAL is the only method that retains virtually perfect feasibility on extremely nonlinear constraints, and matches or outperforms current methods on synthetic and engineering benchmarks. Because it only requires a single detached projection step, PAL trains 2.5x faster than the canonical repair-based method (DC3) on its own ACOPF benchmark. PAL can also be trained when constraints are expensive to evaluate (e.g., via neural surrogates), a setting where current unrolled methods are memory-intractable.

Figures & tables

Panel of Figure 1 — Figure 1: PAL vs. SOTA repair-aware training. From raw model predictions $\hat{y}$ , SOTA iterates projector $\mathcal{P}$ and evaluator $\Phi$ for $N$ times per training iteration and then uses a soft penalty on post-repair constraint violation $\tilde{c}^{(N)}$ . We instead use a single and detached projection step to $\tilde{y}^{(1)}$ . PAL then uses the post-projection residual $\tilde{c}^{(1)}$ to adaptively drive $\bar{w}_{c}$ , which acts as an adaptive penalty on the raw prediction’s constraint residual $\hat{c}$ .

Figure 2: Two-constraint toy problem where

h_{1}

is stiff but linear and therefore trivial to project, and

h_{2}

is non-stiff but nonlinear and difficult to project (more details in Appendix H ). Top: constraint violations before projection

h(\hat{y})

. Bottom: constraint violations after the first projection step

h(\tilde{y}^{(1)})

, equivalent to

R_{1,k}

. Left: ALM (adaptive penalty method) trains without projection and penalizes constraints only according to pre-projection magnitude

h(\hat{y})

, i.e., cannot distinguish constraints by repair difficulty. Center: Under soft-loss training (SnareNet, ENFORCE, FSNet, …), the projector drives

h(\tilde{y}^{(N)})\approx 0

in training so that constraint pressure on the neural backbone is low, resulting in high violations before and after the first projection step. Right: PAL exerts the right amount of training pressure onto

h(\hat{y})

(and thus the neural backbone) to drive post-projection residuals

h_{k}(\tilde{y}^{(1)})

below threshold

\tau_{c}

(here

10^{-7}

	Bench.	ALM+ Bolt-On	ENFORCE	DC3	FSNet	SnareNet	PAL [Ours]
Feas. (%)	S1	$100\pm 0$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S2	$100\pm 0$	$100\pm 0$	$97\pm 7$	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S3	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S4	$100\pm 1$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S5	$100\pm 0$	—	—	$51\pm 31$	$100\pm 0$	$100\pm 0$
	S6	$100\pm 0$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$

Table 1: Synthetic stress-test experiments. Means and standard deviations across 10 seeds. “—”: failure on 2 or more seeds. Forward and backward evaluator passes

N_{\phi}^{f}

and

N_{\phi}^{b}

per training step are only shown for extreme cases S1 (

K=1

) and S6 (

K=90

), with full results in Table 12 .

Panel of Table 2 — Table 2: Case studies in science and engineering. Mem: peak GPU memory (GB) at batchsize=1 . Time: training wall-clock. 10 seeds each, IPOPT 5 restarts $\times$ 64 queries. $\ast$ : Maximum constraint violation. Bold marks the best value among feasible methods. $\ddagger$ : 49 GB on 3 seeds and >96 GB (OOM) on 7 seeds in the first two training epochs. $\lx@sectionsign$ : batchsize=8 . $\dagger$ : 0% feasibility. : Non-convergence. : Structurally inapplicable. Baseline failure modes are discussed in Appendix L .

Panel of Figure 3 — Figure 3: Feasibility, objective and constraint training pressure $\|\nabla_{\theta}L_{c}\|/\|\nabla_{\theta}L\|$ over curvature $\kappa$ . PAL retains best-in-class feasibility at high curvature despite a single repair step, because its adaptive loss raises training pressure in response to first-step residuals.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$y,\;\hat{y},\;\tilde{y}^{(i)}$	design variable, raw neural backbone prediction, $i$ -th projection iterate
$z$	input condition or latent embedding
$f,\;h,\;g$	objective, equality constraints, inequality constraints
$c_{k},\;c_{\mathcal{A}}$	per-constraint residual, active residual vector
$\mathcal{A}$	active index set
$J_{\mathcal{A}}$	active constraint Jacobian $\partial c_{\mathcal{A}}/\partial y$

Appendix

Table 3: Notation summary.

\delta

\lambda_{\min}^{\text{proj}}

\gamma

D_{\text{max}}

\tau_{c}

\tau_{d}

f

Dec.

Obj. (max)

\downarrow

Feas. (%)

\uparrow

\|c(\hat{y})\|_{\infty}\,\downarrow

Base

10^{-3}

10^{-6}

10^{-2}

10^{-4}

10^{-4}

10^{-4}

\hat{y}

✓

0.08

\pm

0.01

100

\pm

0.06

\pm

0.02

0.07

\pm

0.01

100

\pm

0.06

\pm

0.02

10^{-4}

0.07

\pm

0.01

100

\pm

0.05

\pm

0.01

10^{-2}

0.10

\pm

0.02

100

\pm

0.06

\pm

0.02

0.11

\pm

0.09

100

\pm

0.06

\pm

0.02

10^{-3}

0.06

\pm

0.03

100

\pm

0.09

\pm

0.06

Appendix

Table 4: Ablation of PAL hyperparameters on synthetic benchmarks (

n=10

seeds). Blank cells inherit the Base value. Bold

=

best per column. The

f

column lists the argument used inside the objective term of the training loss (

\hat{y}

\tilde{y}^{(1)}

). Dec. marks whether the multiplier rule’s decay half is active:

\checkmark

= bidirectional (default),

✗

= monotone-only growth.

	PAL ( $\tau_{c}{=}10^{-4}$ )			PAL ( $\tau_{c}{=}10^{-2}$ )
$\kappa$	Feas. (%)	Obj.	$\tfrac{\\|\nabla L_{c}\\|}{\\|\nabla L\\|}$ (%)	Feas. (%)	Obj.	$\tfrac{\\|\nabla L_{c}\\|}{\\|\nabla L\\|}$ (%)
$\kappa{=}0$	100 $\pm$ 0	0.01 $\pm$ 0.00	0.0	100 $\pm$ 0	0.01 $\pm$ 0.00	0.0
$\kappa{=}1$	100 $\pm$ 0	0.01 $\pm$ 0.00	23	100 $\pm$ 0	0.01 $\pm$ 0.00	0.0
$\kappa{=}10$	100 $\pm$ 0	0.05 $\pm$ 0.01	63	100 $\pm$ 0	0.03 $\pm$ 0.00	25
$\kappa{=}10^{2}$	100 $\pm$ 0	0.66 $\pm$ 0.13	72	99.9 $\pm$ 0.1	0.48 $\pm$ 0.05	62
$\kappa{=}10^{3}$	100 $\pm$ 0	0.96 $\pm$ 0.40	89	99.8 $\pm$ 0.2	0.42 $\pm$ 0.20	79

Appendix

Table 5: Full results for high-curvature experiment.

\kappa

from 0 to

10^{6}

, 10 seeds, all methods trained and evaluated in float64 .

\|\nabla_{\theta}L_{c}\|/\|\nabla_{\theta}L\|

is the share of constraint training pressure. Bold marks the best feasibility at each

\kappa

across all methods.

\dagger

: gradient ratio carried over from the source ALM training run. Plain ALM without Bolt-On is omitted because it is 0% feasible at every

\kappa

. DC3 uses a Newton completion with Yamashita-Fukushima damping, since the undamped completion diverges for every

\kappa>0

	PAL- LM	PAL- SQP	PAL- IP
S1	$0.02\pm 0.02$	$0.01\pm 0.01$	$0.01\pm 0.01$
S2	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.02\pm 0.00$
S3	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
S4	$0.01\pm 0.00$	$0.01\pm 0.00$	$0.05\pm 0.02$
S5	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
S6	$0.08\pm 0.01$	$0.07\pm 0.01$	$0.21\pm 0.02$

Appendix

Table 6: Objective at convergence (

f^{\star}=0

) for the three repair step mechanisms on the synthetic benchmarks S1 – S6 . Shown is mean

\pm

standard deviation across 10 seeds. All methods use default hyperparameters and consistently reach

100\%

constraint satisfaction on S1 – S6 , except for IP on S3 (

86\pm 5\%

Panel of Figure 4 — Figure 4: Geometric intuition. The projector eliminates gradient flow for $f(\tilde{y}^{(1)})$ along direction $v$ (orthogonal to local constraint manifold $\mathcal{M}$ ) because, for an idealized projector, $\partial\tilde{y}^{(1)}/\partial\hat{y}\cdot\delta v=0$ . This leaves only gradients of $f(\tilde{y}^{(1)})$ which are tangent to the constraint manifold (direction $u$ ) to be backpropagated through the projection step into the neural backbone $N_{\theta}$ .

Figure 5: PAL loss-placement comparison on the toy problem of Appendix H . Rows from top: pre-projection residuals

h(\hat{y})

, post-projection residuals

h(\tilde{y}^{(1)})

, objective values

f

, and objective gradient magnitudes (computed on a fixed evaluation batch). Columns:

f(\hat{y})

throughout (PAL default),

f(\tilde{y}^{(1)})

throughout, and a midway switch from

f(\hat{y})

f(\tilde{y}^{(1)})

t=2500

(dotted vertical line). When using

f(\tilde{y}^{(1)})

throughout, the objective never escapes the initialisation neighbourhood due to vanishing gradients normal to the manifold. The right column, having trained under

f(\hat{y})

first, retains its objective and feasibility quality after the switch, demonstrating that the pathology is specific to early training.

	$N{=}1$	$N{=}2$	$N{=}5$	$N{=}10$	$N{=}100$
float64
S1 (sphere track)	$8.48\cdot 10^{-4}$	$2.46\cdot 10^{-7}$	$7.55\cdot 10^{-16}$	$7.55\cdot 10^{-16}$	$7.55\cdot 10^{-16}$
S2 (active-set switch)	$1.36\cdot 10^{-4}$	$3.41\cdot 10^{-6}$	$2.22\cdot 10^{-16}$	$2.22\cdot 10^{-16}$	$2.22\cdot 10^{-16}$
S3 (ill-cond. tube)	$8.42\cdot 10^{-6}$	$1.03\cdot 10^{-7}$	$3.54\cdot 10^{-15}$	$2.22\cdot 10^{-17}$	$2.01\cdot 10^{-17}$
S4 (qv coupling)	$1.01\cdot 10^{-2}$	$1.26\cdot 10^{-5}$	$8.44\cdot 10^{-16}$	$9.10\cdot 10^{-16}$	$8.22\cdot 10^{-16}$
S5 (overdetermined)	$6.77\cdot 10^{-3}$	$4.98\cdot 10^{-4}$	$6.33\cdot 10^{-8}$	$8.81\cdot 10^{-9}$	$8.83\cdot 10^{-11}$

Appendix

Table 7: Maximum constraint violation over the test set against the number of inference repair steps

N

, over 10 seeds, for the trained PAL models from the paper on the synthetic benchmarks S1 – S6 .

Figure 6: Candidate chip placement for E4 (left) and full results (right). PAL and ALM+Bolt-On run at 10 seeds, FSNet and SnareNet at 5 seeds. All methods use canonical default hyperparameters without per-method tuning .

Figure 7: E1 unmanned aircraft benchmark, candidate design. Left: outer mould-line geometry from the analytical Bézier representation, with the decoded internal wingbox (skin, spars, ribs) overlaid in blue/green. Top right: surface pressure coefficient $c_{p}$ predicted by the FiLM-modulated BlendedNet surrogate $\Phi^{\theta}_{\text{cp}}$ ( Sung et al., 2025 ) at iso-sliced surface points. Bottom right: structural tip deflection $|w(y)|$ from beam integration of the pressure loads through the predicted section properties, warp-amplified $5\times$ for visualisation. The tip-deflection constraint reads $w_{\text{tip}}$ at $y=b/2$ .

Panel of Figure 8 — Figure 8: E1 evaluator $\Phi$ , from the design vector $y$ to the objective $f$ and constraints $c$ . Dashed: geometry processing. Plain: analytical. Grey: neural surrogate. Hatched: generative model. Double border: constraint or objective.

Figure 9: E2 urban planning benchmark, three views of a candidate design. Left: top-down site plan with the $750\times 750\,$ m buildable plot (green dashed) and the 10 building footprints. Clearance is evaluated pairwise and reported as the number of pairs with overlapping $d_{\min}$ -inflated AABBs. Middle: tail-mean wind speed predicted by the WinDiNet surrogate over the full $1100\times 1100\,$ m raster domain. Right: hard pedestrian-comfort zones derived from the tail-mean speed (calm $<1\,$ m/s, acceptable $1$ – $5$ , uncomfortable $5$ – $15$ , danger $>15$ ). The danger constraint forbids any cell entering the red band.

Figure 10: E2 candidate design rendered in 3D with PyVista ( Sullivan & Kaszynski, 2019 ) , with the pedestrian-comfort field draped at ground level. Building heights are decoded from the design vector. The colour map shows tail-mean wind speed over the comfort range $0$ – $15\,$ m/s, with green/yellow indicating calm and acceptable zones and red indicating the danger band that the surrogate constraint forbids.

Panel of Figure 11 — Figure 11: E2 evaluator $\Phi$ , from the design vector $y$ to the objective $f$ and constraints $c$ . Dashed: geometry processing. Plain: analytical. Grey: neural surrogate. Hatched: generative model. Double border: constraint or objective.

Method	BO winner configuration
ALM	lr $8.3{\cdot}10^{-5}$ ; $\gamma$ $8.8{\cdot}10^{-4}$ ; $\alpha$ $0.995$ ; eps $10^{-4}{}$
ALM+Bolt-On	proj_delta $10^{-5}{}$ ; proj_max_iters $20$ $\dagger$
ENFORCE (v4)	lr $2.5{\cdot}10^{-4}{}$ ; eps_chol $2.3{\cdot}10^{-5}$ ; max_it $100$ ; training_tolerance $0.01$ ; weight_loss_displacement $5.0$ ; weighting_option $5$ ; ada_np_auto_activation true (fixed: epoch_start_hard_constrained $0$ , eps $10^{-8}$ , inference_tolerance $10^{-6}$ )
DC3	lr $1.75{\cdot}10^{-5}{}$ ; soft_weight $10$ ; corr_train_steps $10$ ; corr_lr $1.8{\cdot}10^{-6}$
FSNet	lr $2.0{\cdot}10^{-4}$ ; dist_weight $5.0$ ; weight_decay $3.0{\cdot}10^{-5}$
SnareNet	lr $2.5{\cdot}10^{-4}{}$ ; $\lambda$ $2.6{\cdot}10^{-3}$ ; soft_weight $1.0$ ; soft_epochs $1000$ ; decay_epochs $1000$

Appendix

Table 8: Selected hyperparameter values per method from the Bayesian optimization sweep.

\dagger

: For ALM+Bolt-On, the optimized hyperparameters of frozen ALM models were reused, i.e., only hyperparameters of the projection were tuned for ALM+Bolt-On itself.

	Method	Train (s)	Total (s)	Predict (s)	$/$ query (ms)
S1	PAL	106	247	0.0119	0.185
	ALM	58	131	0.00734	0.115
	ALM+Bolt-On	58 †	10	0.0507	0.792
	ENFORCE v4	57	63	0.0152	0.237
	FSNet	1720	1909	0.176	2.75
	SnareNet	149	310	0.0506	0.791

Appendix

Table 9: Wall-clock times on S1 – S6 . The total time reported includes initialization, training, evaluation, and artifact saving. The overall prediction time is for 64 samples (queries). We report the median over

10

seeds (counting non-diverged seeds only). “—” indicates fully diverged or structurally inapplicable cells. † For ALM+Bolt-On, training reuses ALM checkpoints and the reported total time is inference-time projection only.

Benchmark	Method	Median wall-clock
E1 BWB	PAL	$\sim$ 8 min
	ALM	$\sim$ 70 s
E2 Urban Planning	PAL	$\sim$ 106 min
	ALM+Bolt-On	$\sim$ 65 min
	ALM	$\sim$ 32 s

Appendix

Table 10: End-to-end wall-clock time on engineering benchmarks. On E2 , PAL suffers from sequential Jacobian assembly (loop over

K

	Default	E1 (Unmanned Aircraft)	E2 (Urban Planning)
Training epochs	—	200	500
Batch size	—	32	16
Learning rate ( $N_{\theta}$ )	—	$10^{-4}$	$10^{-3}$
PAL
$\gamma$	$10^{-2}$	$\bm{10^{-1}}$	$\bm{10^{-1}}$
$D_{\max}$	$1.0$	$\bm{5.0}$	$\bm{5.0}$

Appendix

Table 11: Hyperparameters for PAL and ALM on E1 and E2 . Bold marks deviations from defaults. ALM defaults follow Basir & Senocak (2023) . For both benchmarks, we performed minor hand-tuning of

\gamma_{\mathrm{ALM}}

and

\alpha

but found defaults to work relatively well.

	Bench.	ALM+ Bolt-On	ENFORCE	DC3	FSNet	SnareNet	PAL [Ours]
Feas. (%)	S1	$100\pm 0$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S2	$100\pm 0$	$100\pm 0$	$97\pm 7$	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S3	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S4	$100\pm 1$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$
	S5	$100\pm 0$	—	—	$51\pm 31$	$100\pm 0$	$100\pm 0$
	S6	$100\pm 0$	$100\pm 0$	—	$100\pm 0$	$100\pm 0$	$100\pm 0$

Appendix

Table 12: Full version of Table 1 : synthetic stress-test experiments for five baselines and PAL, including evaluator passes per training step on all benchmarks. Means and standard deviations across 10 seeds. “—” indicates failure on 2 or more seeds.

Constraint Satisfaction

Constraint satisfaction studies how to find or optimize assignments that obey interacting rules across discrete and structured problems.

See all 86 papers

Improving Feasibility via Fast Autoencoder-Based Projections
Apr 3, 2026Maria Chzhen, Priya L. DontiConstrained Optimization Constrained RL
University of Toronto Toronto, ON, Canada · Massachusetts Institute of Technology Cambridge, MA, USA
HardNet++: Nonlinear Constraint Enforcement in Neural Networks
Apr 21, 2026Andrea Goertzen, Kaveh Alim, Youngjae Min +1Nonlinear MPC Neural Network Optimization
Massachusetts Institute of Technology
Constraint Learning by Optimizing over Trained Neural Networks: A Pragmatic Tutorial
Oct 26, 2026Jiaxiao Fang, Thiago SerraConstrained Optimization
University of Iowa

Apr 3, 2026J/K move · Enter open · S save

Improving Feasibility via Fast Autoencoder-Based Projections

Maria Chzhen, Priya L. Donti

University of Toronto Toronto, ON, Canada · Massachusetts Institute of Technology Cambridge, MA, USA

Constrained Optimization Constrained RL Autoencoders

Open paper AskPDF arXiv

Enforcing complex (e.g., nonconvex) operational constraints is a critical challenge in real-world learning and control systems. However, existing methods struggle to efficiently enforce general classes of constraints. To address this, we propose a novel data-driven amortized approach that uses a trained autoencoder as an approximate projector to provide fast corrections to infeasible predictions. Specifically, we train an autoencoder using an adversarial objective to learn a structured, convex latent representation of the feasible set. This enables rapid correction of neural network outputs by projecting their associated latent representations onto a simple convex shape before decoding into the original feasible set. We test our approach on a diverse suite of constrained optimization and reinforcement learning problems with challenging nonconvex constraints. Results show that our method effectively enforces constraints at a low computational cost, offering a practical alternative to expensive feasibility correction techniques based on traditional solvers.

Abstract

Constraint Satisfaction

Explore similar work

Improving Feasibility via Fast Autoencoder-Based Projections

HardNet++: Nonlinear Constraint Enforcement in Neural Networks

Constraint Learning by Optimizing over Trained Neural Networks: A Pragmatic Tutorial