Soft-Radial Projection for Constrained End-to-End Learning
Organizations: Risk Analytics and Optimization Chair, EPFL
Abstract
Constrained end-to-end learning trains neural networks whose outputs must satisfy feasibility constraints, such as resource, budget, or operational limits. While projection and optimization layers guarantee constraint satisfaction, boundary-based projections can introduce unfavorable optimization geometry: regions of the unconstrained output space are mapped to the same active face of the feasible set. This makes the resulting map locally rank-deficient, suppressing gradients in constrained directions and potentially impairing optimization - a bottleneck known as gradient saturation. We propose Soft-Radial Projection (SRP), a differentiable reparameterization layer that maps unconstrained network outputs into the relative interior of a convex feasible set by smoothly rescaling rays from a strictly feasible anchor point. Unlike hard radial or orthogonal projections, the resulting map is one-to-one and has a full-rank Jacobian almost everywhere, while guaranteeing strict feasibility. We prove that constrained networks equipped with SRP retain universal approximation guarantees. For polytopes and balls, the layer admits a closed-form forward pass, while for general convex sets, the ray-boundary intersection reduces to a one-dimensional search. Experiments on decision-focused learning benchmarks demonstrate strong solution quality at low computational cost, with particularly favorable results in portfolio optimization and substantially lower inference latency than differentiable optimization layers in the evaluated settings.
Figures & tables
| Set | Method | SR (Net) | Turnover | Training [s] | Inference [ms] |
| Softmax | |||||
| O-Proj | |||||
| DC3 | |||||
| HardNet | |||||
| RAYEN | |||||
| SRP |
| Set | Method | Served Rate | Training [s] | Inference [ms] |
| Softmax | ||||
| O-Proj | ||||
| DC3 | ||||
| HardNet | ||||
| RAYEN | ||||
| SRP |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Constraint Scope | Input Dep. | Satisfaction | Computation | Gradient Landscape | Univ. Approx. |
| Soft-Penalty | Any | Yes | No (Penalty) | Closed-Form | Smooth (infeasible) | Yes |
| DC3 [ 13 ] | General | Yes | Approx. ⋆ | Iterative (GD) | Step-size sensitive | Unknown |
| RAYEN [ 8 ] | Convex (Lin/QC/SOC/LMI) | Limited | Strict | Closed-Form | Clipped / Flat | Unknown |
| HardNet-Aff [ 16 ] | Affine | Yes | Strict | Closed-Form | Saturated (Boundary) | Yes |
| HP [ 15 ] | Ball-Homeomorphic | Yes | Strict | Bisection | Iterative search | Yes |
| HoP (Polar) [ 31 ] | Star-Convex | Yes | Strict (Int.) | Closed-Form † | Polar stagnation | Unknown |
| Method | Rel. gap | Training [s] | Inference [ms] | |
| 0.1 | O-Proj | |||
| 0.1 | DC3 | |||
| 0.1 | HardNet | |||
| 0.1 | RAYEN | |||
| 0.1 | SRP | |||
| 0.01 | O-Proj |
| Method | Rel. gap | Training [s] | Inference [ms] | |
| 0.1 | O-Proj | |||
| 0.1 | DC3 | |||
| 0.1 | HardNet | |||
| 0.1 | RAYEN | |||
| 0.1 | SRP | |||
| 0.01 | O-Proj |
| Matched HR | SRP | |
| 1 | ||
| 2 | ||
| 4 | ||
| 8 | ||
| 16 |
| Method | Training protocol | Relative gap |
| SRP | End-to-end (50 epochs) | |
| Pretrained, zero-shot | ||
| Pretrained, fine-tuned (20+30 epochs) | ||
| RAYEN | End-to-end (50 epochs) | |
| Pretrained, zero-shot | ||
| Pretrained, fine-tuned (20+30 epochs) |
| Parameter | Portfolio Optimization | Ride-Sharing Dispatch |
| Architecture & Training | ||
| Base Architecture | LSTM | MLP |
| Hidden Units | ||
| Dropout Rate | ||
| Optimizer | Adam | |
| Learning Rate | ||
| Universe | Split | Median | MaxDD | AC(1) | ||||
| Global | Train | |||||||
| Global | Val | |||||||
| Global | Test | |||||||
| Liquid | Train | |||||||
| Liquid | Val | |||||||
| Liquid | Test |
| Universe | Set | Method | SR (Net) | Turnover | Training [s] | Inference [ms] |
| Global | GLinSAT | |||||
| T-SKM-Net |
| Universe | Set | Method | SR (Net) | Turnover | Training [s] | Inference [ms] |
| Global | Softmax | |||||
| Global | O-Proj | |||||
| DC3 | ||||||
| HardNet | ||||||
| RAYEN | ||||||
| SRP |
| Universe | Set | Method | SR (Net) | Turnover | Training [s] | Inference [ms] |
| Global | Softmax | |||||
| Global | O-Proj | |||||
| DC3 | ||||||
| HardNet | ||||||
| RAYEN | ||||||
| SRP |
| Set | Method | Served Rate | Training [s] | Inference [ms] |
| Softmax | ||||
| O-Proj | ||||
| DC3 | ||||
| HardNet | ||||
| RAYEN | ||||
| SRP |
| Method | Rel. gap | Training [s] | Inference [ms] | |
| 8 | O-Proj | |||
| 8 | RAYEN | |||
| 8 | SRP | |||
| 12 | O-Proj | |||
| 12 | RAYEN | |||
| 12 | SRP |
| Scenario | Method | Rel. gap | Training [s] | Inference [ms] |
| 14 Bus | O-Proj | |||
| 14 Bus | RAYEN | |||
| 14 Bus | SRP | |||
| 118 Bus | O-Proj | |||
| 118 Bus | RAYEN | |||
| 118 Bus | SRP |
| Scenario | Method | Rel. gap | Training [s] | Inference [ms] | Feas. Rate | Anchor Precompute [ms] |
| 14 Bus | RAYEN | |||||
| 14 Bus | SRP |
Constrained Optimization
Constrained optimization finds the best decisions while satisfying equality, inequality, resource, safety, or fairness requirements.
See all 363 papers