cs.LGJan 23, 2026

Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processing

Authors: Francesca Lanzillotta, Chiara Albisani, Davide Pucci, Daniele Baracchi, Alessandro Piva, Matteo Lapucci

Organizations: The authors are with the Department of Information Engineering (DINFO) of the University of Florence, via di S. Marta 3, Firenze, 50139, Italy.

Abstract

In many learning tasks, certain requirements on the processing of individual data samples can arguably be formalized, in an intelligible way, as strict constraints of the underlying optimization problem, rather than by means of arbitrary penalties that often require heavy trial-and-error tuning. In this paper we show that, in these scenarios, learning can be carried out exploiting a sequential penalty method that allows to explicitly deal with constraints. For the proposed algorithm we show that, under classical assumptions and in the considered learning scenario, penalty subproblems can provably be solved in finite time with suitable guarantees of approximate stationarity in expectation; we also show that, when the latter condition is guaranteed for the subproblems, the overall sequential scheme possesses almost sure convergence properties to KKT-stationary points of (general) constrained problems. The results of experiments on sandbox and real-world image processing tasks show that the method is indeed viable to be used in practical deep learning scenarios.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Position: Adopt Constraints Over Fixed Penalties in Deep Learning

    May 27, 2025Juan Ramirez, Meraj Hashemizadeh, Simon Lacoste-JulienDeep Learning OptimizationConstrained Optimization

  2. Stochastic Penalty-Barrier Method for Constrained Machine Learning

    May 18, 2026Adam Bosák, Andrii Kliachkin, Gilles Bareilles +4Constrained Optimization

  3. Constrained Learning with Universally Learnable Concept Classes

    Aug 9, 2026Herlock SeyedAbolfazl Rahimi, Spyridon Pougkakiotis, Dionysis KalogeriasPrimal-Dual OptimizationReproducing Kernel Hilbert Spaces