A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

Yuchen Fang; Jihun Kim; Sen Na; James Demmel; Javad Lavaei

math.OCMar 10, 2026

A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

Authors: Yuchen Fang, Jihun Kim, Sen Na, James Demmel, Javad Lavaei

Organizations: Department of Mathematics, University of California, Berkeley · Department of IEOR, University of California, Berkeley · School of Industrial and Systems Engineering, Georgia Tech · Department of EECS, University of California, Berkeley

Abstract

In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.

Figures & tables

Panel of Figure 1 — Figure 1: Relative KKT residuals of TR-IP-SSQP with different barrier parameter setups. In every plot, each box corresponds to TR-IP-SSQP with one kind of Hessian construction.

Panel of Figure 2 — Figure 2: Performance Profile of TR-IP-SSQP when $\theta_{k}=0.9999^{k}$ . Each trajectory represents one algorithm configuration.

Panel of Figure 3 — Figure 3: Performance Profile of TR-IP-SSQP and Fully-TR-IP-SSQP when $\theta_{k}=0.9999^{k}$ . Each trajectory represents one algorithm configuration.

Panel of Figure 4 — Figure 4: Performance Profile of TR-IP-SSQP and Fully-TR-IP-SSQP over 40 constrained Logistic regression problem instances. Each trajectory represents one method.

Constrained Optimization

Constrained optimization finds the best decisions while satisfying equality, inequality, resource, safety, or fairness requirements.

See all 335 papers

TR-SSQP: A Trust-Region Method for Constrained Stochastic Optimization under Heavy-Tailed Noise
Sep 25, 2026Haoxuan Wang, Yuchen Fang, Sen NaStochastic Optimization Stochastic Optimization Convergence
School of Industrial and Systems Engineering, Georgia Institute of Technology · Department of Mathematics, University of California, Berkeley
Stochastic Trust-Region Methods for Over-parameterized Models
Apr 15, 2026Aike Yang, Hao WangTrust-Region Optimization Nonconvex Stochastic Optimization
ShanghaiTech University, Shanghai, China
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
May 12, 2026Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3Nonconvex Stochastic Optimization Muon Optimizer
KU Leuven, Department of Electrical Engineering, STADIUS Center for Dynamical Systems, Signal Processing, and Data Analytics · ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Laboratory for Information and Inference Systems (LIONS) · CVN, CentraleSup´elec, Universit´e Paris-Saclay, Inria

Sep 25, 2026J/K move · Enter open · S save

TR-SSQP: A Trust-Region Method for Constrained Stochastic Optimization under Heavy-Tailed Noise

Haoxuan Wang, Yuchen Fang, Sen Na

School of Industrial and Systems Engineering, Georgia Institute of Technology · Department of Mathematics, University of California, Berkeley

Stochastic Optimization Stochastic Optimization Convergence Heavy-Tailed Stochastic Optimization Trust-Region Optimization

Open paper AskPDF arXiv

We consider stochastic nonlinear optimization problems with deterministic equality constraints. While unconstrained stochastic optimization is well understood, the interplay between optimality and feasibility in the constrained setting poses significant challenges. Moreover, existing theoretical guarantees for constrained stochastic methods predominantly rely on bounded-variance assumptions, leaving the heavy-tailed noise regime largely unexplored. To address this gap, we propose a novel trust-region method within the stochastic sequential quadratic programming framework, termed TR-SSQP. Our method employs a normal-tangential decomposition in the step computation to balance optimality and feasibility. In addition, we incorporate a normalization mechanism in the design of the trust-region radius, together with Polyak momentum for gradient estimation, ensuring stable updates without gradient clipping. When the trust-region radius and the momentum parameter decay at appropriate rates, we establish global almost-sure convergence of the method. To the best of our knowledge, this is the first asymptotic convergence result for constrained stochastic optimization under heavy-tailed noise. We demonstrate the promising performance of the proposed method through extensive numerical experiments, including comparisons among its variants and with existing constrained stochastic optimization methods.

Abstract

Constrained Optimization

Explore similar work

TR-SSQP: A Trust-Region Method for Constrained Stochastic Optimization under Heavy-Tailed Noise

Stochastic Trust-Region Methods for Over-parameterized Models

Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives