math.OCJun 18, 2026

Beyond Averaging in John Ellipsoid Approximation: High-Accuracy Algorithms in the Leverage-Score Model

Authors: Xiaoyu LiJunwei YuJiaojiao JiangJunbin GaoAndi Han

Organizations: University of New South Wales · University of California, Berkeley · University of Sydney

Abstract

The John ellipsoid of a symmetric polytope P={xRd:Ax1}P=\{\mathbf{x}\in\mathbb{R}^d:\|\mathbf{A}\mathbf{x}\|_\infty\le1\}, ARn×d\mathbf{A}\in\mathbb{R}^{n\times d}, is computed by a long line of leverage-score algorithms, from Cohen, Cousins, Lee and Yang (COLT 2019) to its successors [WY24, CLS+25], all reaching a (1+ε)(1+\varepsilon)-approximation in Θ(ε1log(n/d))Θ(\varepsilon^{-1}\log(n/d)) iterations. We separate this complexity into three costs the modern line conflates (certification, identification, and accuracy) and locate the historical ε1\varepsilon^{-1} in the first alone. In the equivalent D-optimal-design form minpΔnlogdet(ipiaiai)\min_{\mathbf{p}\inΔ_n}-\log\det(\sum_i p_i\mathbf{a}_i\mathbf{a}_i^\top), the leverage-score oracle is exactly the first-order oracle and the (1+ε)(1+\varepsilon)-John guarantee the Frank-Wolfe gap g(p)εdg(\mathbf{p})\le\varepsilon d; through this dictionary the costs come apart. The ε1\varepsilon^{-1} is a certification artifact: the uniform average of the iterates, the certificate used throughout the line, has gap exactly Θ(1/T)Θ(1/T), however cheap each iteration is made. Pointed instead at the last iterate the same oracle is fast: a warm-started accelerated method reaches the guarantee in C(A)+O(κlog(1/ε))C(\mathbf{A})+O(\sqrtκ\log(1/\varepsilon)) queries after an ε\varepsilon-independent setup C(A)C(\mathbf{A}), and once the optimal face is identified the facial problem is an unconstrained self-concordant minimization whose Hessian the oracle recovers exactly, so damped Newton needs only O(loglog(1/ε))O(\log\log(1/\varepsilon)) steps, for a total of C(A)+O(d2loglog(1/ε))C(\mathbf{A})+O(d^2\log\log(1/\varepsilon)) queries. The accuracy dependence is thus doubly logarithmic after an ε\varepsilon-independent, condition-dependent setup; the open problem is the remaining identification cost (a condition-free bound on reaching the optimal face) and lower bounds. Accuracy is not the obstruction.

Explore similar work

Nov 4, 2025math.OC

Accelerated Frank-Wolfe Algorithms: Complementarity Conditions and Sparsity

We develop new accelerated first-order algorithms in the Frank-Wolfe (FW) family for minimizing smooth convex functions over compact convex sets, with a focus on two prominent constraint classes: (1) polytopes and (2) matrix domains given by the spectrahedron and nuclear-norm balls. A key technical ingredient is a complementarity condition that captures solution sparsity---face dimension for polytopes and rank for matrices. We present two algorithms: (1) a purely linear optimization oracle (LOO) method for polytopes that has optimal worst-case first-order (FO) oracle complexity and, aside of a finite \emph{burn-in} phase and up to a logarithmic factor, has LOO complexity that scales with r/εr/\sqrtε, where εε is the target accuracy and rr is the solution sparsity (independently of the ambient dimension), and (2) a hybrid scheme that combines FW with a sparse projection oracle (e.g., low-rank SVDs for matrix domains with low-rank solutions), which also has optimal FO oracle complexity, and after a finite burn-in phase, only requires O(1/ε)O(1/\sqrtε) sparse projections and LOO calls (independently of both the ambient dimension and the sparsity level of optimal solutions). Our results close a gap on how to accelerate recent advancements in linearly-converging FW algorithms for strongly convex optimization, without paying the price of the dimension.
Dan Garber
May 29, 2026math.OC

Wall-Clock Complexity for Zeroth-Order Optimization with Tunable Oracle Fidelity

Zeroth-order (black-box) optimization is applied when gradients are unavailable and objective evaluations rely on expensive simulations. In many such applications, the oracle fidelity is tunable: higher-accuracy queries reduce noise but incur higher computational costs. To capture this trade-off, we study an accuracy-aware wall-clock model where each query with fidelity δδ has a cost c(δ)c(δ), and we minimize the total time Ttotal=k=1Nc(δk)T_{\mathrm{total}} = \sum_{k=1}^{N} c(δ_k), subject to a target accuracy constraint. We show how the choice of oracle type, noise model, and optimization scheme induces explicit wall-clock-optimal choices for the algorithmic parameters. For instance, we demonstrate that accelerated methods can be wall-clock inferior to non-accelerated schemes. Furthermore, we characterize the conditions under which a constant fidelity strategy is optimal in the Big-O sense. Our framework provides a unified methodology to translate convergence guarantees into practical fidelity and batching recommendations.
Alexandra Suvorikova, Igor Pavlov, Artem Vasin +4
Jul 21, 2026cs.DS

Stronger Memory-Query Tradeoffs for Convex Optimization: The Limitations of Subquadratic Memory

We prove two lower bounds for the first order oracle complexity of minimizing a dd-dimensional 11-Lipschitz convex function over the unit ball with mm bits of memory. We first show that any such (possibly randomized) algorithm must make Ω~(d2m)\tildeΩ(\frac{d^2}{\sqrt{m}}) oracle queries. For deterministic optimization algorithms, we show that Ω~(min{d1.6,d8/3m2/3})\tildeΩ(\min\{d^{1.6},\frac{d^{8/3}}{m^{2/3}}\}) queries are required. For all memory regimes of interest, these improves upon the previous best known lower bounds of Ω~(max{d8/3m4/3,d4/3m1/6})\tildeΩ(\max\{\frac{d^{8/3}}{m^{4/3}},\frac{d^{4/3}}{m^{1/6}}\}) and Ω~(d5/3m1/3)\tildeΩ(\frac{d^{5/3}}{m^{1/3}}) for randomized and deterministic algorithms respectively. Notably, due to existing upper bounds, our lower bound for deterministic algorithms is the first to show a sharp oracle complexity phase transition around md2m\approx d^2, where a polylogarithmic change in memory leads to a poly(d)\mathsf{poly}(d) change in the number of required oracle calls. Further, when the suboptimality is polynomially small in dd, our lower bound randomized algorithms is the first to show that Ω~(d2)\tildeΩ(d^2) memory is necessary to nearly match the optimal query complexity among algorithms without memory constraints. Previously, such a result was only known for the regime where the suboptimality is quasipolynomially small in dd.
Michael Menart, Aleksandar Nikolov, Ohad Shamir