stat.MLJul 8, 2026

Statistical inverse learning and \ell^1-regularization

Authors: Abhishake RastogiTatiana A. BubbaTapio HelinLuca Ratti

Abstract

We study the recovery of sparse functions from finite, noisy, and indirect observations in the framework of statistical inverse learning. The unknown is modeled as an element of 1\ell^1, and observations are generated through a possibly nonlinear forward operator A:1HA:\ell^1\to H, where HH is a vector-valued reproducing kernel Hilbert space. We propose an 1\ell^1-regularized empirical risk minimizer and develop a theoretical analysis of its statistical properties. Under mild assumptions, we establish almost-sure consistency and derive non-asymptotic high-probability convergence rates in both the prediction and 1\ell^1 reconstruction norms. The rates depend on the source smoothness parameter rr, characterized by a variational source condition, and the effective dimension exponent bb, describing the polynomial spectral decay of the covariance operator. We further prove matching minimax lower bounds, showing that the obtained convergence rates are optimal. To relate the theory to practical sparsity models, we consider finitely smoothing operators of the form A=GSA=G\circ S, where SS is a synthesis operator, and show that approximation-space assumptions imply the required variational source conditions. In particular, we prove that membership in the approximation space ktk_t is equivalent to polynomial decay of the best nn-term approximation error. Finally, we verify the assumptions for two representative inverse problems: reaction coefficient identification in elliptic PDEs and sparse computed tomography. For filtered Radon transforms, we derive explicit effective-dimension asymptotics, yielding concrete convergence rates for standard image models and sparsifying systems.

Explore similar work

May 11, 2025cs.LG

Learning from samples: inverse problems over measures

We study inverse problems where an unknown potential is observed only through samples from the measure it induces by a convex variational principle. Such problems arise in learning costs, energies, and dynamics from distributional data, but the associated forward solution map is typically nonlinear and implicit. We show that its optimality gap nevertheless yields convex empirical objectives for finite-dimensional potential classes, and we introduce sharpened Fenchel--Young losses that add a data-dependent discrepancy inside the forward problem. This keeps the estimator calibrated while improving the local geometry of the loss. Our main stability theorem separates the inverse error analysis into measurement error, forward perturbation, and empirical curvature. We instantiate this principle for inverse entropic unbalanced optimal transport and for inverse Jordan--Kinderlehrer--Otto (JKO) learning from independent snapshot samples, obtaining high-probability parameter recovery bounds. JKO schemes discretize Wasserstein gradient flows through a sequence of variational problems over measures, making them a natural language for population dynamics observed through snapshots. In this JKO case, the sharpened objective reduces to an unbalanced transport problem, which also clarifies the connection between variational gap losses and quadratic iJKO^\star surrogates. Numerical experiments illustrate the conditioning effect of sharpening and its benefits for sparse inverse-gradient-flow recovery.
Francisco Andrade, Gabriel Peyré, Clarice Poon
Feb 18, 2026math.ST

Separating Oblivious and Adaptive Models of Variable Selection

Sparse recovery is among the most well-studied problems in learning theory and high-dimensional statistics. In this work, we investigate the statistical and computational landscapes of sparse recovery with \ell_\infty error guarantees. This variant of the problem is motivated by \emph{variable selection} tasks, where the goal is to estimate the support of a kk-sparse signal in Rd\mathbb{R}^d. Our main contribution is a provable separation between the \emph{oblivious} (for each'') and \emph{adaptive} (for all'') models of \ell_\infty sparse recovery. We show that under an oblivious model, the optimal \ell_\infty error is attainable in near-linear time with klogd\approx k\log d samples, whereas in an adaptive model, k2\gtrsim k^2 samples are necessary for any algorithm to achieve this bound. This establishes a surprising contrast with the standard 2\ell_2 setting, where klogd\approx k \log d samples suffice even for adaptive sparse recovery. We conclude with a preliminary examination of a \emph{partially-adaptive} model, where we show nontrivial variable selection guarantees are possible with klogd\approx k\log d measurements.
Ziyun Chen, Jerry Li, Kevin Tian +1
Jun 22, 2026math.ST

Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence

We develop a comprehensive theory for regularized M-estimation in reproducing kernel Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the estimator, covering a wide range of convex and non-convex losses, including bounded robust losses. We further prove sharp rates of convergence with an explicit bias-variance decomposition governed by a novel complexity measure. We show that the variance is independent of misspecification, while the bias depends on a source condition parameter known in the learning literature. For tensor product Sobolev spaces we obtain new rates that connect to spaces of functions with dominating mixed smoothness, substantially extending existing results and explaining why these estimators circumvent the curse of dimensionality. Our methodology, combining elements from both functional analysis and empirical process theory, allows for an asymptotic linearisation of the objective function that avoids both closed-form solutions and global Lipschitz assumptions, and may be of independent interest. The estimators are implemented in C++ and theory is supported by numerical experiments.
Ioannis Kalogridis