stat.MLJul 12, 2026

Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization

Authors: Raziyeh Takbiri

Abstract

We consider the recovery of a pair of sparse vectors from a limited number of nonlinear observations of their superposition: yi=g(\inner\bai\bPhi\bw+\bPsi\bz)+eiy_i=g(\inner{\ba_i}{\bPhi\bw^\ast+\bPsi\bz^\ast})+e_i, i=1,,mi=1,\dots,m, with mnm\ll n, incoherent orthonormal bases \bPhi,\bPsi\bPhi,\bPsi, a scalar link gg, and noise eie_i that may be heavy-tailed or contaminated. We propose a regularization-based framework combining a Huberized data fidelity with generalized folded-concave penalties (SCAD, MCP), and a two-block proximal alternating algorithm with backtracking (NLD-PALM) whose whole iterate sequence provably converges to critical points under the Kurdyka--Łojasiewicz property, with local linear rates. On the statistical side we establish restricted strong convexity of the Huberized nonlinear loss through an exact sign-definite decomposition, and derive estimation error bounds of order σslog(n)/mσ\sqrt{s\log(n)/m} that hold at \emph{every} localized stationary point, an oracle rate σs/mσ\sqrt{s/m} free of logn\log n and shrinkage bias under a beta-min condition, and a co-equal recovery theorem for \emph{unknown} monotone links via a linear surrogate and a clipped Plan--Vershynin decoupling. The estimator requires no knowledge of the sparsity levels, and its guarantees hold under symmetric noise with only finite variance. Experiments at n=512n=512 under a frozen data-driven regularization rule show an earlier phase transition than convex 1\ell_1 demixing and greedy hard-thresholding baselines, a 35×35\times accuracy advantage over squared-loss estimation under 5%5\% gross outliers, and successful demixing of spike-plus-background signals observed through a saturating amplifier.

Explore similar work

Apr 25, 2026cs.IT

A Unified Fractional Regularization Framework for Sparse Recovery

We propose a unified fractional regularization framework for sparse signal recovery based on the 1/pq\ell_1/\ell_p^q model. This model generalizes several widely used sparsity-promoting regularizers and provides additional flexibility through the parameters pp and qq. Our main theoretical contribution is the characterization of the equivalence between the first-order stationary points of the 1/pq\ell_1/\ell_p^q formulation and the subtractive 1αp\ell_1-α\ell_p model, thereby offering a unified perspective on these nonconvex regularizers. In addition, we establish a new sufficient recovery condition under the Restricted Isometry Property (RIP), which shows that the proposed framework can provide relaxed recovery guarantees and improved robustness. To solve the resulting nonconvex problem, we develop a majorization--minimization (MM) algorithm and prove its convergence by using the Kurdyka--Łojasiewicz (KL) property. Numerical experiments on sparse recovery problems with different sensing matrices and MRI reconstruction demonstrate that the proposed approach outperforms existing methods in recovery accuracy.
Yinhao Zhao, Haoyu He, Chuanqi Ma +1
Feb 18, 2026math.ST

Separating Oblivious and Adaptive Models of Variable Selection

Sparse recovery is among the most well-studied problems in learning theory and high-dimensional statistics. In this work, we investigate the statistical and computational landscapes of sparse recovery with \ell_\infty error guarantees. This variant of the problem is motivated by \emph{variable selection} tasks, where the goal is to estimate the support of a kk-sparse signal in Rd\mathbb{R}^d. Our main contribution is a provable separation between the \emph{oblivious} (for each'') and \emph{adaptive} (for all'') models of \ell_\infty sparse recovery. We show that under an oblivious model, the optimal \ell_\infty error is attainable in near-linear time with klogd\approx k\log d samples, whereas in an adaptive model, k2\gtrsim k^2 samples are necessary for any algorithm to achieve this bound. This establishes a surprising contrast with the standard 2\ell_2 setting, where klogd\approx k \log d samples suffice even for adaptive sparse recovery. We conclude with a preliminary examination of a \emph{partially-adaptive} model, where we show nontrivial variable selection guarantees are possible with klogd\approx k\log d measurements.
Ziyun Chen, Jerry Li, Kevin Tian +1
Sep 1, 2025stat.ML

The Price of Sparsity: Sufficient Conditions for Sparse Recovery using Sparse and Sparsified Measurements

We consider the problem of support recovery for sparse binary signals from noisy linear measurements. For sparse Gaussian measurement matrices we identify sufficient conditions on the minimal sample size for maximum-likelihood recovery in the high-SNR regime ds/pds/p \to \infty, where pp denotes the signal dimension, ss the number of non-zero components of the signal, and dd the expected number of non-zero components per row of measurement. Combined with known lower bounds, this yields an information-theoretic threshold of order slog(p/s)/log(ds/p)s\log(p/s) / \log(ds/p), making explicit the price of measurement sparsity. In particular, we highlight a regime where the sample-complexity loss from measurement sparsity is logarithmic while the computational gain is nearly linear. Second, we study recovery after sparsifying an originally dense Gaussian design: the observations are generated from the dense design, while estimation uses an independently sparsified design and a rescaled response. In the proportional regime s=αps=αp, d=ψpd=ψp, we prove that, for every fixed target error level δδ and every slack ε>0\varepsilon>0, a sample size of order p/ψ2p/ψ^2 is sufficient for support recovery for arbitrarily small ψψ.
Youssef Chaabouni, David Gamarnik