Variational Garrote for Sparse Inverse Problems

Kanghun Lee; Hyungjoon Soh; Junghyo Jo

stat.MLMar 13, 2026

Variational Garrote for Sparse Inverse Problems

Authors: Kanghun Lee, Hyungjoon Soh, Junghyo Jo

Organizations: Department of Science Education, Seoul National University, Seoul, 08826, Korea · Center for Theoretical Physics and Artificial Intelligence Institute, Seoul National University, Seoul, 08826, Korea · School of Computational Sciences, Korea Institute for Advanced Study, Seoul, 02455, Korea

Abstract

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.

Figures & tables

Task	Forward Operator $\mathbf{A}$	Information Bottleneck	Sparsity Control
Signal Resampling	Subsampling / masking operator	Sampling ratio $R=M/N$	transform sparsity (DCT)
Signal Denoising	Identity ( $\mathbf{A}=\mathbf{I}$ )	Noise amplitude $\alpha$	transform sparsity (DCT)
Sparse-view CT Reconstruction	Discrete Radon transform	Number of angles $K$	image-domain sparsity

Table 1: Inverse problems considered in this work expressed within a unified sparse reconstruction framework. All tasks share the same sparse regression formulation but differ in the forward operator and the form of information bottleneck.

Figure 1: Conceptual illustration of sparse reconstruction tasks. (a) Signal resampling: a synthetic sinusoidal signal (black) is observed through random subsampling (red markers) and reconstructed using sparsity-based regularization (red dashed line). The inset shows the magnitude of the discrete cosine transform (DCT) spectrum. (b) Signal denoising: a clean signal (black) corrupted by additive Gaussian noise (gray) and its reconstructed estimate (red dashed line). The inset shows the corresponding DCT spectra of the noisy and reconstructed signals.

Figure 2: Train–generalization (

\mathcal{E}_{\mathrm{train}}

vs.

\mathcal{E}_{\mathrm{gen}}

) error curves for LASSO and Variational Garrote (VG). Top row: signal resampling experiments for the synthetic signal (a) and real flute signal (b). Bottom row: signal denoising experiments for the synthetic (c) and real (d) datasets. For each task, the left plot shows the relationship between training error and generalization error on log–log scales. Solid lines denote VG and dashed lines denote LASSO. Curve colors indicate different sampling ratios (top) or noise amplitudes (bottom). Markers indicate the hyperparameter setting achieving the minimum generalization error. The dotted diagonal line represents

\mathcal{E}_{\mathrm{train}}=\mathcal{E}_{\mathrm{gen}}

. The adjacent right plot explicitly shows this minimum generalization error as a function of the sampling ratio or noise amplitude.

Figure 3: Qualitative comparison of sparse-view CT reconstructions ( $K=40$ projection angles). Rows correspond to four datasets: Shepp–Logan phantom, LIDC-IDRI chest CT, BraTS brain MRI, and Walnut CT. Columns show the ground truth image, reconstructions obtained with filtered back-projection (FBP), LASSO, and the Variational Garrote (VG), together with their corresponding absolute error maps (“Diff”). Brighter colors in the difference maps indicate larger reconstruction errors.

Figure 4: Reconstruction error versus angular resolution in sparse-view CT. Mean squared error (MSE) between reconstructed and ground-truth images is shown as a function of the number of projection angles

K

for four datasets: (a) Shepp–Logan phantom, (b) LIDC-IDRI chest CT, (c) BraTS brain MRI, and (d) Walnut CT. Filtered back-projection (FBP), LASSO, and Variational Garrote (VG) are compared. Each point corresponds to the reconstruction obtained with the hyperparameter that minimizes MSE for the given

K

. Markers show the mean over 10 trials and error bars denote standard deviation.

Explore similar work

Statistical inverse learning and $\ell^1$ -regularization
Jul 8, 2026Abhishake Rastogi, Tatiana A. Bubba, Tapio Helin +1Sparse Recovery Statistical Learning Theory
Department of Computational Engineering, LUT University, Lappeenranta, Finland. · Department of Mathematics and Computer Science, University of Ferrara, Ferrara, Italy.
A Stability Benchmark of Generative Regularizers for Inverse Problems
May 11, 2026Alexander Denker, Johannes Hertrich, Sebastian NeumayerGenerative Modeling Diffusion-Based Inverse Problems
Helmholtz Imaging, Deutsches Elektronen-Synchrotron DESY, Germany · ENS Paris & Inria, France · Chemnitz University of Technology, Germany
Inverse-Free Sparse Variational Gaussian Processes
Apr 1, 2026Stefano Cortinovis, Laurence Aitchison, Stefanos Eleftheriadis +1Gaussian Processes Variational Inference
University of Oxford · University of Bristol

Jul 8, 2026J/K move · Enter open · S save

Statistical inverse learning and $\ell^1$ -regularization

Abhishake Rastogi, Tatiana A. Bubba, Tapio Helin, Luca Ratti

Department of Computational Engineering, LUT University, Lappeenranta, Finland. · Department of Mathematics and Computer Science, University of Ferrara, Ferrara, Italy.

Sparse Recovery Statistical Learning Theory

Open paper AskPDF arXiv

We study the recovery of sparse functions from finite, noisy, and indirect observations in the framework of statistical inverse learning. The unknown is modeled as an element of

\ell^1

, and observations are generated through a possibly nonlinear forward operator

A:\ell^1\to H

, where

H

is a vector-valued reproducing kernel Hilbert space. We propose an

\ell^1

-regularized empirical risk minimizer and develop a theoretical analysis of its statistical properties. Under mild assumptions, we establish almost-sure consistency and derive non-asymptotic high-probability convergence rates in both the prediction and

\ell^1

reconstruction norms. The rates depend on the source smoothness parameter

r

, characterized by a variational source condition, and the effective dimension exponent

b

, describing the polynomial spectral decay of the covariance operator. We further prove matching minimax lower bounds, showing that the obtained convergence rates are optimal. To relate the theory to practical sparsity models, we consider finitely smoothing operators of the form

A=G\circ S

, where

S

is a synthesis operator, and show that approximation-space assumptions imply the required variational source conditions. In particular, we prove that membership in the approximation space

k_t

is equivalent to polynomial decay of the best

n

-term approximation error. Finally, we verify the assumptions for two representative inverse problems: reaction coefficient identification in elliptic PDEs and sparse computed tomography. For filtered Radon transforms, we derive explicit effective-dimension asymptotics, yielding concrete convergence rates for standard image models and sparsifying systems.

Abstract

Explore similar work

Statistical inverse learning and ℓ1\ell^1ℓ1-regularization

A Stability Benchmark of Generative Regularizers for Inverse Problems

Inverse-Free Sparse Variational Gaussian Processes

Statistical inverse learning and $\ell^1$ -regularization