Adaptive regularization parameter selection for high-dimensional inverse problems: A Bayesian approach with Tucker low-rank constraints

Qing-Mei Yang; Da-Qing Zhang

cs.LGMar 17, 2026

Adaptive regularization parameter selection for high-dimensional inverse problems: A Bayesian approach with Tucker low-rank constraints

Authors: Qing-Mei Yang, Da-Qing Zhang

Organizations: School of Science, University of Science and Technology Liaoning, Anshan, Liaoning Province, 114051, PR China

Abstract

This paper introduces a novel variational Bayesian method that integrates Tucker decomposition for efficient high-dimensional inverse problem solving. The method reduces computational complexity by transforming variational inference from a high-dimensional space to a lower-dimensional core tensor space via Tucker decomposition. A key innovation is the introduction of per-mode precision parameters, enabling adaptive regularization for anisotropic structures. For instance, in directional image deblurring, learned parameters align with physical anisotropy, applying stronger regularization to critical directions (e.g., row vs. column axes). The method further estimates noise levels from data, eliminating reliance on prior knowledge of noise parameters (unlike conventional benchmarks such as the discrepancy principle (DP)). Experimental evaluations across 2D deblurring, 3D heat conduction, and Fredholm integral equations demonstrate consistent improvements in quantitative metrics (PSNR, SSIM) and qualitative visualizations (error maps, precision parameter trends) compared to L-curve criterion, generalized cross-validation (GCV), unbiased predictive risk estimator (UPRE), and DP. The approach scales to problems with 110,000 variables and outperforms existing methods by 0.73-2.09 dB in deblurring tasks and 6.75 dB in 3D heat conduction. Limitations include sensitivity to rank selection in Tucker decomposition and the need for theoretical analysis. Future work will explore automated rank selection and theoretical guarantees. This method bridges Bayesian theory and scalable computation, offering practical solutions for large-scale inverse problems in imaging, remote sensing, and scientific computing.

Figures & tables

	Direct VB	Tucker-VB
Posterior covariance size	$n\times n$	$n_{\mathcal{G}}\times n_{\mathcal{G}}$
Matrix inversion	$O(n^{3})$	$O(n_{\mathcal{G}}^{3})$
Storage	$O(n^{2})$	$O(n_{\mathcal{G}}^{2})$
Per-iteration cost	$O(n^{3}+mn^{2})$	$O(n_{\mathcal{G}}^{3}+mn_{\mathcal{G}}^{2})$

Table 1: Complexity comparison

Noise Level $\sigma$	Tucker-VB	VB	Relative Gap
0.01	$0.0319\pm 0.0007$	$0.0250\pm 0.0006$	$+27.6\%$
0.02	$0.0420\pm 0.0022$	$0.0349\pm 0.0017$	$+20.3\%$
0.05	$0.0830\pm 0.0069$	$0.0730\pm 0.0050$	$+13.7\%$
0.10	$0.1548\pm 0.0138$	$0.1391\pm 0.0101$	$+11.3\%$
0.20	$0.2809\pm 0.0244$	$0.2598\pm 0.0192$	$+8.1\%$

Table 2: Reconstruction error comparison across noise levels

Noise Level $\sigma$	Tucker-VB $\hat{\sigma}$	Rel. Error	VB $\hat{\sigma}$	Rel. Error
0.01	$0.0103\pm 0.0001$	$+3.0\%$	$0.0101\pm 0.0001$	$+1.0\%$
0.02	$0.0201\pm 0.0002$	$+0.5\%$	$0.0200\pm 0.0002$	$0.0\%$
0.05	$0.0499\pm 0.0006$	$-0.2\%$	$0.0499\pm 0.0006$	$-0.2\%$
0.10	$0.0997\pm 0.0012$	$-0.3\%$	$0.0997\pm 0.0012$	$-0.3\%$
0.20	$0.1992\pm 0.0024$	$-0.4\%$	$0.1993\pm 0.0024$	$-0.4\%$

Table 3: Noise estimation accuracy across noise levels

Dimension $n$	Problem Size $n^{2}$	Tucker-VB	VB	Speedup
30	900	$0.0012\pm 0.0031$ s	$0.109\pm 0.020$ s	$91\times$
50	2,500	$0.0016\pm 0.0030$ s	$5.70\pm 0.38$ s	$3,563\times$
80	6,400	$0.0027\pm 0.0031$ s	$111.6\pm 7.6$ s	$41,333\times$
100	10,000	$0.0019\pm 0.0037$ s	N/A	—
150	22,500	$0.0050\pm 0.0056$ s	N/A	—
200	40,000	$0.0085\pm 0.0045$ s	N/A	—

Table 4: Computation time comparison across problem dimensions

Figure 1: Reconstruction error curves from 500 independent trials at six noise levels: L-curve (gray), GCV (orange), UPRE (green), DP (purple), and Tucker-VB (red).

Figure 2: (A) Relative reconstruction error versus noise level

\sigma

for five regularization methods; shaded regions indicate

\pm 1

standard deviation. (B) Percentage improvement of Tucker-VB over other methods.

Noise Level	Method	PSNR (dB)	SSIM	$\Delta$ PSNR (dB)	$\Delta$ SSIM
$\sigma=0.01$	L-curve	17.80	0.9022	+12.47	+0.0919
	GCV	26.82	0.9870	+3.45	+0.0071
	UPRE	26.66	0.9865	+3.61	+0.0076
	DP	29.54	0.9931	+0.73	+0.0010
	Tucker-VB	30.27	0.9941	—	—
$\sigma=0.03$	L-curve	16.87	0.8800	+10.68	+0.1089

Table 5: Reconstruction performance comparison across noise levels

Figure 3: Anisotropic Gaussian deblurring results ( $\sigma=0.01$ ). (A) Original image; (B) Blurred and noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 4: Anisotropic Gaussian deblurring results ( $\sigma=0.03$ ). (A) Original image; (B) Blurred and noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 5: Precision parameters

\alpha_{k}

learned by Tucker-VB. Left column: row-direction precision

\alpha_{k}^{\mathrm{row}}

; right column: column-direction precision

\alpha_{l}^{\mathrm{col}}

. Top row:

\sigma=0.01

; bottom row:

\sigma=0.03

	PSNR (dB)			Tucker-VB Gain (dB)
Method	1% noise	5% noise	10% noise	1% noise	5% noise	10% noise
L-curve	36.74	33.38	30.99	+12.81	+8.01	+7.88
GCV	36.97	30.38	27.26	+12.58	+11.01	+11.61
UPRE	36.95	30.39	27.31	+12.60	+11.00	+11.56
DP	42.72	35.41	31.42	+6.83	+5.98	+7.45
Tucker-VB	49.55	41.39	38.87	—	—	—

Table 6: Comparison of regularization methods for 3D backward heat equation inverse problem

Figure 6: Reconstruction results for 3D backward heat equation at $\sigma=0.01$ (1% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 7: Reconstruction results for 3D backward heat equation at $\sigma=0.05$ (5% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 8: Reconstruction results for 3D backward heat equation at $\sigma=0.1$ (10% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 9: Precision parameters learned by Tucker-VB at different noise levels. (A)

\sigma=0.01

; (B)

\sigma=0.05

; (C)

\sigma=0.1

Explore similar work

May 28, 2026

Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems

Solving high-dimensional PDE-governed inverse problems is often challenging due to complex non-Gaussian posterior distributions, expensive forward model evaluations, and misspecified prior information. To address these issues, we propose a deep adaptive dimension-reduction Bayesian inference framework based on the Variational Flow (VF) model. Since standard normalizing flows are restricted by bijective mappings and cannot directly reduce dimensions, VF overcomes this limitation by integrating VAE-based nonlinear dimension reduction with dual normalizing flows for the latent prior and encoder. This design provides a strictly higher evidence lower bound than VAE and allows more flexible approximation of complex posterior distributions. We further introduce an iterative prior updating strategy that gradually moves the prior mean toward high-probability posterior regions, avoiding manual prior tuning. These components form a closed adaptive loop together with an adaptively fine-tuned Fourier Neural Operator (FNO) surrogate: VF generates posterior-concentrated samples to refine the surrogate, while the updated surrogate further improves posterior inference. Numerical experiments on a 100-dimensional Rosenbrock problem and three standard PDE-governed inverse problems show that our method delivers competitive or superior accuracy compared with MCMC, UKI, and SVGD baselines across all tested configurations, with the most pronounced advantages emerging in challenging scenarios such as high-noise observations and high-dimensional parameter spaces.

Yueyang Wang, Xili Wang, Kejun Tang +3

School of Mathematical Sciences, Peking University · School of Sciences, Great Bay University · Department of Mathematics, Louisiana State University +1

Bayesian Inverse Problems Dimensionality Reduction PDE Inverse Problems

Ask PDF

Mar 13, 2026

Variational Garrote for Sparse Inverse Problems

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.

Kanghun Lee, Hyungjoon Soh, Junghyo Jo

Department of Science Education, Seoul National University, Seoul, 08826, Korea · Center for Theoretical Physics and Artificial Intelligence Institute, Seoul National University, Seoul, 08826, Korea · School of Computational Sciences, Korea Institute for Advanced Study, Seoul, 02455, Korea

Sparse Recovery Variational Inference

Ask PDF

Jun 2, 2026

Bayesian Tensor Decomposition with Diffusion Model Prior

Low-rank tensor decomposition (TD) is usually effective on clean, fully observed data, but it often degrades under severe missingness or noise. Low-rankness is itself a useful but limited structural prior, and additional handcrafted priors (e.g., sparsity or smoothness) still fall short of capturing the rich statistics of real-world data. To compensate for this weak inductive bias under heavy corruption, one would like to inject a learned, data-driven prior; however, the state-of-the-art diffusion models are not readily compatible with current TD and tractable posterior inference. To address these challenges, we introduce DiffBCP, a hybrid-prior Bayesian CP decomposition framework that couples a cumulative shrinkage process prior over the CP factors for automatic rank selection with an off-the-shelf pre-trained diffusion model as an implicit data prior on the reconstructed tensor. To make posterior inference tractable despite the coupling among the likelihood, low-rank constraint, and diffusion prior, we develop a split Gibbs sampler: CP factors admit conjugate updates, while the diffusion block is sampled via low-rank-guided denoising. A noise-adaptive coupling schedule further reduces sensitivity to hand-tuned annealing. Experiments on image inpainting and denoising, including high-resolution out-of-distribution images, show consistent gains over Bayesian, nonlinear, and plug-and-play TD baselines.

Zerui Tao, Qibin Zhao

RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan.

Bayesian Inverse Problems Tensor Decomposition Diffusion Posterior Sampling

Ask PDF