Adaptive regularization parameter selection for high-dimensional inverse problems: A Bayesian approach with Tucker low-rank constraints

Qing-Mei Yang; Da-Qing Zhang

cs.LGMar 17, 2026

Adaptive regularization parameter selection for high-dimensional inverse problems: A Bayesian approach with Tucker low-rank constraints

Authors: Qing-Mei Yang, Da-Qing Zhang

Organizations: School of Science, University of Science and Technology Liaoning, Anshan, Liaoning Province, 114051, PR China

Abstract

This paper introduces a novel variational Bayesian method that integrates Tucker decomposition for efficient high-dimensional inverse problem solving. The method reduces computational complexity by transforming variational inference from a high-dimensional space to a lower-dimensional core tensor space via Tucker decomposition. A key innovation is the introduction of per-mode precision parameters, enabling adaptive regularization for anisotropic structures. For instance, in directional image deblurring, learned parameters align with physical anisotropy, applying stronger regularization to critical directions (e.g., row vs. column axes). The method further estimates noise levels from data, eliminating reliance on prior knowledge of noise parameters (unlike conventional benchmarks such as the discrepancy principle (DP)). Experimental evaluations across 2D deblurring, 3D heat conduction, and Fredholm integral equations demonstrate consistent improvements in quantitative metrics (PSNR, SSIM) and qualitative visualizations (error maps, precision parameter trends) compared to L-curve criterion, generalized cross-validation (GCV), unbiased predictive risk estimator (UPRE), and DP. The approach scales to problems with 110,000 variables and outperforms existing methods by 0.73-2.09 dB in deblurring tasks and 6.75 dB in 3D heat conduction. Limitations include sensitivity to rank selection in Tucker decomposition and the need for theoretical analysis. Future work will explore automated rank selection and theoretical guarantees. This method bridges Bayesian theory and scalable computation, offering practical solutions for large-scale inverse problems in imaging, remote sensing, and scientific computing.

Figures & tables

	Direct VB	Tucker-VB
Posterior covariance size	$n\times n$	$n_{\mathcal{G}}\times n_{\mathcal{G}}$
Matrix inversion	$O(n^{3})$	$O(n_{\mathcal{G}}^{3})$
Storage	$O(n^{2})$	$O(n_{\mathcal{G}}^{2})$
Per-iteration cost	$O(n^{3}+mn^{2})$	$O(n_{\mathcal{G}}^{3}+mn_{\mathcal{G}}^{2})$

Table 1: Complexity comparison

Noise Level $\sigma$	Tucker-VB	VB	Relative Gap
0.01	$0.0319\pm 0.0007$	$0.0250\pm 0.0006$	$+27.6\%$
0.02	$0.0420\pm 0.0022$	$0.0349\pm 0.0017$	$+20.3\%$
0.05	$0.0830\pm 0.0069$	$0.0730\pm 0.0050$	$+13.7\%$
0.10	$0.1548\pm 0.0138$	$0.1391\pm 0.0101$	$+11.3\%$
0.20	$0.2809\pm 0.0244$	$0.2598\pm 0.0192$	$+8.1\%$

Table 2: Reconstruction error comparison across noise levels

Noise Level $\sigma$	Tucker-VB $\hat{\sigma}$	Rel. Error	VB $\hat{\sigma}$	Rel. Error
0.01	$0.0103\pm 0.0001$	$+3.0\%$	$0.0101\pm 0.0001$	$+1.0\%$
0.02	$0.0201\pm 0.0002$	$+0.5\%$	$0.0200\pm 0.0002$	$0.0\%$
0.05	$0.0499\pm 0.0006$	$-0.2\%$	$0.0499\pm 0.0006$	$-0.2\%$
0.10	$0.0997\pm 0.0012$	$-0.3\%$	$0.0997\pm 0.0012$	$-0.3\%$
0.20	$0.1992\pm 0.0024$	$-0.4\%$	$0.1993\pm 0.0024$	$-0.4\%$

Table 3: Noise estimation accuracy across noise levels

Dimension $n$	Problem Size $n^{2}$	Tucker-VB	VB	Speedup
30	900	$0.0012\pm 0.0031$ s	$0.109\pm 0.020$ s	$91\times$
50	2,500	$0.0016\pm 0.0030$ s	$5.70\pm 0.38$ s	$3,563\times$
80	6,400	$0.0027\pm 0.0031$ s	$111.6\pm 7.6$ s	$41,333\times$
100	10,000	$0.0019\pm 0.0037$ s	N/A	—
150	22,500	$0.0050\pm 0.0056$ s	N/A	—
200	40,000	$0.0085\pm 0.0045$ s	N/A	—

Table 4: Computation time comparison across problem dimensions

Figure 1: Reconstruction error curves from 500 independent trials at six noise levels: L-curve (gray), GCV (orange), UPRE (green), DP (purple), and Tucker-VB (red).

Figure 2: (A) Relative reconstruction error versus noise level

\sigma

for five regularization methods; shaded regions indicate

\pm 1

standard deviation. (B) Percentage improvement of Tucker-VB over other methods.

Noise Level	Method	PSNR (dB)	SSIM	$\Delta$ PSNR (dB)	$\Delta$ SSIM
$\sigma=0.01$	L-curve	17.80	0.9022	+12.47	+0.0919
	GCV	26.82	0.9870	+3.45	+0.0071
	UPRE	26.66	0.9865	+3.61	+0.0076
	DP	29.54	0.9931	+0.73	+0.0010
	Tucker-VB	30.27	0.9941	—	—
$\sigma=0.03$	L-curve	16.87	0.8800	+10.68	+0.1089

Table 5: Reconstruction performance comparison across noise levels

Figure 3: Anisotropic Gaussian deblurring results ( $\sigma=0.01$ ). (A) Original image; (B) Blurred and noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 4: Anisotropic Gaussian deblurring results ( $\sigma=0.03$ ). (A) Original image; (B) Blurred and noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 5: Precision parameters

\alpha_{k}

learned by Tucker-VB. Left column: row-direction precision

\alpha_{k}^{\mathrm{row}}

; right column: column-direction precision

\alpha_{l}^{\mathrm{col}}

. Top row:

\sigma=0.01

; bottom row:

\sigma=0.03

	PSNR (dB)			Tucker-VB Gain (dB)
Method	1% noise	5% noise	10% noise	1% noise	5% noise	10% noise
L-curve	36.74	33.38	30.99	+12.81	+8.01	+7.88
GCV	36.97	30.38	27.26	+12.58	+11.01	+11.61
UPRE	36.95	30.39	27.31	+12.60	+11.00	+11.56
DP	42.72	35.41	31.42	+6.83	+5.98	+7.45
Tucker-VB	49.55	41.39	38.87	—	—	—

Table 6: Comparison of regularization methods for 3D backward heat equation inverse problem

Figure 6: Reconstruction results for 3D backward heat equation at $\sigma=0.01$ (1% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 7: Reconstruction results for 3D backward heat equation at $\sigma=0.05$ (5% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 8: Reconstruction results for 3D backward heat equation at $\sigma=0.1$ (10% noise). (A) True initial temperature field; (B) Noisy observation; (C) L-curve reconstruction; (D) GCV reconstruction; (E) UPRE reconstruction; (F) DP reconstruction; (G) Tucker-VB reconstruction; (H) Tucker-VB absolute error map.

Figure 9: Precision parameters learned by Tucker-VB at different noise levels. (A)

\sigma=0.01

; (B)

\sigma=0.05

; (C)

\sigma=0.1

Explore similar work

Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems
May 28, 2026Yueyang Wang, Xili Wang, Kejun Tang +3Bayesian Inverse Problems Dimensionality Reduction
School of Mathematical Sciences, Peking University · School of Sciences, Great Bay University · Department of Mathematics, Louisiana State University +1
Variational Garrote for Sparse Inverse Problems
Mar 13, 2026Kanghun Lee, Hyungjoon Soh, Junghyo JoSparse Recovery Variational Inference
Department of Science Education, Seoul National University, Seoul, 08826, Korea · Center for Theoretical Physics and Artificial Intelligence Institute, Seoul National University, Seoul, 08826, Korea · School of Computational Sciences, Korea Institute for Advanced Study, Seoul, 02455, Korea
Bayesian Tensor Decomposition with Diffusion Model Prior
Jun 2, 2026Zerui Tao, Qibin ZhaoBayesian Inverse Problems Tensor Decomposition
RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan.

May 28, 2026J/K move · Enter open · S save

Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems

Yueyang Wang, Xili Wang, Kejun Tang, Xiaoliang Wan+2

School of Mathematical Sciences, Peking University · School of Sciences, Great Bay University · Department of Mathematics, Louisiana State University+1

Bayesian Inverse Problems Dimensionality Reduction PDE Inverse Problems

Open paper AskPDF arXiv

Solving high-dimensional PDE-governed inverse problems is often challenging due to complex non-Gaussian posterior distributions, expensive forward model evaluations, and misspecified prior information. To address these issues, we propose a deep adaptive dimension-reduction Bayesian inference framework based on the Variational Flow (VF) model. Since standard normalizing flows are restricted by bijective mappings and cannot directly reduce dimensions, VF overcomes this limitation by integrating VAE-based nonlinear dimension reduction with dual normalizing flows for the latent prior and encoder. This design provides a strictly higher evidence lower bound than VAE and allows more flexible approximation of complex posterior distributions. We further introduce an iterative prior updating strategy that gradually moves the prior mean toward high-probability posterior regions, avoiding manual prior tuning. These components form a closed adaptive loop together with an adaptively fine-tuned Fourier Neural Operator (FNO) surrogate: VF generates posterior-concentrated samples to refine the surrogate, while the updated surrogate further improves posterior inference. Numerical experiments on a 100-dimensional Rosenbrock problem and three standard PDE-governed inverse problems show that our method delivers competitive or superior accuracy compared with MCMC, UKI, and SVGD baselines across all tested configurations, with the most pronounced advantages emerging in challenging scenarios such as high-noise observations and high-dimensional parameter spaces.

Abstract

Explore similar work

Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems

Variational Garrote for Sparse Inverse Problems

Bayesian Tensor Decomposition with Diffusion Model Prior