Sparse Bayesian Deep Functional Learning with Structured Region Selection

Xiaoxian Zhu; Yingmeng Li; Shuangge Ma; Mengyun Wu

cs.LGFeb 24, 2026

Sparse Bayesian Deep Functional Learning with Structured Region Selection

Authors: Xiaoxian Zhu, Yingmeng Li, Shuangge Ma, Mengyun Wu

Organizations: School of Statistics and Data Science, Shanghai University of Finance and Economics · Department of Biostatistics, Yale School of Public Health

Abstract

In modern applications such as ECG monitoring, neuroimaging, wearable sensing, and industrial equipment diagnostics, complex and continuously structured data are ubiquitous, presenting both challenges and opportunities for functional data analysis. However, existing methods face a critical trade-off: conventional functional models are limited by linearity, whereas deep learning approaches lack interpretable region selection for sparse effects. To bridge these gaps, we propose a sparse Bayesian functional deep neural network (sBayFDNN). It learns adaptive functional embeddings through a deep Bayesian architecture to capture complex nonlinear relationships, while a structured prior enables interpretable, region-wise selection of influential domains with quantified uncertainty. Theoretically, we establish rigorous approximation error bounds, posterior consistency, and region selection consistency. These results provide the first theoretical guarantees for a Bayesian deep functional model, ensuring its reliability and statistical rigor. Empirically, comprehensive simulations and real-world studies confirm the effectiveness and superiority of sBayFDNN. Crucially, sBayFDNN excels in recognizing intricate dependencies for accurate predictions and more precisely identifies functionally meaningful regions, capabilities fundamentally beyond existing approaches.

Figures & tables

Figure 1: Workflow of sBayFDNN. The functional predictor

X_{i}(t)

is projected onto locally supported B-spline bases to form spline features

\bm{x}_{i}=(x_{i1},\ldots,x_{i,J_{n}})^{\top}

. A DNN with a spike-and-slab prior on the first-layer weight columns yields feature-wise posterior inclusion probabilities (PIPs)

q_{j}

, which are thresholded to select spline features and mapped back to the function domain to produce an estimated active region

\widehat{\Omega}

Figure 2: F1 scores across

g

functions, SNR settings, and

\beta(t)

scenarios.

Figure 3: PIPs from sBayFDNN in the high-SNR, Medium-

\beta

, logistic-link scenario.

Figure 4: RMSE across

g

functions, SNR settings, and

\beta(t)

scenarios.

	ECG					Tecator
Method	RMSE	MAE	F1	Recall	Precision	RMSE	MAE	F1	Recall	Precision
sBayFDNN	12.069	8.711	0.634	1.000	0.464	2.138	1.594	0.339	1.000	0.204
FNN	12.991	9.239	–	–	–	2.217	1.613	–	–	–
AdaFNN	14.083	10.198	–	–	–	3.027	2.299	–	–	–
cFuSIM	17.677	12.861	0.501	1.000	0.334	3.932	3.283	0.137	0.977	0.074
BFRS	16.297	11.805	0.396	0.784	0.265	2.691	2.250	0.211	0.714	0.124

Table 1: Performance on ECG and Tecator datasets. F1, Recall, and Precision are reported only for methods that output an estimated active region; otherwise shown as “–”. Best results within each dataset/metric are in bold.

	Bike		IHPC
Method	RMSE	MAE	RMSE	MAE
sBayFDNN	0.618	0.497	0.536	0.409
FNN	0.699	0.535	0.549	0.409
AdaFNN	0.720	0.577	0.552	0.420
cFuSIM	0.693	0.539	0.548	0.411
BFRS	0.749	0.550	0.552	0.416

Table 2: Performance on Bike rental and IHPC datasets.

Figure 5: PIPs from sBayFDNN for ECG and Tecator datasets.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

\widehat{\bm{\theta}}_{J,r}\in\arg\min_{\bm{\theta}}\;\mathcal{L}_{n,J}(\bm{\theta})=\frac{1}{2\sigma_{\varepsilon}^{2}}\sum_{(X_{i},Y_{i})\in\mathcal{D}_{\mathrm{tr}}}\!\Bigl(Y_{i}-F_{\bm{\theta}}(\bm{\eta}_{J}(X_{i}))\Bigr)^{2}-\log\pi(\bm{\theta}).

Appendix

Algorithm 1 Sparse DNN elicitation with projection-size selection

Scenario	Mean Rank	Top-1	Top-2	Top-3	Within 15% of Oracle F1	Within 30% of Oracle F1
Complex $\beta(t)$ , Composite $g$	3.85	11.0%	21.4%	28.6%	50.0%	92.9%
Simple $\beta(t)$ , Composite $g$	2.5	28.6%	42.9%	64.3%	100.0%	100.0%
Moderate $\beta(t)$ , Composite $g$	3.33	33.3%	33.3%	33.3%	66.7%	100.0%
Overall	3.17	17.6%	23.5%	44.1%	73.5%	97.1%

Appendix

Table 3: Top-

k

effectiveness of evidence-based

J_{n}

selection (seed-level; F1 criterion). Candidate set size per run is 8.

SNR	$J_{n}$	MeanLen	Curve Roughness	RMSE	F1	Recall	Precision	MinLen/Mesh
$10$	20	0.5500(0.1843)	0.0056(0.0014)	0.5587(0.0640)	0.5573(0.1017)	1.0000(0.0000)	0.3926(0.0924)	11.0000(3.6857)
	40	0.2800(0.0307)	0.0077(0.0005)	0.5515(0.0518)	0.8366(0.0523)	1.0000(0.0000)	0.7225(0.0780)	11.2000(1.2262)
	60	0.2286(0.0300)	0.0077(0.0005)	0.5522(0.0493)	0.9175(0.0437)	0.9807(0.0456)	0.8684(0.0823)	13.7143(1.7981)
	80	0.2087(0.0337)	0.0077(0.0012)	0.5677(0.0496)	0.9323(0.0480)	0.9500(0.0568)	0.9245(0.0921)	16.6947(2.6985)
	100	0.1782(0.0352)	0.0084(0.0015)	0.5708(0.0513)	0.9095(0.0477)	0.8867(0.0875)	0.9462(0.0783)	17.1875(4.9006)
	120	0.1489(0.0402)	0.0083(0.0014)	0.5698(0.0535)	0.8803(0.0576)	0.8229(0.0997)	0.9601(0.0652)	16.4276(6.9745)

Appendix

Table 4: Simulation results under different

J_{n}

settings for the scenario with simple

\beta(t)

, composite link function

g

, and two SNR levels (reported as mean(sd) over 50 reps).

Hyperparameter	Value	RMSE	F1	Recall	Precision
$\lambda_{n}$	$1\times 10^{-6}$	0.6747(0.1180)	0.7868(0.1472)	0.7574(0.1582)	0.8349(0.1261)
	$3\times 10^{-6}$	0.6813(0.1273)	0.8182(0.2748)	0.7859(0.2667)	0.8532(0.1764)
	$1\times 10^{-5}$	0.6693(0.1201)	0.7982(0.1071)	0.8884(0.1045)	0.7346(0.1007)
	$3\times 10^{-5}$	0.6931(0.1237)	0.7165(0.1995)	0.6845(0.2687)	0.8470(0.0874)
	$1\times 10^{-4}$	0.6743(0.1183)	0.7947(0.1382)	0.7746(0.1438)	0.8315(0.1228)
$\sigma_{0}^{2}$	$1\times 10^{-6}$	0.6542(0.1271)	0.5463(0.0754)	0.9990(0.0010)	0.3826(0.0732)

Appendix

Table 5: Sensitivity analysis of the spike-and-slab hyperparameters (macro-averaged over scenarios). Results are reported as mean(sd).

$\tau$	F1	Recall	Precision	$n_{\mathrm{selected}}$
0.1	0.8256(0.0217)	0.9930(0.0031)	0.7251(0.0316)	11.51(0.48)
0.2	0.8969(0.0137)	0.9573(0.0083)	0.8579(0.0244)	8.30(0.21)
0.3	0.8969(0.0137)	0.9573(0.0083)	0.8579(0.0244)	8.30(0.21)
0.4	0.8781(0.0126)	0.8610(0.0164)	0.9146(0.0201)	5.83(0.19)
0.5	0.8781(0.0126)	0.8610(0.0164)	0.9146(0.0201)	5.83(0.19)
0.6	0.7428(0.0281)	0.6498(0.0340)	0.9201(0.0159)	3.32(0.22)

Appendix

Table 6: Sensitivity analysis of the PIP threshold

\tau

(macro-averaged over scenarios). Results are reported as mean(sd).

$T_{obs}$	Projection ( $\times 10^{-3}$ s)	Training	Evidence	Runtime/Repeat
200	2.0 (5.0)	83.8 (22.1)	41.3 (3.6)	125.1 (22.8)
400	2.0 (4.0)	80.3 (24.8)	41.5 (7.6)	121.7 (20.1)
600	2.0 (4.0)	105.9 (66.0)	38.8 (7.0)	144.7 (62.8)
1000	3.0 (6.0)	68.4 (18.0)	36.3 (3.7)	104.7 (15.1)
1500	2.0 (4.0)	68.0 (18.5)	36.2 (3.0)	104.2 (15.8)
2000	2.0 (5.0)	67.6 (19.3)	36.1 (3.1)	103.7 (16.7)

Appendix

Table 7: Runtime vs. observed points

T_{obs}

at fixed

n=1000

. Entries are mean(sd) over 5 repeats (seconds).

$n$	Projection ( $\times 10^{-3}$ s)	Train	Evidence	Runtime/Repeat
1000	2.0 (5.0)	83.8 (22.1)	41.3 (3.6)	125.1 (22.8)
2000	2.0 (5.0)	122.2 (83.3)	80.3 (56.9)	202.5 (58.3)
3000	3.0 (6.0)	151.2 (97.5)	49.4 (11.5)	200.6 (96.3)

Appendix

Table 8: Runtime vs. sample size

n

at fixed

T_{obs}=200

. Entries are mean(sd) over 5 repeats (seconds).

Figure 6: Cross-seed stability of the plug-in PIPs and the resulting selected regions under the simple

\beta(t)

and complex

g

setting, shown for both low and high SNR. Higher values indicate better stability for PIP Corr , whereas lower values indicate better stability for PIP MeanDiff and Interval SymDiff ; for

\Delta

F1, values closer to zero indicate smaller seed-induced variation.

Figure 7: Illustration of the three types of coefficient functions

\beta(t)

used in simulations: Simple (single interior bump), Medium (single boundary bump), and Complex (two separated oscillating bumps).

Figure 8: Recall values across

g

functions, SNR settings, and

\beta(t)

scenarios.

Figure 9: Precision values across

g

functions, SNR settings, and

\beta(t)

scenarios.

SNR	$\beta(t)$ Setting	Method	RMSE	F1	Recall	Precision
2.0	Complex	sBayFDNN	1.019(0.071)	0.641(0.066)	0.811(0.238)	0.601(0.169)
		FNN	1.100(0.108)	–	–	–
		AdaFNN	1.175(0.209)	–	–	–
		cFuSIM	1.023(0.068)	0.488(0.053)	0.766(0.124)	0.358(0.079)
		BFRS	1.073(0.081)	0.517(0.268)	0.503(0.307)	0.583(0.270)
		SLoS	1.072(0.078)	0.621(0.060)	0.912(0.106)	0.472(0.044)

Appendix

Table 9: Simulation results under scenarios with composite

g

and lower SNR levels. Prediction and selection metrics are reported as mean(standard deviation).

SNR	Method	RMSE	F1	Recall	Precision
5	sBayFDNN	0.049(0.006)	0.907(0.196)	0.873(0.268)	1.000(0.000)
	FNN	0.049(0.004)	–	–	–
	AdaFNN	0.051(0.002)	–	–	–
	cFuSIM	0.051(0.003)	0.824(0.006)	0.701(0.009)	1.000(0.000)
	BFRS	0.053(0.003)	0.000(0.000)	0.000(0.000)	0.000(0.000)
	SLoS	0.053(0.003)	0.000(0.000)	0.000(0.000)	0.000(0.000)

Appendix

Table 10: Simulation results for the stress setting with FGAM mean under two noise levels. Prediction and selection metrics are reported as mean(standard deviation).

SNR	Method	RMSE	F1	Recall	Precision
5	sBayFDNN	0.658(0.077)	0.921(0.044)	0.953(0.053)	0.900(0.092)
	FNN	0.704(0.084)	–	–	–
	AdaFNN	0.796(0.295)	–	–	–
	cFuSIM	0.661(0.093)	0.736(0.138)	0.988(0.035)	0.602(0.169)
	BFRS	0.777(0.076)	0.802(0.054)	0.852(0.196)	0.811(0.137)
	SLoS	0.777(0.077)	0.625(0.021)	1.000(0.000)	0.455(0.022)

Appendix

Table 11: Simulation results for the stress setting with non-Gaussian error, simple

\beta(t)

, and composite link function

g

under two noise levels. Prediction and selection metrics are reported as mean(standard deviation).

$J_{n}$	MeanLen	Curve Roughness	RMSE	F1	Recall	Precision	MinLen/Mesh
20	1.0000	0.0000	14.2331	0.3333	1.0000	0.2000	20.0000
40	1.0000	0.0410	12.8696	0.3333	1.0000	0.2000	40.0000
60	1.0000	0.1559	12.3688	0.3333	1.0000	0.2000	60.0000
80	1.0000	0.1063	12.3784	0.3333	1.0000	0.2000	80.0000
100	0.2935	0.1818	12.1960	0.5083	1.0000	0.3407	1.0870
120	0.1845	0.2235	12.1353	0.5308	1.0000	0.3613	1.0714

Appendix

Table 12: Results for ECG dataset under different

J_{n}

settings.

	sBayFDNN	AdaFNN	FNN	cFuSim	BFRS	SLoS
Runtime	673.641 (27.851)	513.336 (23.642)	$>$ 20 min	$>$ 30 min	819.805 (37.151)	104.144 (12.219)

Appendix

Table 13: IHPC runtime (mean (sd), seconds) comparison across methods under (AMD Ryzen 5 5600H CPU, NVIDIA RTX 3050 Ti Laptop GPU with 4GB VRAM, and 16GB RAM).

Explore similar work

Apr 8, 2026

Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension

Deep neural networks have emerged as powerful tools for learning operators defined over infinite-dimensional function spaces. However, existing theories frequently encounter difficulties related to dimensionality and limited interpretability. This work investigates how sparsity can help address these challenges in functional learning, a central ingredient in operator learning. We propose a framework that employs convolutional architectures to extract sparse features from a finite number of samples, together with deep fully connected networks to effectively approximate nonlinear functionals. Using universal discretization methods, we show that sparse approximators enable stable recovery from discrete samples. In addition, both the deterministic and the random sampling schemes are sufficient for our analysis. These findings lead to improved approximation rates and reduced sample sizes in various function spaces, including those with fast frequency decay and mixed smoothness. They also provide new theoretical insights into how sparsity can alleviate the curse of dimensionality in functional learning.

Jianfei Li, Shuo Huang, Han Feng +2

Department of Mathematics, Ludwig-Maximilians-Universität München (LMU Munich), Akademiestr. 7, 80799 München · Department of Mathematics, City University of Hong Kong, Hong Kong, China · School of Mathematics and Statistics, The University of Sydney, Sydney, New South Wales 2006, Australia +3

Neural Network Approximation Theory Sparse Neural Networks Neural Operators

Ask PDF

Sep 5, 2026

Functional Attentive Interpretable Regression

In function-on-function regression, the coefficient surface

β(s,t)

may exhibit complex support structure---from localized patches to global patterns such as disconnected regions, bands, or rings---where effect similarity does not align with Euclidean proximity. Projection-based methods that rely on fixed basis expansions can obscure such structure, while direct smoothing approaches risk oversmoothing the surface and its boundaries. We propose Functional Attentive Interpretable Regression (FAIR), which represents

β(s,t)

directly through coordinate features and uses self-attention to learn effect-adaptive neighborhoods, enabling information sharing at both local and global scales. A scalar compression network maps these learned representations to the coefficient surface. Sparsity and smoothness penalties applied over these neighborhoods promote localized support with coherent boundaries. We establish a sieve equivalence to tensor-product spline spaces and derive convergence rates. Simulations and applications to oceanographic and hydrological data demonstrate that FAIR recovers support geometry more accurately than existing methods while achieving superior prediction, particularly under sparse sampling.

Haixu Wang, Tianyu Guan, Jiguo Cao

Department of Mathematics and Statistics, University of Calgary · Department of Mathematics and Statistics, York University · Department of Statistics and Actuarial Science, Simon Fraser University

Nonparametric Regression

Ask PDF

Feb 25, 2026

Function-Space Empirical Bayes Regularisation with Student's t Priors

Bayesian deep learning (BDL) has emerged as a principled approach to produce reliable uncertainty estimates by integrating deep neural networks with Bayesian inference, and the selection of informative prior distributions remains a significant challenge. Various function-space variational inference (FSVI) regularisation methods have been presented, assigning meaningful priors over model predictions. However, these methods typically rely on a Gaussian prior, which fails to capture the heavy-tailed statistical characteristics inherent in neural network outputs. By contrast, this work proposes a novel function-space empirical Bayes regularisation framework -- termed ST-FS-EB -- which employs heavy-tailed Student's

t

priors in both parameter and function spaces. Also, we approximate the posterior distribution through variational inference (VI), inducing an evidence lower bound (ELBO) objective based on Monte Carlo (MC) dropout. Furthermore, the proposed method is evaluated against various VI-based BDL baselines, and the results demonstrate its robust performance in in-distribution prediction, out-of-distribution (OOD) detection and handling distribution shifts.

Pengcheng Hao, Ercan Engin Kuruoglu

Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Shenzhen, China

Bayesian Neural Networks Bayesian Inference Empirical Bayes Variational Inference

Ask PDF