Sparse Bayesian Deep Functional Learning with Structured Region Selection

Xiaoxian Zhu; Yingmeng Li; Shuangge Ma; Mengyun Wu

cs.LGFeb 24, 2026

Sparse Bayesian Deep Functional Learning with Structured Region Selection

Authors: Xiaoxian Zhu, Yingmeng Li, Shuangge Ma, Mengyun Wu

Organizations: School of Statistics and Data Science, Shanghai University of Finance and Economics · Department of Biostatistics, Yale School of Public Health

Abstract

In modern applications such as ECG monitoring, neuroimaging, wearable sensing, and industrial equipment diagnostics, complex and continuously structured data are ubiquitous, presenting both challenges and opportunities for functional data analysis. However, existing methods face a critical trade-off: conventional functional models are limited by linearity, whereas deep learning approaches lack interpretable region selection for sparse effects. To bridge these gaps, we propose a sparse Bayesian functional deep neural network (sBayFDNN). It learns adaptive functional embeddings through a deep Bayesian architecture to capture complex nonlinear relationships, while a structured prior enables interpretable, region-wise selection of influential domains with quantified uncertainty. Theoretically, we establish rigorous approximation error bounds, posterior consistency, and region selection consistency. These results provide the first theoretical guarantees for a Bayesian deep functional model, ensuring its reliability and statistical rigor. Empirically, comprehensive simulations and real-world studies confirm the effectiveness and superiority of sBayFDNN. Crucially, sBayFDNN excels in recognizing intricate dependencies for accurate predictions and more precisely identifies functionally meaningful regions, capabilities fundamentally beyond existing approaches.

Figures & tables

Figure 1: Workflow of sBayFDNN. The functional predictor

X_{i}(t)

is projected onto locally supported B-spline bases to form spline features

\bm{x}_{i}=(x_{i1},\ldots,x_{i,J_{n}})^{\top}

. A DNN with a spike-and-slab prior on the first-layer weight columns yields feature-wise posterior inclusion probabilities (PIPs)

q_{j}

, which are thresholded to select spline features and mapped back to the function domain to produce an estimated active region

\widehat{\Omega}

Figure 2: F1 scores across

g

functions, SNR settings, and

\beta(t)

scenarios.

Figure 3: PIPs from sBayFDNN in the high-SNR, Medium-

\beta

, logistic-link scenario.

Figure 4: RMSE across

g

functions, SNR settings, and

\beta(t)

scenarios.

	ECG					Tecator
Method	RMSE	MAE	F1	Recall	Precision	RMSE	MAE	F1	Recall	Precision
sBayFDNN	12.069	8.711	0.634	1.000	0.464	2.138	1.594	0.339	1.000	0.204
FNN	12.991	9.239	–	–	–	2.217	1.613	–	–	–
AdaFNN	14.083	10.198	–	–	–	3.027	2.299	–	–	–
cFuSIM	17.677	12.861	0.501	1.000	0.334	3.932	3.283	0.137	0.977	0.074
BFRS	16.297	11.805	0.396	0.784	0.265	2.691	2.250	0.211	0.714	0.124

Table 1: Performance on ECG and Tecator datasets. F1, Recall, and Precision are reported only for methods that output an estimated active region; otherwise shown as “–”. Best results within each dataset/metric are in bold.

	Bike		IHPC
Method	RMSE	MAE	RMSE	MAE
sBayFDNN	0.618	0.497	0.536	0.409
FNN	0.699	0.535	0.549	0.409
AdaFNN	0.720	0.577	0.552	0.420
cFuSIM	0.693	0.539	0.548	0.411
BFRS	0.749	0.550	0.552	0.416

Table 2: Performance on Bike rental and IHPC datasets.

Figure 5: PIPs from sBayFDNN for ECG and Tecator datasets.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

\widehat{\bm{\theta}}_{J,r}\in\arg\min_{\bm{\theta}}\;\mathcal{L}_{n,J}(\bm{\theta})=\frac{1}{2\sigma_{\varepsilon}^{2}}\sum_{(X_{i},Y_{i})\in\mathcal{D}_{\mathrm{tr}}}\!\Bigl(Y_{i}-F_{\bm{\theta}}(\bm{\eta}_{J}(X_{i}))\Bigr)^{2}-\log\pi(\bm{\theta}).

Appendix

Algorithm 1 Sparse DNN elicitation with projection-size selection

Scenario	Mean Rank	Top-1	Top-2	Top-3	Within 15% of Oracle F1	Within 30% of Oracle F1
Complex $\beta(t)$ , Composite $g$	3.85	11.0%	21.4%	28.6%	50.0%	92.9%
Simple $\beta(t)$ , Composite $g$	2.5	28.6%	42.9%	64.3%	100.0%	100.0%
Moderate $\beta(t)$ , Composite $g$	3.33	33.3%	33.3%	33.3%	66.7%	100.0%
Overall	3.17	17.6%	23.5%	44.1%	73.5%	97.1%

Appendix

Table 3: Top-

k

effectiveness of evidence-based

J_{n}

selection (seed-level; F1 criterion). Candidate set size per run is 8.

SNR	$J_{n}$	MeanLen	Curve Roughness	RMSE	F1	Recall	Precision	MinLen/Mesh
$10$	20	0.5500(0.1843)	0.0056(0.0014)	0.5587(0.0640)	0.5573(0.1017)	1.0000(0.0000)	0.3926(0.0924)	11.0000(3.6857)
	40	0.2800(0.0307)	0.0077(0.0005)	0.5515(0.0518)	0.8366(0.0523)	1.0000(0.0000)	0.7225(0.0780)	11.2000(1.2262)
	60	0.2286(0.0300)	0.0077(0.0005)	0.5522(0.0493)	0.9175(0.0437)	0.9807(0.0456)	0.8684(0.0823)	13.7143(1.7981)
	80	0.2087(0.0337)	0.0077(0.0012)	0.5677(0.0496)	0.9323(0.0480)	0.9500(0.0568)	0.9245(0.0921)	16.6947(2.6985)
	100	0.1782(0.0352)	0.0084(0.0015)	0.5708(0.0513)	0.9095(0.0477)	0.8867(0.0875)	0.9462(0.0783)	17.1875(4.9006)
	120	0.1489(0.0402)	0.0083(0.0014)	0.5698(0.0535)	0.8803(0.0576)	0.8229(0.0997)	0.9601(0.0652)	16.4276(6.9745)

Appendix

Table 4: Simulation results under different

J_{n}

settings for the scenario with simple

\beta(t)

, composite link function

g

, and two SNR levels (reported as mean(sd) over 50 reps).

Hyperparameter	Value	RMSE	F1	Recall	Precision
$\lambda_{n}$	$1\times 10^{-6}$	0.6747(0.1180)	0.7868(0.1472)	0.7574(0.1582)	0.8349(0.1261)
	$3\times 10^{-6}$	0.6813(0.1273)	0.8182(0.2748)	0.7859(0.2667)	0.8532(0.1764)
	$1\times 10^{-5}$	0.6693(0.1201)	0.7982(0.1071)	0.8884(0.1045)	0.7346(0.1007)
	$3\times 10^{-5}$	0.6931(0.1237)	0.7165(0.1995)	0.6845(0.2687)	0.8470(0.0874)
	$1\times 10^{-4}$	0.6743(0.1183)	0.7947(0.1382)	0.7746(0.1438)	0.8315(0.1228)
$\sigma_{0}^{2}$	$1\times 10^{-6}$	0.6542(0.1271)	0.5463(0.0754)	0.9990(0.0010)	0.3826(0.0732)

Appendix

Table 5: Sensitivity analysis of the spike-and-slab hyperparameters (macro-averaged over scenarios). Results are reported as mean(sd).

$\tau$	F1	Recall	Precision	$n_{\mathrm{selected}}$
0.1	0.8256(0.0217)	0.9930(0.0031)	0.7251(0.0316)	11.51(0.48)
0.2	0.8969(0.0137)	0.9573(0.0083)	0.8579(0.0244)	8.30(0.21)
0.3	0.8969(0.0137)	0.9573(0.0083)	0.8579(0.0244)	8.30(0.21)
0.4	0.8781(0.0126)	0.8610(0.0164)	0.9146(0.0201)	5.83(0.19)
0.5	0.8781(0.0126)	0.8610(0.0164)	0.9146(0.0201)	5.83(0.19)
0.6	0.7428(0.0281)	0.6498(0.0340)	0.9201(0.0159)	3.32(0.22)

Appendix

Table 6: Sensitivity analysis of the PIP threshold

\tau

(macro-averaged over scenarios). Results are reported as mean(sd).

$T_{obs}$	Projection ( $\times 10^{-3}$ s)	Training	Evidence	Runtime/Repeat
200	2.0 (5.0)	83.8 (22.1)	41.3 (3.6)	125.1 (22.8)
400	2.0 (4.0)	80.3 (24.8)	41.5 (7.6)	121.7 (20.1)
600	2.0 (4.0)	105.9 (66.0)	38.8 (7.0)	144.7 (62.8)
1000	3.0 (6.0)	68.4 (18.0)	36.3 (3.7)	104.7 (15.1)
1500	2.0 (4.0)	68.0 (18.5)	36.2 (3.0)	104.2 (15.8)
2000	2.0 (5.0)	67.6 (19.3)	36.1 (3.1)	103.7 (16.7)

Appendix

Table 7: Runtime vs. observed points

T_{obs}

at fixed

n=1000

. Entries are mean(sd) over 5 repeats (seconds).

$n$	Projection ( $\times 10^{-3}$ s)	Train	Evidence	Runtime/Repeat
1000	2.0 (5.0)	83.8 (22.1)	41.3 (3.6)	125.1 (22.8)
2000	2.0 (5.0)	122.2 (83.3)	80.3 (56.9)	202.5 (58.3)
3000	3.0 (6.0)	151.2 (97.5)	49.4 (11.5)	200.6 (96.3)

Appendix

Table 8: Runtime vs. sample size

n

at fixed

T_{obs}=200

. Entries are mean(sd) over 5 repeats (seconds).

Figure 6: Cross-seed stability of the plug-in PIPs and the resulting selected regions under the simple

\beta(t)

and complex

g

setting, shown for both low and high SNR. Higher values indicate better stability for PIP Corr , whereas lower values indicate better stability for PIP MeanDiff and Interval SymDiff ; for

\Delta

F1, values closer to zero indicate smaller seed-induced variation.

Figure 7: Illustration of the three types of coefficient functions

\beta(t)

used in simulations: Simple (single interior bump), Medium (single boundary bump), and Complex (two separated oscillating bumps).

Figure 8: Recall values across

g

functions, SNR settings, and

\beta(t)

scenarios.

Figure 9: Precision values across

g

functions, SNR settings, and

\beta(t)

scenarios.

SNR	$\beta(t)$ Setting	Method	RMSE	F1	Recall	Precision
2.0	Complex	sBayFDNN	1.019(0.071)	0.641(0.066)	0.811(0.238)	0.601(0.169)
		FNN	1.100(0.108)	–	–	–
		AdaFNN	1.175(0.209)	–	–	–
		cFuSIM	1.023(0.068)	0.488(0.053)	0.766(0.124)	0.358(0.079)
		BFRS	1.073(0.081)	0.517(0.268)	0.503(0.307)	0.583(0.270)
		SLoS	1.072(0.078)	0.621(0.060)	0.912(0.106)	0.472(0.044)

Appendix

Table 9: Simulation results under scenarios with composite

g

and lower SNR levels. Prediction and selection metrics are reported as mean(standard deviation).

SNR	Method	RMSE	F1	Recall	Precision
5	sBayFDNN	0.049(0.006)	0.907(0.196)	0.873(0.268)	1.000(0.000)
	FNN	0.049(0.004)	–	–	–
	AdaFNN	0.051(0.002)	–	–	–
	cFuSIM	0.051(0.003)	0.824(0.006)	0.701(0.009)	1.000(0.000)
	BFRS	0.053(0.003)	0.000(0.000)	0.000(0.000)	0.000(0.000)
	SLoS	0.053(0.003)	0.000(0.000)	0.000(0.000)	0.000(0.000)

Appendix

Table 10: Simulation results for the stress setting with FGAM mean under two noise levels. Prediction and selection metrics are reported as mean(standard deviation).

SNR	Method	RMSE	F1	Recall	Precision
5	sBayFDNN	0.658(0.077)	0.921(0.044)	0.953(0.053)	0.900(0.092)
	FNN	0.704(0.084)	–	–	–
	AdaFNN	0.796(0.295)	–	–	–
	cFuSIM	0.661(0.093)	0.736(0.138)	0.988(0.035)	0.602(0.169)
	BFRS	0.777(0.076)	0.802(0.054)	0.852(0.196)	0.811(0.137)
	SLoS	0.777(0.077)	0.625(0.021)	1.000(0.000)	0.455(0.022)

Appendix

Table 11: Simulation results for the stress setting with non-Gaussian error, simple

\beta(t)

, and composite link function

g

under two noise levels. Prediction and selection metrics are reported as mean(standard deviation).

$J_{n}$	MeanLen	Curve Roughness	RMSE	F1	Recall	Precision	MinLen/Mesh
20	1.0000	0.0000	14.2331	0.3333	1.0000	0.2000	20.0000
40	1.0000	0.0410	12.8696	0.3333	1.0000	0.2000	40.0000
60	1.0000	0.1559	12.3688	0.3333	1.0000	0.2000	60.0000
80	1.0000	0.1063	12.3784	0.3333	1.0000	0.2000	80.0000
100	0.2935	0.1818	12.1960	0.5083	1.0000	0.3407	1.0870
120	0.1845	0.2235	12.1353	0.5308	1.0000	0.3613	1.0714

Appendix

Table 12: Results for ECG dataset under different

J_{n}

settings.

	sBayFDNN	AdaFNN	FNN	cFuSim	BFRS	SLoS
Runtime	673.641 (27.851)	513.336 (23.642)	$>$ 20 min	$>$ 30 min	819.805 (37.151)	104.144 (12.219)

Appendix

Table 13: IHPC runtime (mean (sd), seconds) comparison across methods under (AMD Ryzen 5 5600H CPU, NVIDIA RTX 3050 Ti Laptop GPU with 4GB VRAM, and 16GB RAM).

Explore similar work

Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension
Apr 8, 2026Jianfei Li, Shuo Huang, Han Feng +2Neural Network Approximation Theory Sparse Neural Networks
Department of Mathematics, Ludwig-Maximilians-Universität München (LMU Munich), Akademiestr. 7, 80799 München · Department of Mathematics, City University of Hong Kong, Hong Kong, China · School of Mathematics and Statistics, The University of Sydney, Sydney, New South Wales 2006, Australia +3
Functional Attentive Interpretable Regression
Sep 5, 2026Haixu Wang, Tianyu Guan, Jiguo CaoNonparametric Regression
Department of Mathematics and Statistics, University of Calgary · Department of Mathematics and Statistics, York University · Department of Statistics and Actuarial Science, Simon Fraser University
Function-Space Empirical Bayes Regularisation with Student's t Priors
Feb 25, 2026Pengcheng Hao, Ercan Engin KuruogluBayesian Neural Networks Bayesian Inference
Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Shenzhen, China

Apr 8, 2026J/K move · Enter open · S save

Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension

Jianfei Li, Shuo Huang, Han Feng, Ding-Xuan Zhou+1

Department of Mathematics, Ludwig-Maximilians-Universität München (LMU Munich), Akademiestr. 7, 80799 München · Department of Mathematics, City University of Hong Kong, Hong Kong, China · School of Mathematics and Statistics, The University of Sydney, Sydney, New South Wales 2006, Australia+3

Neural Network Approximation Theory Sparse Neural Networks Neural Operators

Open paper AskPDF arXiv

Deep neural networks have emerged as powerful tools for learning operators defined over infinite-dimensional function spaces. However, existing theories frequently encounter difficulties related to dimensionality and limited interpretability. This work investigates how sparsity can help address these challenges in functional learning, a central ingredient in operator learning. We propose a framework that employs convolutional architectures to extract sparse features from a finite number of samples, together with deep fully connected networks to effectively approximate nonlinear functionals. Using universal discretization methods, we show that sparse approximators enable stable recovery from discrete samples. In addition, both the deterministic and the random sampling schemes are sufficient for our analysis. These findings lead to improved approximation rates and reduced sample sizes in various function spaces, including those with fast frequency decay and mixed smoothness. They also provide new theoretical insights into how sparsity can alleviate the curse of dimensionality in functional learning.

Abstract

Explore similar work

Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension

Functional Attentive Interpretable Regression

Function-Space Empirical Bayes Regularisation with Student's t Priors