Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

Kazuma Sawaya

stat.MLDec 4, 2025

Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

Authors: Kazuma Sawaya

Organizations: The University of Tokyo, Bunkyo, Tokyo, Japan

Abstract

We develop a flexible feature selection framework based on deep neural networks that approximately controls the false discovery rate (FDR), a measure of Type-I error. The method applies to architectures whose first layer is fully connected. From the second layer onward, it accommodates multilayer perceptrons (MLPs) of arbitrary width and depth, convolutional and recurrent networks, attention mechanisms, residual connections, and dropout. The procedure also accommodates stochastic gradient descent with data-independent initializations and learning rates. To the best of our knowledge, this is the first work to provide a theoretical guarantee of FDR control for feature selection within such a general deep learning setting. Our analysis is built upon a multi-index data-generating model and an asymptotic regime in which the feature dimension $n$ diverges faster than the latent dimension $q^{*}$ , while the sample size, the number of training iterations, the network depth, and hidden layer widths are left unrestricted. Under this setting, we show that each coordinate of the gradient-based feature-importance vector admits a marginal normal approximation, thereby supporting the validity of asymptotic FDR control. As a theoretical limitation, we assume $\mathbf{B}$ -right orthogonal invariance of the design matrix, and we discuss broader generalizations. We also present numerical experiments that underscore the theoretical findings.

Figures & tables

Figure 1 . A schematic illustration of $\bm{\xi}^{(t)}$ for $q^{*}=1$ and $n=3$ . $\bm{U}\in\mathbb{R}^{n\times n}$ is any orthogonal matrix such that $\bm{U}\bm{B}=\bm{B}$ (i.e., rotation around $\bm{B}$ ).

\displaystyle\xi_{js}^{(T)}=\sum_{i=1}^{m/2}\frac{\partial}{\partial x_{ij}^{(s)}}f_{\mathcal{W}_{s}^{(T)}}(\bm{x}_{i}^{(s)}),\qquad j\in[n].

Algorithm 1 pseudocode for the selection procedure

Figure 2 . Histograms of the empirical distribution of

{\sqrt{n}\xi_{j}^{(10)}}/{\|\bm{P}_{\bm{B}}^{\perp}\bm{\xi}^{(10)}\|}

for

j\in S^{\mathsf{c}}

. The solid black curve shows the

\mathcal{N}(0,1)

density. The solid red curve represents a normal density fitted to the histograms, and the dotted blue line indicates the empirical mean.

Figure 3 . Empirical false discovery rate (left) and power (right) for feature selection using the artificial data and models defined in Section 5.1 . The Transformer is trained for 40,000 updates, and the other models are trained for 2,000 updates. The horizontal axis shows the number of training updates, divided by 20 for the Transformer. Solid curves represent averages over 20 independent runs, and shaded regions indicate one standard deviation around the mean.

	MLP		1D-CNN		LSTM		DeepLINK		NGM
Design	FDR	Power	FDR	Power	FDR	Power	FDR	Power	FDR	Power
N(0,1)	0.075 (0.019)	0.830 (0.049)	0.094 (0.028)	0.998 (0.002)	0.101 (0.025)	1.000 (0.000)	0.089 (0.021)	0.999 (0.000)	0.086 (0.015)	0.851 (0.013)
t(3)	0.043 (0.042)	0.363 (0.305)	0.047 (0.044)	0.512 (0.382)	0.072 (0.040)	0.843 (0.343)	0.081 (0.030)	0.870 (0.218)	0.063 (0.031)	0.599 (0.151)
Spiked	0.076 (0.034)	0.808 (0.070)	0.099 (0.028)	0.998 (0.003)	0.104 (0.026)	1.000 (0.000)	0.087 (0.021)	1.000 (0.000)	0.080 (0.016)	0.847 (0.017)

Table 1 . Performance comparison for

m=2000

	MLP		1D-CNN		LSTM		DeepLINK		NGM
Design	FDR	Power	FDR	Power	FDR	Power	FDR	Power	FDR	Power
N(0,1)	0.060 (0.030)	0.186 (0.069)	0.075 (0.031)	0.546 (0.093)	0.072 (0.024)	0.637 (0.089)	0.065 (0.021)	0.467 (0.122)	0.131 (0.031)	0.658 (0.041)
t(3)	0.053 (0.057)	0.080 (0.068)	0.040 (0.039)	0.122 (0.111)	0.061 (0.043)	0.168 (0.130)	0.064 (0.076)	0.085 (0.107)	0.144 (0.612)	0.401 (0.127)
Spiked	0.047 (0.036)	0.187 (0.066)	0.070 (0.023)	0.571 (0.094)	0.072 (0.021)	0.682 (0.089)	0.063 (0.028)	0.492 (0.126)	0.151 (0.031)	0.674 (0.046)

Table 2 . Performance comparison for

m=1000

	MLP		1D-CNN		LSTM		DeepLINK		NGM
Design	FDR	Power	FDR	Power	FDR	Power	FDR	Power	FDR	Power
N(0,1)	0.087 (0.092)	0.031 (0.029)	0.087 (0.079)	0.057 (0.033)	0.047 (0.068)	0.040 (0.034)	0.019 (0.58)	0.003 (0.010)	0.242 (0.049)	0.380 (0.093)
t(3)	0.074 (0.133)	0.012 (0.015)	0.041 (0.098)	0.009 (0.010)	0.058 (0.098)	0.016 (0.012)	0.017 (0.061)	0.004 (0.013)	0.325 (0.04)	0.428 (0.076)
Spiked	0.116 (0.223)	0.028 (0.033)	0.138 (0.214)	0.059 (0.046)	0.029 (0.059)	0.037 (0.036)	0.026 (0.065)	0.020 (0.049)	0.264 (0.039)	0.435 (0.056)

Table 3 . Performance comparison for

m=500

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4 . Histograms of the empirical distribution of

{\sqrt{n}\xi_{j}^{(10)}}/{\|\bm{P}_{\bm{B}}^{\perp}\bm{\xi}^{(10)}\|}

for

j\in S^{\mathsf{c}}

. The solid black curve shows the

\mathcal{N}(0,1)

density. The solid red curve represents a normal density fitted to the histograms.

Figure 5 . Results for the FDR/power (left) and the training loss (right) when performing feature selection at each iteration. The solid curves represent averages over 20 independent runs, and the shaded areas indicate one standard deviation around the mean.

Figure 6 . QQ-plots of

{\sqrt{n}\xi_{j}^{(10)}}/{\|\bm{P}_{\bm{B}}^{\perp}\bm{\xi}^{(10)}\|}

under the settings of Figure 2 .

Figure 7 . Training loss trajectories of each method across iterations (corresponding to the methods plotted in Figure 3 ).

Figure 8 . Histograms of the empirical distribution of

{\sqrt{n}\xi_{j}^{(10)}}/{\|\bm{P}_{\bm{B}}^{\perp}\bm{\xi}^{(10)}\|}

for

j\in S^{\mathsf{c}}

under the multi-class classification model. The solid black curve shows the

\mathcal{N}(0,1)

density. The solid red curve represents a normal density fitted to the histograms, and the dotted blue line indicates the empirical mean.

Figure 9 . QQ-plots of

{\sqrt{n}\xi_{j}^{(10)}}/{\|\bm{P}_{\bm{B}}^{\perp}\bm{\xi}^{(10)}\|}

under the multi-class classification model.

Figure 10 . Results for the FDR/power (left) and the training loss (right) when performing feature selection at each iteration under the multi-class classification model. The solid curves represent averages over 20 independent runs, and the shaded areas indicate one standard deviation around the mean.

Figure 11 . Under the same setting as in Figure 3 , we present the false discovery proportion and power of the selected features at each SGD step for the MLP architecture, for each choice of

\psi(u,v)\in\{\min(u,v),\max(u,v),u+v,uv\}

Figure 12 . Under the same setting as in Figure 3 , we present the false discovery proportion and power of the selected features at each SGD step for the 1D-CNN architecture, for each choice of

\psi(u,v)\in\{\min(u,v),\max(u,v),u+v,uv\}

Figure 13 . Under the same setting as in Figure 3 , we present the false discovery proportion and power of the selected features at each SGD step for the LSTM architecture, for each choice of

\psi(u,v)\in\{\min(u,v),\max(u,v),u+v,uv\}

Figure 14 . Under the same setting as in Figure 3 (plotted as dotted lines), we present the false discovery proportion and power of the selected features at each SGD step for the multiple data-splitting method [ 10 ] over 20 random splits for each architecture.

Explore similar work

Aug 2, 2026stat.ML

Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP

Most FDR-controlled feature selection methods are designed for coordinate-wise hypotheses, where each feature has a single weight or importance score. This abstraction fails in sequential and grouped models, where one original feature is represented by a block of sub-features, such as lags, recurrent states, or attention-based interactions. We propose a grouped-feature FDR control framework for such settings. For grouped linear models, we construct null-symmetric block-level mirror statistics with matrix-valued perturbations. For neural sequential models, we combine Permutation SHAP derivatives as model-agnostic block-level importance scores with kernel-based dependence measure. The framework is model-agnostic across network architectures, does not require specifying the covariate distribution, and reduces to Gaussian Mirror or Neural Gaussian Mirror when the block size is one. We prove FDR control for low- and high-dimensional grouped linear models and asymptotic symmetry of smoothed Permutation SHAP derivatives under fixed fitted nonlinear models. Experiments on simulated and real-world datasets show reliable FDR control and improved power under correlated grouped-feature signals.

Jiaan Han, Junxiao Chen, Yanzhe Fu

Columbia Business School Columbia University · Department of Statistics University of Hong Kong

Feature Importance False Discovery Rate Shapley Additive Explanations Importance Gaussian Primitives

Ask PDF

Jun 3, 2026stat.ML

Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks

The deep neural network is a widely used framework in machine learning that has been widely applied in various fields. However, deep neural networks often involve a large number of parameters and inputs, many of which may be irrelevant to the goal or true output. These parameters and input variables not only increase computational complexity, but also contribute to additional computational cost. One solution to this problem is knockoff methods, which have proven successful in controlling false discovery rates in high-dimensional regression. Building on the knockoff methods and using the regularised neural network, this paper proposes three variable screening methods under the condition of controlling false discovery rates: one layer filter, multiple layers filter, and variable weight aggregation filter. In comparison with existing algorithms, we find that our algorithms show satisfactory performance.

Wenyu Liao, Yiqing Shi, Fang Xie

Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China

False Discovery Rate Screening Regularization Discovery Simplification

Ask PDF

Jun 5, 2026cs.LG

Bias in Filter Feature Selection Evaluation: A Meta-Analysis of Datasets, Baselines, and Experimental Design Choices

Background: Since 1990 many feature selection methods have been proposed across heterogeneous applications. To validate the usefulness of a new method, it needs to be compared against at least one baseline method from the existing literature on a feature selection task using at least one dataset. Recent developments in tabular Deep Learning (DL) and data valuation in Machine Learning (ML) suggest that the evaluation of new methods, algorithms, and models may be consciously or unconsciously biased. We hypothesise that a similar trend exists in feature selection (FS), particularly in filter feature selection (FFS). The aim of this study is therefore to examine FFS studies to identify factors that influence the evaluation and that might consist entry point for biases in order to recommend stronger principles for FFS evaluation. Methods: We analyse a sample of 28 high profile FFS studies published between 1994 and 2025. The analysis provides reflections on how to examine FFS studies, highlights lessons learned throughout the process, and gives five evidence-based recommendations for future FFS evaluation. Results: Multivariate Linear Regression analysis achieved a score of

R^2=0.33

. It means that 33% of the variance in the performance of new methods against chosen baselines (win rate) is explained by the number of datasets (#Datasets), the number of baselines (#Baselines), and the number of new methods (#NewMethods). Discussion:

R^2=0.33

is considered medium explanation; which is promising given that this is the first such study. The medium explanation result is due to the fact that win rate is influenced by additional factors such as the maturity of the feature selection domain, the type of datasets and baselines, and the simplicity of the regression model used to explain the relationship.

Malick Ebiele, Malika Bendechache, Rob Brennan

1*Computer Science, University College Dublin, Dublin, Dublin, Ireland. · 3ADAPT Centre, Dublin, Dublin, Ireland. · 2Computer Science, University of Galway, Galway, Galway, Ireland.

Feature Selection False Discovery Rate Tabular Learning

Ask PDF