Beyond False Discovery Rate: A Stepdown Group SLOPE Approach for Grouped Variable Selection

Xuelin Zhang; Jingxuan Liang; Xinyue Liu; Hong Chen; Biqin Song

stat.MEMar 1, 2026

Beyond False Discovery Rate: A Stepdown Group SLOPE Approach for Grouped Variable Selection

Authors: Xuelin Zhang, Jingxuan Liang, Xinyue Liu, Hong Chen, Biqin Song

Organizations: College of Informatics, Huazhong Agricultural University, Wuhan, 430070, Hubei Province, China.

Abstract

High-dimensional feature selection is routinely required to balance statistical power with strict control of multiple-error metrics such as the k-Family-Wise Error Rate (k-FWER) and the False Discovery Proportion (FDP), yet some existing frameworks are confined to the narrower goal of controlling the expected False Discovery Rate (FDR) and can not exploit the group-structure of the covariates, such as Sorted L-One Penalized Estimation (SLOPE). We introduce the Group Stepdown SLOPE, a unified optimization procedure which is capable of embedding the Lehmann-Romano stepdown rules into SLOPE to achieve finite-sample guarantees under k-FWER and FDP thresholds. Specifically, we derive closed-form regularization sequences under orthogonal designs that provably bound k-FWER and FDP at user-specified levels, and extend these results to grouped settings via gk-SLOPE and gF-SLOPE, which control the analogous group-level errors gk-FWER and gFDP. For non-orthogonal general designs, we provide a calibrated data-driven sequence inspired by Gaussian approximation and Monte-Carlo correction, preserving convexity and scalability. Extensive simulations are conducted across sparse, correlated, and group-structured regimes. Empirical results corroborate our theoretical findings that the proposed methods achieve nominal error control, while yielding markedly higher power than competing stepdown procedures, thereby confirming the practical value of the theoretical advances.

Figures & tables

	FDR 1 Control	FDP 1 Control	$k$ -FWER 1 Control	Group-wise
SLOPE ( Bogdan et al., 2015 ; Su and Candès, 2016 ; Larsson et al., 2020a ; Dupuis and Tardivel, 2024 ; Minsker et al., 2024 )	$\surd$	$\times$	$\times$	$\times$
g-SLOPE ( Brzyski et al., 2019 ; Wei et al., 2021 ; Chen and Luo, 2023 ; Feser and Evangelou, 2023 )	$\surd$	$\times$	$\times$	$\surd$
Stepdown (FDP) ( Lehmann and Romano, 2005 )	$\times$	$\surd$	$\times$	$\times$
Stepdown ( $k$ -FWER) ( Lehmann and Romano, 2005 )	$\times$	$\times$	$\surd$	$\times$
F-SLOPE (Ours)	$\surd$	$\surd$	$\times$	$\times$
$k$ -SLOPE (Ours)	$\surd$	$\times$	$\surd$	$\times$

Table 1: Algorithmic properties (

\surd

- has the given information,

\times

- hasn’t the given information). 1 1 footnotetext: The group-level definitions of these three criteria (gFDR, gFDP and g

k

-FWER) are considered with respect to g-SLOPE, gF-SLOPE and g

k

-SLOPE.

t	$k$ -SLOPE			F-SLOPE
t	$\mathrm{Prob}(\mathrm{FDP}>\gamma)$	FDR	Power	$\mathrm{Prob}(\mathrm{FDP}>\gamma)$	FDR	Power
50	0.001	0.006	1.000	0.003	0.007	1.000
100	0.000	0.002	0.998	0.002	0.005	1.000
200	0.001	0.001	1.000	0.000	0.007	1.000
300	0.002	0.001	1.000	0.000	0.005	0.995
400	0.000	0.001	0.995	0.001	0.004	0.994

Table 2: Results for controlled feature selection under the orthogonal design (different

t

and fixed

k=5

Figure 1:

k

-FWER provided by different approaches for controlled feature selection under orthogonal design (with different

k

and

t

). The value in the small square is the size of

k

-FWER. The darker the color, the larger the

k

-FWER and vice versa.

Figure 2: Result for controlled feature selection on the simulated data. The black dashed lines indicate the target FDR level. Constance for

k

-SLOPE is

k=6

in the second column (from left to right). The value in the small square is the size of

k

-FWER in the third and fourth columns (from left to right). The darker the color, the larger the

k

-FWER and vice versa.

$t$	F-SLOPE	$k$ -SLOPE	Sd (FDP)	Sd ( $k$ -FWER)
10	0.00	0.04	0.02	0.08
20	0.03	0.00	0.00	0.03
30	0.00	0.01	0.01	0.01
40	0.00	0.00	0.00	0.00
50	0.01	0.00	0.00	0.00
60	0.00	0.00	0.00	0.00

Table 3:

\mathrm{Prob(FDP}>\gamma

) results on the simulated data for multiple mean testing (

k=6

)

$k/t$	10	30	40	50	60	70	80
2	0.02	0.02	0.00	0.04	0.04	0.07	0.07
4	0.00	0.00	0.01	0.00	0.00	0.00	0.01
6	0.00	0.00	0.00	0.00	0.00	0.00	0.00
8	0.00	0.00	0.01	0.00	0.00	0.00	0.01

Table 4:

k

-FWER of

k

-SLOPE (

m=2n

) on the simulated data under the weak signals

$k/t$	30	40	50	60	70	80
2	0.05	0.03	0.10	0.12	0.07	0.13
4	0.00	0.01	0.00	0.00	0.01	0.06
6	0.00	0.00	0.00	0.00	0.00	0.01
8	0.00	0.01	0.00	0.02	0.00	0.00

Table 5: FDR of F-SLOPE (

m=2n

) on the simulated data under the weak signals

Figure 3: Power and FDR of F-SLOPE under Gaussian design (different

t

). The black dashed line indicates the target FDR level.

Figure 4: The g

k

-FWER provided by different approaches for controlled feature selection under orthogonal design. The value in the small square is the size of g

k

-FWER. The darker the color, the larger the g

k

-FWER and vice versa.

Figure 5: The gFDR of g-SLOPE, g

k

-SLOPE and gF-SLOPE on the simulated data in orthogonal design when

k=15

$t$	$m=2n$		$m=n/2$
$t$	weak	moder	weak	moder
10	0.03	0.00	0.05	0.00
20	0.07	0.00	0.03	0.01
30	0.01	0.00	0.00	0.00
40	0.00	0.01	0.00	0.00
50	0.00	0.00	0.00	0.00

Table 6: Prob(FDP

>\gamma

) of F-SLOPE on the simulated data under the weak and moderate signals (different

t

the number	50	100	150	200
g-SLOPE	0.51	0.73	0.98	1.00
g $k$ -SLOPE	0.01	0.00	0.00	0.01
gFSLOPE	0.01	0.00	0.00	0.00

Table 7: Prob(FDP

>\gamma

) of g-SLOPE, g

k

-SLOPE and gF-SLOPE on the simulated data.

the number	50	100	150	200	250
g-SLOPE	0.8	0.9	0.93	0.95	0.96
g $k$ -SLOPE	0.99	0.98	1	0.99	1
gF-SLOPE	1	0.99	1	1	0.98

Table 8: Power values of g-SLOPE, g

k

-SLOPE and gF-SLOPE in orthogonal design.

Figure 6: The g

k

-FWER provided by different approaches for controlled feature selection under orthogonal design. The value in the small square is the size of g

k

-FWER. The darker the color, the larger the g

k

-FWER and vice versa.

Figure 7: The gFDR of g-SLOPE, g

k

-SLOPE and gF-SLOPE on the simulated data as

k=6

Size	10	20	30	40	50
g-SLOPE	0.42(0.32)	0.28(0.38)	0.46(0.48)	0.24(0.44)	0.48(0.56)
gk-SLOPE	0.16(0.00)	0.08(0.00)	0.12(0.02)	0.02(0.00)	0.1(0.00)
gF-SLOPE	0.16(0.06)	0.08(0.00)	0.12(0.00)	0.02(0.00)	0.1(0.00)

Table 9: Results of Prob(FDP

>\gamma

) on the simulated data. The values in different group configurations are shown in parentheses.

Size	50	100	150	200	250
g-SLOPE	0.68	0.87	0.9	0.92	0.93
g $k$ -SLOPE	1	1	1	1	1
gF-SLOPE	1	1	1	1	1

Table 10: Results of power values under general design.

Method	MSE ( $\times 10^{-2}$ )	Time (s)	gk-FWER	Prob(gFDP $\geq$ 0.1)
g-SLOPE	2.84 ± 0.21	2.34 ± 0.82	0.82	0.48
Group Lasso	2.79 ± 0.19	0.82 ± 0.02	0.91	0.63
g $k$ -SLOPE	2.61 ± 0.15	1.51 ± 0.43	0.12	0.17
gF-SLOPE	2.60 ± 0.14	1.59 ± 0.47	0.19	0.11

Table 11: Results on real-world ADNI data.

Feature Selection

Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.

See all 196 papers

Aug 2, 2026

Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP

Most FDR-controlled feature selection methods are designed for coordinate-wise hypotheses, where each feature has a single weight or importance score. This abstraction fails in sequential and grouped models, where one original feature is represented by a block of sub-features, such as lags, recurrent states, or attention-based interactions. We propose a grouped-feature FDR control framework for such settings. For grouped linear models, we construct null-symmetric block-level mirror statistics with matrix-valued perturbations. For neural sequential models, we combine Permutation SHAP derivatives as model-agnostic block-level importance scores with kernel-based dependence measure. The framework is model-agnostic across network architectures, does not require specifying the covariate distribution, and reduces to Gaussian Mirror or Neural Gaussian Mirror when the block size is one. We prove FDR control for low- and high-dimensional grouped linear models and asymptotic symmetry of smoothed Permutation SHAP derivatives under fixed fitted nonlinear models. Experiments on simulated and real-world datasets show reliable FDR control and improved power under correlated grouped-feature signals.

Jiaan Han, Junxiao Chen, Yanzhe Fu

Columbia Business School Columbia University · Department of Statistics University of Hong Kong

FDR Control Feature Selection

Ask PDF

Dec 4, 2025

Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

We develop a flexible feature selection framework based on deep neural networks that approximately controls the false discovery rate (FDR), a measure of Type-I error. The method applies to architectures whose first layer is fully connected. From the second layer onward, it accommodates multilayer perceptrons (MLPs) of arbitrary width and depth, convolutional and recurrent networks, attention mechanisms, residual connections, and dropout. The procedure also accommodates stochastic gradient descent with data-independent initializations and learning rates. To the best of our knowledge, this is the first work to provide a theoretical guarantee of FDR control for feature selection within such a general deep learning setting. Our analysis is built upon a multi-index data-generating model and an asymptotic regime in which the feature dimension

n

diverges faster than the latent dimension

q^{*}

, while the sample size, the number of training iterations, the network depth, and hidden layer widths are left unrestricted. Under this setting, we show that each coordinate of the gradient-based feature-importance vector admits a marginal normal approximation, thereby supporting the validity of asymptotic FDR control. As a theoretical limitation, we assume

\mathbf{B}

-right orthogonal invariance of the design matrix, and we discuss broader generalizations. We also present numerical experiments that underscore the theoretical findings.

Kazuma Sawaya

The University of Tokyo, Bunkyo, Tokyo, Japan

FDR Control Feature Selection Statistical Learning Theory Multiple Hypothesis Testing

Ask PDF

Mar 30, 2026

KGroups: A Versatile Univariate Max-Relevance Min-Redundancy Feature Selection Algorithm for High-dimensional Biological Data

This paper proposes a new univariate filter feature selection (FFS) algorithm called KGroups. The majority of work in the literature focuses on investigating the relevance or redundancy estimations of feature selection (FS) methods. This has shown promising results and a real improvement of FFS methods' predictive performance. However, limited efforts have been made to investigate alternative FFS algorithms. This raises the following question: how much of the FFS methods' predictive performance depends on the selection algorithm rather than the relevance or the redundancy estimations? The majority of FFS methods fall into two categories: relevance maximisation (Max-Rel, also known as KBest) or simultaneous relevance maximisation and redundancy minimisation (mRMR). KBest is a univariate FFS algorithm that employs sorting (descending) for selection. mRMR is a multivariate FFS algorithm that employs an incremental search algorithm for selection. In this paper, we propose a new univariate mRMR called KGroups that employs clustering for selection. Extensive experiments on 14 high-dimensional biological benchmark datasets showed that KGroups achieves similar predictive performance compared to multivariate mRMR while being up to 821 times faster. KGroups is parameterisable, which leaves room for further predictive performance improvement through hyperparameter finetuning, unlike mRMR and KBest. KGroups outperforms KBest.

Malick Ebiele, Malika Bendechache, Rob Brennan

Computer Science, University College Dublin, Dublin, Dublin, Ireland. · ADAPT Centre, Dublin, Dublin, Ireland. · Computer Science, University of Galway, Galway, Galway, Ireland.

Bioinformatics Feature Selection

Ask PDF

Abstract

Feature Selection

Explore similar work

Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP

Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

KGroups: A Versatile Univariate Max-Relevance Min-Redundancy Feature Selection Algorithm for High-dimensional Biological Data