Hide&Seek: Learning to Explain in an End-to-End Differentiable Network

Tal Ellinson; Hadi Mohasel Afshar; Sally Cripps

stat.MLAug 17, 2026

Hide&Seek: Learning to Explain in an End-to-End Differentiable Network

Authors: Tal Ellinson, Hadi Mohasel Afshar, Sally Cripps

Organizations: Human Technology Institute; School of Mathematical and Physical Sciences, University of Technology Sydney, Australia.

Abstract

Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.

Figures & tables

	Hide&Seek	REAL-x	INVASE	L2X
Adaptive number of important features	✓	✓	✓	×
End-to-end differentiable	✓	×	×	✓
Avoids information leakage	✓	✓	×	×

Table 1: Comparison of similar instance-wise feature selection models. All four models use a selector–predictor framework.

Figure 2: The annealing schedule for

\lambda_{t}

over

t

epochs, showing the impact of different choices of

q=\{0,1,2,3\}

\lambda_{t}=\left(\frac{t}{T}\right)^{q}\lambda_{\text{max}}

on the loss function ( 12 ). (a) The functional form of

\lambda_{t}

. (b) The cross-entropy term

-\sum_{c=1}^{C}y_{c}\log\hat{y}_{c}

vs t. (c) The regularized parsimony term

\frac{\lambda_{t}}{d}\|{\boldsymbol{\mathbf{m}}}\|_{1}

vs t. (d) The combined loss

\ell(\boldsymbol{\mathbf{\alpha}},\boldsymbol{\mathbf{\beta}},t)

vs t. The metrics in (b)-(d) are calculated on a hold-out validation set using the same

\lambda_{\text{max}}

and data as in the Syn4 experiment in section 4.1 . See Appendix D.7 for other choices of

\lambda_{\text{max}}

and more detail.

Model	Hide&Seek		INVASE		REAL-x		SHAP		LIME		L2X		RForest		LASSO
	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR
Syn1	100	0	100	0	100	9	64	36	19	81	30	70	100	0	0	100
Syn2	100	0	100	0	100	4	95	5	100	0	75	25	100	0	50	50
Syn3	99	0	91	0	82	1	92	8	96	4	43	57	100	0	75	25
Syn4	99	4	90	4	98	15	72	38	54	51	44	65	67	40	58	55
Syn5	97	3	83	1	88	9	74	38	50	54	53	57	67	40	56	50

Table 2: Performance of eight algorithms across six datasets. Each TPR and FDR metric is the median of 20 experiments. See Appendix C for boxplots.

Method	Run time (hh:mm:ss)
L2X	00:00:03
Hide&Seek	00:00:05
REAL-x	00:01:16
INVASE	01:18:52

Table 3: Typical run times of selector–predictor models for the synthetic data, with IWFS metrics reported in table 2 . Times include training (10,000 samples), prediction, and feature importance attribution (10,000 samples). See Appendix B.1 for hardware and Appendix D.5 for more detail.

Figure 3: Switch accuracy. The percentage of instances where the switch-feature,

X_{11}

, was correctly identified as important. Each boxplot represents the distribution across 20 runs.

Figure 4: Model performance under the multicollinear setting. Each point represents the average IWFS F1 score across Syn1-6 for a given model and a given value of

\rho

. For these synthetic datasets,

\mathbf{X}\sim\mathcal{N}(\mathbf{0},\Sigma)

, where

\Sigma_{i,j}=\rho

for

i\neq j

and

\Sigma_{i,j}=1

for

i=j

Model	Hide&Seek			INVASE			REAL-X
	TPR	FDR	F1	TPR	FDR	F1	TPR	FDR	F1
Syn1	100	1	99	100	0	100	100	22	86
Syn2	67	26	67	100	0	100	49	13	59
Syn3	100	0	100	98	0	99	63	11	69
Syn4	87	34	74	58	31	57	63	35	62
Syn5	96	28	81	80	28	74	79	41	66

Table 4: IWFS performance on highly correlated credit default data, where the target is generated as in Section 4.1 . Each metric is the median of 20 experiments.

	AUROC
Model	Hide&Seek	INVASE	REAL-X
Syn1	0.664 $\pm$ 0.002	0.601 $\pm$ 0.002	0.641 $\pm$ 0.003
Syn2	0.885 $\pm$ 0.002	0.863 $\pm$ 0.002	0.878 $\pm$ 0.003
Syn3	0.767 $\pm$ 0.002	0.752 $\pm$ 0.003	0.729 $\pm$ 0.005
Syn4	0.828 $\pm$ 0.001	0.815 $\pm$ 0.002	0.809 $\pm$ 0.002
Syn5	0.700 $\pm$ 0.002	0.675 $\pm$ 0.002	0.663 $\pm$ 0.003

Table 5: Prediction performance for highly correlated credit default data with synthetically generated labels (Syn1-6) and the credit default binary

Y

labels (CDY) that indicate whether or not a customer defaulted on their payment. The metric is mean AUROC (± standard error) across 20 runs.

Panel of Figure 5 — Figure 5: Important patches in MNIST data, as identified by each explanation method. Each 3x3 red square is one of the four most important patches in the image.

Hide&Seek		INVASE		REAL-x		SHAP		L2X		LIME
Gene	Imp.	Gene	Imp.	Gene	Imp.	Gene	Imp.	Gene	Imp.	Gene	Imp.
ESR1	0.999	CCNB2	0.723	ESR1	0.969	ESR1	0.695	PENK	0.640	TUBB	0.044
CCNB2	0.951	ESR1	0.711	CCNB2	0.897	CCNB2	0.402	BIRC3	0.600	HACE1	0.044
STATH	0.827	ZNF775	0.680	NUP210	0.833	C6orf15	0.212	TMEM52	0.590	C6orf26	0.038
C6orf26	0.804	KLF3	0.619	C6orf15	0.758	ZNF385	0.126	HPS4	0.590	PENK	0.033
TUBB	0.784	C6orf15	0.610	SLC25A3	0.754	NUP210	0.121	OTUD3	0.590	ESR1	0.033

Table 6: Top 10 important genes across different explanation methods. Feature importance represents the mean mask size (Hide&Seek, INVASE, REAL-x), mean importance scores (SHAP, LIME), or mean selection frequency (L2X). Standard errors are provided in the appendix.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Comparison of alternative methods for sampling $\hat{\mathbf{x}}$ in the Hide&Seek architecture, showing that marginal sampling outperforms knockoff sampling. (a) can be directly compared to Table 2 in the paper and (b) can be directly compared to fig. 4 .

$\rho$	Hide&Seek	INVASE	L2X	LIME	REAL-x	SHAP
0.0	0.998	0.547	0.577	0.179	1.000	0.813
0.1	0.999	0.059	0.557	0.156	1.000	0.824
0.2	0.996	0.720	0.578	0.196	1.000	0.850
0.3	0.999	0.561	0.547	0.285	1.000	0.815
0.4	1.000	0.695	0.613	0.210	1.000	0.802
0.5	0.999	0.788	0.571	0.339	1.000	0.820

Appendix

Table 7: Switch accuracy (identifying

X_{11}

as important) across

\rho

and models. Each value is the mean switch accuracy across Syn4, Syn5 and Syn6 for a single run of each. Note that some variability in results is expected, as indicated by the boxplots in Figure 3 .

Panel of Figure 7 — Figure 7: Distributions of (a) mean TPR, (b) mean FDR, and (c) mean F1 scores for feature identification across Syn1-6. Each boxplot shows the distribution across 20 experiments. The medians of the TPR and FDR boxplots are reported in Table 2 .

Figure 8: The mask distribution for the 6 synthetic experiments in Section 4.1 , for one of the 20 experiments. Shown are the histograms and associated KDE plots.

X_{11}

is the switch-feature, used in Syn4–6.

Figure 9: Predictive performance (AUROC) of the models in Section 4.1 .

Panel of Figure 10 — Figure 10: $\lambda$ sensitivity across different models, with AUROC (predicting $Y$ ) and F1 (identifying important features) averaged over Syn1-6.

Model	k	Syn4		Syn5
		TPR	FDR	TPR	FDR
Hide&Seek		99	4	97	3
SHAP	5	72	38	74	38
	4	60	35	63	34
	3	46	34	48	32
LIME	5	54	51	50	54

Appendix

Table 8: TPR and FDR for Syn4–5 for different values of

k

, as explained in Section 4.1 . Each metric is the median of 20 experiments.

Method	Run time (hh:mm:ss)
SHAP	00:00:02
L2X	00:00:03
Hide&Seek	00:00:05
REAL-x	00:01:16
LIME	00:10:54
INVASE	01:18:52

Appendix

Table 9: Typical model run times on the synthetic data, with IWFS results reported in table 2 . Times include training (10,000 samples), prediction, and feature attribution (10,000 samples). All models were run on identical hardware, described in Appendix B.1 .

Method	Source	Syn3
INVASE	Paper	$-10\sin\bigl(2X_{7}\bigr)+2\|X_{8}\|+X_{9}+\exp\bigl(-X_{10}\bigr)$
	Code	$-10\sin\bigl(0.2X_{7}\bigr)+\|X_{8}\|+X_{9}+\exp\bigl(-X_{10}\bigr)-2.4$
L2X	Paper	$-100\sin\bigl(2X_{1}\bigr)+2\|X_{2}\|+X_{3}+\exp\bigl(-X_{4}\bigr)$
	Code	$-100\sin\bigl(0.2X_{1}\bigr)+\|X_{2}\|+X_{3}+\exp\bigl(-X_{4}\bigr)-2.4$
Hide&Seek	Both	$-10\sin\bigl(0.2X_{7}\bigr)+\|X_{8}\|+X_{9}+\exp\bigl(-X_{10}\bigr)-2.4$

Appendix

Table 10: Paper and code expressions for Syn3 model in INVASE, L2X, and Hide&Seek.

$q$	TPR	FDR	F1	AUROC
0.5	38.34	5.33	44.45	0.68
1.0	78.43	2.71	82.88	0.79
2.0	97.66	2.49	97.26	0.83
3.0	98.53	4.30	96.67	0.83
4.0	98.78	5.44	96.13	0.83
5.0	98.88	6.65	95.44	0.83

Appendix

Table 11: Sensitivity analysis for choices of

q

\lambda_{t}=\left(\frac{t}{T}\right)^{q}\lambda_{\text{max}}

. TPR, FDR and F1 are instance-wise feature selection metrics and AUROC is a prediction metric. Each value represents the mean across the six synthetic datasets in Section 4.1 and 20 seeds.

\lambda_{\text{max}}=0.3

for all runs.

Figure 11: Parsimony-weight annealing analysis. These graphs show the impact of different choices of

q=\{0,1,2,3\}

\lambda_{t}=\left(\frac{t}{T}\right)^{q}\lambda_{\text{max}}

on the loss function ( 12 ), for different values of

\lambda_{\text{max}}

. Each row shows (a) The cross-entropy term

\sum_{c=1}^{C}y_{c}\log\hat{y}_{c}

vs t. (b) The regularized parsimony term

\frac{\lambda_{t}}{d}\|{\boldsymbol{\mathbf{m}}}\|_{1}

vs t. (c) The combined loss

\ell(\boldsymbol{\mathbf{\alpha}},\boldsymbol{\mathbf{\beta}},t)

vs t. See section D.7 for more detail.

Model	Hide&Seek		INVASE		REAL-x		SHAP		LIME		L2X
	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR	TPR	FDR
Syn1	100	0	100	0	100	0	98	2	24	76	100	0
Syn2	100	0	100	0	87	6	100	0	100	0	100	0
Syn3	99	0	100	0	94	0	100	0	98	2	85	15
Syn4	100	1	90	1	95	4	70	38	55	49	83	32
Syn5	98	1	84	1	89	1	73	36	50	53	90	29

Appendix

Table 12: Performance of IWFS algorithms on Syn1-6 when trained on 1,000,000 samples.

Model	Hide&Seek	Hide&Seek ${}_{\text{ens}}$	INVASE	INVASE ${}_{\text{ideal}}$	REAL-x	SHAP	LIME	L2X
	F1	F1	F1	F1	F1	F1	F1	F1
Syn1	90	100	0	100	48	14	2	2
Syn2	96	100	100	100	72	92	100	10
Syn3	97	99	100	100	93	88	78	4
Syn4	66	73	53	63	68	56	25	4
Syn5	77	79	63	75	68	53	25	4

Appendix

Table 13: F1 values for high-dimensional (100 features) synthetic datasets. The target is the same function of

\{X_{1},\dots,X_{11}\}

as in the earlier experiments. This experiment includes 89 extra unimportant features of independent noise. Each F1 value is the median of 20 experiments. Hide&Seek

{}_{\text{ens}}

is an ensemble of 10 independent Hide&Seek models with 90% column subsampling and INVASE

{}_{\text{ideal}}

represents INVASE’s performance if the ideal

\lambda

is known.

	Hide&Seek	INVASE	REAL-x	SHAP	LIME	L2X
TPR	97	84	79	74	56	52
FDR	17	17	10	26	44	48
F1	88	83	83	74	56	52

Appendix

Table 14: IWFS performance on California Housing data. Each TPR, FDR and F1 metric is the median of 10 runs of each model on the same test data.

Figure 12: The correlation matrix for the 23 input features in the Credit default data experiments. The 11 features surrounded by the black box are used for $\{X_{1},\dots,X_{11}\}$ in generating Syn4–6 in Section 4.4 .

100 genes
OSTbeta	STATH	MAPK10	PLEKHG5	ERO1L	ZNF711	ZNF385	OR52E8	SLC5A11
P4HA3	LHFPL4	MGC33657	CAPZB	RBM15B	C1orf176	KLF3	OLFM4	NBR2
CCDC64	NUP210	HEMGN	SLC25A3	LEF1	MVD	OTUD3	KIAA1949	SLC44A3
ZNF775	THY1	DYNC1I2	CYP1A1	SPTA1	CLEC4M	RXFP3	TSHR	C7
CRYBB2	PPAPDC3	TXNL4B	CHST9	HACE1	AYTL1	PRSS35	ZNF408	DDC
CSTL1	OR2F1	C12orf50	SH3YL1	SNUPN	COL25A1	HPS4	ZFPM1	OAS2

Appendix

Table 15: 100 genes used for analysis in section 4.6 to match the same experiment as in Covert et al. (2021) .

Hide&Seek		INVASE		REAL-x
Gene	Importance $\pm$ SEM	Gene	Importance $\pm$ SEM	Gene	Importance $\pm$ SEM
ESR1	$0.999\pm 0.000$	CCNB2	$0.723\pm 0.037$	ESR1	$0.969\pm 0.003$
CCNB2	$0.951\pm 0.014$	ESR1	$0.711\pm 0.040$	CCNB2	$0.897\pm 0.006$
STATH	$0.827\pm 0.018$	ZNF775	$0.680\pm 0.039$	NUP210	$0.833\pm 0.010$
C6orf26	$0.804\pm 0.017$	KLF3	$0.619\pm 0.038$	C6orf15	$0.758\pm 0.013$
TUBB	$0.784\pm 0.024$	C6orf15	$0.610\pm 0.040$	SLC25A3	$0.754\pm 0.010$

Appendix

Table 16: Detailed gene importance (mean

\pm

standard error). Importance represents mean mask size (Hide&Seek, INVASE, REAL-x), mean importance scores (SHAP, LIME), or mean selection frequency (L2X).

Feature Selection

Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.

See all 187 papers

Model-Agnostic Feature Selection via LOCO-Guided Adaptive Minipatch Sampling
Sep 21, 2026Xuhui Liu, Lili ZhengAdaptive Sampling Feature Selection
Department of Statistics, University of Illinois Urbana-Champaign
Hypernetworks for Dynamic Feature Selection
May 12, 2026Javier Fumanal-Idocin, Raquel Fernandez-Peralta, Javier Andreu-PerezSet Transformer Hypernetworks
University of Essex Wivenhoe Park, Colchester, United Kingdom · Slovak Academy of Sciences Štefánikova 49, 814 73 Bratislava, Slovakia
Instance-Level Difficulty: A Missing Perspective in Machine Unlearning
Oct 3, 2024Hammad Rizwan, Mahtab Sarvmaili, Hassan Sajjad +1Machine Unlearning Machine Unlearning Evaluation
Department of Computer Science, Dalhousie University, Canada.

Sep 21, 2026J/K move · Enter open · S save

Model-Agnostic Feature Selection via LOCO-Guided Adaptive Minipatch Sampling

Xuhui Liu, Lili Zheng

Department of Statistics, University of Illinois Urbana-Champaign

Adaptive Sampling Feature Selection

Open paper AskPDF arXiv

Black-box machine learning models increasingly deliver strong predictions, but extracting useful information from them, such as a set of important features, remains challenging. Existing model-agnostic methods primarily estimate feature importance or conduct inference on it rather than directly selecting features, whereas many feature selection methods are model-specific or rely on the model-X assumption. We introduce LOCO-guided Adaptive Minipatch Sampling (LAMPS), a model-agnostic ensemble framework that uses any black-box regression algorithm as its base learner to select features important for predicting the response. The base learner need only produce predictions and need not perform feature selection itself. LAMPS operates within a minipatch ensemble framework that subsamples both observations and features, allowing leave-one-covariate-out (LOCO) feature importance scores to be easily computed. It adaptively concentrates minipatch sampling on features with high LOCO scores while maintaining exploration. The resulting sampling probabilities rapidly separate signal from noise features after a few iterations, enabling selection through simple thresholding. We establish that LAMPS achieves exact feature selection in high-dimensional settings, provided that the base predictive models are sufficiently well trained on average. Extensive experiments on synthetic and real data show that LAMPS outperforms state-of-the-art feature selection methods, with particularly strong performance in the presence of correlated features.

Abstract

Feature Selection

Explore similar work

Model-Agnostic Feature Selection via LOCO-Guided Adaptive Minipatch Sampling

Hypernetworks for Dynamic Feature Selection

Instance-Level Difficulty: A Missing Perspective in Machine Unlearning