Min-Marginal Power Indices for Multi-Objective Feature Selection

Wenjie Shi; Chi Zhao

cs.LGOct 24, 2026

Min-Marginal Power Indices for Multi-Objective Feature Selection

Authors: Wenjie Shi, Chi Zhao

Organizations: Saint Petersburg State University 7/9 Universitetskaya nab., Saint Petersburg, 199034, Russia

Abstract

Game-theoretic feature selection scores each feature by its average marginal contribution across coalitions. An average shows whether a feature helps overall, but not whether, in some coalition, adding it turns an acceptable model into an unacceptable one. Building on minimum marginal aggregation, we define the Min-Marginal Power Index (MinPI) on a binary utility-threshold game. For a fixed quota, MinPI takes the value -1, 0, or +1 and labels each feature harmful, neutral, or decisive. A harmful label carries its own evidence, an evaluated coalition in which adding the feature drops utility below the quota, so no score cutoff is needed; finite-budget estimators keep this property but may miss harmful features. We use MinPI to propose feature removals, which are accepted only if they pass a validation gate, and to prefilter multi-objective search. On six benchmark datasets over five seeds, MinPI removed 12.5% of analyzed features with almost no loss of held-out utility. On Madelon and Abalone, where it removed the most, its removals cost far less held-out utility than random removals of the same size. Prefiltering increased the dataset-averaged hypervolume of multi-objective search in all five seeds. MinPI is therefore a conservative feature-deletion step whose gains are largest where the data contain many removable features.

Figures & tables

Dataset	Native dim.	Cap	Samples	Task / utility
Spambase	57	57	4,601	binary / ROC AUC
Madelon	500	100	2,600	binary / ROC AUC
ISOLET	617	100	7,797	26-class / OVR AUC
Gisette	5,000	100	7,000	binary / ROC AUC
Credit-Card-Fraud	29	29	284,807	binary / ROC AUC
Abalone	8	8	4,177	regression / $R^{2}$

Table 1: Datasets and configured dimension cap. The realized analyzed dimension can be smaller after training-only variance filtering. OVR: one-vs-rest.

Figure 1: Validation-gated matched removal. (a) Mean descending ranks (1 is best) of the six methods from their five-seed mean $\Delta U$ (full participates but is omitted). (b) Five-seed mean $\Delta U$ ; positive favors removal. Controls delete the accepted MinPI-matched count, with 30 random-matched trials averaged per seed.

Dataset	Harmful	Removed	Compression (%)	Held-out $\Delta U$	Raw pass
Abalone	4.4	2.2	$27.50\pm 25.62$	$-.00565\pm.01077$	$3/5$
Credit-Card-Fraud	0.8	0.8	$2.76\pm 2.89$	$-.00078\pm.00335$	$5/5$
Gisette	0.6	0.6	$.60\pm.89$	$-.00011\pm.00016$	$5/5$
ISOLET	0.0	0.0	$.00\pm.00$	$.00000\pm.00000$	$5/5$
Madelon	48.0	43.6	$43.60\pm 31.96$	$+.00469\pm.00497$	$4/5$
Spambase	0.4	0.4	$.72\pm.99$	$-.00003\pm.00007$	$5/5$

Table 2: MinPI removal at

\alpha=.95

over five seeds. Compression and held-out

\Delta U

are mean

\pm

SD; “raw pass” counts validation-gate passes before backoff. Harmful and removed are mean counts of features flagged before the gate and removed after it.

Dataset	Full	MinPI	SHAP	MinShap	Banzhaf	Random
Madelon	.7876	.7923	.7985	.7861	.7884	.7144
Abalone	.5397	.5340	.5333	.5251	.5174	.5027
Credit	.9734	.9726	.9728	.9736	.9736	.9736
Gisette	.9887	.9886	.9887	.9889	.9887	.9887
ISOLET	.9935	.9935	.9935	.9935	.9935	.9935
Spambase	.9858	.9858	.9858	.9856	.9859	.9858

Table 3: Five-seed mean held-out utility after each variant deletes the accepted MinPI-matched count; utility is AUC/OVR-AUC/

R^{2}

by task, higher is better. SHAP/MinShap/Banzhaf/Random/Credit abbreviate SHAP-bottom/MinShap-bottom/Banzhaf-bottom/random-matched/Credit-Card-Fraud.

Figure 2: Overview of MinPI. (a) Exact minimum marginals in the constructed game equation 6 :

f_{1}=-1

, the rest neutral. (b) Detection probability vs budget for Algorithm A (

p_{-}=1/30=.0333

) and Algorithm B on the oracle seed coalition

\{f_{0},f_{2}\}

(

\rho=.129

); illustrative, not real-data guidance evidence. (c) Seed-42 held-out fronts of full vs prefiltered NSGA-II with the

\alpha

points on Madelon and Credit-Card-Fraud; connecting lines only order

\alpha

Figure 3: Hypervolume convergence of full vs MinPI-prefiltered NSGA-II under the matched 800-evaluation budget. Each panel is a dataset; blue/green lines are five-seed means of cumulative-best validation hypervolume with

\pm

1-SD shading and reference point

(1.1,1.1)

. Curves coincide on most datasets and separate on Madelon; the dataset-averaged held-out change is +0.016. No dataset-wise comparison is significant after Holm correction.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	$\alpha=.85$	$\alpha=.90$	$\alpha=.95$	$\alpha=.99$
Abalone	$(.375,+.0041)$	$(.500,-.0198)$	$(.500,+.0010)$	$(0,0)$
Credit-Card-Fraud	$(0,0)$	$(0,0)$	$(.034,-.0045)$	$(.586,-.0057)$
Gisette	$(0,0)$	$(0,0)$	$(0,0)$	$(.090,+.0002)$
ISOLET	$(0,0)$	$(0,0)$	$(0,0)$	$(.010,-.000005)$
Madelon	$(.240,+.0077)$	$(.490,+.0227)$	$(.560,-.0003)$	$(.510,-.0054)$
Spambase	$(0,0)$	$(0,0)$	$(0,0)$	$(.196,-.0012)$

Appendix

Table 4: Quota operating points, shown as (compression, held-out

\Delta U

). Every cell has

n=1

Feature Selection

Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.

See all 200 papers

MinShap: A Modified Shapley Value Approach for Feature Selection
Apr 16, 2026Chenghui Zheng, Garvesh RaskuttiShapley Value Attribution Feature Selection
Department of Statistics, University of Wisconsin - Madison, Madison, WI 53706
Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection
May 20, 2026Mathieu Cherpitel, Thomas Bäck, Martijn R. Tannemaat +1Feature Selection Multi-Objective Optimization
1LIACS, Leiden University, Leiden, The Netherlands · 2LUMC, Leiden University, Leiden, The Netherlands
Bias in Filter Feature Selection Evaluation: A Meta-Analysis of Datasets, Baselines, and Experimental Design Choices
Jun 5, 2026Malick Ebiele, Malika Bendechache, Rob BrennanFeature Selection
1*Computer Science, University College Dublin, Dublin, Dublin, Ireland. · 3ADAPT Centre, Dublin, Dublin, Ireland. · 2Computer Science, University of Galway, Galway, Galway, Ireland.

Apr 16, 2026J/K move · Enter open · S save

MinShap: A Modified Shapley Value Approach for Feature Selection

Chenghui Zheng, Garvesh Raskutti

Department of Statistics, University of Wisconsin - Madison, Madison, WI 53706

Shapley Value Attribution Feature Selection Statistical Learning Theory Multiple Hypothesis Testing

Open paper AskPDF arXiv

Feature selection is a classical problem in statistics and machine learning, and it continues to remain an extremely challenging problem especially in the context of unknown non-linear relationships with dependent features. On the other hand, Shapley values are a classic solution concept from cooperative game theory that is widely used for feature attribution in general non-linear models with highly-dependent features. However, Shapley values are not naturally suited for feature selection since they tend to capture both direct effects from each feature to the response and indirect effects through other features. In this paper, we combine the advantages of Shapley values and adapt them to feature selection by proposing MinShap, a modification of the Shapley value framework along with a suite of other related algorithms. In particular for MinShap, instead of taking the average marginal contributions over permutations of features, considers the minimum marginal contribution across permutations. We provide a theoretical foundation motivated by the faithfulness assumption in DAG (directed acyclic graphical models), a guarantee for the Type I error of MinShap, and show through numerical simulations and real data experiments that MinShap tends to outperform state-of-the-art feature selection algorithms such as LOCO, GCM and Lasso in terms of both accuracy and stability. We also introduce a suite of algorithms related to MinShap by using the multiple testing/p-value perspective that improves performance in lower-sample settings and provide supporting theoretical guarantees.

Abstract

Feature Selection

Explore similar work

MinShap: A Modified Shapley Value Approach for Feature Selection

Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection

Bias in Filter Feature Selection Evaluation: A Meta-Analysis of Datasets, Baselines, and Experimental Design Choices