Measuring Variable Importance Using Proportional Inclusive Marginal Effects

Youngseo Cho; V. Roshan Joseph

stat.MLOct 12, 2026

Measuring Variable Importance Using Proportional Inclusive Marginal Effects

Authors: Youngseo Cho, V. Roshan Joseph

Organizations: H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, GA, USA

Abstract

Shapley-based measures are widely used for measuring variable importance in statistics and machine learning. However, these measures have a serious issue--a nonactive variable can get a positive importance measure when it is correlated with an active variable. Herin et al. recently proposed a measure called proportional marginal effects (PME) that mitigates this spurious effect. However, PME is developed for deterministic models and has high computational cost. In this article, we propose a new measure called proportional inclusive marginal effects (PRIME) that overcomes these two limitations of PME. The key idea in PRIME is to use a novel model-free variable selection technique to identify the active variables and include only them in the computation of the importance measure. Analysis of test functions and real datasets show that PRIME can accurately and reliably quantify variable importance from noisy input-output data.

Figures & tables

Linear regression	Global sensitivity analysis	Nonparametric data analysis
$t^{2}$	Total Sobol’ indices	LOCO, FIRST
LMG	Shapley effects	SHAP, SAGE, SPVIM, FIRST-SHAP
PMVD	PME	PRIME (proposed measure)

Table 1: Analogy among some of the variable importance measures in linear regression, global sensitivity analysis, and real data analysis.

Figure 1: Allocation to

X_{3}

in ( 18 ). Curves show (A) Shapley effects and (B) PME as the coefficient

\beta

increases from 0 to 1 for four correlation settings.

Figure 2: Local-constant and local-linear residuals for

m(z+u)=u+u^{2}

u\in[-h,h]

. Panels A and B show the deterministic residuals; Panel C gives their integrated squares. Local linearization removes

h^{2}/3

, leaving

4h^{4}/45

Figure 3: Exact active-set recovery for Friedman function at

p=100

based on 50 replications.

Figure 4: Variable importance for Friedman function. Mean absolute SHAP is in response units. Diamonds mark the method-specific population PME targets: unit-signal PME for PRIME and response-variance PME for PME-NA. Computational times shown in red are median elapsed seconds per replication. Thirteen estimates of PME-NA lie outside the displayed range.

$n$	$\rho$	PRIME	PRIME-Full	PME-NA
100	0	0.101	0.140	0.644
100	0.5	0.107	0.143	0.404
100	0.9	0.103	0.134	0.748
200	0	0.067	0.104	0.428
200	0.5	0.062	0.094	0.398
200	0.9	0.083	0.109	0.773

Table 2: Mean relative half-

L_{1}

PME error over 30 datasets per condition.

Figure 5: Variable importance for Concrete. Mean absolute SHAP is in response units. Median computation times are shown in red. Twenty-nine estimates of PME-NA lie outside the displayed range.

Dataset	Noise	Negative entries	Full range	Outside plot range
Friedman	FALSE	0/300	$[0.030,\,0.319]$	0
	TRUE	116/300	$[-2.618,\,3.771]$	13
Concrete	FALSE	0/800	$[0.021,\,0.575]$	0
	TRUE	314/800	$[-3.583,\,3.080]$	29

Table S1: Native six-neighbor PME-kNN outputs under both noise settings.

Figure S1: Active-set coverage for Branin and Ackley–3 at

p=100

, with 50 paired replications per cell. Columns identify the response and rows the predictor law.

Figure S2: Exact active-set recovery at

p=100

. Each point is the proportion of 50 paired replications with

\widehat{A}=A

Figure S3: PME error after coordinate selection at

n=p=100

over 30 paired replications. Rows identify the predictor law and columns the response; boxes use completed allocations.

Scenario	Fixed ridge	BIC-gated	Empty
Ackley–3, $n=p=100$	0.067	0.967	1.000
Friedman, $n=p=100$	0.233	0.767	0.900
Suppressor, $n=100$ , $\sigma=0.1$	0.780	0.720	0.040
Suppressor, $n=500$ , $\sigma=0.5$	0.780	0.680	0.160

Table S2: Active-set coverage under alternative forward initializations.

Figure S4: Normalized RMSE of local-linear and local-constant risk estimates on the known active set

A

over 100 paired replications. Each point is one response–predictor-law setting; the diagonal marks equal error.

Figure S5: PME estimator calibration with paired inactive proxies. Panel A shows allocation error; Panel B shows elapsed time. Full-coordinate LL-PME uses local-linear subset-value estimation without screening.

Method	Mean loss	Active-set coverage	Proxy allocation	Completed
PRIME	0.040	0.758	0.0012	120/120
FIRST + 2NN-PME	0.103	0.458	0.0073	120/120
NNVS + 2NN-PME	0.078	0.492	0.0129	120/120
LASSO + 2NN-PME	0.213	0.225	0.0083	93/120
FOCI + 2NN-PME	0.102	0.450	0.0103	118/120
FORD + 2NN-PME	0.086	0.483	0.0053	120/120

Table S3: Correlated-proxy performance pooled over Branin and Bratley.

Figure S6: PME and Shapley computed from the same full-coordinate local-linear subset estimates at

n=200

and

\rho=0.5

. Diamonds mark population values; the dotted line separates active inputs and proxies.

Figure S7: Total proxy allocation as dependence increases. Points and bands show the median and 5th–95th percentiles over 30 datasets; dashed lines and open diamonds show population values.

Rule	Game estimator	Mean error	Mean proxy mass
PME	Full local-linear	0.09379	0.00014
PME	Matched- $K$ local-constant	0.44029	0.00082
PME	Two-neighbor local-constant	0.16836	0.00056
Shapley	Full local-linear	0.08641	0.10749
Shapley	Matched- $K$ local-constant	0.17978	0.11888
Shapley	Two-neighbor local-constant	0.10243	0.13831

Table S4: Estimator sensitivity at

n=200

and

\rho=0.5

over 30 paired Friedman datasets.

$n$	$\rho$	Raw game	Projected game
100	0	-0.16239	0.08757
100	0.5	-0.03505	0.14592
100	0.9	0.36126	0.40458
200	0	-0.12646	0.05681
200	0.5	0.02922	0.13831
200	0.9	0.36040	0.39743

Table S5: Shapley proxy allocation before and after monotone projection of the controlled two-neighbor game, over 30 datasets per condition.

Figure S8: PME error as inactive coordinates accumulate at

n=500

. Points and bars show medians and interquartile ranges over completed runs.

Response	Method	Median loss	Total time (s)	Selected $\|\widehat{A}\|$	Completed
OTL	PRIME	0.024	5.83	4.0	30/30
	FIRST + 2NN-PME	0.043	2.89	3.0	30/30
	NNVS + 2NN-PME	0.089	1.58	3.0	29/30
	LASSO + 2NN-PME	0.052	0.53	5.0	30/30
	FOCI + 2NN-PME	0.043	2.96	4.0	30/30
	FORD + 2NN-PME	0.043	7.11	3.0	30/30

Table S6: Screen–reduce–allocate pipelines at

n=p=500

over 30 paired runs.

Split ratio $r$	Observations $n_{Q}$	Selected-set Jaccard	Active recall	Speedup
2	250	1.000	1.000	1.64 [1.48, 1.72]
3	167	1.000	1.000	2.29 [2.03, 2.44]
5	100	1.000	1.000	3.52 [2.96, 3.78]

Table S7: Approximation using representative observations at

n=500

over 60 paired datasets per split ratio.

Figure S9: Observed-data diagnostics: Panel A compares bootstrap-median PRIME and full-coordinate LL-PME allocations; Panel B compares paired held-out RMSE for full and screened predictor sets.

Data	$n$	$p$	Selected $d$	Subset-value evaluations		Test-RMSE ratio
				PRIME	Full-coordinate
Concrete	1030	8	5	32	256	0.868
Airfoil	1503	5	3	8	32	0.806
Auto MPG	392	8	5	32	256	0.991

Table S8: Observed-data comparison under bootstrap allocation and nested cross-validation.

Feature Attribution

Feature attribution estimates which input features or internal components contribute to a model’s predictions.

See all 234 papers

Trustworthy Feature Importance Avoids Unrestricted Permutations
Apr 13, 2026Emanuele Borgonovo, Francesco Cappelli, Xuefei Lu +2Feature Attribution Perturbation-Based Feature Attribution
Department of Decision Sciences and Bocconi Institute for Data Science and Analytics, Bocconi University, Milan, Italy · Department of Decision Science, Bocconi University, Milan, Italy · SKEMA Business School, Université Côte d’Azur, France +2
Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
Feb 11, 2026Justin Whitehouse, Ayush Sawarni, Vasilis SyrgkanisSHAP Feature Attribution Shapley Value Attribution
Management Science and Engineering, Stanford University
cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Context
Feb 23, 2026Jörg Martin, Stefan HaufeFeature Attribution Shapley Value Attribution
Physikalisch-Technische Bundesanstalt, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · Charité – Universitätsmedizin Berlin, Berlin, Germany

Apr 13, 2026J/K move · Enter open · S save

Trustworthy Feature Importance Avoids Unrestricted Permutations

Emanuele Borgonovo, Francesco Cappelli, Xuefei Lu, Elmar Plischke+1

Department of Decision Sciences and Bocconi Institute for Data Science and Analytics, Bocconi University, Milan, Italy · Department of Decision Science, Bocconi University, Milan, Italy · SKEMA Business School, Université Côte d’Azur, France+2

Feature Attribution Perturbation-Based Feature Attribution

Open paper AskPDF arXiv

Feature importance methods using unrestricted permutations are flawed due to extrapolation errors; such errors appear in all non-trivial variable importance approaches. We propose three new approaches: conditional model reliance and Knockoffs with Gaussian transformation, and restricted ALE plot designs. Theoretical and numerical results show our strategies reduce/eliminate extrapolation.

Abstract

Feature Attribution

Explore similar work

Trustworthy Feature Importance Avoids Unrestricted Permutations

Statistical Inference and Learning for Shapley Additive Explanations (SHAP)

cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Context