Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations

Adriana Laurindo Monteiro; Jean-Michel Loubes

cs.LGMar 16, 2026

Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations

Authors: Adriana Laurindo Monteiro, Jean-Michel Loubes

Organizations: Independent Researcher · INRIA Regalia & ANITI · Institut de Math´ematiques de Toulouse, France · Centre INRIA de l’Universit´e de Bordeaux

Abstract

The growing use of Machine Learning (ML) tools comes with critical challenges, such as limited model explainability. We propose a global explainability framework that leverages Optimal Transport and Distributionally Robust Optimization to analyze how ML algorithms respond to constrained data perturbations. Our approach enforces constraints on feature-level statistics (e.g., brightness, age distribution), generating realistic perturbations that preserve semantic structure. We provide a model-agnostic diagnostic bench that applies to both tabular and image domains with solid theoretical guarantees. We validate the approach on real-world datasets providing interpretable robustness diagnostics that complement standard evaluation and fairness auditing tools.

Figures & tables

Figure 1: Impact of a feature in positive classifications: portion of predicted 1’s as a function of mean Age . Red marker represents original dataset.

Panel of Figure 2 — Figure 2: (a) Multiple mean changes impact on higher income ( $\geq$ 50k). As $\tau$ increases (respectively decreases), the values of the selected feature increases (respectively decreases). $\tau=0$ represents no projection. (b) Simultaneous mean perturbations for Gradient Boosting indicating a positive joint influence of features Education-Num and Hours per week on positive classifications. Color scale represents the proportion of predicted 1’s. (c) Disparate Impact variation with respect to Capital Gain mean constraint. The red star represents the DI of the original dataset and the gray area denotes the confidence interval.

Figure 3: Random test sample of fashion classes. First row shows from left to right T-shirt/top, Shirt, Bag, Pullover and Coat . Second row displays Sandal, Sneakers, Ankle Boots, Dress, Trouser.

Figure 4: Portion of positive classifications for each class by perturbation parameter $\tau$ . Red marker represents original dataset.

Figure 5: Perturbation of test sample of classes Sandal, Shirt, Sneaker, Bag and Ankle boot (from left to right). First row shows the darker perturbation $(\tau=-1)$ , middle row is the original image $(\tau=0)$ and third row is the brightest perturbation $(\tau=1)$ .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: (Top - Average price) average of price predicted for projected datasets with respect to $\tau$ . (Bottom - Price variance) with respect to $\tau$ . There is no projection when $\tau=0$ . The higher (respectively lower) value for $\tau$ , the higher (respectively lower) the value of the feature.

Figure 7: Percentage of unchanged predictions versus perturbation parameter $\tau$ . Red marker represents original dataset.

Figure 8: (Left) SHAP values of a specific individual (Middle) SHAP values averaged across the dataset in descent order. (Right) Feature importance score with importance weight.

Figure 9: (Left) Multiple mean stress obtained with entropic variable projection. (Middle) Stressing the feature Age . (Right) Stressing the feature Hours per week .

Explainability Evaluation

Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.

See all 321 papers

Informative Perturbation Selection for Uncertainty-Aware Post-hoc Explanations
Mar 16, 2026Sumedha Chugh, Ranjitha Prasad, Nazreen ShahPerturbation-Based Feature Attribution Uncertainty Quantification
Indraprastha Institute of Information Technology Delhi, India
Optimized Instance Alteration for Explaining and Assessing Robustness of Classifiers
Jul 7, 2026Evgenii Kuriabov, David Miller, Jia LiCounterfactual Explanations Adversarial Robustness
Department of Statistics, The Pennsylvania State University, University Park, PA 16802, USA · Department of Electrical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
eXplaining to Learn (eX2L): Regularization Using Contrastive Visual Explanation Pairs for Distribution Shifts
May 7, 2026Paulo Mario P. Medina, Jose Marie Antonio Miñoza, Sebastian C. IbañezDistribution Shift Robustness Interpretable ML
Center for AI Research

Mar 16, 2026J/K move · Enter open · S save

Informative Perturbation Selection for Uncertainty-Aware Post-hoc Explanations

Sumedha Chugh, Ranjitha Prasad, Nazreen Shah

Indraprastha Institute of Information Technology Delhi, India

Perturbation-Based Feature Attribution Uncertainty Quantification

Open paper AskPDF arXiv

Trust and ethical concerns due to the widespread deployment of opaque machine learning (ML) models motivating the need for reliable model explanations. Post-hoc model-agnostic explanation methods addresses this challenge by learning a surrogate model that approximates the behavior of the deployed black-box ML model in the locality of a sample of interest. In post-hoc scenarios, neither the underlying model parameters nor the training are available, and hence, this local neighborhood must be constructed by generating perturbed inputs in the neighborhood of the sample of interest, and its corresponding model predictions. We propose Expected Active Gain for Local Explanations (EAGLE), a post-hoc model-agnostic explanation framework that formulates perturbation selection as an information-theoretic active learning problem. By adaptively sampling perturbations that maximize the expected information gain, EAGLE efficiently learns a linear surrogate explainable model while producing feature importance scores along with the uncertainty/confidence estimates. Theoretically, we establish that cumulative information gain scales as

\mathcal{O}(d \log t)

, where

d

is the feature dimension and

t

represents the number of samples, and that the sample complexity grows linearly with

d

and logarithmically with the confidence parameter

1/δ

. Empirical results on tabular and image datasets corroborate our theoretical findings and demonstrate that EAGLE improves explanation reproducibility across runs, achieves higher neighborhood stability, and improves perturbation sample quality as compared to state-of-the-art baselines such as Tilia, US-LIME, GLIME and BayesLIME.

Abstract

Explainability Evaluation

Explore similar work

Informative Perturbation Selection for Uncertainty-Aware Post-hoc Explanations

Optimized Instance Alteration for Explaining and Assessing Robustness of Classifiers

eXplaining to Learn (eX2L): Regularization Using Contrastive Visual Explanation Pairs for Distribution Shifts