Navigating Disagreement Among Criteria for Evaluating Explanation Faithfulness
Organizations: Max Planck Institute for Security and Privacy, Germany · Yale University, USA
Abstract
Faithfulness requires that algorithmic explanations accurately reflect the model's behavior. Yet existing operationalizations of faithfulness can disagree about whether the same explanation is faithful, leaving developers and auditors to choose among conflicting evaluations. In this paper, we investigate disagreement among evaluation criteria and offer a practical path forward for evaluating explanation faithfulness. We first identify how operationalizations of faithfulness differ. We introduce a faithfulness functional that locates their disagreements in what an explanation claims about the model, which comparisons bear on that claim, and how agreement across those comparisons should be aggregated, and we reconstruct existing evaluation criteria within it. We argue these reconstructions support understanding faithfulness as a common though incompletely specified relation between the model and explanation, so the differences reflect choices left open by the general requirement. Justifying an operationalization then comes down to justifying these choices, which, on our view, draws on explanatory, institutional, and normative considerations. We support this argument with a case study of adverse action notices, reconstructing the evaluation implicit in an industry-proposed procedure and identifying which of its choices the setting justifies and which remain open. Our work gives auditors and practitioners a way to navigate disagreeing evaluations of faithfulness. It also explains why the literature should expect multiple operationalizations, while showing how concrete settings can narrow the choice among them. More broadly, our analysis shows how a shared requirement of faithfulness can support contextually justified evaluations despite disagreement among existing metrics.
Figures & tables
| Operationalization | Comparison | Assessment | Aggregation |
| Correspondence: attribution values should predict or track observed score changes. | |||
| Explanation infidelity ( Yeh et al., 2019 ) The attribution vector should predict the score change associated with each perturbation. | |||
| Faithfulness correlation ( Bhatt et al., 2020 ) Attribution sums should track the score changes caused by replacing feature subsets. | |||
| Dependence: disturbing the identified features should move the score. | |||
| Comprehensiveness ( DeYoung et al., 2020 ) Removing the selected rationale should suppress the model score. | |||
| AOPC ( Samek et al., 2017 ) Removing earlier-ranked features should rapidly suppress the model score. | |||
Explainability Evaluation
Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.
See all 347 papers