cs.CVNov 1, 2026

Vocabulary Aligned Explanations for Computer Vision Models

Authors: Erfan Mirhaji, Nikhil Krishnaswamy, Jill Zarestky, Lisa Mason, Nathaniel Blanchard, Sarath Sreedharan

Abstract

Optimizing vision models for accuracy does not force the model to learn and utilize human-understandable and domain-specific concepts and rules. We propose explaining image classifications through domain-specific concepts, visual evidence, and expert knowledge expressed as logical rules. We introduce FLUENT (Framework for Localized, Uncertainty-Aware Explanations in Normative Terminology), a post-hoc framework that ranks expert-authored rules combining jointly required, alternative, and absent concepts. To account for prediction errors, FLUENT combines predicted labels and concept states with empirically estimated likelihoods and concept priors. It scores each rule by calculating support over satisfying assignments to the concepts mentioned in that rule, then normalizes these scores to express relative confidence among candidate explanations. The selected rule is illustrated through occlusion-based highlights for positive concept mentions in the query image and localized examples in retrieved reference images for negative mentions. These illustrations are intended to show end users unfamiliar with the domain what the concepts named in an explanation look like. FLUENT combines domain knowledge and model accuracy, and delivers visual concept highlights and explicit rule confidence to explain an image in the vocabulary of an application domain.

Figures & tables

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VisionLogic: Discovering and Grounding Decision-Relevant Visual Concepts

    Mar 13, 2025Chuqin Geng, Yuhe Jiang, Li Zhang +5Explainable Artificial IntelligenceConcept-Based Explanations

  2. Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models

    May 7, 2026Ronaldo Canizales, Divya Gopinath, Corina Păsăreanu +1Abductive ReasoningConcept-Based Explanations

  3. Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation

    Jul 8, 2026Ahsan Habib Akash, Dipkamal Bhusal, Stacey Jones +3Concept-Based ExplanationsNeural Network Interpretability