cs.CVNov 9, 2026

How a Model's Architecture and Performance Influences Explainability: A Study of Grad-CAM and CRAFT in CNN-Based Image Classification

Authors: Kilian Göller, Haadia Amjad, Steffen Seitz, Carsten Knoll, Ronald Tetzlaff

Organizations: Fundamentals of Electronics, TUD Dresden University of Technology, Dresden, Germany

Abstract

Deep learning models, particularly convolutional neural networks (CNNs), are widely adopted in computer vision, yet their opaque decision-making raises concerns in high-risk applications like healthcare and autonomous systems, where trust and accountability are critical. Therefore, robust explainable AI (XAI) methods are desirable to provide transparent insights into model decisions. To ensure these methods are reliable and effective across diverse scenarios, it is important to understand how model performance and architectural complexity influence explanation quality. Our study takes an initial step toward systematically investigating the impact of these two factors on key XAI evaluation metrics. We constructed a pipeline using 10 subsets of ImageNet-1K, each containing 80 unique classes, on each of which we trained three CNN architectures of increasing structural complexity: VGG16, ResNet50, and ConvNeXt-Tiny. For each individual model, we analyzed 10 training checkpoints representing increasing levels of model accuracy using two XAI methods: Grad-CAM (feature-attribution-based) and CRAFT (concept-based). We assessed these methods across four evaluation categories: Faithfulness, Correctness, Sensitivity, and Simplicity. Our findings reveal that as training progresses, the complex architectures (ResNet50 and ConvNeXt-Tiny) exhibit improved Faithfulness and lower Sensitivity, indicating more robust explanations. In contrast, VGG16's simpler architecture yields simpler but less faithful and correct explanations.

Figures & tables

Explore similar work

CardsList
  1. Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

    Oct 8, 2026Marco Stefani, Edoardo Parpaiola, Matteo Bino +5Neural Network InterpretabilitySaliency Map Evaluation

  2. SCAN: Visual Explanations with Self-Confidence and Analysis Networks

    Mar 4, 2026Gwanghee Lee, Sungyoon Jeong, Kyoungson JhangNeural Network InterpretabilityExplainable Image Classification

  3. Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality

    May 11, 2026Mateusz Cedro, Marcin ChlebusNeural Network InterpretabilityExplainability Evaluation