We present Feature-Align CNN (FA-CNN), a prototype CNN architecture with intrinsic class attribution through end-to-end feature alignment. Our intuition is that the use of unordered operations such as Linear and Conv2D layers cause unnecessary shuffling and mixing of semantic concepts, thereby making raw feature maps difficult to understand. We introduce two new order preserving layers, the dampened skip connection, and the global average pooling classifier head. These layers force the model to maintain an end-to-end feature alignment from the raw input pixels all the way to final class logits. This end-to-end alignment enhances the interpretability of the model by allowing the raw feature maps to intrinsically exhibit class attribution. We prove theoretically that FA-CNN penultimate feature maps are identical to Grad-CAM saliency maps. Moreover, we prove that these feature maps slowly morph layer-by-layer over network depth, showing the evolution of features through network depth toward penultimate class activations. FA-CNN performs well on benchmark image classification datasets. Moreover, we compare the averaged FA-CNN raw feature maps against Grad-CAM and permutation methods in a percent pixels removed interpretability task. We conclude this work with a discussion and future, including limitations and extensions toward hybrid models.
Figures & tables
Figure 1: Comarison of averaged FA-CNN raw featuremaps (Left) versus Grad-CAM maps (Right). Perturbation methods RISE and K-OCC are also shown (Far right). Deepest layers are equivalent (correl 1.0), and middle layers show strong similarity. Aligned features also show gradual morphing layer-by-layer through depth.
Figure 2: Diagram showing a single order-preserving FA-CNN Layer using the dampened skip connection.
Figure 3: Diagram showing The 3D Global Average Pooling head to compute pre-logits from Penultimate Feature Maps.
Method
CIFAR-10
CIFAR-100
ImageNet-100
FA-CNN ( Pooling head)
93.13
77.03
81.78
FA-CNN (Linear head)
93.80
75.32
77.50
Vanilla-CNN
92.97
75.07
76.98
ResNet-18
93.57
73.96
77.74
ResNet-50
91.50
72.45
76.76
ResNet-18 (standard recipe)
94.89
77.31
83.44
Table 1: Top-1 validation accuracy (%) across datasets for all models. Models with minimal training recipe (top) reference models with standard recipe (bottom).
Figure 4: Classification Accuracy with Percent Pixels Removed on Layer 24 (top) and Layer 18 (bottom) on ImangeNet-100 Dataset.
Figure 5: Comarison of averaged FA-CNN raw featuremaps (Left) versus Grad-CAM maps (Right) and Perturbation methods (Far right).
Deep Learning has revolutionized machine learning, reaching unprecedented levels of accuracy, but at the cost of reduced interpretability. Especially in image processing systems, deep networks transform local pixel information into more global concepts in a highly obscured manner. Explainable AI methods for image processing try to shed light on this issue by highlighting the regions of the image that are important for the prediction task. Among these, Class Activation Mapping (CAM) and its gradient-based variants compute attributions based on the feature map and upscale them to the image resolution, assuming that feature map locations are influenced only by underlying regions. Perturbation-based methods, such as CorrRISE, on the other hand, try to provide pixel-level attributions by perturbing the input with fixed patches and checking how the output of the network changes. In this work, we propose Feature Activation Map Explanation (FAME), which combines both worlds by using network gradients to compute changes to the input image, manipulating it in a gradient-driven way rather than using fixed patches. We apply this technique on two common tasks, image classification and face recognition, and show that CAM's above-mentioned assumption does not hold for deeper networks. We qualitatively and quantitively show that FAME produces attribution maps that are competitive state-of-the-art systems. Our code is available: {\footnotesize https://github.com/AIML-IfI/fame.}
Deep Learning models can include billions of parameters or more, making it difficult to explain their internal transformations and outputs. However, explainability is increasing in importance due to the use of AI in crucial applications. This paper focuses on the interpretability of convolutional neural networks (CNNs). Building on the popular gradient based method LayerCAM for extracting internal features in CNNs, we propose an improved method named HiRA-CAM, and show that it outperforms both LayerCAM and Grad-CAM on creating useful saliency maps for object classification. The main feature of HiRA-CAM is its adaptive use of activation maps from all the layers of the CNN to arrive at a more focused saliency map.
Manasi Nerurkar, Ali A. Minai
Dept. of Electrical and Computer Engineering University of Cincinnati Cincinnati, USA
Although standard Convolutional Neural Networks (CNNs) can be mathematically reinterpreted as Self-Explainable Models (SEMs), their built-in prototypes do not on their own accurately represent the data. Replacing the final linear layer with a k-means-based classifier addresses this limitation without compromising performance. This work introduces a common formalization of k-means-based post-hoc explanations for the classifier, the encoder's final output (B4), and combinations of intermediate feature activations. The latter approach leverages the spatial consistency of convolutional receptive fields to generate concept-based explanation maps, which are supported by gradient-free feature attribution maps. Empirical evaluation with a ResNet34 shows that using shallower, less compressed feature activations, such as those from the last three blocks (B234), results in a trade-off between semantic fidelity and a slight reduction in predictive performance.
Ahcène Boubekki, Line H. Clemmensen
Department of Mathematical Sciences, University of Copenhagen, Denmark