We present Feature-Align CNN (FA-CNN), a prototype CNN architecture with intrinsic class attribution through end-to-end feature alignment. Our intuition is that the use of unordered operations such as Linear and Conv2D layers cause unnecessary shuffling and mixing of semantic concepts, thereby making raw feature maps difficult to understand. We introduce two new order preserving layers, the dampened skip connection, and the global average pooling classifier head. These layers force the model to maintain an end-to-end feature alignment from the raw input pixels all the way to final class logits. This end-to-end alignment enhances the interpretability of the model by allowing the raw feature maps to intrinsically exhibit class attribution. We prove theoretically that FA-CNN penultimate feature maps are identical to Grad-CAM saliency maps. Moreover, we prove that these feature maps slowly morph layer-by-layer over network depth, showing the evolution of features through network depth toward penultimate class activations. FA-CNN performs well on benchmark image classification datasets. Moreover, we compare the averaged FA-CNN raw feature maps against Grad-CAM and permutation methods in a percent pixels removed interpretability task. We conclude this work with a discussion and future, including limitations and extensions toward hybrid models.
Figures & tables
Figure 1: Comarison of averaged FA-CNN raw featuremaps (Left) versus Grad-CAM maps (Right). Perturbation methods RISE and K-OCC are also shown (Far right). Deepest layers are equivalent (correl 1.0), and middle layers show strong similarity. Aligned features also show gradual morphing layer-by-layer through depth.
Figure 2: Diagram showing a single order-preserving FA-CNN Layer using the dampened skip connection.
Figure 3: Diagram showing The 3D Global Average Pooling head to compute pre-logits from Penultimate Feature Maps.
Method
CIFAR-10
CIFAR-100
ImageNet-100
FA-CNN ( Pooling head)
93.13
77.03
81.78
FA-CNN (Linear head)
93.80
75.32
77.50
Vanilla-CNN
92.97
75.07
76.98
ResNet-18
93.57
73.96
77.74
ResNet-50
91.50
72.45
76.76
ResNet-18 (standard recipe)
94.89
77.31
83.44
Table 1: Top-1 validation accuracy (%) across datasets for all models. Models with minimal training recipe (top) reference models with standard recipe (bottom).
Figure 4: Classification Accuracy with Percent Pixels Removed on Layer 24 (top) and Layer 18 (bottom) on ImangeNet-100 Dataset.
Figure 5: Comarison of averaged FA-CNN raw featuremaps (Left) versus Grad-CAM maps (Right) and Perturbation methods (Far right).