stat.MLMar 5, 2025

Exploring specialization and sensitivity of convolutional neural networks under simultaneous image augmentations

Authors: Pavel Kharyuk, Sergey Matveev, Ivan Oseledets

Organizations: Marchuk Institute of Numerical Mathematics, Gubkin str., 8, 119333, Moscow, Russia · Skolkovo Institute of Science and Technology, Bolshoy Boulevard, 30, p.1, 121205, Moscow, Russia · Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, Leninskie gory, 1/52, 119991, Moscow, Russia · Artificial Intelligence Research Institute (AIRI), Moscow, Russia

Abstract

Drawing parallels with the way biological networks are studied, the authors adapt the treatment--control paradigm to explainable artificial intelligence research and enrich it through multi-parametric input alterations. A framework for investigating internal inference impacted by input data augmentations is proposed to address the real-world scenario with multiple simultaneously applied treatments instead of observing a single effect. The internal changes in network operation are reflected in activation changes measured by variance, which can be decomposed into components related to each augmentation by employing Sobol indices and Shapley values. To facilitate distinguishing model-related specifics from the self-interactions of augmentations, an auxiliary variable responsible for their permutations has been incorporated into the analysis. The resulting quantities enable one to visualize sensitivity to different variables and use them as features for further analysis and processing, including guided masking of activations. In addition, a method for single-class sensitivity analysis is introduced through a pipeline that involves class selection according to their match with the prediction bias generated by targeted damaging of the activations. Three convolutional networks, namely AlexNet, VGG11, and ResNet18, have been evaluated via the designed framework, with two vision transformers, ViT-b-16 and Swin-t, partially complementing the findings. Two sets of augmentations, consisting of five and seven distinct input transformations applied simultaneously, have been utilized as sources of variance in the image domain. Relying on the observed parallels, the authors assume that the developed framework can potentially be transferred to the study of biological neural networks in complex environments, as well as enhance the design of autonomous decision-making systems.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 14, 2026

How Data Augmentation Shapes Neural Representations

Data augmentation is widely recognized for improving generalization in deep networks, yet its impact on the geometry of learned representations remains poorly understood. In this work, we characterize how different data augmentation strategies reshape internal representations in neural networks. Using tools from shape analysis, we embed network hidden representations into a metric space where distance is invariant to scaling, translation, rotation and reflection. We show that increasing augmentation strength leads to well-behaved trajectories in this space, and that different augmentation types steer representations in distinct directions. Moreover, we investigate how neural representation shapes are distorted along data augmentation trajectories, and show that insights from neural geometry can predict which representations provide the most improvement when ensembling models. Our results reveal shared geometric patterns across architectures and seeds, and suggest that analyzing shape-space trajectories offers a principled tool for understanding and comparing data augmentation methods.
Oct 1, 2025

Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI

Perturbation-based explainability methods face criticism due to their reliance on out-of-distribution mutants. This raises doubts about the quality of the explanations. In this paper, we introduce a novel forward pass paradigm, Activation-Deactivation (AD), which obviates the need for perturbation of the input. AD replaces perturbation of input features with switching off parts of the model corresponding to to the intended perturbations. We implement ConvAD, an AD approximation algorithm for CNNs. ConvAD is a drop-in mechanism that can be easily added to any trained CNN and, without any additional training, generates more robust and more transferable explanations. We provide evaluation results across multiple architectures, datasets, methods and perturbation strategies, demonstrating the superior quality of ConvAD compared to the SOTA.
Aug 16, 2025

Grad-CAM for Visualizing Attention Regions of PCA and SVM Layers in Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are an effective approach for classification tasks, particularly when the training dataset is large. Although CNNs have long been considered a black-box classification method, they can be used as a white-box method through visualization techniques such as Grad-CAM. When the training samples are limited, incorporating a Principal Component Analysis (PCA) layer and/or a Support Vector Machine (SVM) classifier into a CNN can effectively improve the classification performance. However, a conventional Grad-CAM cannot be directly applied to PCA and/or SVM layers. Generating attention regions for PCA and/or SVM layers in CNNs is important to facilitate the development of white-box methods. Therefore, we propose PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and SVM-Grad-CAM'', a method for visualizing attention regions in an SVM classifier layer. Solving a closed-form Jacobian problem comprising partial derivatives from the last convolutional layer to the PCA and/or SVM layers is necessary to complete the proposed methods analytically. In this paper, we present the exact closed-form Jacobian and visualization results of the proposed methods applied to several major datasets. In addition, the insertion and deletion metrics, which are major evaluation metrics in explainable AI, were applied to PCA- and SVM-Grad-CAM. The results suggest that the proposed method can successfully visualize the attention regions of PCA and SVM.