Humans have been shown to use a "straightened" encoding to represent the natural visual world as it evolves in time (Hénaff et al. 2019). In the context of discrete video sequences, "straightened" means that changes between frames follow a more linear path in representation space at progressively deeper levels of processing. While deep convolutional networks are often proposed as models of human visual processing, many do not straighten natural videos. In this paper, we explore the relationship between network architecture, differing types of robustness, biologically-inspired filtering mechanisms, and representational straightness in response to time-varying input; we identify strengths and limitations of straightness as a useful way of evaluating neural network representations. We find that (1) adversarial training leads to straighter representations in both CNN and transformer-based architectures but (2) this effect is task-dependent, not generalizing to tasks such as segmentation and frame-prediction, where straight representations are not favorable for predictions; and nor to other types of robustness. In addition, (3) straighter representations impart temporal stability to class predictions, even for out-of-distribution data. Finally, (4) biologically-inspired elements increase straightness in the early stages of a network, but do not guarantee increased straightness in downstream layers of CNNs. We show that straightness is an easily computed measure of representational robustness and stability, as well as a hallmark of human representations with benefits for computer vision models.
Figures & tables
Figure 1: Schematic illustration of the representation of a discrete video sequence becoming progressively straighter as information is processed through a visual processing pipeline, starting from the highly nonlinear trajectory of typical video frames in pixel space.
Figure 2: Illustration of how the curvature measure is distinct from cosine similarity. (Left) Three points are sampled along a trajectory in time ( xt−1,xt,xt+1 ). The angle θ between neighboring x samples is their cosine similarity. (Right) Curvature ct is the angle between v1 and vt+1 , where vt is the difference between xt and xt−1 .
Figure 3: Output Representation layer curvature for a variety of network architectures and training schemes. The x-axis ticks describe different models and are formatted as: [architecture][training scheme][training dataset]. The datasets are IN:ImageNet ( Deng et al., 2009 ) , RIN:RestrictedImageNet ( Santurkar et al., 2019 ) ,CF:CIFAR10 ( Krizhevsky et al., 2014 ) , VOC:PASCAL VOC 2012 ( Everingham et al., 2010 ) , CIT:Cityscapes ( Cordts et al., 2016 ) , KITTI:KITTI ( Geiger et al., 2012 ) . Exact model details can be found in the supplemental data. The y-axis shows curvature of each model at the output representation layer relative to the curvature of the input video. For image classification models the output representation is the penultimate layer.
Figure 4: Deep network models vary in curvature for each layer over model depth. Curvature is shown for a variety of model. Filled circles indicate non-adversarially trained models, and open circles indicate adversarially-trained robust models. All adversarial models show output curvature below zero, indicating output straightening. All non-adversarial models have output curvature above zero, indicating output representation is less straight than the input pixels; All adversarial models shown have output curvature below zero, indicating output representation is more straight than input.
Figure 5: Left : White-box adversarial attacks reduce curvature in the output layer of ResNet50 models. Increased attack strength ( ϵ ) decreases curvature, with l2 attacks most reducing curvature in the resulting model. Data on symlog x-scale to show ϵ=0 : data to left of line on linear scale to show ϵ=0 , and right of line on log scale. Right : For a variety of robustly trained models, Improved AutoAttack accuracy for APGD-CE is predicted by larger decreases in curvature from the pixel layer for adversarially robust CIFAR-trained ResNet50 models. We plot CIFAR-trained models trained with l2 norm, ϵ=0,0.25,0.5,1.0 .
Figure 6: Left : Curvature distribution for Adversarially Robust (Robust) and Non-Robust (Regular) ResNet when tested on VGGSound dataset. Robust model shows lower curvature over all videos. Right : Standard Deviation of true label prediction for same networks. Robust model shows overall lower variability in probability for the true label.
Figure 7: Visual Transformer models vary in curvature over model blocks. All ViT models (standard and DINO, base-patch-16) show output curvature higher than the pixel input. For DINO and standard ViT models, this increase is consistent throughout the layers. For CrossViT (18 dagger 408) the representation forks in the middle stage. One path increases in curvature, while the other decreases below input, but after recombination output is curved above pixel input.
Figure 8: Segmentation models vary in curvature for each layer over model depth. Curvature is shown for various segmentation model types. Filled circles indicate non-adversarially trained models, and open circles indicate adversarially-trained robust models.
Figure 9: Biologically-inspired network models result in straightened representations compared to input pixels. Non-parametric multi-stage filter models PeriphNet and Henaffbio progressively straighten. The VOneNetCornets model straightens the representation in its V1-block up until its noise layer, where curvature increases dramatically and does not recover in downstream convolutional blocks. Adversarially-trained CrossViT model shows progressive straightening.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Change in curvature for different numbers of principle components . Each subplot shows layer-wise change in curvature. Left is a standard ResNet50 trained on ImageNet ( Deng et al., 2009 ) classification. The natural line is the curvature measures on the natural videos from ( Hénaff et al., 2019 ) that we used for all model analyses in the paper. The contrast and artificial sequences were additional video sequences psychophysically tested by ( Hénaff et al., 2019 ) where humans tend to have little change in curvature for contrast but large increase in curvature for artificial. Right is an adversarially robust (projected gradient descent with l2 norm and ϵ=3 ) ResNet50 trained on ImageNet. All rows below the top row show curvature when taking the first 2 , 5 , 10 , and 100 principle components (PCs) of each representation layer activations. For PCs less than the number of frames in each video ( 11 ), we performed PCA on the pixel-level video in addition to the model activations when computing the change in curvature. For all PCs, the standard ResNet50 has a higher curvature for natural video sequences than the adversarially robust.
Figure 11: Final stage curvature for a variety of network architecture and training schemes. Model families show very similar curvature to that of their output representation layers 3 .
Figure 12: ResNet50 models trained on ImageNet
Figure 13: ResNet50 models trained on CIFAR10
Figure 14: ResNet50 models trained on Restricted ImageNet (a subset of ImageNet)
Figure 20: DL3 segmentation Models trained on VOC ( Everingham et al., 2010 )
Figure 21: PSP segmentation models trained on CIT ( Cordts et al., 2016 )
Figure 22: PSP segmentation models trained on VOC ( Everingham et al., 2010 )
Figure 23: Left : While adversarial attacks with greater ϵ strength impart many desirable robustness properties on a network, adversarial training does not improve test accuracy, often decreasing test accuracy on the within-domain test set for a given model. (Data plotted on symlog scale.) Right : While stronger adversarial attacks decrease curvature, improved test accuracy for a model is not predictive of output curvature reduction. Rather, within a given model training/test set, increased test accuracy predicts a smaller curvature reduction in the output layer.
Figure 24: Curvature plots for three of the top five performing models for ImageNet-C (corrupted) dataset benchmark. MAE is a masked autoencoder transformer network ( He et al., 2022 ) , mae_dat is the mae model with additional discreet adversarial training ( Mao et al., 2022 ) (note this is a modified version of traditional adversarial training for visual models), and hat is a visual transformer optimized to process high-frequency image components Bai et al. (2022) . Although these models are robust to image corruptions, they do not reduce curvature. The mae_dat and hat models also represent the top two performing models on Stylized ImageNet, indicating that robustness to shape/texture transforms like those in Stylized ImageNet ( Geirhos et al., 2018 ) is also not indicative of curvature reduction.
Figure 25: Curvature plots for three model architectures modified to be shift-invariant using the anti-aliasing method from ( Zhang, 2019 ) . These models are robust to location shifting, but do not show significant curvature reduction from input, as seen in adversarially-trained robust models
Figure 26: Curvature plots for ResNet50 variants trained: without robust training (Standard ResNet50), with shift-invariance robustness as in ( Zhang, 2019 ) (Shift Invariant ResNet50), and with adversarial training (Adversarially Robust ResNet50). Shift invariant robustness only slightly reduces curvature as compared to the standard model, and does not reduce output curvature from the input data, while adversarial robustness strongly reduces curvature reducing it below that of the input data.
Figure 27: Curvature plots for AlexNet variants trained: without robust training (Standard AlexNet), with shift-invariance robustness as in ( Zhang, 2019 ) (Shift Invariant AlexNet), and with adversarial training (Adversarially Robust AlexNet ( Liu et al., 2021 ) ). The small size of AlexNet limits its ability to benefit from adversarial attacks ( Liu et al., 2021 ) . This may explain why, unlike other adversarially trained models tested, curvature is not reduced significantly from the input. Shift-invariance in this model however, does appear to reduce output curvature as compared to the standard model.
Figure 28: PredNet change in curvature across model layers measured in-distribution (top: KITTI dataset) and out-of-distribution (bottom: ( Hénaff et al., 2019 ) stimuli) . Generally, all PredNet models have lower curvature values for KITTI, the in-distribution dataset, than the ( Hénaff et al., 2019 ) stimuli, but the curvature follows similar trends across dataset. All PredNet models are trained on KITTI. The blue, orange, and green lines are the baseline PredNet models developed in ( Lotter et al., 2016 ) . The baseline models have a lower change in curvature on the KITTI dataset than the ( Hénaff et al., 2019 ) stimuli. However, the change in curvature is generally close to 0 for both datasets – indicating little straightening. The red and purple lines show PredNet models that were trained to straighten layers 2 and 3 respectively on the KITTI dataset. Those models show a larger drop in curvature on the KITTI dataset than ( Hénaff et al., 2019 ) stimuli at layers 2 and 3 , but the models overall straighten in only two layers for both datasets.
Figure 29: Example Predictions for Original and Straighten PredNet . Both models were trained on the KITTI dataset ( Geiger et al., 2012 ) . The straightened PredNet was trained with penalty on its R2 layer if there was high curvature.
Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.
In computational vision science, Convolutional Neural Networks (CNNs) have emerged as a popular model of biological vision because of the alignment they can exhibit with neural and behavioral data in humans and animals. However, it remains unclear to what extent this alignment persists for visual tasks that extend beyond the canonical object recognition paradigm based on well defined semantic content. In this study, we diverge from the common object-centric view by focusing on another aspect of vision: texture perception. We consider textures of different complexity generated with three different algorithms from the same source images. Using a rank-based statistic, we quantify the information encoded in the internal representations of a CNN and three Vision Transformers (ViTs), and we compare the similarity of these representations to those inferred from human psychophysics data. We find that the representation of textures is aligned in different ViTs, but not between the ViTs and the CNN; that ViTs form similar representations for textures of different complexity; that human performance in recognizing textures can be better predicted from ViTs representations rather than CNN representations. Taken together, these results suggest that ViTs may capture more faithfully than CNNs how texture patterns are visually processed by humans, and that the representations of texture stimuli in computational models may be driven by the network architecture.
Ludovica de Paolis, Marco Baroni, Alessandro Laio +1
Department of Neuroscience, International School for Advanced Studies (SISSA), Trieste, Italy · Department of Language and Translation Sciences, Pompeu Fabra University, Barcelona, Spain · ICREA, Barcelona, Spain +1
Deep convolutional neural networks (DCNNs) have rivaled humans on many visual tasks, yet they remain vulnerable to near-imperceptible perturbations generated by adversarial attacks. Recent work shows that aligning DCNN representations with human visual cortex activity improves adversarial robustness, but the mechanisms driving this advantage are unclear. One hypothesis suggests that neural alignment confers robustness by biasing models away from brittle high-frequency details and towards the low spatial frequencies (LSF). However, recent work shows that human object recognition critically depends on a narrow, mid-frequency "human channel". Interestingly, this band was partially preserved in prior LSF-focused studies. Here, we investigate whether a spectral bias towards the LSF or the human channel is the primary driver of the adversarial robustness observed in neurally aligned DCNNs. We first show that DCNNs aligned to higher-order regions of the human ventral visual stream systematically increase reliance on both LSF and the human channel. However, directly steering DCNNs towards these bands revealed a clear dissociation. Biasing models towards the human channel, either alone or together with LSF, does not improve robustness and even impairs it. LSF bias produced some robustness gains, but such improvements are modest despite inducing much larger shifts in spatial-frequency reliance than neurally aligned models. Spatial-frequency-biased models overall show little, if any, increase in similarity to human neural representational geometry. Together, our results suggest that altered spatial-frequency reliance is likely an emergent property of learning more human-like representations rather than the primary mechanism by which neural alignment confers adversarial robustness, and motivate the need for future research examining representational properties beyond spatial-frequency profiles.
Zhenan Shao, Tianyu Ren, Chengxiao Wang +2
Department of Psychology, University of Illinois Urbana-Champaign · The Beckman Institute, University of Illinois Urbana-Champaign · Department of Cognitive Science, Johns Hopkins University +2
Progress in AI has largely been driven by methods that assume less. As compute and data increase, approaches with weaker inductive biases generally outperform those with stronger assumptions. This is particularly characteristic of the field of Visual Representation Learning, where approaches have gone from being dominated by Supervised Learning, to Weakly Supervised Learning, to the now widespread success of Self-Supervised Learning without human labels. Yet, even modern Self-Supervised Learning approaches still depend on strong inductive biases such as augmentations, masking, or cropping. If this trend holds, even these remaining biases should become bottlenecks at scale -- and our experiments confirm this: the optimal strength of inductive biases decreases as data grows. This motivates the search for approaches that rely on fewer assumptions. To this end, we introduce Temporal Difference in Vision (TDV), a new paradigm for self-supervised learning from video that avoids existing inductive biases, relying instead on a causal assumption that the past causes the future. TDV functions by jointly training an image encoder and a motion encoder so that the current frame's representation plus the encoded motion equals the next frame's representation. Despite not leveraging any strong inductive biases, TDV matches state-of-the-art recipes on dense spatial tasks, laying the foundation for representation learning without strong assumptions.