cs.CVSep 5, 2026

Image-Scale Robustness and Visual Recognition Performance: A Cross-Architecture Analysis

Authors: Anish Monsley Kirupakaran

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The sensitivity of visual recognition models to changes in image scale is well established, yet the factors governing this sensitivity across heterogeneous architectures remain unclear. In this work, we investigate whether scale robustness exhibits a common quantitative structure across modern vision models. We evaluate 20 pretrained ImageNet-1K classifiers spanning seven architectural families, including convolutional, mobile, efficient, and Transformer-based architectures. By systematically reducing input image scale, we construct scale-accuracy response curves and define a characteristic scale as a compact measure of the onset of substantial recognition degradation. We then examine the relationship between characteristic scale and baseline recognition accuracy, model parameter count, architectural family, and representation stability. A strong inverse association is observed between baseline accuracy and characteristic scale (Pearson r = -0.890, R^2= 0.792, p < 10^-6). This relationship remains stable under bootstrap resampling, leave-one-architecture-out analysis, and leave-one-family-out analysis. In contrast, parameter count provides negligible additional explanatory power after controlling for baseline accuracy (p = 0.80), while architectural family does not provide significant incremental explanatory power. Furthermore, characteristic scale shows essentially no association with representation stability (r = -0.003, p = 0.991). These results indicate that, across the studied models, scale robustness is strongly organized by baseline recognition performance rather than simply by model size, architectural family, or representation stability. The study provides an empirical framework for characterizing scale robustness across vision architectures and identifies a reproducible accuracy-scale regularity that warrants further theoretical investigation.

Figures & tables

Figure 1: Each point represents one architecture. The fitted regression demonstrates a strong negative association between baseline accuracy and characteristic scale ( r=−0.890r=-0.890 , R2=0.792R^{2}=0.792 , p<10−6p<10^{-6} ). Higher-performing architectures generally retain their recognition performance at smaller image scales.
Figure 2: Robustness of the accuracy–scale relationship. (a) Bootstrap distribution of Pearson correlation between baseline accuracy and characteristic scale over 10,000 resamples, with the observed correlation and 95% confidence interval indicated. (b) Leave-one-architecture-out correlations demonstrate that the negative association persists across all individual architecture exclusions.
Figure 3: Robustness residuals quantify the difference between observed characteristic scale and the value predicted from baseline accuracy. Positive residuals indicate greater-than-expected scale tolerance, whereas negative residuals indicate lower-than-expected tolerance..
Figure 4: Characteristic scale versus representation stability. No significant association is observed between characteristic scale and representation-stability AUC ( r=−0.003r=-0.003 , p=0.991p=0.991 ), indicating that the observed cross-architecture variation in scale robustness is not explained by representation stability alone.
More in this topic

Neural Network Robustness

Neural network robustness studies how models withstand adversarial inputs, data shifts, noise, and hardware faults while maintaining reliable predictions.

See all 238 papers

Explore similar work

CardsList
Oct 19, 2026

On the Limits of Scale: Inverse Scaling in Vision-Language Models

Scaling laws suggest that increasing model size, training data, and compute generally yields monotonic performance improvements. While this often holds at the aggregate level, we present systematic evidence that vision-language models (VLMs) exhibit inverse scaling at the subtask level: specific question categories on which larger models perform worse than smaller counterparts within the same model family. We evaluate 19 models from four distinct model families across eight multimodal benchmarks (nine evaluation splits), answering 29,386 questions per model, 558,334 model-question evaluations in total. Using per-question pairwise analysis across all model pairs in each family, we identify 375 subtask-model-pair combinations exhibiting inverse scaling, spanning 4,342 unique queries where a smaller model answers correctly but a larger model fails, and we find that these patterns replicate across independently developed architectures. We identify 13 subtask-dataset pairs that show inverse scaling across multiple families, consistent with the phenomenon being task-driven rather than model-specific. The most affected domains in our study include multi-image spatial reasoning, visual perception, and university-level STEM knowledge, with accuracy gaps exceeding 20 percentage points in several cases, while diagram understanding appears comparatively resistant. These findings challenge the assumption that increasing model size uniformly improves multimodal capabilities and suggest that model selection should be task-aware.

Rohit Saxena, Alessandro Suglia, Marek Rei +1

School of Informatics, University of Edinburgh, United Kingdom · Imperial College London, United Kingdom · Miniml.AI, United Kingdom

Reasoning for VLMsVLM Evaluation
AskPDF
Mar 4, 2026

Revisiting the Role of Foundation Models in Cell-Level Histopathological Image Analysis under Small-Patch Constraints -- Effects of Training Data Scale and Blur Perturbations on CNNs and Vision Transformers

Background and objective: Cell-level pathological image analysis requires working with extremely small image patches (40x40 pixels), far below standard ImageNet resolutions. It remains unclear whether modern deep learning architectures and foundation models can learn robust and scalable representations under this constraint. We systematically evaluated architectural suitability and data-scale effects for small-patch cell classification. Methods: We analyzed 303 colorectal cancer specimens with CD103/CD8 immunostaining, generating 185,432 annotated cell images. Eight task-specific architectures were trained from scratch at multiple data scales (FlagLimit: 256--16,384 samples per class), and three foundation models were evaluated via linear probing and fine-tuning after resizing inputs to 224x224 pixels. Robustness to blur was assessed using pre- and post-resize Gaussian perturbations. Results: Task-specific models improved consistently with increasing data scale, whereas foundation models saturated at moderate sample sizes. A Vision Transformer optimized for small patches (CustomViT) achieved the highest accuracy, outperforming all foundation models with substantially lower inference cost. Blur robustness was comparable across architectures, with no qualitative advantage observed for foundation models. Conclusion: For cell-level classification under extreme spatial constraints, task-specific architectures are more effective and efficient than foundation models once sufficient training data are available. Higher clean accuracy does not imply superior robustness, and large pre-trained models offer limited benefit in the small-patch regime.

Hiroki Kagiyama, Toru Nagasaka, Yukari Adachi +5

Division of Gastrointestinal Surgery, Department of Surgery, Kobe University Graduate School of Medicine, 7-5-2 Kusunoki-cho, Chuo-ku,, Kobe 650-0017, Hyogo, Japan · Association of Medical Artificial Intelligence Curation, 505, Sakae Members Office Building, 4-16-8 Sakae, Naka-ku, Nagoya 460-0008, Aichi, Japan · Center for Medical Education and Clinical Training, Kindai University Faculty of Medicine, 1-14-1 Miharadai, Minami-ku, Sakai 590-0111, Osaka, Japan +1

Histopathology Image ClassificationComputational PathologyPathology Foundation ModelsVision Transformer
AskPDF
May 19, 2026

MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space

Modern vision models achieve strong performance on standard benchmarks, yet their aggregate accuracy reveals little about which scene properties drive their predictions. Existing robustness benchmarks provide important stress tests, but typically manipulate global 2D image properties, rely on entangled real-world variation, or cover only a limited set of 3D objects and scene parameters. We introduce MAPS (Manifolds of Artificial Parametric Scenes), a scalable instrument for controlled attribution of vision model behavior to scene parameters. MAPS comprises 2,618 curated photorealistic 3D meshes validated for recognizability across 560 ImageNet classes and provides a Blender-based rendering pipeline for on-demand image generation under continuous variation of nine independent scene-factors spanning background, camera, and lighting, extensible to other factors. To showcase its applicability, we use MAPS to evaluate 20 convolutional and transformer-based models by quantifying their reliance on these scene factors through regression-based sensitivity analysis. We find a near-universal failure axis across all tested architectures: camera distance and elevation consistently dominate recognition failure regardless of ImageNet accuracy. However, the full sensitivity structure reveals that modern CNNs and transformers cluster together, distinct from older architectures, suggesting that fine-grained architectural design choices, rather than the coarse CNN-versus-transformer distinction, are the stronger determinant of sensitivity profiles.

Santiago Galella, Pamela Osuna-Vargas, Maren Wehrheim +3

FIAS & Institute of Computer Science · York University · Mila & Department of Biology +1

Image Classification
AskPDF