cs.CVNov 8, 2026

ADC-RANet: Improving Resolution Adaptive Neural Networks with Attention, Distillation and Calibration

Authors: Jeferson Severino de Araujo, Lilian Hollard, Eric Granger, Marco Pedersoli

Organizations: LIVIA, Department of Systems Engineering, ETS Montreal, Canada

Abstract

Dynamic neural networks are a class of adaptive models in which the computation assigned to each input sample is determined dynamically, potentially reducing computational cost while maintaining similar performance. For image classification, the Resolution Adaptive Network (RANet) is a strong approach that progressively allocates computational resources across input scales based on sample difficulty. In this work, we introduce RANet with Attention, Distillation and Calibration (ADC-RANet), an enhanced version of RANet that improves performance across all computational budgets. Our approach combines three tightly connected components: bottom-up channel attention, top-down cascaded knowledge distillation, and overconfidence correction using MixUp, in a synergistic design that coordinates both pathways. Experiments on CIFAR-100 and ImageNet datasets show consistent gains while preserving the efficiency of RANet. Our code is available: https://github.com/jefersonseverino/ADC-RANet

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Nov 28, 2023

DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectors

Achieving constant accuracy in object detection is challenging due to the inherent variability of object sizes. One effective approach to this problem involves optimizing input resolution, referred to as a multi-resolution strategy. Previous approaches to resolution optimization have often been based on pre-defined resolutions with manual selection. However, there is a lack of study on run-time resolution optimization for existing architectures. This paper introduces DyRA, a dynamic resolution adjustment network providing an image-specific scale factor for existing detectors. This network is co-trained with detectors utilizing specially designed loss functions, namely ParetoScaleLoss and BalanceLoss. ParetoScaleLoss determines an adaptive scale factor for robustness, while BalanceLoss optimizes overall scale factors according to the localization performance of the detector. The loss function is devised to minimize the accuracy drop across contrasting objectives of different-sized objects for scaling. Our proposed network can improve accuracy across various models, including RetinaNet, Faster-RCNN, FCOS, DINO, and H-Deformable-DETR. The code is available at https://github.com/DaEunFullGrace/DyRA.git.
Feb 9, 2026

ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling

Scaling network depth has been a central driver behind the success of modern foundation models, yet recent investigations suggest that deep layers are often underutilized. This paper revisits the default mechanism for deepening neural networks, namely residual connections, from an optimization perspective. Rigorous analysis proves that the layout of residual connections can fundamentally shape convergence behavior, and even induces an exponential gap in convergence rates. Prompted by this insight, we introduce adaptive neural connection reassignment (ANCRe), a principled and lightweight framework that parameterizes and learns residual connectivities from the data. ANCRe adaptively reassigns residual connections with negligible computational and memory overhead (<1%<1\%), while enabling more effective utilization of network depth. Extensive numerical tests across pre-training of large language models, diffusion models, and deep ResNets demonstrate consistently accelerated convergence, boosted performance, and enhanced depth efficiency over conventional residual connections.
Feb 27, 2026

RAViT: Resolution-Adaptive Vision Transformer

Vision transformers have recently made a breakthrough in computer vision showing excellent performance in terms of precision for numerous applications. However, their computational cost is very high compared to alternative approaches such as Convolutional Neural Networks. To address this problem, we propose a novel framework for image classification called RAViT based on a multi-branch network that operates on several copies of the same image with different resolutions to reduce the computational cost while preserving the overall accuracy. Furthermore, our framework includes an early exit mechanism that makes our model adaptive and allows to choose the appropriate trade-off between accuracy and computational cost at run-time. For example in a two-branch architecture, the original image is first resized to reduce its resolution, then a prediction is performed on it using a first transformer and the resulting prediction is reused together with the original-size image to perform a final prediction on a second transformer with less computation than a classical Vision transformer architecture. The early-exit process allows the model to make a final prediction at intermediate branches, saving even more computation. We evaluated our approach on CIFAR-10, Tiny ImageNet, and ImageNet. We obtained an equivalent accuracy to the classical Vision transformer model with only around 70% of FLOPs.