Achieving constant accuracy in object detection is challenging due to the inherent variability of object sizes. One effective approach to this problem involves optimizing input resolution, referred to as a multi-resolution strategy. Previous approaches to resolution optimization have often been based on pre-defined resolutions with manual selection. However, there is a lack of study on run-time resolution optimization for existing architectures. This paper introduces DyRA, a dynamic resolution adjustment network providing an image-specific scale factor for existing detectors. This network is co-trained with detectors utilizing specially designed loss functions, namely ParetoScaleLoss and BalanceLoss. ParetoScaleLoss determines an adaptive scale factor for robustness, while BalanceLoss optimizes overall scale factors according to the localization performance of the detector. The loss function is devised to minimize the accuracy drop across contrasting objectives of different-sized objects for scaling. Our proposed network can improve accuracy across various models, including RetinaNet, Faster-RCNN, FCOS, DINO, and H-Deformable-DETR. The code is available at https://github.com/DaEunFullGrace/DyRA.git.
Figures & tables
Figure 1: Motivation for judicious image scaling. Scaling the resolution to align with the network’s capacity can enhance the detection accuracy of extreme scales.
Figure 2: (a) The overall architecture consists of two networks: DyRA and the detector. First, DyRA aims to estimate the scale factor from the given image; then, the detector classifies and localizes objects from the resized image as a general inference process. The proposed network is optimized by two additional loss functions, which are ParetoScaleLoss and BalacneLoss . (b) ParetoScaleLoss optimizes an image-specific scale factor based on ScaleLoss , which decides a box-specific scale factor. These loss functions adjust the factor based on the relative location within the boundary sizes for up-/down-scaling. BalanceLoss modifies the boundaries based on the detector’s localization performance.
Figure 3: (a) The x -axis denotes the size ratio, and the y -axis is the predicted scale factor from DyRA . The network is optimized by two additional loss functions, which are ParetoScaleLoss and BalacneLoss . (b) By the average of B , boxes are divided into two groups: blue (to be upscaled) and green (to be downscaled) . If the loss value of the blue region is larger than the green , the μ(B) will be moved to the right to reduce the loss value of the blue region.
Figure 4: Distribution of scale factors and examples of images. The x -axis is a square root of averaged box sizes within each image in the COCO.
Figure 5: Accuracy gain of each class of COCO trained with various detectors with DyRA .
Figure 6: Per box localization loss on FCOS (ResNet50). Boxes are equally divided into 3 groups, indicating small , medium , and large objects.
Figure 7: (a) BalanceLoss can help to converge γ . (b) Network-specific γ .
Figure 8: (a) P.O. can help to detect small objects. (b) The small-sized scales S exhibit higher adaptiveness of scale factors in training process.
Figure 9: (a) Scale factors over training iterations. (b) The absence of proposed loss functions significantly reduces the accuracy of DINO.
Figure 10: (a) Convergence of ParetoScaleLoss in RetinaNet. (b) The bar indicates the difference in localization losses between two different settings in RetinaNet. The gray bar indicates that the model without weight shows a lower localization loss, while the green bar denotes that the model with variable weight produces a lower value.
Figure 11: (a) Scheduling function plot. (b) Obtained accuracy from multi-step and ConstCosine learning rate scheduling.
Modern object detectors are static, fixed-depth networks optimized for a single operating point, requiring separate models for different deployment scenarios. We present an any-depth detection framework that enables a single network to span a continuous range of accuracy--efficiency trade-offs by controlling depth at inference time without retraining. Each backbone and neck stage is divided into an essential path, which always executes, and a skippable refinement path; this decomposition preserves the full multi-scale feature hierarchy at every depth configuration, unlike conventional early exiting that discards entire stages. To train such a network, jointly optimizing many sub-networks of varying depth introduces conflicting gradient signals. We address this via self-distillation between only the two extremes, with prediction-level and feature-level alignment losses that enforce stage-wise modularity, ensuring the outputs of each stage remain compatible regardless of the paths taken. Instantiated on RT-DETR and YOLOv12, our full-depth configurations match or surpass their respective SOTA baselines with negligible parameter overhead, while the most efficient configurations achieve up to 1.82× speedup at a cost of only 2.0 AP, all from a single set of weights.
Large-scale foundation models achieve strong performance across diverse tasks, but their size makes inference costly, largely due to dense matrix multiplications. Prior work reduces this cost by replacing dense weight matrices with efficient structured forms such as low-rank factorizations. However, these methods approximate weights rather than the output activations that determine inference accuracy. Consequently, small weight-space errors can be amplified by input activations, producing large output errors. In this work, we propose DyRA, an input-adaptive method that improves structured matrix multiplication approximation by correcting residual output errors during inference. We show that matrix multiplication can be approximated more effectively by directly optimizing low-rank factors of the output. DyRA builds on this insight by dynamically approximating and correcting the output error introduced by structured weight approximations. This combines efficient structured computation with input-dependent correction, yielding a more faithful approximation of full matrix multiplication under the same computational budget. Across vision, speech, and language models, DyRA consistently improves the accuracy-efficiency trade-off over structured weight approximations alone. Notably, DyRA achieves a 1.5× end-to-end GPU speedup for DINOv3 while reducing accuracy degradation by more than 3× relative to weight-only baselines.
Daewon Chae, Hyunwon Chung, Changwoo Lee +1
Department of EECS, University of Michigan · Qualcomm AI Research
Real-time object detectors achieve remarkable accuracy under controlled conditions, yet degrade sharply on non-ideal inputs: fisheye distortion, game-renderedcharacters, aerial viewpoints, and 360°panoramas. We present YOLOv14, a detection framework with four adaptive mechanisms designed for specific types of inputvariation:(1) Deformable Area-Attention with windowed computation and shiftedwindows for geometric distortion;(2) Multi-level Game2Real Alignment with progressive adversarial training for domain shift;(3) View-Aware Contrastive Learning with adaptive temperature for viewpoint invariance; and (4) Scene-Adaptive Augmentation with dynamic loss balancing for scene diversity. Together, YOLOv14 achieves 49.1 mAP on COCO val2017 at 2.91 ms (T4 GPU), and delivers substantial gains on fisheye (+4.1 mAP), panorama (+6.6 mAP), drone (+6.4 mAP), andour synthesized game-character benchmark (+26.1 mAP). We release code and models to facilitate reproducible research.
Jinling Jia, Jian Lu, Jone Yawl +1
Nanjing University of Posts and Telecommunications