Deploying high-performance dense prediction models on resource-constrained edge devices remains challenging due to strict computation and memory budgets. In practice, lightweight systems for object detection, instance segmentation, and pose estimation are still dominated by CNN-based architectures such as YOLO, while compact Vision Transformers (ViTs) often struggle to achieve comparable accuracy-efficiency trade-offs, even with large-scale pretraining. We argue that this gap arises primarily from insufficient task-specific representation learning in small-scale ViTs, rather than from an inherent mismatch between ViTs and edge dense prediction. To address this issue, we introduce EdgeCrafter, a unified compact ViT framework for edge dense prediction centered on ECDet, a detection model built on a distilled compact backbone and an edge-friendly encoder-decoder design. The resulting detection-distilled representation transfers directly to instance segmentation and human pose estimation through lightweight task-specific prediction modules. Without using task annotations beyond COCO, ECDet-S achieves 51.7 box AP with fewer than 10M parameters, while ECInsSeg-X and ECPose-X reach 48.4 mask AP and 74.8 keypoint AP, respectively. As a complementary but more compute-intensive setting, Objects365 detection pretraining consistently improves performance across all scales, with the X variants reaching 59.9 box AP, 49.8 mask AP, and 75.9 keypoint AP. These results show that compact ViTs, when combined with task-specialized distillation and edge-aware design, can be a practical and competitive solution for edge dense prediction. Code is available at: https://intellindust-ai-lab.github.io/projects/EdgeCrafter
Figures & tables
Figure 1: Comprehensive evaluation of EdgeCrafter. (a) Accuracy–efficiency comparison across object detection, human pose estimation, and instance segmentation on COCO ( Lin et al., 2014 ) . The top and bottom rows plot model parameters and FLOPs, respectively, against AP. Methods marked with ∗ use Objects365 pretraining ( Shao et al., 2019 ) . (b) Training dynamics and downstream AP for ECDet-T under different backbone initialization strategies. ViT-T follows the ImageNet-21K ( Ridnik et al., 2021 ) training strategy of Steiner et al. (2022) . Supervised ImageNet-21K pretraining is weaker than training from scratch for this compact model, while task-specialized distillation yields substantially stronger downstream performance.
Figure 2: Overview of the EdgeCrafter pipeline. Stage 1: A pretrained DINOv3 backbone ( Siméoni et al., 2025 ) is adapted to object detection to create a task-specialized teacher within the ECDet formulation. Stage 2: The resulting teacher distills its detection-oriented representation into compact ECViT student backbones through feature alignment on a large image collection. Stage 3: The distilled students are used to instantiate the ECDet model family at different scales (S/M/L/X), and the same distilled backbone and encoder are further reused for instance segmentation and human pose estimation with lightweight task-specific heads. The key idea is that detection serves as the representation-learning stage, while the learned backbone transfers directly to other dense prediction tasks.
Figure 3: Architecture of ECDet. A frozen detection-specialized teacher distills the compact ECViT backbone, whose convolutional stem replaces standard large-stride patch embedding. The final two transformer features are fused, converted into stride- 8 , stride- 16 , and stride- 32 feature maps, refined by the hybrid encoder, and decoded into object predictions.
Teacher Arch.
COCO Pre-training
Distillation Dataset
AP (%)
Teacher capacity
DINOv3-S
✓
IN-1K + COCO
54.0
DINOv3-B
✓
IN-1K + COCO
54.3
DINOv3-L
✓
IN-1K + COCO
52.6
Teacher task adaptation
DINOv3-B
✗
IN-1K + COCO
53.5
Table 5: Unified ablation on teacher design and distillation data. All experiments use ECViT-T+ as the student. The teacher is initialized from DINOv3 ( Siméoni et al., 2025 ) , and “COCO pre-training” indicates that the teacher is first adapted as the ECDet backbone on COCO before distillation.
Table 5
Patch Embedding
Params (M)
FLOPs (G)
Latency (ms)
AP
AP S
AP M
AP L
Vanilla
19.0
82.5
7.87
53.5
33.7
58.2
72.8
ConvStem ( d=1 )
19.2
84.9
7.98
54.3
35.9
59.1
72.7
ConvStem ( d=2 )
19.2
84.9
-
53.0
33.7
57.9
72.5
ConvStem ( d=3 )
19.2
84.9
-
53.6
33.2
58.6
72.6
Table 8: Effect of patch embedding design and stem receptive field. Vanilla uses a single 16×16 stride-16 convolution. ConvStem follows the convolutional-stem design used in the final model and consists of four 3×3 stride-2 convolutions ( Xiao et al., 2021 ) . Here, d denotes the dilation rate of the final stem convolution, which controls the effective receptive field of the patch embedding stage. All results are reported on ECDet-M.
Fusion Method
Fused Layers
Params (M)
FLOPs (G)
Latency (ms)
AP
Mean
L11
19.2
84.9
7.98
54.1
Mean
L10→11
19.2
84.9
7.98
54.3
Mean
L9→11
19.2
84.9
7.98
54.3
Mean
L6→11
19.2
84.9
8.11
54.1
Concat
L10→11
19.5
87.0
8.05
54.2
STA ( Huang et al., 2025a )
L5 , L7 , L11
19.4
86.3
8.48
54.3
Table 9: Effect of layer fusion strategy. We compare simple mean fusion over different late-layer ranges with concatenation and the STA fusion module from DEIMv2 ( Huang et al., 2025a ) . The gray row marks the adopted default setting in ECDet-M.
Student
Teacher
AP
ECViT-T
–
46.6
Co-DETR-Swin-S ( )
50.4
DINOv3-S
50.7
ECTeacher-S
51.7
Table 10: Comparison of different teacher models. We additionally evaluate Co-DETR with a Swin-S backbone ( Zong et al., 2023 ; Liu et al., 2021 ) as a strong detection teacher, while keeping the ECDet-S student architecture and overall training budget unchanged.
Method
#Epochs
Detection
Distillation
Total GPU Hours ↓
\rowfont YOLO26-M
80
∼ 24
–
∼ 24
YOLO11-M
500
∼ 133
–
∼ 133
RT-DETRv4-M
102
∼ 190
–
∼ 190
ECDet-M
62
∼ 96
∼ 34
∼ 130
Table 11: Comparison of training cost on COCO. All models are trained on 4 × NVIDIA RTX 4090 GPUs . Gray results indicate pre-training on Objects365 ( Shao et al., 2019 ) followed by fine-tuning on COCO.
The Vision Transformer (ViT) achieves remarkable accuracy across visual tasks but remains computationally expensive for edge deployment. This paper presents MicroViTv2, a lightweight Vision Transformer optimized for real-device efficiency. Built upon the original MicroViT, the proposed model is designed based on reparameterized design, specifically Reparameterized Patch Embedding (RepEmbed) and Reparameterized Depth-Wise convolution mixer (RepDW) for faster inference, and introduces the Single Depth-Wise Transposed Attention (SDTA) to capture long-range dependencies with minimal redundancy. Despite slightly higher FLOPs, MicroViTv2 improves accuracy up to 0.5% compared to its predecessor and surpassing MobileViTv2, EdgeNeXt, and EfficientViT while maintaining fast inference and high energy efficiency on Jetson AGX Orin. Experiments on ImageNet-1K and COCO demonstrate that hardware-aware design and structural re-parameterization are key to achieving high accuracy and low energy consumption, validating the need to evaluate efficiency beyond FLOPs. Code is available at https://github.com/novendrastywn/MicroViT.
Novendra Setyawan, Chi-Chia Sun, Mao-Hsiu Hsu +2
Department of Electro-Optics, National Formosa University, Taiwan · Department of Electrical Engineering, University of Muhammadiyah Malang, Indonesia · Department of Electrical Engineering, National Taipei University, Taiwan +1
In recent years, vision transformers (ViTs) have emerged as powerful and promising techniques for computer vision tasks such as image classification, object detection, and segmentation. Unlike convolutional neural networks (CNNs), which rely on hierarchical feature extraction, ViTs treat images as sequences of patches and leverage self-attention mechanisms. However, their high computational complexity and memory demands pose significant challenges for deployment on resource-constrained edge devices. To address these limitations, extensive research has focused on model compression techniques and hardware-aware acceleration strategies. Nonetheless, a comprehensive review that systematically categorizes these techniques and their trade-offs in accuracy, efficiency, and hardware adaptability for edge deployment remains lacking. This survey bridges this gap by providing a structured analysis of model compression techniques, software tools for inference on edge, and hardware acceleration strategies for ViTs. We discuss their impact on accuracy, efficiency, and hardware adaptability, highlighting key challenges and emerging research directions to advance ViT deployment on edge platforms, including graphics processing units (GPUs), application-specific integrated circuit (ASICs), and field-programmable gate arrays (FPGAs). The goal is to inspire further research with a contemporary guide on optimizing ViTs for efficient deployment on edge devices.
Shaibal Saha, Lanyu Xu
Department of Computer Science and Engineering, Oakland University, , Rochester Hills, 48309, Michigan, USA
Vision Transformers (ViTs) have demonstrated strong representation capability in image classification. However, their quadratic self-attention complexity and large parameter counts limit deployment on resource-constrained mobile and edge devices. This paper introduces UtVAA, an ultra-tiny Vision Transformer architecture designed for efficient visual recognition under strict computational budgets. It incorporates a novel Affix Attention block that combines depthwise-pointwise local feature extraction, linear self-attention, coordinate attention for spatial dependency modelling, and a lightweight ternary fusion strategy to integrate local and global representations. In addition, Dilated Bottleneck blocks expand the receptive field using dilated depthwise separable convolutions while maintaining low FLOPs and stable optimisation through residual connections. UtVAA is implemented in scalable Tiny, Medium, and Large variants, with the smallest model containing 204.67K parameters and 53.95M FLOPs. Experimental results on CIFAR-10, CIFAR-100, PlantVillage-Tomato and SLIF-Tomato datasets show that UtVAA achieves competitive accuracy within a sub-million-parameter regime. Overall, the results demonstrate that transformer-based vision models can be redesigned into ultra-tiny architectures without significant loss in discriminative performance, making UtVAA suitable for mobile and edge deployment. Code is available at https://github.com/romiyal/UtVAA