Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies

Shaibal Saha; Lanyu Xu

cs.CVFeb 26, 2025

Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies

Authors: Shaibal Saha, Lanyu Xu

Organizations: Department of Computer Science and Engineering, Oakland University, , Rochester Hills, 48309, Michigan, USA

Abstract

In recent years, vision transformers (ViTs) have emerged as powerful and promising techniques for computer vision tasks such as image classification, object detection, and segmentation. Unlike convolutional neural networks (CNNs), which rely on hierarchical feature extraction, ViTs treat images as sequences of patches and leverage self-attention mechanisms. However, their high computational complexity and memory demands pose significant challenges for deployment on resource-constrained edge devices. To address these limitations, extensive research has focused on model compression techniques and hardware-aware acceleration strategies. Nonetheless, a comprehensive review that systematically categorizes these techniques and their trade-offs in accuracy, efficiency, and hardware adaptability for edge deployment remains lacking. This survey bridges this gap by providing a structured analysis of model compression techniques, software tools for inference on edge, and hardware acceleration strategies for ViTs. We discuss their impact on accuracy, efficiency, and hardware adaptability, highlighting key challenges and emerging research directions to advance ViT deployment on edge platforms, including graphics processing units (GPUs), application-specific integrated circuit (ASICs), and field-programmable gate arrays (FPGAs). The goal is to inspire further research with a contemporary guide on optimizing ViTs for efficient deployment on edge devices.

Figures & tables

Figure 1: (a) The prevalence of transformer-based models in computer vision has led to a substantial increase in research publications. (b) Given their high computational complexity, model compression techniques are critical for reducing redundancy and improving efficiency. These advancements are essential for optimizing ViTs for hardware acceleration and real-world deployment on resource-constrained platforms [ 18 ] .

Survey	Year	Scope
Survey	Year	Model Compression	Software Tools	Evaluation Metrics	Hardware Accelerators
[ 24 ]	2022	✓	✗	✗	✗
[ 25 ]	2022	✓✓	✗	✗	✓
[ 26 ]	2024	✓✓	✗	✓	✗
[ 27 ]	2024	✓✓	✗	✗	✓
Our Survey	2025	✓✓	✓✓	✓✓	✓✓

Table 1: Comparison of existing surveys on model compression and acceleration techniques for ViTs. ✓✓ indicates a comprehensive discussion, while ✓ denotes a limited discussion.

Figure 2: Core components and tools are covered in this survey. This survey focuses on three main parts—model compression techniques (pruning, KD, and quantization), accelerating techniques (using FPGA, GPU, and ASIC), and associated software tools (toolkits, libraries, and inference engines)—to provide a comprehensive understanding of efficient ViT deployment.

Figure 3: Static Pruning vs Dynamic Pruning

Method	Importance-Based	Token Pruning	Structured	Cascade
VTP [ 38 ]	✓	✗	✗	✗
Patch Slimming [ 39 ]	✓	✓	✗	Top-down
WDPruning [ 40 ]	✓	✗	✓	✗
SP-ViT [ 37 ]	✗	✓	✓	✗
UVC [ 41 ]	✓	✗	✓	✗
CP-ViT [ 36 ]	✓	✓	✓	✓

Table 2: Categorization of ViT pruning methods across key perspectives. Many methods span multiple categories.

Figure 4: The overview of importance-based pruning techniques for ViTs. VTP [ 38 ] prunes dimensionality in the multi-head self-attention (MSA) and multi-layer perception (MLP) modules using important scores. WDPruning [ 40 ] applies a binary mask ( $N$ ) to the MSA module (a), followed by pruning of (b) attention heads and (c) linear projection channels. Patch Slimming [ 39 ] removes redundant patches in a top-down, layer-wise manner.

Figure 5: The overview of token pruning based on SP-ViT [ 37 ] .

Algorithm	Method	Top-1 Accuracy (%) on ViT Models
Algorithm	Method	DeiT-T	DeiT-S	DeiT-B	Swin-S	T2T-ViT-14
Baseline	-	72.2	79.8	81.8	83.2	81.5
VTP [ 38 ]	Dimension-based	-	-	80.7 ( $\downarrow$ 1.34%)	-	-
WDPruning [ 40 ]	Width-depth pruning	70.34 ( $\downarrow$ 2.58%)	78.3 ( $\downarrow$ 1.88%)	80.8 ( $\downarrow$ 1.22%)	81.8 ( $\downarrow$ 1.68%)	-
MD [ 51 ]	Dependency based pruning by Gaussian-search	-	79.9 ( $\uparrow$ 0.13%)	82.3 ( $\downarrow$ 0.61%)	-	81.7 ( $\uparrow$ 0.24%)
SPViT [ 37 ]	Pruning based on token score	-	-	-	82.7 ( $\downarrow$ 0.60%)	-

Table 3: Top-1 (%) accuracy comparison of pruning techniques across different ViT models. ’

\downarrow

’ denotes accuracy loss from the baseline ViT model, and ’

\uparrow

’ denotes the accuracy gain after applying the pruning technique. All results are evaluated on the ImageNet-1k [ 45 ] dataset.

Algorithm	GFLOPS ( $\downarrow$ )
Algorithm	DeiT-T	DeiT-S	DeiT-B	Swin-S	T2T-ViT-14
Baseline	1.3	4.6	17.6	8.7	4.8
VTP [ 38 ]	-	-	10.0 ( $\downarrow$ 43.2%)	-	-
WDPruning [ 40 ]	0.7 ( $\downarrow$ 46.2%)	2.6 ( $\downarrow$ 43.5%)	9.9 ( $\downarrow$ 43.8%)	6.3 ( $\downarrow$ 27.6%)	-
MD [ 51 ]	-	2.9 ( $\downarrow$ 37.0%)	11.2 ( $\downarrow$ 36.4%)	-	2.9 ( $\downarrow$ 39.6%)
SPViT [ 37 ]	-	-	-	6.4 ( $\downarrow$ 26.4%)	-

Table 4: GFLOPS reduction comparison of pruning techniques across different ViT models. ’

\downarrow

’ denotes the reduction rate from the baseline ViT model.

Algorithm	Params (M $\downarrow$ )
Algorithm	DeiT-T	DeiT-S	DeiT-B	Swin-S
Baseline	5.6	22	86	50
VTP [ 38 ]	-	-	47.3 ( $\downarrow$ 45.0%)	-
WDPruning [ 40 ]	3.5 ( $\downarrow$ 37.5%)	13.3 ( $\downarrow$ 39.5%)	55.3 ( $\downarrow$ 35.7%)	32.8 ( $\downarrow$ 34.4%)
NViT [ 42 ]	3.5 ( $\downarrow$ 37.5%)	10.5 ( $\downarrow$ 52.3%)	17.0 ( $\downarrow$ 80.3%)	15.0 ( $\downarrow$ 70.0%)
SAViT [ 47 ]	4.2 ( $\downarrow$ 25.0%)	14.7 ( $\downarrow$ 33.2%)	51.9 ( $\downarrow$ 39.6%)	-

Table 5: Parameter (Million) reduction comparison of pruning techniques across different ViT models. ’

\downarrow

’ denotes parameter reduction rate from the baseline ViT model.

Panel of Figure 6 — Figure 6: An illustration depicting the three types of KD: logit-based, relation-based, and feature-based between teacher and student models.

Type	Algorithm	Method	Loss function	Dataset	Teachers		Students
					Models	Top-1 (%)	Models	Top-1 (%)
Relation	TAT [ 17 ]	One-to-all spatial matching KD	Vanilla distillation + L 2 loss	ImageNet-1k [ 45 ]	ResNet34	72.4%	ResNet18	72.1 ( $\uparrow$ 2.0%)
	Co-advise [ 57 ]	Co-advised by lightweight teachers	Kull back divergence + Cross entropy loss	ImageNet-1k [ 45 ]	ResNet18	83.4%	CiT-Ti	88.0 ( $\uparrow$ 1.5%)
Feature	Manifold [ 58 ]	Patch-level manifold space method	Manifold distillation loss	ImageNet-1k [ 45 ]	CaiT-S24	83.4%	DeiT-T	76.5 ( $\uparrow$ 4.3%)
						83.4%	DeiT-S	82.2 ( $\uparrow$ 2.3%)
					Swin-S	83.2%	Swin-T	82.2 ( $\uparrow$ 1.0%)

Table 6: Results of KD techniques applied to ViT models for classification tasks. MSE refers to Mean Squared Error loss. ’

\uparrow

’ in the student model indicates the performance improvement over the baseline student model.

Algorithm	Pretrained Dataset	Dataset	Students
			Without KD		With KD
			Models	AP box	Models	AP box
Manifold [ 58 ]	ImageNet-1k [ 45 ]	COCO-2017	Swin-T	43.7	Swin-T	44.7( $\uparrow$ 1.0)
MiniViT [ 59 ]	-	COCO-2017	Swin-T	48.1	Swin-T	48.6( $\uparrow$ 0.5)

Table 7: Results of KD techniques applied to ViT models for object detection tasks.

Algorithm	Dataset	Metrics	Teachers		Students		Proposed
Algorithm	Dataset	Metrics	Models	Result	Models	Result	Models	Result
Manifold [ 58 ]	ADE20K [ 63 ]	mIoU	Swin-S + UPerNet	47.64	Swin-S + UPerNet	44.51	Swin-T + UPerNet	45.66 ( $\uparrow$ 2.58%)
TAT [ 17 ]	COCO- Stuff10k [ 64 ]	mIoU	ResNet18	33.10	ResNet18	26.33	ResNet18	28.75 ( $\uparrow$ 9.09%)
TAT [ 17 ]	Pascal VOC [ 65 ]	mIoU	ResNet18	78.43	ResNet18	72.07	ResNet18	75.76 ( $\uparrow$ 9.28%)

Table 8: Results of KD techniques applied to ViT models for segmentation tasks.

Aspects of quantization	Techniques	Advantages	Limitation
Quantization schemes	Uniform [ 66 , 67 ]	Simple implementation & fast inference	Inflexible to data distribution
Quantization schemes	Non-uniform [ 68 , 69 ]	High accuracy & Efficient bandwidth	Complex implementation & data-dependent
Quantization approaches	PTQ [ 14 , 70 ]	No retraining required & low overhead	Accuracy drop
Quantization approaches	QAT [ 71 , 72 , 73 ]	Preserves accuracy & robust	Requires retraining & computational intensive
Calibration methods	Static [ 74 , 75 ]	Lightweight	Accuracy drop
Calibration methods	Dynamic [ 76 , 77 ]	Adaptive to input variations	Runtime overhead

Table 9: Categorization of quantization techniques across five key aspects: schemes, approaches, calibration methods, granularity levels, and deployment strategies.

Figure 7: An overview of the quantization framework in PTQ4ViT [ 66 ] , which integrates twin uniform quantization and a Hessian-guided metric for determining scaling factors. Building on this, APQ-ViT [ 88 ] addresses the critical quantization error caused by Hessian-based scaling through a block-wise evaluation.

Figure 8: Comparison of using full precision Softmax and log-int-softmax in quantized MSA inference in FQ-ViT [ 14 ] .

Algorithm	Code	Key point	Backbone	Dataset	Results
					Baseline	W-bit	A-bit	Top-1 accuracy
PTQ [ 70 ]	✗	Introduce a mixed-precision weights strategy	ViT-B	CIFAR-10	Percentile	6 MP	6 MP	96.83 ( $+3.35$ )
						8 MP	8 MP	97.79 ( $+2.03$ )
				CIFAR-100	Percentile	6 MP	6 MP	83.99 ( $+3.15$ )
						8 MP	8 MP	85.76 ( $+2.48$ )
				ImageNet-1k [ 45 ]	Percentile	6 MP	6 MP	75.26 ( $+3.68$ )

Table 10: Results of different post-training quantization(classification) techniques proposed for ViTs. MP denotes mixed precision; W-bit refers to weight bit-widths and A-bit refers to activation bit-widths. Here, Baseline refers to the closest comparable results for classification tasks.

Algorithm	Backbone	Results
Algorithm	Backbone	Baseline	W-bit	A-bit	mAP	AP box
PTQ [ 70 ]	DETR	Easyquant [ 107 ]	6 MP	6 MP	40.5( $+1.5$ )	-
PTQ [ 70 ]	DETR	Easyquant [ 107 ]	8 MP	8 MP	41.7( $+1.3$ )	-
APQ-ViT [ 88 ]	Mask-RCNN+Swin-T	PTQ4ViT [ 66 ]	6	6	-	45.4 ( $+39.6$ )
APQ-ViT [ 88 ]	Mask-RCNN+Swin-T	PTQ4ViT [ 66 ]	4	4	-	23.7 ( $+16.8$ )
FQ-ViT [ 14 ]	Mask-RCNN+Swin-S	OMSE [ 108 ]	8	8	47.8(+5.2)	-

Table 11: Results of different quantization (object detection) techniques proposed for ViTs. The algorithms are experimented on COCO 2017 [ 106 ] datasets for object detection tasks. Here, Baseline refers to the closest comparable results for object detection. MP denotes mixed precision.

Algorithm	Code	Key point	Dataset	Results
				Baseline	Model	W-bit	A-bit	Top-1 (%)	AP box
Q-ViT [ 73 ]	✓	Enables 3-bit ViT QAT.	ImageNet-1k [ 45 ]	LSQ [ 111 ]	DeiT-B	4	4	83.0 ( $+2.1$ )	-
						2	2	74.2 ( $+3.9$ )	-
					Swin-S	4	4	84.4 ( $+1.9$ )	-
						2	2	76.9 ( $+4.5$ )	-
Q-DETR [ 84 ]	✓	KD to enhance quantized DETR’s representation	PASCAL VOC [ 112 ]	LSQ [ 111 ]	TR50	2	2	-	50.7 ( $+8.1$ )

Table 12: Results of different QAT techniques proposed for ViTs. The results of QD-BEV [ 72 ] are the mean average precision (mAP) rather than AP box .

Tool	Type	Key Features				Supported Edge Devices
Tool	Type	MO	MC	HLS	IR	FPGA	GPU	CPU
Xilinx Vivado design suite 1	Toolkit	✗	✓	✓	✗	Xilinx FPGA	No	No
Intel Quartus Prime 2	Toolkit	✗	✓	✓	✗	Intel FPGA	No	No
OpenVINO 4	Toolkit	✓	✗	✗	✓	Intel FPGA	Yes	Yes
Vitis AI 3	Engine	✓	✓	✗	✗	Xilinx FPGA	No	No
FINN 6	Engine	✓	✓	✗	✓	Xilinx FPGA	No	No

Table 13: The overview of popular software tools for deploying deep learning models on different hardware architectures.

Model	Experiment Hardware	Non-linear Operations			Retrain
Model	Experiment Hardware	Softmax	GELU	LayerNorm	Retrain
FQ-ViT [ 14 ]	-	✓	✗	✓	✗
I-ViT [ 93 ]	RTX 2080 Ti GPU	✓	✓	✓	✓
EdgeKernel [ 119 ]	Apple A13 and M1 chips	✓	✓	✓	✗
PackQViT [ 109 ]	Snapdragon 870 SoC (Mobile Phone)	✓	✓	✓	✓
SOLE [ 120 ]	ASIC 28nm	✓	✓	✓	✗

Table 14: Overview of ViT models utilizing integer approximations for non-linear operations to enhance inference efficiency and avoid dequantization.

Approaches	Framework	Retrain	Baseline Models	Hardware device		Key optimization
				Baseline	Experiment	Software	Hardware
SW-HW co-design	VAQF [ 122 ]	✗	DeiT-B/S/T	Intel i7-9800X $\bigstar$	ZCU102 $\clubsuit$	Quantization	Traditional resources
				TITAN RTX $\spadesuit$
	Auto-ViT-Acc [ 123 ]	✗	DeiT-B/S/T	A100 $\spadesuit$	ZCU102 $\clubsuit$	Mixed quantization	Traditional resources
	HeatViT [ 20 ]	✓	DeiT-B/S/T	A100 $\spadesuit$	ZCU102 $\clubsuit$	Adaptive token pruning	Motivated from [ 123 ]
	EQ-ViT [ 124 ]	✓	DeiT-T	ZCU102 $\clubsuit$	VCK190 $\blacklozenge$	Kernel-level profiling	Spatial & heterogeneous accelerators

Table 15: The overview of current accelerating techniques for ViT on edge devices. In the baseline models, B denotes Base; S denotes Small, and T denotes Tiny versions. Five types of hardware devices are used in existing SW-HW co-design frameworks: GPU (

\spadesuit

), EdgeGPU (

\blacktriangle

), CPU (

\bigstar

), FPGA (

\clubsuit

), and AMD Versal Adaptive Compute Acceleration (ACAP) (

\blacklozenge

Framework	Baseline	Effort Modulation	Prediction Mechanism	Accuracy Top1(%)	GPP Compatible
ViTCoD [ 23 ]	DeiT-S	Constant	Norm Score	78.1	✗
HeatViT [ 20 ]	DeiT-S	Constant	Head level	79.1	✗
PIVOT [ 132 ]	DeiT-S	Input-aware	Entropy Metric	79.4	✓
VAQF [ 122 ]	DeiT-S	Hardware-driven	Hardware-aware	79.5	✗

Table 16: Compatibilities comparison of SOTA SW-HW co-design accelerating techniques. Here, GPP denotes general-purpose platforms such as CPU and GPU.

Framework	Device	Precision	Frequency (MHZ)	Resource Utilization				FPS	Power (W)	Energy (FPS/W)	Throughput (GOPs)	Speedup ( $\uparrow$ )
Framework	Device	Precision	Frequency (MHZ)	BRAM	DSP	KLUT	KFF	FPS	Power (W)	Energy (FPS/W)	Throughput (GOPs)	Speedup ( $\uparrow$ )
VAQF [ 122 ]	ZCU102 $\clubsuit$	W1A8	150	565.5	1564	143	110	24.8	8.7	2.85	861.2	-
ViA [ 21 ]	U50 $\clubsuit$	FP16	300	1002	2420	258	257	-	39	7.94 $\lozenge$	309.6	59.5 $\times$
ViTA [ 22 ]	ZC7020 $\clubsuit$	INT8	150	-	-	-	-	8.7	0.88	3.13	-	2 $\times$
Auto-ViT-Acc [ 123 ]	ZCU102 $\clubsuit$	W8A8+W4A8	150	-	1556	186	-	34.0	9.40	3.66	1181.5	-
HeatViT [ 20 ]	ZCU102 $\clubsuit$	INT8	150	528.6	2066	161.4	101.8	11.4	54.8	4.83	-	4.89 $\times$

Table 17: A comparative analysis of hardware performance for various ViT acceleration techniques, deployed on FPGA (

\clubsuit

) and ACAP (

\blacklozenge

) platforms. Performance metrics include Frames Per Second (FPS), Energy Efficiency, and Throughput. The notation

\intercal

represents FPS measured as images per second, while

\lozenge

denotes energy efficiency in GOP/J. Additionally, * indicates power consumption measured in Watt-Seconds (W·S), providing deeper insights into the trade-offs between computation speed and energy usage.

Framework	Baseline	Dataset	Accuracy
Framework	Baseline	Dataset	Top-1(%)	mIoU
VAQF [ 122 ]	DeiT-B	ImageNet-1K [ 45 ]	77.6	-
Auto-ViT-Acc [ 123 ]	DeiT-B	ImageNet-1K [ 45 ]	81.8	-
HeatViT [ 20 ]	DeiT-T	ImageNet-1K [ 45 ]	72.1	-
EQ-ViT [ 124 ]	DeiT-T		74.5	-
ViTCoD [ 23 ]	DeiT-T		70.0

Table 18: Comparison of accuracy across different ViT acceleration techniques using DeiT-B (Base), DeiT-T (Tiny), ViT-B (Base), and ViT-S (Small) as baseline models. Results include Top-1 accuracy on ImageNet-1K and mean Intersection over Union (mIoU) for segmentation benchmarks.

Efficient ViTs

Efficient Vision Transformers reduce the compute, memory, energy, or latency needed to process images and video while retaining useful accuracy.

See all 239 papers

FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers
Jun 30, 2026Hubert Dymarkowski, Xingjian Fu, Rappy Saha +2Hybrid CNN-Transformer Architectures Edge Inference
School of Computing Science, University of Glasgow, Scotland, UK
MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers
May 11, 2026Novendra Setyawan, Chi-Chia Sun, Mao-Hsiu Hsu +2Energy-Efficient ML Edge Inference
Department of Electro-Optics, National Formosa University, Taiwan · Department of Electrical Engineering, University of Muhammadiyah Malang, Indonesia · Department of Electrical Engineering, National Taipei University, Taiwan +1
Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation
May 13, 2026Kieran Carrigg, Sigur de Vries, Amirhossein Sadough +1Efficient Neural Network Inference Efficient ViTs
Department of Machine Learning and Neural Computing Donders Institute for Brain, Cognition, and Behaviour Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands

Jun 30, 2026J/K move · Enter open · S save

FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers

Hubert Dymarkowski, Xingjian Fu, Rappy Saha, Jude Haris+1

School of Computing Science, University of Glasgow, Scotland, UK

Hybrid CNN-Transformer Architectures Edge Inference Efficient ViTs FPGA-Based Neural Network Acceleration

Open paper AskPDF arXiv

Deploying Vision Transformer (ViT) models on edge platforms remains challenging due to their high computational demands and the architectural heterogeneity of modern hybrid ViT models, which incorporate both fully connected and convolutional layers. This heterogeneity leads to significant variation in tensor shapes, requiring flexible and efficient FPGA-based acceleration. In this paper, we present FlexViT, a reconfigurable FPGA accelerator for efficient ViT inference on resource-constrained edge devices. Built on the SECDA-TFLite framework, FlexViT employs a hardware-software co-design approach that maps both fully connected and convolutional layers onto a unified high-throughput INT8 GEMM engine using a runtime im2col transformation. To efficiently support diverse layer configurations, we propose a dual-mode dataflow that dynamically switches between input and weight reuse by reconfiguring the compute array at runtime. We further introduce a depth-first tiling strategy that completes accumulation in a single pass, eliminating off-chip partial-sum transfers and reducing memory bandwidth requirements. We implement FlexViT on a PYNQ-Z2 FPGA and evaluate it across a representative set of ViT models. FlexViT achieves up to 2.74x speedup on accelerator-executed layers, translating into up to 1.40x end-to-end speedup compared to CPU-only execution. The code is available at: https://github.com/gicLAB/FlexViT

Abstract

Efficient ViTs

Explore similar work

FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers

MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers

Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation