Efficient Vision Transformer

Efficient Vision Transformers (ViTs) aim to overcome the computational limitations of standard ViTs while maintaining their strong performance in computer vision tasks. Current research focuses on developing novel attention mechanisms (e.g., polynomial attention), token reduction strategies (e.g., learnable token merging, dynamic token idling), and adaptive computation techniques to reduce the number of tokens processed based on image complexity. These advancements are significant because they enable the deployment of ViTs in resource-constrained environments like mobile devices and embedded systems, broadening their applicability in various fields.

Papers

October 19, 2024

EViT-Unet: U-Net Like Efficient Vision Transformer for Medical Image Segmentation on Mobile and Edge Devices
Xin Li, Wenhui Zhu, Xuanzhao Dong, Oana M. Dumitrascu, Yalin Wang
Vision Transformer Transformer Based Medical Image Segmentation Based Approach CNN Model U Net U Shaped Network Convolutional Operation Efficient Vision Transformer Encrypted Vision Transformer

August 30, 2024

Vote&Mix: Plug-and-Play Token Reduction for Efficient Vision Transformer
Shuai Peng, Di Fu, Baole Wei, Yong Cao, Liangcai Gao, Zhi Tang
Vision Transformer Efficient Vision Transformer Token Reduction

July 21, 2024

Efficient Visual Transformer by Learnable Token Merging
Yancheng Wang, Yingzhen Yang
Vision Transformer Token Merging Efficient Vision Transformer Various Vision Transformer

July 16, 2024

PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer
Pierre-David Letourneau, Manish Kumar Singh, Hsin-Pai Cheng, Shizhong Han, Yunxiao Shi, Dalton Jones, Matthew Harper Langston, Hong Cai, Fatih Porikli
Self Attention Attention Mechanism Efficient Vision Transformer

May 16, 2024

LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation
Wentao Jiang, Jing Zhang, Di Wang, Qiming Zhang, Zengmao Wang, Bo Du
Remote Sensing Image Visual Token Image Token Efficient Vision Transformer Learnable Token Dual Cross Attention Sparse Token

January 3, 2024

TPC-ViT: Token Propagation Controller for Efficient Vision Transformer
Wentao Zhu
Vision Transformer Computer Vision Task Efficient Vision Transformer Token Reduction Token Propagation

November 20, 2023

Tiny-VBF: Resource-Efficient Vision Transformer based Lightweight Beamformer for Ultrasound Single-Angle Plane Wave Imaging
Abdul Rahoof, Vivek Chaturvedi, Mahesh Raveendranatha Panicker, Muhammad Shafique
Deep Learning Architecture Adaptive Beamforming Efficient Vision Transformer Plane Wave Imaging

November 2, 2023

Efficient Vision Transformer for Accurate Traffic Sign Detection
Javad Mirzapour Kaleybar, Hooman Khaloo, Avaz Naghipour
Vision Transformer Traffic Sign Recognition Efficient Vision Transformer Traffic Sign Detection Vision Transformer Variant

October 9, 2023

No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
Xuwei Xu, Changlin Li, Yudong Chen, Xiaojun Chang, Jiajun Liu, Sen Wang
Vision Transformer Supervised ImageNet Computer Vision Task K TOKEN Image Token Efficient Vision Transformer Encrypted Vision Transformer

September 21, 2023

DualToken-ViT: Position-aware Efficient Vision Transformer with Dual Token Fusion
Zhenzhen Chu, Jiayu Chen, Cen Chen, Chengyu Wang, Ziheng Wu, Jun Huang, Weining Qian
Vision Transformer Self Attention Token Fusion Efficient Vision Transformer

June 10, 2023

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer
Haoran You, Huihong Shi, Yipin Guo, Yingyan, Lin
Vision Transformer Mixture Component Model Reduction Efficient Vision Transformer Simple Primitive Transformer Based Vision Shift Invariant Kernel

June 7, 2023

Efficient Vision Transformer for Human Pose Estimation via Patch Selection
Kaleab A. Kinfu, Rene Vidal
Convolutional Neural Network Vision Transformer High Resolution Image Human Pose Estimation 2 Dimensional Human Pose Estimation Efficient Vision Transformer Patch Selection

May 24, 2023

Predicting Token Impact Towards Efficient Vision Transformer
Hong Wang, Su Yang, Xiaoke Huang, Weishan Zhang
Self Attention Module Efficient Vision Transformer Token Importance

May 17, 2023

CageViT: Convolutional Activation Guided Efficient Vision Transformer
Hao Zheng, Jinbao Wang, Xiantong Zhen, Hong Chen, Jingkuan Song, Feng Zheng
Deep Convolutional Neural Network Efficient Vision Transformer Net CAGE Gated Linear Deep Convolutional Activation Feature

February 22, 2023

A residual dense vision transformer for medical image super-resolution with segmentation-based perceptual loss fine-tuning
Jin Zhu, Guang Yang, Pietro Lio
Vision Transformer Super Resolution Image Super Resolution Perceptual Loss Efficient Vision Transformer Medical Image Super Resolution Dense Residual Connected Transformer

November 14, 2022

Fcaformer: Forward Cross Attention in Hybrid Vision Transformer
Haokui Zhang, Wenze Hu, Xiaoyu Wang
Vision Transformer Cross Attention Sparse Attention Transformer Based Architecture Efficient Vision Transformer

April 18, 2022

Temporally Efficient Vision Transformer for Video Instance Segmentation
Shusheng Yang, Xinggang Wang, Yu Li, Yuxin Fang, Jiemin Fang, Wenyu Liu, Xun Zhao, Ying Shan
Vision Transformer Video Instance Segmentation Efficient Vision Transformer

December 14, 2021

AdaViT: Adaptive Tokens for Efficient Vision Transformer
Hongxu Yin, Arash Vahdat, Jose Alvarez, Arun Mallya, Jan Kautz, Pavlo Molchanov
Vision Transformer Image Classification Inference Cost Adaptive Token Efficient Vision Transformer