Fast Vision Transformer

Fast Vision Transformers (FViTs) aim to overcome the computational limitations of standard Vision Transformers (ViTs) while maintaining high accuracy for computer vision tasks. Research focuses on developing efficient architectures through techniques like hierarchical attention, optimized attention mechanisms (e.g., incorporating Gabor filters or cascaded group attention), and generative architecture search, leading to models such as FasterViT, TurboViT, and EfficientViT. These advancements enable faster inference speeds and lower memory consumption, making ViTs more suitable for real-time applications in robotics, mobile devices, and other resource-constrained environments.

Papers

February 17, 2024

FViT: A Focal Vision Transformer with Gabor Filter
Yulong Shi, Mingwei Sun, Yongshuai Wang, Rui Wang, Hui Sun, Zengqiang Chen
Vision Transformer Gabor Filter Focal Transformer Fast Vision Transformer

November 23, 2023

FViT-Grasp: Grasping Objects With Using Fast Vision Transformers
Arda Sarp Yenicesu, Berk Cicek, Ozgur S. Oguz
Arbitrary Object Vision Based Robotic Grasping Fast Vision Transformer

August 22, 2023

TurboViT: Generating Fast Vision Transformers via Generative Architecture Search
Alexander Wong, Saad Abbasi, Saeejith Nair
Neural Architecture Search Hierarchical Vision Transformer Vision Transformer Architecture Vision Transformer Network Fast Vision Transformer

June 9, 2023

FasterViT: Fast Vision Transformers with Hierarchical Attention
Ali Hatamizadeh, Greg Heinrich, Hongxu Yin, Andrew Tao, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov
Self Attention Hierarchical Attention Fast Vision Transformer EfficientViT SAM

May 11, 2023

EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, Yixuan Yuan
Vision Transformer Memory Efficient Fast Vision Transformer EfficientViT SAM

March 17, 2023

ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
Vision Transformer Neural Architecture Search Vision Transformer Network Fast Vision Transformer Super Network Training Elastica Model

May 26, 2022

Fast Vision Transformers with HiLo Attention
Zizheng Pan, Jianfei Cai, Bohan Zhuang
Vision Transformer Self Attention Vision Task Attention Layer Multi Head Self Attention Attention Head Fast Vision Transformer

December 27, 2021

SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Zhenglun Kong, Peiyan Dong, Xiaolong Ma, Xin Meng, Mengshu Sun, Wei Niu, Xuan Shen, Geng Yuan, Bin Ren, Minghai Qin, Hao Tang, Yanzhi Wang
Vision Transformer DNN Architecture Token Pruning Convolutional Structure Fast Vision Transformer Vision Transformer Pruning Soft Filter Pruning