ViT Architecture

Vision Transformers (ViTs) are a powerful class of neural networks increasingly used for image analysis, offering performance comparable to or exceeding convolutional neural networks (CNNs). Current research focuses on improving ViT efficiency through techniques like structured pruning to reduce computational cost and power consumption, as well as exploring hybrid CNN-ViT architectures to leverage the strengths of both approaches. These advancements aim to make ViTs more practical for deployment in resource-constrained environments and broaden their applicability across various computer vision tasks, including object detection, segmentation, and 3D vision.

Papers

July 2, 2024

LPViT: Low-Power Semi-structured Pruning for Vision Transformers
Kaixin Xu, Zhe Wang, Chunyun Chen, Xue Geng, Jie Lin, Xulei Yang, Min Wu, Xiaoli Li, Weisi Lin
Vision Transformer Structured Pruning Block Sparsity ViT Architecture Block Wise Pruning

February 5, 2024

Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A survey
Haruna Yunusa, Shiyin Qin, Abdulrahman Hamman Adama Chukkol, Abdulganiyu Abdu Yusuf, Isah Bello, Adamu Lawan
Convolutional Neural Network Timely Survey Vision Transformer Computer Vision Hybrid Convolutional Emerging Synergy ViT Architecture Vision Architecture

November 9, 2023

FMViT: A multiple-frequency mixing Vision Transformer
Wei Tan, Yifeng Geng, Xuansong Xie
Vision Transformer Convolutional Block Hybrid CNN Transformer Multi Band ViT Architecture

October 11, 2023

MatFormer: Nested Transformer for Elastic Inference
Devvrit, Sneha Kudugunta, Aditya Kusupati, Tim Dettmers, Kaifeng Chen, Inderjit Dhillon, Yulia Tsvetkov, Hannaneh Hajishirzi, Sham Kakade, Ali Farhadi, Prateek Jain
ViT Architecture Batch Inference

May 5, 2023

Semantic Segmentation using Vision Transformers: A survey
Hans Thisanke, Chamli Deshan, Kavindu Chamith, Sachith Seneviratne, Rajith Vidanaarachchi, Damayanthi Herath
Convolutional Neural Network Timely Survey Vision Transformer Semantic Segmentation Traditional Convolutional ViT Architecture

December 15, 2022

FlexiViT: One Model for All Patch Sizes
Lucas Beyer, Pavel Izmailov, Alexander Kolesnikov, Mathilde Caron, Simon Kornblith, Xiaohua Zhai, Matthias Minderer, Michael Tschannen, Ibrahim Alabdulmohsin, Filip Pavetic
Vision Transformer Full Model Pre Trained Model Patch Selection ViT Architecture

October 27, 2022

ViT-CAT: Parallel Vision Transformers with Cross Attention Fusion for Popularity Prediction in MEC Networks
Zohreh HajiAkhondi-Meybodi, Arash Mohammadi, Ming Hou, Jamshid Abouei, Konstantinos N. Plataniotis
Pre Trained ViT Popularity Prediction Edge Caching Cross Attention Fusion Parallel Transformer ViT LoS V2X ViT Architecture

September 15, 2022

Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Yi Wang, Zhiwen Fan, Tianlong Chen, Hehe Fan, Zhangyang Wang
Vision Transformer 3D Vision Task ViT Architecture

February 24, 2022

Auto-scaling Vision Transformers without Training
Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wang, Denny Zhou
Vision Transformer ImageNet 1k Auto Scaling Convolutional Counterpart ViT Architecture

December 17, 2021

A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation
Wuyang Chen, Xianzhi Du, Fan Yang, Lucas Beyer, Xiaohua Zhai, Tsung-Yi Lin, Huizhong Chen, Jing Li, Xiaodan Song, Zhangyang Wang, Denny Zhou
Convolutional Neural Network Vision Transformer Instance Segmentation Object Localization ViT Architecture Simple Vision Transformer