Vision Transformer Architecture

Vision Transformers (ViTs) adapt the Transformer architecture, initially successful in natural language processing, to image analysis, aiming to improve upon convolutional neural networks (CNNs) by capturing long-range dependencies within images. Current research focuses on enhancing ViT efficiency through techniques like adaptive token sampling, sparse regularization, and hybrid CNN-Transformer designs, as well as exploring their application in diverse fields such as medical image registration, weather forecasting, and object recognition in challenging scenarios (e.g., UAV imagery). The resulting models demonstrate strong performance across various computer vision tasks, offering a powerful alternative to traditional CNNs and impacting fields requiring high-accuracy image analysis.

Papers

October 24, 2024

PESFormer: Boosting Macro- and Micro-expression Spotting with Direct Timestamp Encoding
Wang-Wang Yu, Kai-Fu Yang, Xiangrui Hu, Jingwen Jiang, Hong-Mei Yan, Yong-Jie Li
Untrimmed Video Vision Transformer Architecture Global Scale Temporal Expression

September 14, 2024

Investigation of Hierarchical Spectral Vision Transformer Architecture for Classification of Hyperspectral Imagery
Wei Liu, Saurabh Prasad, Melba Crawford
Vision Transformer Classification Code Hyperspectral Image Comprehensive Investigation Vision Transformer Architecture Spectral Transformer

August 10, 2024

Transformer-based Neuro-Animator for Qualitative Simulation of Soft Body Movement
Somnuk Phon-Amnuaisuk
Real World Transformer Model Human Cognition Vision Transformer Architecture Neural Avatar Soft Body Simulation Based Approach

April 17, 2024

JointViT: Modeling Oxygen Saturation Levels with Joint Supervision on Long-Tailed OCTA
Zeyu Zhang, Xuyin Qi, Mingxi Chen, Guangxi Li, Ryan Pham, Ayub Qassim, Ella Berry, Zhibin Liao, Owen Siggs, Robert Mclaughlin, Jamie Craig, Minh-Son To
Loss Function Optical Coherence Tomography Vision Transformer Architecture Co Supervision

March 20, 2024

Retina Vision Transformer (RetinaViT): Introducing Scaled Patches into Vision Transformers
Yuyang Shu, Michael E. Bain
Vision Transformer Vision Transformer Architecture Spatial Frequency Scene Composition Multi Scale Patch

February 1, 2024

Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics
Eyup B. Unlu, Marçal Comajoan Cara, Gopal Ramesh Dahale, Zhongtian Dong, Roy T. Forestano, Sergei Gleyzer, Daniel Justice, Kyoungchul Kong, Tom Magorsch, Konstantin T. Matchev, Katia Matcheva
Vision Transformer High Energy Physic Event Classification Vision Transformer Architecture

November 24, 2023

GeoViT: A Versatile Vision Transformer Architecture for Geospatial Image Analysis
Madhav Khirwar, Ankur Narang
Vision Transformer Satellite Imagery Vision Transformer Architecture Satellite Data Greenhouse Gas Emission

November 5, 2023

Rotation Invariant Transformer for Recognizing Object in UAVs
Shuoyi Chen, Mang Ye, Bo Du
Arbitrary Object Various Fast Moving Drone Vision Transformer Architecture Rotation Augmentation

August 22, 2023

TurboViT: Generating Fast Vision Transformers via Generative Architecture Search
Alexander Wong, Saad Abbasi, Saeejith Nair
Neural Architecture Search Hierarchical Vision Transformer Vision Transformer Architecture Vision Transformer Network Fast Vision Transformer

July 22, 2023

Sparse then Prune: Toward Efficient Vision Transformers
Yogi Prasetyo, Novanto Yudistira, Agus Wahyu Widodo
Vision Transformer Many Sparse Edge Pruning Model Pruning Vision Transformer Architecture Vision Transformer Model Sparse Regularization

May 30, 2023

Vision Transformers for Mobile Applications: A Short Survey
Nahid Alam, Steven Kolawole, Simardeep Sethi, Nishant Bansali, Karina Nguyen
Vision Transformer Vision Task Mobile Application Pre Trained ViT Vision Transformer Architecture

May 26, 2023

Do We Really Need a Large Number of Visual Prompts?
Youngeun Kim, Yuhang Li, Abhishek Moitra, Ruokai Yin, Priyadarshini Panda
Visual Prompt Parameter Efficient Transfer Learning Visual Prompt Tuning Vision Transformer Architecture Large Number

May 17, 2023

A survey of the Vision Transformers and its CNN-Transformer based Variants
Asifullah Khan, Zunaira Rauf, Anabia Sohail, Abdul Rehman, Hifsa Asif, Aqsa Asif, Umair Farooq
Timely Survey Vision Transformer Transformer Architecture Natural Language Oriented Variant CNN Transformer Vision Transformer Architecture

May 12, 2023

ViT Unified: Joint Fingerprint Recognition and Presentation Attack Detection
Steven A. Grosz, Kanishka P. Wijewardena, Anil K. Jain
Anti Spoofing Presentation Attack Detection Presentation Attack ViT Model Fingerprint Recognition Vision Transformer Architecture

April 14, 2023

Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentation
Alexander Koenig, Maximilian Schambach, Johannes Otterbach
Vision Transformer Self Distillation Unsupervised Semantic Segmentation Vision Transformer Architecture Stego Image

November 17, 2022

Detecting Arbitrary Keypoints on Limbs and Skis with Sparse Partly Correct Segmentation Masks
Katja Ludwig, Daniel Kienzle, Julian Lorenz, Rainer Lienhart
Many Sparse Segmentation Mask Semantic Mask Vision Transformer Architecture Arbitrary Keypoints Lower Limb

October 27, 2022

Masked Vision-Language Transformer in Fashion
Ge-Peng Ji, Mingcheng Zhuge, Dehong Gao, Deng-Ping Fan, Christos Sakaridis, Luc Van Gool
Vision Language Alignment Vision Transformer Architecture Multi Modal Input Fashion Domain Vision Language Transformer Fashion Knowledge

September 22, 2022

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning
Manuel Goulão, Arlindo L. Oliveira
Convolutional Neural Network Vision Transformer Deep Reinforcement Learning Vision Paper Learned Representation Self Supervised Pretraining Self Supervised Method Vision Transformer Architecture

September 12, 2022

Vision Transformer with Convolutional Encoder-Decoder for Hand Gesture Recognition using 24 GHz Doppler Radar
Kavinda Kehelella, Gayangana Leelarathne, Dhanuka Marasinghe, Nisal Kariyawasam, Viduneth Ariyarathna, Arjuna Madanayake, Ranga Rodrigo, Chamira U. S. Edussooriya
Vision Transformer Encoder Decoder Hand Gesture Recognition Convolutional Encoder Vision Transformer Architecture Millimeter Wave Radar Micro Doppler

August 2, 2022

A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting
Alabi Bojesomo, Hasan Al Marzouqi, Panos Liatsis
Swin Transformer Video Transformer Spatiotemporal Forecasting Vision Transformer Architecture Meteorological Satellite Weather4cast Challenge

Vision Transformer Architecture

Papers

PESFormer: Boosting Macro- and Micro-expression Spotting with Direct Timestamp Encoding

Investigation of Hierarchical Spectral Vision Transformer Architecture for Classification of Hyperspectral Imagery

Transformer-based Neuro-Animator for Qualitative Simulation of Soft Body Movement

JointViT: Modeling Oxygen Saturation Levels with Joint Supervision on Long-Tailed OCTA

Retina Vision Transformer (RetinaViT): Introducing Scaled Patches into Vision Transformers

Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics

GeoViT: A Versatile Vision Transformer Architecture for Geospatial Image Analysis

Rotation Invariant Transformer for Recognizing Object in UAVs

TurboViT: Generating Fast Vision Transformers via Generative Architecture Search

Sparse then Prune: Toward Efficient Vision Transformers

Vision Transformers for Mobile Applications: A Short Survey

Do We Really Need a Large Number of Visual Prompts?

A survey of the Vision Transformers and its CNN-Transformer based Variants

ViT Unified: Joint Fingerprint Recognition and Presentation Attack Detection

Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentation

Detecting Arbitrary Keypoints on Limbs and Skis with Sparse Partly Correct Segmentation Masks

Masked Vision-Language Transformer in Fashion

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

Vision Transformer with Convolutional Encoder-Decoder for Hand Gesture Recognition using 24 GHz Doppler Radar

A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting