Vision Transformer Variant

Vision Transformer (ViT) variants are being actively developed to improve the efficiency and accuracy of various computer vision tasks. Current research focuses on creating more efficient architectures, such as those incorporating local attention mechanisms and parameter-sharing strategies to reduce computational costs while maintaining performance, particularly for large-scale datasets and 3D medical imaging. These advancements are impacting diverse fields, including autonomous driving, medical image analysis, and general object detection, by enabling faster and more accurate processing of visual data. The development of efficient ViT variants is crucial for deploying these powerful models in resource-constrained environments and real-world applications.

Papers

November 2, 2023

Efficient Vision Transformer for Accurate Traffic Sign Detection
Javad Mirzapour Kaleybar, Hooman Khaloo, Avaz Naghipour
Vision Transformer Traffic Sign Recognition Efficient Vision Transformer Traffic Sign Detection Vision Transformer Variant

October 10, 2023

Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing
Wei Dong, Dawei Yan, Zhijun Lin, Peng Wang
Vision Transformer Pre Trained Model Efficient Adaptation Efficient Model Adaptation Large Pre Trained Parameter Efficient Adapter Conditional Adapter Vision Transformer Variant

June 14, 2023

When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants
Anuj Diwan, Eunsol Choi, David Harwath
Self Attention Attention Mechanism Speech Analysis Global Attention Self Supervised Speech Model Vision Transformer Variant

March 14, 2023

Efficiently Training Vision Transformers on Structural MRI Scans for Alzheimer's Disease Detection
Nikhil J. Dhinagar, Sophia I. Thomopoulos, Emily Laltoo, Paul M. Thompson
Vision Transformer Transformer Model Alzheimer'S Disease Deep Learning Architecture Disease Detection Structural MRI Brain Image Vision Transformer Variant

March 15, 2022

Smoothing Matters: Momentum Transformer for Domain Adaptive Semantic Segmentation
Runfa Chen, Yu Rong, Shangmin Guo, Jiaqi Han, Fuchun Sun, Tingyang Xu, Wenbing Huang
Semantic Segmentation Computer Vision Pseudo Labeling Target Domain Domain Adaptive Semantic Segmentation Momentum Transformer Vision Transformer Variant

January 3, 2022

D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation
Yixuan Wu, Kuanlun Liao, Jintai Chen, Jinhong Wang, Danny Z. Chen, Honghao Gao, Jian Wu
Transformer Based Medical Image Segmentation 3D Medical Image Segmentation Dilated Convolution X Former Vision Transformer Variant