Vision Transformer

Vision Transformers (ViTs) adapt the transformer architecture, initially designed for natural language processing, to image analysis by treating images as sequences of patches. Current research focuses on improving ViT efficiency and robustness through techniques like token pruning, attention engineering, and hybrid models combining ViTs with convolutional neural networks or other architectures (e.g., Mamba). These advancements are driving progress in various applications, including medical image analysis, object detection, and spatiotemporal prediction, by offering improved accuracy and efficiency compared to traditional convolutional neural networks in specific tasks.

Papers

January 30, 2024

ViTree: Single-path Neural Tree for Step-wise Interpretable Fine-grained Visual Categorization
Danning Lao, Qi Liu, Jiazi Bu, Junchi Yan, Wei Shen
Vision Transformer Inherent Interpretability Model Interpretability Fine Grained Visual Neural Tree

January 29, 2024

Cutup and Detect: Human Fall Detection on Cutup Untrimmed Videos Using a Large Foundational Video Understanding Model
Till Grutschus, Ola Karrar, Emir Esenov, Ekta Vats
Vision Transformer Video Understanding Temporal Action Localization Fall Detection Action Classification Cut and Approximate Untrimmed Video

January 28, 2024

Intriguing Equivalence Structures of the Embedding Space of Vision Transformers
Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu
Vision Transformer Adversarial Perturbation Meaningful Representation Deep Space Representation Space Model Equivalence

January 27, 2024

SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
Foozhan Ataiefard, Walid Ahmed, Habib Hajimolahoseini, Saina Asani, Farnoosh Javadi, Mohammad Hassanpour, Omar Mohamed Awad, Austin Wen, Kangling Liu, Yang Liu
Vision Transformer Transformer Model CNN Model Multi Head Self Attention ViT Model Skip Connection Skip and Recover

January 26, 2024

MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
Yu-Shan Tai, An-Yeu, Wu
Vision Transformer Quantization Operator Low Bit Quantization GPUSQ ViT

January 23, 2024

January 22, 2024

OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning
Chu Myaet Thwal, Minh N. H. Nguyen, Ye Lin Tun, Seong Tae Kim, My T. Thai, Choong Seon Hong
Vision Transformer Lightweight Vision Transformer Lightweight CNN Transformer Device Vision Task

January 20, 2024

LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation
Navin Ranjan, Andreas Savakis
Vision Transformer Mixed Precision Quantization Layer Wise Relevance Propagation Vision Transformer Quantization Bit Width Quantization

January 19, 2024

PhotoBot: Reference-Guided Interactive Photography via Natural Language
Oliver Limoyo, Jimmy Li, Dmitriy Rivkin, Jonathan Kelly, Gregory Dudek
Vision Transformer Natural Language Visual Language Model Camera System Image Acquisition

January 17, 2024

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, Xinggang Wang
Vision Transformer Representation Learning Visual Representation Visual Representation Learning Vision Foundation Model Visual Understanding Vision Mamba Bidirectional State Space Model

January 16, 2024

January 13, 2024

Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIs
Zhaonian Zhang, Richard Jiang
Vision Transformer Brain Age ViT Lens 3D Awareness MRI Scan Brain Aging

January 12, 2024

January 11, 2024

Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection
Wei Ye, Chaoya Jiang, Haiyang Xu, Chenhao Ye, Chenliang Li, Ming Yan, Shikun Zhang, Songhang Huang, Fei Huang
Vision Transformer Vision Language Attention Network

Vision Transformer

Papers

ViTree: Single-path Neural Tree for Step-wise Interpretable Fine-grained Visual Categorization

Cutup and Detect: Human Fall Detection on Cutup Untrimmed Videos Using a Large Foundational Video Understanding Model

Intriguing Equivalence Structures of the Embedding Space of Vision Transformers

SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection

MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer

On the Efficacy of Text-Based Input Modalities for Action Anticipation

EL-VIT: Probing Vision Transformer with Interactive Visualization

OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning

LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation

PhotoBot: Reference-Guided Interactive Photography via Natural Language

Reconstructing the Invisible: Video Frame Restoration through Siamese Masked Conditional Variational Autoencoder

Supervised Fine-tuning in turn Improves Visual Foundation Models

A Comparative Analysis on Metaheuristic Algorithms Based Vision Transformer Model for Early Detection of Alzheimer's Disease

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

B-Cos Aligned Transformers Learn Human-Interpretable Features

Statistical Test for Attention Map in Vision Transformer

Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIs

UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer

Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention

Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection