Vision Transformer

Vision Transformers (ViTs) adapt the transformer architecture, initially designed for natural language processing, to image analysis by treating images as sequences of patches. Current research focuses on improving ViT efficiency and robustness through techniques like token pruning, attention engineering, and hybrid models combining ViTs with convolutional neural networks or other architectures (e.g., Mamba). These advancements are driving progress in various applications, including medical image analysis, object detection, and spatiotemporal prediction, by offering improved accuracy and efficiency compared to traditional convolutional neural networks in specific tasks.

Papers

June 27, 2023

Towards predicting Pedestrian Evacuation Time and Density from Floorplans using a Vision Transformer
Patrick Berggold, Stavros Nousias, Rohit K. Dubey, André Borrmann
Vision Transformer Data Generation Particle Density Floor Plan Density Map Large Scale Synthetic Dataset Pedestrian Simulation Crowd Evacuation

June 26, 2023

FeSViBS: Federated Split Learning of Vision Transformer with Block Sampling
Faris Almalik, Naif Alkhunaizi, Ibrahim Almakky, Karthik Nandakumar
Vision Transformer Split Learning Medical Image Classification Task Federated Split Learning

June 23, 2023

Swin-Free: Achieving Better Cross-Window Attention and Efficiency with Size-varying Window
Jinkyu Koo, John Yang, Le An, Gwenaelle Cunha Sergio, Su Inn Park
Vision Transformer High Efficiency Human Attention Transformer Model Swin Transformer

June 22, 2023

Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing
Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort
Language Model Vision Transformer Transformer Model Attention Head Quantized Transformer

June 21, 2023

June 19, 2023

June 16, 2023

Group Orthogonalization Regularization For Vision Models Adaptation and Robustness
Yoav Kurtz, Noga Bar, Raja Giryes
Vision Transformer Native Robustness Adversarial Training Regularization Model Convolutional Filter Orthogonality Regularization Performance Guaranteed Regularization

June 15, 2023

June 13, 2023

Reviving Shift Equivariance in Vision Transformers
Peijian Ding, Davit Soselia, Thomas Armstrong, Jiahao Su, Furong Huang
Vision Transformer Transformer Model Shift Equivariance

June 12, 2023

June 11, 2023

June 10, 2023

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer
Haoran You, Huihong Shi, Yipin Guo, Yingyan, Lin
Vision Transformer Mixture Component Model Reduction Efficient Vision Transformer Simple Primitive Transformer Based Vision Shift Invariant Kernel

June 9, 2023

Vision Transformer

Papers

Towards predicting Pedestrian Evacuation Time and Density from Floorplans using a Vision Transformer

FeSViBS: Federated Split Learning of Vision Transformer with Block Sampling

Swin-Free: Achieving Better Cross-Window Attention and Efficiency with Size-varying Window

Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing

Inter-Instance Similarity Modeling for Contrastive Learning

ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining

RaViTT: Random Vision Transformer Tokens

B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers

Vision Transformer with Attention Map Hallucination and FFN Compaction

Group Orthogonalization Regularization For Vision Models Adaptation and Robustness

Seeing the Pose in the Pixels: Learning Pose-Aware Representations in Vision Transformers

ViP: A Differentially Private Foundation Model for Computer Vision

Reviving Shift Equivariance in Vision Transformers

Revisiting Token Pruning for Object Detection and Instance Segmentation

Unmasking Deepfakes: Masked Autoencoding Spatiotemporal Transformers for Enhanced Video Forgery Detection

$E(2)$-Equivariant Vision Transformer

2-D SSM: A General Spatial Layer for Visual Transformers

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer

FLSL: Feature-level Self-supervised Learning

Customizing General-Purpose Foundation Models for Medical Report Generation