Vision Transformer Network

Vision Transformer (ViT) networks leverage the attention mechanism to process image data as a sequence of patches, enabling powerful feature extraction and global context understanding. Current research focuses on improving ViT efficiency through architectural innovations like hierarchical designs and adaptive token processing, as well as exploring effective training strategies such as self-supervised learning and novel initialization methods. These advancements are driving improvements in various applications, including image classification, object detection, semantic segmentation, and medical image analysis, often surpassing the performance of convolutional neural networks in specific tasks. The resulting models are finding use in diverse fields like autonomous driving, drone technology, and healthcare.

Papers

September 18, 2024

NT-ViT: Neural Transcoding Vision Transformers for EEG-to-fMRI Synthesis
Romeo Lanzino, Federico Fontana, Luigi Cinque, Francesco Scarcello, Atsuto Maki
Vision Transformer Neural Topic 3D Brain ViT Lens Vision Transformer Network EEG Representation

August 22, 2024

AT-SNN: Adaptive Tokens for Vision Transformer on Spiking Neural Network
Donghwa Kang, Youngmoon Lee, Eun-Kyu Lee, Brent Kang, Jinkyu Lee, Hyeongboo Baek
Convolutional Neural Network Vision Transformer Spiking Neural Network Traditional RNNs Deep Transformer Adaptive Token Vision Transformer Network

May 29, 2024

MDS-ViTNet: Improving saliency prediction for Eye-Tracking with Vision Transformer
Polezhaev Ignat, Goncharenko Igor, Iurina Natalya
Vision Transformer Eye Tracking Saliency Prediction Vision Transformer Network Accurate Saliency

April 30, 2024

C2FDrone: Coarse-to-Fine Drone-to-Drone Detection using Vision Transformer Networks
Sairam VC Rebbapragada, Pranoy Panda, Vineeth N Balasubramanian
Vision Transformer Drone Detection Drone Dataset Vision Transformer Network Coarse Detection

April 27, 2024

CLFT: Camera-LiDAR Fusion Transformer for Semantic Segmentation in Autonomous Driving
Junyi Gu, Mauro Bellone, Tomáš Pivoňka, Raivo Sell
Vision Transformer Semantic Segmentation Autonomous Driving Camera LiDAR Fusion Vision Transformer Network LiDAR Fusion Transformer Fusion

March 28, 2024

January 23, 2024

Convolutional Initialization for Data-Efficient Vision Transformers
Jianqiao Zheng, Xueqian Li, Simon Lucey
Convolutional Neural Network Transformer Network Vision Transformer Network Initialization Bias

November 19, 2023

Shape-Sensitive Loss for Catheter and Guidewire Segmentation
Chayun Kongtongvattana, Baoru Huang, Jingxuan Kang, Hoan Nguyen, Olajide Olufemi, Anh Nguyen
Vision Transformer X Ray Image Vision Transformer Network Central Venous Catheter Guidewire Segmentation Shape Aware Loss Function

November 2, 2023

Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition
Hamid Ahmadabadi, Omid Nejati Manzari, Ahmad Ayatollahi
Vision Transformer Deep Convolutional Neural Network Action Recognition Human Action Recognition CNN Transformer Vision Transformer Network Distilling Knowledge

August 22, 2023

TurboViT: Generating Fast Vision Transformers via Generative Architecture Search
Alexander Wong, Saad Abbasi, Saeejith Nair
Neural Architecture Search Hierarchical Vision Transformer Vision Transformer Architecture Vision Transformer Network Fast Vision Transformer

July 18, 2023

MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments
Spyros Gidaris, Andrei Bursuc, Oriane Simeoni, Antonin Vobecky, Nikos Komodakis, Matthieu Cord, Patrick Pérez
Self Supervised Learning Self Supervised Representation Learning Image Modeling Codebook Learning Self Supervised Representation Vision Transformer Network

April 28, 2023

DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation
Yousef Yeganeh, Azade Farshad, Peter Weinberger, Seyed-Ahmad Ahmadi, Ehsan Adeli, Nassir Navab
Medical Image Segmentation Convolutional Layer Transformer Based Architecture Hybrid Transformer Vision Transformer Network Complex Transformer

March 23, 2023

Scaled Quantization for the Vision Transformer
Yangyang Chang, Gerald E. Sobelman
Vision Transformer Quantization Operator Quantization Technique Vision Transformer Network Integer Quantization Quantization Scale

March 17, 2023

ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
Vision Transformer Neural Architecture Search Vision Transformer Network Fast Vision Transformer Super Network Training Elastica Model

February 18, 2023

VITAL: Vision Transformer Neural Networks for Accurate Smartphone Heterogeneity Resilient Indoor Localization
Danish Gufran, Saideep Tiku, Sudeep Pasricha
Vision Transformer Localization Accuracy Indoor Localization Vision Transformer Network Localization Network Fingerprinting Localization Localization Algorithm

November 23, 2022

An ensemble of VisNet, Transformer-M, and pretraining models for molecular property prediction in OGB Large-Scale Challenge @ NeurIPS 2022
Yusong Wang, Shaoning Li, Zun Wang, Xinheng He, Bin Shao, Tie-Yan Liu, Tong Wang
Large Scale Molecular Graph Molecular Property Prediction 3D Pre Training Vision Transformer Network Transformer XL Graph Regression Open Graph Benchmark Quantum Chemical Property

July 18, 2022

TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers
Jihao Liu, Boxiao Liu, Hang Zhou, Hongsheng Li, Yu Liu
Vision Transformer Data Augmentation Token Level Vision Transformer Network Token Mixer Image Blending Various Vision Transformer

March 24, 2022

Keypoints Tracking via Transformer Networks
Oleksii Nasypanyi, Francois Rameau
Transformer Network Keypoint Detector Supervised Keypoint Vision Transformer Network Sparse Keypoints Keypoints Representation Keypoint Tracking

December 27, 2021

Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction
Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li
Energy Based Saliency Prediction Vision Transformer Network Generative Vision Transformer

Vision Transformer Network

Papers

NT-ViT: Neural Transcoding Vision Transformers for EEG-to-fMRI Synthesis

AT-SNN: Adaptive Tokens for Vision Transformer on Spiking Neural Network

MDS-ViTNet: Improving saliency prediction for Eye-Tracking with Vision Transformer

C2FDrone: Coarse-to-Fine Drone-to-Drone Detection using Vision Transformer Networks

CLFT: Camera-LiDAR Fusion Transformer for Semantic Segmentation in Autonomous Driving

Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights

Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection

Convolutional Initialization for Data-Efficient Vision Transformers

Shape-Sensitive Loss for Catheter and Guidewire Segmentation

Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition

TurboViT: Generating Fast Vision Transformers via Generative Architecture Search

MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments

DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation

Scaled Quantization for the Vision Transformer

ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices

VITAL: Vision Transformer Neural Networks for Accurate Smartphone Heterogeneity Resilient Indoor Localization

An ensemble of VisNet, Transformer-M, and pretraining models for molecular property prediction in OGB Large-Scale Challenge @ NeurIPS 2022

TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers

Keypoints Tracking via Transformer Networks

Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction