Pre Trained Vision Transformer

Pre-trained Vision Transformers (ViTs) leverage the power of large-scale pre-training to efficiently adapt to diverse downstream computer vision tasks, focusing on parameter and computational efficiency. Current research emphasizes methods like low-rank adaptation, visual prompt tuning, and multiple-exit strategies to minimize the number of trainable parameters and optimize inference speed, particularly for resource-constrained environments like onboard satellite processing and mobile devices. These advancements are significantly impacting various fields, including remote sensing, medical image analysis, and robotics, by enabling high-accuracy visual recognition with reduced computational demands.

Papers

May 17, 2024

Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformers
Sheng Yang, Jiawang Bai, Kuofeng Gao, Yong Yang, Yiming Li, Shu-tao Xia
Vision Transformer Backdoor Attack Complex Prompt Vision Task Pre Trained Vision Transformer Backdoor Effect

May 6, 2024

April 14, 2024

Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers
Diana-Nicoleta Grigore, Mariana-Iuliana Georgescu, Jon Alvarez Justo, Tor Johansen, Andreea Iuliana Ionescu, Radu Tudor Ionescu
Vision Transformer Knowledge Distillation Adaptation Concern Mutual Distillation Balancing Weight Pre Trained Vision Transformer Self Supervised Transformer

April 13, 2024

Rethinking Low-Rank Adaptation in Vision: Exploring Head-Level Responsiveness across Diverse Tasks
Yibo Zhong, Yao Zhou
Vision Transformer Pre Trained Vision Transformer Hierarchical Transformer Parameter Efficient Transfer Learning Importance Score

April 9, 2024

Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers
Dipam Goswami, Bartłomiej Twardowski, Joost van de Weijer
Class Incremental Learning Pre Trained Vision Transformer Shot Class Shot Continual Learning Shot Data Order Statistic

April 8, 2024

iVPT: Improving Task-relevant Information Sharing in Visual Prompt Tuning by Cross-layer Dynamic Connection
Nan Zhou, Jiaxin Chen, Di Huang
Task Specific Pre Trained Vision Transformer Visual Prompt Tuning Cross Layer Information Sharing

March 28, 2024

Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach
Wei Dong, Xing Zhang, Bihui Chen, Dawei Yan, Zhijun Lin, Qingsen Yan, Peng Wang, Yang Yang
Vision Transformer Pre Trained Model Pre Trained Parameter Efficient Fine Tuning Pre Trained Vision Transformer

February 27, 2024

LSPT: Long-term Spatial Prompt Tuning for Visual Representation Learning
Shentong Mo, Yansen Wang, Xufang Luo, Dongsheng Li
Visual Representation Learning Pre Trained Vision Transformer Self Supervised Vision Transformer Visual Prompt Tuning Spatial Prompt

February 8, 2024

Memory Consolidation Enables Long-Context Video Understanding
Ivana Balažević, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, Olivier J. Hénaff
Pre Trained Vision Transformer Memory Efficient Long Video Transformer Encoders Long Form Video Understanding Memory Consolidation

February 4, 2024

January 26, 2024

PL-FSCIL: Harnessing the Power of Prompts for Few-Shot Class-Incremental Learning
Songsong Tian, Lusi Li, Weijun Li, Hang Ran, Li Li, Xin Ning
Real Power Prompt Learning Class Incremental Learning Complex Prompt Attention Layer Visual Prompt Pre Trained Vision Transformer Shot Class Incremental Learning

January 14, 2024

MIMIC: Mask Image Pre-training with Mix Contrastive Fine-tuning for Facial Expression Recognition
Fan Zhang, Xiaobao Guo, Xiaojiang Peng, Alex Kot
Facial Expression Recognition Expression Recognition Pre Trained Vision Transformer Mask Guided Based Mix Face Image Datasets Contrastive Fine Tuning

January 3, 2024

FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers
Zheng Yuan, Jie Zhang, Shiguang Shan
Vision Transformer Native Robustness Adversarial Robustness Model Robustness Pre Trained Vision Transformer Layer Normalization Adversarial Fine Tuning

December 3, 2023

Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERT
Saurav Sengupta, Donald E. Brown
Histopathology Image Pre Trained Vision Transformer Bidirectional Encoder Representation From Transformer Caption Generation Report Generation Bidirectional Encoder Representation Tissue Classification

November 15, 2023

Generalizable Imitation Learning Through Pre-Trained Representations
Wei-Di Chang, Francois Hogan, David Meger, Gregory Dudek
Strong Generalization Imitation Learning Pre Trained Vision Transformer Pre Trained Representation Patch Embeddings

November 10, 2023

Automatic Report Generation for Histopathology images using pre-trained Vision Transformers
Saurav Sengupta, Donald E. Brown
Deep Learning Histopathology Image Pre Trained Vision Transformer Report Generation Hierarchical Vision Transformer

October 31, 2023

Medi-CAT: Contrastive Adversarial Training for Medical Image Classification
Pervaiz Iqbal Khan, Andreas Dengel, Sheraz Ahmed
Neural Network Medical Image Classification Medical Image Datasets Pre Trained Vision Transformer Large Scale Medical

October 30, 2023

MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decoder
Md Motiur Rahman, Shiva Shokouhmand, Smriti Bhatt, Miad Faezipour
Medical Image Segmentation Segmentation Performance Pre Trained Vision Transformer MEG Decoder Attention Mixing

Pre Trained Vision Transformer

Papers

Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformers

Class-relevant Patch Embedding Selection for Few-Shot Image Classification

Intra-task Mutual Attention based Vision Transformer for Few-Shot Learning

Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers

Rethinking Low-Rank Adaptation in Vision: Exploring Head-Level Responsiveness across Diverse Tasks

Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers

iVPT: Improving Task-relevant Information Sharing in Visual Prompt Tuning by Cross-layer Dynamic Connection

Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach

LSPT: Long-term Spatial Prompt Tuning for Visual Representation Learning

Memory Consolidation Enables Long-Context Video Understanding

ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation

Learning Semantic Proxies from Visual Prompts for Parameter-Efficient Fine-Tuning in Deep Metric Learning

PL-FSCIL: Harnessing the Power of Prompts for Few-Shot Class-Incremental Learning

MIMIC: Mask Image Pre-training with Mix Contrastive Fine-tuning for Facial Expression Recognition

FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers

Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERT

Generalizable Imitation Learning Through Pre-Trained Representations

Automatic Report Generation for Histopathology images using pre-trained Vision Transformers

Medi-CAT: Contrastive Adversarial Training for Medical Image Classification

MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decoder