Generative Vision Transformer

Generative Vision Transformers (GenViTs) are a rapidly evolving area of research focusing on leveraging the strengths of vision transformers for image and video generation tasks. Current work explores architectures that combine transformers with convolutional neural networks or diffusion models, often incorporating techniques like masked token modeling and prompt tuning to improve efficiency and generalization across diverse datasets. These models are being applied to various applications, including deepfake detection and video synthesis, demonstrating improved performance and efficiency compared to previous methods. The ability of GenViTs to perform both generative and discriminative tasks within a single framework represents a significant advancement in the field.

Papers

July 13, 2023

Deepfake Video Detection Using Generative Convolutional Vision Transformer
Deressa Wodajo, Solomon Atnafu, Zahid Akhtar
Deepfake Detection Deep Fake Deepfake Video DeepFake Video Detection Generative Vision Transformer

December 10, 2022

MAGVIT: Masked Generative Video Transformer
Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G. Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, Lu Jiang
Spatial Token Masked Generative Video Generation Benchmark Generative Vision Transformer

October 3, 2022

Visual Prompt Tuning for Generative Transfer Learning
Kihyuk Sohn, Yuan Hao, José Lezama, Luisa Polania, Huiwen Chang, Han Zhang, Irfan Essa, Lu Jiang
Vision Transformer Visual Prompt Generative Image Model Generative Vision Transformer Generative Transfer

August 16, 2022

Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model
Xiulong Yang, Sheng-Min Shih, Yinlin Fu, Xiaoting Zhao, Shihao Ji
Generative Model Vision Transformer Generative Modeling Denoising Diffusion Probabilistic Model Generative Vision Transformer

December 27, 2021

Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction
Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li
Energy Based Saliency Prediction Vision Transformer Network Generative Vision Transformer

Generative Vision Transformer

Papers

Deepfake Video Detection Using Generative Convolutional Vision Transformer

MAGVIT: Masked Generative Video Transformer

Visual Prompt Tuning for Generative Transfer Learning

Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model

Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction