Shifted Window Transformer

Shifted window transformers are a class of deep learning models designed to improve the efficiency and performance of vision transformers, particularly in resource-constrained environments or for high-resolution data. Current research focuses on adapting this architecture for various tasks, including image captioning, medical image analysis (e.g., bone pathology detection, breast mass segmentation, image quality assessment), and other domains like genomic analysis and traffic prediction. These advancements demonstrate the versatility of shifted window transformers and their potential to enhance the accuracy and speed of numerous applications, ranging from autonomous vehicles to healthcare diagnostics.

Papers

August 25, 2024

Shifted Window Fourier Transform And Retention For Image Captioning
Jia Cheng Hu, Roberto Cavicchioli, Alessandro Capotondi
Transformer Architecture Image Captioning Transformer Based Architecture Image Captioning Model Customer Retention Shifted Window Transformer

July 24, 2024

Domain Generalized Recaptured Screen Image Identification Using SWIN Transformer
Preeti Mehta, Aman Sagar, Suchi Kumari
Data Augmentation Domain Shift Screen Content Dual Adversarial Shifted Window Transformer Generalization Model

February 2, 2024

Faster Inference of Integer SWIN Transformer by Removing the GELU Activation
Mohammadreza Tayaranian, Seyyed Hasan Mozafari, James J. Clark, Brett Meyer, Warren Gross
Neural Network Fast Inference Vision Transformer Model Integer Quantization Shifted Window Transformer

October 11, 2023

Impact of Label Types on Training SWIN Models with Overhead Imagery
Ryan Ford, Kenneth Hutchison, Nicholas Felts, Benjamin Cheng, Jesse Lew, Kyle Jackson
Global Impact Object Detection Model Model Training Object Segmentation Segmentation Label Overhead Imagery Shifted Window Transformer Label Description

August 24, 2023

DeepLOC: Deep Learning-based Bone Pathology Localization and Classification in Wrist X-ray Images
Razan Dibo, Andrey Galichin, Pavel Astashev, Dmitry V. Dylov, Oleg Y. Rogov
Classification Code Medical Image Analysis Shifted Window Transformer Bone Lesion

July 14, 2023

HEAL-SWIN: A Vision Transformer On The Sphere
Oscar Carlsson, Jan E. Gerken, Hampus Linander, Heiner Spieß, Fredrik Ohlsson, Christoffer Petersson, Daniel Persson
Vision Transformer Spherical Surface Shifted Window Transformer

June 8, 2023

Genomic Interpreter: A Hierarchical Genomic Deep Neural Network with 1D Shifted Window Transformer
Zehui Li, Akashaditya Das, William A V Beardall, Yiren Zhao, Guy-Bart Stan
Gene Expression Prediction Shifted Window Transformer

November 16, 2022

SWIN-SFTNet : Spatial Feature Expansion and Aggregation using Swin Transformer For Whole Breast micro-mass segmentation
Sharif Amit Kamran, Khondker Fariha Hossain, Alireza Tavakkoli, George Bebis, Sal Baker
Swin Transformer Mammogram Classification Shifted Window Transformer Dense Breast

August 11, 2022

Shifted Windows Transformers for Medical Image Quality Assessment
Caner Ozer, Arda Guler, Aysel Turkvatan Cansever, Deniz Alis, Ercan Karaarslan, Ilkay Oksuz
Image Quality Swin Transformer Shifted Window Transformer

January 17, 2022

SwinUNet3D -- A Hierarchical Architecture for Deep Traffic Prediction using Shifted Window Transformers
Alabi Bojesomo, Hasan Al Marzouqi, Panos Liatsis
Vision Transformer Swin Transformer Traffic Prediction Traffic Forecasting Spatiotemporal Forecasting Shifted Window Transformer

Shifted Window Transformer

Papers

Shifted Window Fourier Transform And Retention For Image Captioning

Domain Generalized Recaptured Screen Image Identification Using SWIN Transformer

Faster Inference of Integer SWIN Transformer by Removing the GELU Activation

Impact of Label Types on Training SWIN Models with Overhead Imagery

DeepLOC: Deep Learning-based Bone Pathology Localization and Classification in Wrist X-ray Images

HEAL-SWIN: A Vision Transformer On The Sphere

Genomic Interpreter: A Hierarchical Genomic Deep Neural Network with 1D Shifted Window Transformer

SWIN-SFTNet : Spatial Feature Expansion and Aggregation using Swin Transformer For Whole Breast micro-mass segmentation

Shifted Windows Transformers for Medical Image Quality Assessment

SwinUNet3D -- A Hierarchical Architecture for Deep Traffic Prediction using Shifted Window Transformers