High Performance Transformer

High-performance transformers are being actively developed to improve efficiency and accuracy across diverse applications, addressing limitations in processing speed and local information capture inherent in standard transformer architectures. Current research focuses on optimizing transformer models for specific tasks, such as image restoration (using techniques like high-frequency injection), natural language processing (through padding-free algorithms and architectural enhancements), and medical image analysis (leveraging Swin Transformers and GANs). These advancements are significantly impacting fields like computer vision, natural language processing, and medical imaging by enabling faster, more accurate, and memory-efficient solutions for complex problems.

Papers

August 27, 2024

FastTextSpotter: A High-Efficiency Transformer for Multilingual Scene Text Spotting
Alloy Das, Sanket Biswas, Umapada Pal, Josep Lladós, Saumik Bhattacharya
Robust Version Swin Transformer Character Recognition Multilingual Scenario FastText Model High Performance Transformer

March 30, 2024

Look-Around Before You Leap: High-Frequency Injected Transformer for Image Restoration
Shihao Zhou, Duosheng Chen, Jinshan Pan, Jufeng Yang
Image Restoration Glance Annotation Transformer Based Approach Leap Forward Convolutional Operation Spatial Resolution Enhancement High Performance Transformer

March 4, 2024

Human Evaluation of English--Irish Transformer-Based NMT
Séamus Lankford, Haithem Afli, Andy Way
Transformer Based Machine Translation Human Evaluation Machine Translation System High Performance Transformer Transformer Based Neural Machine Translation

November 9, 2023

High-Performance Transformers for Table Structure Recognition Need Early Convolutions
ShengYun Peng, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau
Hybrid CNN Transformer Table Structure Recognition Convolutional Backbone High Performance Transformer

February 25, 2023

Cybersecurity Challenges of Power Transformers
Hossein Rahimpour, Joe Tusek, Alsharif Abuadbba, Aruna Seneviratne, Toan Phung, Ahmed Musleh, Boyu Liu
Power Grid Cyber Attack Threat Landscape Security Challenge High Performance Transformer Power Transformer

November 25, 2022

TRAC: A Textual Benchmark for Reasoning about Actions and Change
Weinan He, Canming Huang, Zhanhao Xiao, Yongmei Liu
Complex Reasoning Transformer Based Language Model Past Action Pre Change Information Text Benchmark Structural Generalization High Performance Transformer

October 6, 2022

ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs
Yujia Zhai, Chengquan Jiang, Leyuan Wang, Xiaoying Jia, Shang Zhang, Zizhong Chen, Xin Liu, Yibo Zhu
Transformer Model BERT Model Transformer Based Framework Long Input High Performance Transformer

January 23, 2022

Fast MRI Reconstruction: How Powerful Transformers Are?
Jiahao Huang, Yinzhe Wu, Huanjun Wu, Guang Yang
Generative Adversarial Network Fast Magnetic Resonance Imaging Fast Magnetic Resonance Imaging Reconstruction High Performance Transformer

January 17, 2022

Automatic Segmentation of Head and Neck Tumor: How Powerful Transformers Are?
Ikboljon Sobirov, Otabek Nazarov, Hussain Alasmawi, Mohammad Yaqub
Deep Learning Convolutional Neural Network Segmentation Task Tumor Segmentation Human Head Automatic Segmentation Neck Cancer High Performance Transformer

November 22, 2021

Mesa: A Memory-saving Training Framework for Transformers
Zizheng Pan, Peng Chen, Haoyu He, Jing Liu, Jianfei Cai, Bohan Zhuang
Transformer Megatron Decepticons Activation Quantization Memory Efficient Training Mesa Optimization High Performance Transformer