Transformer Accelerator

Transformer accelerators are specialized hardware designed to efficiently execute the computationally intensive operations within transformer-based neural networks, aiming to improve speed and energy efficiency for applications like natural language processing and computer vision. Current research focuses on optimizing various aspects of transformer architectures, including binarization for reduced model size, sparse matrix multiplication for faster computation, and innovative attention mechanisms to reduce complexity. These advancements are crucial for deploying large transformer models on resource-constrained devices (like edge devices) and for accelerating the training process of even larger models, ultimately impacting the scalability and accessibility of advanced AI applications.

Papers

August 6, 2024

HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration
Pratyush Dhingra, Janardhan Rao Doppa, Partha Pratim Pande
Generative Modeling Transformer Architecture Transformer Model Hardware Accelerator Heterogeneous Computing Transformer Accelerator

July 16, 2024

Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
Yuhao Ji, Chao Fang, Shaobo Ma, Haikuo Shao, Zhongfeng Wang
Transformer Based Transformer Model Hardware Accelerator Edge Deployment Transformer Accelerator

June 1, 2024

Phasor-Driven Acceleration for FFT-based CNNs
Eduardo Reis, Thangarajah Akilan, Mohammed Khalid
Convolutional Neural Network CNN Architecture Fast Fourier Transform Transformer Accelerator

April 2, 2024

Accelerating Transformer Pre-training with 2:4 Sparsity
Yuezhou Hu, Kang Zhao, Weiyu Huang, Jianfei Chen, Jun Zhu
Fine Tuning Sparsity Increase Sparse Training Dense Training Large Scale Transformer Sparse Matrix Multiplication Transformer Accelerator

January 22, 2024

BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge
Yuhao Ji, Chao Fang, Zhongfeng Wang
Extreme Edge Matrix Multiplication FPGA Accelerator Edge Transformer Transformer Accelerator Binary Vision Transformer

January 8, 2024

FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs
Shulin Zeng, Jun Liu, Guohao Dai, Xinhao Yang, Tianyu Fu, Hongyi Wang, Wenheng Ma, Hanbo Sun, Shiyao Li, Zixiao Huang, Yadong Dai, Jintao Li, Zehao Wang, Ruoyu Zhang, Kairui Wen, Xuefei Ning, Yu Wang
Field Programmable Gate Array Large Language Model Inference Efficient Computation Transformer Accelerator

July 7, 2023

ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers
Gamze İslamoğlu, Moritz Scherer, Gianna Paulin, Tim Fischer, Victor J. B. Jung, Angelo Garofalo, Luca Benini
Transformer Model Softmax Function Transformer Network Hardware Acceleration Accelerator Architecture Quantized Transformer Transformer Accelerator

March 27, 2023

TransCODE: Co-design of Transformers and Accelerators for Efficient Training and Inference
Shikhar Tuli, Niraj K. Jha
Transformer Megatron Decepticons Scientific Inference Transformer Architecture Transformer Model Efficient Training Transformer Inference Transformer Accelerator Accelerator Tuning Video Transcoding

March 22, 2023

TRON: Transformer Neural Network Acceleration with Non-Coherent Silicon Photonics
Salma Afifi, Febin Sunny, Mahdi Nikdast, Sudeep Pasricha
Vision Transformer Neural Network Accelerator Silicon Photonics Transformer Accelerator

February 28, 2023

AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers
Shikhar Tuli, Niraj K. Jha
Transformer Megatron Decepticons Transformer Model Transformer Inference Dynamic Inference Accelerator Architecture Transformer Accelerator Sparse Accelerator

August 7, 2022

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Hongwu Peng, Shaoyi Huang, Shiyang Chen, Bingbing Li, Tong Geng, Ang Li, Weiwen Jiang, Wujie Wen, Jinbo Bi, Hang Liu, Caiwen Ding
Transformer Based Transformer Model Field Programmable Gate Array Sparse Attention Length Constraint Transformer Accelerator

Transformer Accelerator

Papers

HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration

Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment

Phasor-Driven Acceleration for FFT-based CNNs

Accelerating Transformer Pre-training with 2:4 Sparsity

BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge

FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs

ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers

TransCODE: Co-design of Transformers and Accelerators for Efficient Training and Inference

TRON: Transformer Neural Network Acceleration with Non-Coherent Silicon Photonics

AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining