Inference Accelerator

Inference accelerators are specialized hardware and software systems designed to significantly speed up the execution of deep neural networks (DNNs), crucial for deploying AI models in resource-constrained environments or real-time applications. Current research focuses on optimizing these accelerators for various DNN architectures, including convolutional neural networks (CNNs), large language models (LLMs), and spiking neural networks (SNNs), through techniques like quantization, efficient dataflow management, and hardware-aware neural architecture search. These advancements are vital for enabling energy-efficient and high-throughput AI inference across diverse applications, from autonomous driving and edge computing to quantum error correction and resource-limited devices.

Papers

November 6, 2024

Stepping Forward on the Last Mile
Chen Feng, Shaojie Zhuo, Xiaopeng Zhang, Ramchalam Kinattinkara Ramakrishnan, Zhaocong Yuan, Andrew Zou Li
Back Propagation Energy Efficient Last Mile Inference Accelerator Forward Gradient

October 3, 2024

DecTrain: Deciding When to Train a DNN Online
Zih-Sing Fu, Soumya Sudhakar, Sertac Karaman, Vivienne Sze
Neural Network Deep Learning Deep Neural Network DNN Framework Monocular Depth Inference Accelerator

April 8, 2024

Exploring Quantization and Mapping Synergy in Hardware-Aware Deep Neural Network Accelerators
Jan Klhufek, Miroslav Safar, Vojtech Mrazek, Zdenek Vasicek, Lukas Sekanina
Convolutional Neural Network Quantization Operator Weight Quantization Model Synergy Inference Accelerator Mixed Quantization

March 4, 2024

Inference Acceleration for Large Language Models on CPUs
Ditto PS, Jithin VG, Adarsh MS
Large Language Model Natural Language Processing Scientific Inference Efficient Inference Inference Accelerator Inference Acceleration

February 22, 2024

An FPGA-Based Accelerator Enabling Efficient Support for CNNs with Arbitrary Kernel Sizes
Miaoxin Wang, Xiao Wu, Jun Lin, Zhongfeng Wang
Convolutional Neural Network Hardware Accelerator Inference Accelerator Large Kernel Convolution Kernel Convolutional Neural Network Kernel Size

January 27, 2024

Bayesian Inference Accelerator for Spiking Neural Networks
Prabodh Katti, Anagha Nimbekar, Chen Li, Amit Acharyya, Bashir M. Al-Hashimi, Bipin Rajendran
Neural Network Spiking Neural Network Bayesian Neural Network Spike Based Inference Accelerator Binary Neural

October 27, 2023

Edge AI Inference in Heterogeneous Constrained Computing: Feasibility and Opportunities
Roberto Morabito, Mallik Tatipamula, Sasu Tarkoma, Mung Chiang
Artificial Intelligence Emerging Opportunity Task Feasibility Inference Framework Edge Inference Inference Accelerator Robust AI Artificial Intelligence Inference

October 2, 2023

Subtractor-Based CNN Inference Accelerator
Victor Gao, Issam Hammad, Kamal El-Sankary, Jason Gu
Direct Convolution MNIST Dataset Inference Accelerator

July 18, 2023

A Cryogenic Memristive Neural Decoder for Fault-tolerant Quantum Error Correction
Victor Yon, Frédéric Marcotte, Pierre-Antoine Mouny, Gebremedhin A. Dagnew, Bohdan Kulchytskyy, Sophie Rochette, Yann Beilliard, Dominique Drouin, Pooya Ronagh
Neural Network Quantum Processor Inference Accelerator Neural Decoder

June 21, 2023

Subgraph Stationary Hardware-Software Inference Co-Design
Payman Behnam, Jianming Tong, Alind Khare, Yangyu Chen, Yue Pan, Pranav Gadikar, Abhimanyu Rajeshkumar Bambhaniya, Tushar Krishna, Alexey Tumanov
Inference Accelerator Latency Accuracy Weight Sharing Supernet Stationary Graph Process

May 30, 2023

Towards Machine Learning and Inference for Resource-constrained MCUs
Yushan Huang, Hamed Haddadi
Machine Learning Machine Learning Model Scientific Inference Machine Learning Inference Inference Accelerator Fish Segmentation

May 17, 2023

AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing
Hadjer Benmeziane, Corey Lammie, Irem Boybat, Malte Rasch, Manuel Le Gallo, Hsinyu Tsai, Ramachandran Muralidhar, Smail Niar, Ouarnoughi Hamza, Vijay Narayanan, Abu Sebastian, Kaoutar El Maghraoui
Deep Neural Network Exact Inference Analog Neural Network Inference Accelerator Analog in Memory Computing

April 24, 2023

Design optimization for high-performance computing using FPGA
Murat Isik, Kayode Inadagbo, Hakan Aktas
Field Programmable Gate Array High Performance Computing Design Optimization Inference Accelerator

February 9, 2023

DeepCAM: A Fully CAM-based Inference Accelerator with Variable Hash Lengths for Energy-efficient Deep Neural Networks
Duy-Thanh Nguyen, Abhiroop Bhattacharjee, Abhishek Moitra, Priyadarshini Panda
Deep Neural Network Deep Image Inference Accelerator Energy Efficient Neural Network

November 24, 2022

Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing
Zhongtian Dong, Nan Li, Alexandros Iosifidis, Qi Zhang
Product Design Scientific Inference Edge Computing Inference Latency Exact Inference Inference Accelerator Fast Prototyping

October 19, 2022

Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge
Ashwin Bhat, Adou Sangbone Assoa, Arijit Raychowdhury
Extreme Edge XAI Method Feature Attribution Multi FPGA Inference Accelerator Gradient Backpropagation

September 12, 2022

Understanding Time Variations of DNN Inference in Autonomous Driving
Liangkai Liu, Yanzhi Wang, Weisong Shi
Deep Neural Network Autonomous Driving DNN Inference Inference Accelerator Temporal Variation

September 9, 2022

ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference
Jing Gong, Hassaan Saadat, Hasindu Gamaarachchi, Haris Javaid, Xiaobo Sharon Hu, Sri Parameswaran
Scientific Inference DNN Architecture DNN Training Fast Simulation Inference Accelerator Approximate Multiplier Neural Network Library

August 27, 2022

Sub-mW Neuromorphic SNN audio processing applications with Rockpool and Xylo
Hannah Bos, Dylan Muir
Spiking Neural Network Neuromorphic Hardware Inference Accelerator Neuromorphic Audio

June 8, 2022

Memory-Oriented Design-Space Exploration of Edge-AI Hardware for XR Applications
Vivek Parmar, Syed Shakib Sarwar, Ziyun Li, Hsien-Hsin S. Lee, Barbara De Salvo, Manan Suri
Edge AI Design Space Exploration Inference Accelerator Non Volatile Memory Query Workload Inference Pipeline XR Application