CNN Inference

CNN inference focuses on efficiently executing pre-trained convolutional neural networks (CNNs), aiming to minimize computational cost and latency while maintaining accuracy. Current research emphasizes optimizing inference for resource-constrained environments like microcontrollers and edge devices, exploring techniques such as sparse computations (e.g., processing only frame differences), model compression (e.g., using autoencoders and attention mechanisms), and distributed inference across multiple devices. These advancements are crucial for deploying CNNs in real-time applications like robotics, IoT, and mobile computing, improving efficiency and expanding the scope of deep learning deployments.

Papers

October 16, 2024

EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge
Motahare Mounesan, Xiaojie Zhang, Saptarshi Debroy
Reinforcement Learning Optimization Purpose Extreme Edge Deep Learning Inference CNN Inference Model Based Deep Reinforcement Learning

August 17, 2024

H2PIPE: High throughput CNN Inference on FPGAs with High-Bandwidth Memory
Mario Doumet, Marius Stan, Mathew Hall, Vaughn Betz
Field Programmable Gate Array CNN Inference Deep Learning Compiler Dataflow Accelerator High Bandwidth Memory

June 7, 2024

Benchmarking Deep Jansen-Rit Parameter Inference: An in Silico Study
Deepa Tilwani, Christian O'Reilly
Neural Recording Silico Study Brain Dynamic CNN Inference Brain Mapping

June 6, 2024

OCCAM: Towards Cost-Efficient and Accuracy-Aware Image Classification Inference
Dujian Ding, Bicheng Xu, Laks V. S. Lakshmanan
Image Classification Image Classifier Image Classification Task Cost Effective CNN Inference Additive Attention Optimal Portfolio

April 8, 2024

TabConv: Low-Computation CNN Inference via Table Lookups
Neelesh Gupta, Narayanan Kannan, Pengmiao Zhang, Viktor Prasanna
Convolutional Neural Network Direct Convolution Lookup Table CNN Inference

January 19, 2024

Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencoder
Nan Li, Alexandros Iosifidis, Qi Zhang
Task Offloading Autoencoder Model Computation Offloading Graph AutoEncoder CNN Inference Semantic Compression Multi Access Edge Computing

October 25, 2023

MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory
Yinan Liang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu
Vision Transformer Transformer Architecture Deep Model 32 Bit Microcontrollers Limited Memory CNN Inference

August 29, 2023

Efficient labeling of solar flux evolution videos by a deep learning model
Subhamoy Chatterjee, Andrés Muñoz-Jaramillo, Derek A. Lamb
Convolutional Neural Network Deep Learning Model Data Labeling CNN Inference Quantifying Emergence

August 10, 2023

Machine Learning aided Computer Architecture Design for CNN Inferencing Systems
Christopher A. Metz
Machine Learning Single GPU Hardware Design Computational Resource Design Space Exploration Machine Learning Accelerator CNN Inference

March 8, 2023

EvConv: Fast CNN Inference on Event Camera Inputs For High-Speed Robot Perception
Sankeerth Durvasula, Yushi Guan, Nandita Vijaykumar
Event Camera Robot Perception Real Time Inference CNN Inference

February 19, 2023

Fixflow: A Framework to Evaluate Fixed-point Arithmetic in Light-Weight CNN Inference
Farhad Taheri, Siavash Bayat-Sarmadi, Hatame Mosanaei-Boorani, Reza Taheri
New Framework Fixed Point CNN Inference Light Weight

December 15, 2022

A Study on the Intersection of GPU Utilization and CNN Inference
Jack Kosaian, Amar Phanishayee
Convolutional Neural Network Study Feature Intersection Scenario CNN Inference GPU Implementation GPU Inference

November 24, 2022

Attention-based Feature Compression for CNN Inference Offloading in Edge Computing
Nan Li, Alexandros Iosifidis, Qi Zhang
Edge Computing Autoencoder Architecture Inference Task Feature Compression CNN Inference Device Edge Co Inference

October 18, 2022

MotionDeltaCNN: Sparse CNN Inference of Frame Differences in Moving Camera Videos
Mathias Parger, Chengcheng Tang, Thomas Neff, Christopher D. Twigg, Cem Keskin, Robert Wang, Markus Steinberger
Camera Motion Deep Motion Extrinsic Camera CNN Inference Per Frame Sparse CNN

July 22, 2022

July 20, 2022

AutoDiCE: Fully Automated Distributed CNN Inference at the Edge
Xiaotian Guo, Andy D. Pimentel, Todor Stefanov
Neural Network Deep Neural Network Extreme Edge CNN Inference

March 8, 2022

DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos
Mathias Parger, Chengcheng Tang, Christopher D. Twigg, Cem Keskin, Robert Wang, Markus Steinberger
Convolutional Neural Network Gameplay Video CNN Inference Sparse Convolutional

February 18, 2022

Towards Enabling Dynamic Convolution Neural Network Inference for Edge Intelligence
Adewale Adeyemo, Travis Sandefur, Tolulope A. Odetola, Syed Rafay Hasan
Field Programmable Gate Array Deep Learning Application Edge Intelligence Convolutional Neural Network Architecture CNN Inference

CNN Inference

Papers

EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge

H2PIPE: High throughput CNN Inference on FPGAs with High-Bandwidth Memory

Benchmarking Deep Jansen-Rit Parameter Inference: An in Silico Study

OCCAM: Towards Cost-Efficient and Accuracy-Aware Image Classification Inference

TabConv: Low-Computation CNN Inference via Table Lookups

Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencoder

MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory

Efficient labeling of solar flux evolution videos by a deep learning model

Machine Learning aided Computer Architecture Design for CNN Inferencing Systems

EvConv: Fast CNN Inference on Event Camera Inputs For High-Speed Robot Perception

Fixflow: A Framework to Evaluate Fixed-point Arithmetic in Light-Weight CNN Inference

A Study on the Intersection of GPU Utilization and CNN Inference

Attention-based Feature Compression for CNN Inference Offloading in Edge Computing

MotionDeltaCNN: Sparse CNN Inference of Frame Differences in Moving Camera Videos

Distributed Deep Learning Inference Acceleration using Seamless Collaboration in Edge Computing

Receptive Field-based Segmentation for Distributed CNN Inference Acceleration in Collaborative Edge Computing

AutoDiCE: Fully Automated Distributed CNN Inference at the Edge

DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos

Towards Enabling Dynamic Convolution Neural Network Inference for Edge Intelligence