DNN Inference

DNN inference focuses on efficiently executing pre-trained deep neural networks, aiming to minimize latency, energy consumption, and memory footprint while maintaining accuracy. Current research emphasizes optimizing inference across diverse hardware platforms (e.g., mobile devices, microcontrollers, edge servers, and cloud), exploring techniques like model compression, adaptive batching, workload partitioning, and mixed-precision computation. These advancements are crucial for deploying DNNs in resource-constrained environments and improving the performance and sustainability of AI applications across various domains, from mobile computing to autonomous systems.

Papers

May 1, 2023

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms
Ziyang Zhang, Huan Li, Yang Zhao, Changyao Lin, Jie Liu
Deep Neural Network Learning Based Edge Inference DNN Inference Maximum Entropy Reinforcement Learning Edge Platform

March 31, 2023

Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Rafael Stahl, Daniel Mueller-Gritschneder, Ulf Schlichtmann
Deep Neural Network TinyML Model DNN Inference Memory Optimization

January 15, 2023

Adaptive Deep Neural Network Inference Optimization with EENet
Fatih Ilhan, Ka-Ho Chow, Sihao Hu, Tiansheng Huang, Selim Tekin, Wenqi Wei, Yanzhao Wu, Myungjin Lee, Ramana Kompella, Hugo Latapie, Gaowen Liu, Ling Liu
Deep Neural Network Supervised ImageNet DNN Model DNN Inference Deep Learning Inference Multi Exit

October 11, 2022

Edge-Cloud Cooperation for DNN Inference via Reinforcement Learning and Supervised Learning
Tinghao Zhang, Zhijun Li, Yongrui Chen, Kwok-Yan Lam, Jun Zhao
Reinforcement Learning Supervised Learning Edge Cloud DNN Inference DNN Compression

September 12, 2022

Understanding Time Variations of DNN Inference in Autonomous Driving
Liangkai Liu, Yanzhi Wang, Weisong Shi
Deep Neural Network Autonomous Driving DNN Inference Inference Accelerator Temporal Variation

July 7, 2022

Efficient Pruning for Machine Learning Under Homomorphic Encryption
Ehud Aharoni, Moran Baruch, Pradip Bose, Alper Buyuktosunoglu, Nir Drucker, Subhankar Pal, Tomer Pelleg, Kanthi Sarpatwar, Hayim Shaul, Omri Soceanu, Roman Vaculin
Edge Pruning Homomorphic Encryption Pruning Method Inference Latency Autoencoder Architecture Machine Permutation DNN Inference Efficient Machine Learning

June 17, 2022

Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes
Matteo Risso, Alessio Burrello, Luca Benini, Enrico Macii, Massimo Poncino, Daniele Jahier Pagliari
Deep Neural Network Neural Architecture Search Mixed Precision Quantization Mixed Precision DNN Inference Edge Resource

December 27, 2021

FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions
Behnam Ghavami, Mani Sadati, Zhenman Fang, Lesley Shannon
DNN Inference

DNN Inference

Papers

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms

Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference

Adaptive Deep Neural Network Inference Optimization with EENet

Edge-Cloud Cooperation for DNN Inference via Reinforcement Learning and Supervised Learning

Understanding Time Variations of DNN Inference in Autonomous Driving

Efficient Pruning for Machine Learning Under Homomorphic Encryption

Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes

FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions