DNN Optimization

DNN optimization focuses on improving the efficiency and effectiveness of training deep neural networks, primarily aiming to enhance accuracy and reduce training time. Current research emphasizes developing sophisticated learning rate scheduling strategies, exploring novel optimization algorithms (like AdaBelief and its variants) that adapt step sizes more effectively, and optimizing model architectures through techniques such as network architecture search (NAS) to better suit hardware constraints (e.g., TinyML). These advancements are crucial for deploying DNNs in resource-limited environments and improving their performance across various applications, from autonomous systems to natural language processing.

Papers

October 24, 2022

Selecting and Composing Learning Rate Policies for Deep Neural Networks
Yanzhao Wu, Ling Liu
Deep Neural Network DNN Model View Selection DNN Training Learning Rate Schedule DNN Optimization

August 23, 2022

DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems
Michael Bechtel, QiTao Weng, Heechul Yun
TinyML Model Convolutional Neural Network Architecture Autonomous Miniature DNN Optimization

August 4, 2022

Leveraging the HW/SW Optimizations and Ecosystems that Drive the AI Revolution
Humberto Carvalho, Pavel Zaykov, Asim Ukaye
Deep Neural Network DNN Model Edge Inference Performance Optimization AI Revolution Ecosystem Level Analysis DNN Optimization

June 22, 2022

Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming
Chuan Wen, Jianing Qian, Jierui Lin, Jiaye Teng, Dinesh Jayaraman, Yang Gao
Deep Learning DNN Framework Fighting Fire Fire Occurrence Visual Imitation Learning Psychological Priming Practice DNN Optimization

March 24, 2022

A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
Supervised ImageNet Step Size Superior Optimizer Copy Suppression Adaptive Optimizers Analytic Calculus Crack AdaBoost Code Adaptive Stepsizes DNN Optimization

DNN Optimization

Papers

Selecting and Composing Learning Rate Policies for Deep Neural Networks

DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems

Leveraging the HW/SW Optimizations and Ecosystems that Drive the AI Revolution

Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming

A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range