Hardware Aware Training

Hardware-aware training (HAT) optimizes machine learning model training to account for the limitations and characteristics of the target hardware, improving efficiency and robustness. Current research focuses on adapting various neural network architectures, including spiking neural networks and multilayer perceptrons, to diverse hardware platforms like neuromorphic processors, in-memory computing systems, and even printed electronics, often employing techniques like backpropagation-through-time and genetic algorithms. This approach is crucial for deploying efficient and reliable AI models on resource-constrained devices, impacting fields ranging from mobile computing and edge AI to specialized hardware accelerators. The ultimate goal is to bridge the gap between software-optimized models and the realities of hardware implementation, leading to more practical and energy-efficient AI systems.

Papers

November 7, 2024

Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach
Yequan Zhao, Hai Li, Ian Young, Zheng Zhang
Training Data Zeroth Order Gradient Computation Backpropagation Free Hardware Aware Training

October 25, 2024

Genetic Motifs as a Blueprint for Mismatch-Tolerant Neuromorphic Computing
Tommaso Boccato, Dmitrii Zendrikov, Nicola Toschi, Giacomo Indiveri
Neuromorphic Computing Neuromorphic Hardware Digital Computing Blueprint Architecture Hardware Aware Training

September 12, 2024

Efficient and Reliable Vector Similarity Search Using Asymmetric Encoding with NAND-Flash for Many-Class Few-Shot Learning
Hao-Wei Chiang, Chi-Tse Huang, Hsiang-Yun Cheng, Po-Hao Tseng, Ming-Hsiu Lee, An-Yeu (Andy)Wu
LeArning Abstract High Efficiency Shot Learning Encoder Decoder Architecture Multi Class Content Addressable Memory Memory Augmented Hardware Aware Training Vector Similarity Search

April 16, 2024

Hardware-aware training of models with synaptic delays for digital event-driven neuromorphic processors
Alberto Patino-Saucedo, Roy Meijer, Amirreza Yousefzadeh, Manil-Dev Gomony, Federico Corradi, Paul Detteter, Laura Garrido-Regife, Bernabe Linares-Barranco, Manolis Sifalakis
Full Model Synaptic Delay Neuromorphic Platform Hardware Aware Training Core Neuromorphic

February 16, 2024

FedKit: Enabling Cross-Platform Federated Learning for Android and iOS
Sichang He, Beilong Tang, Boyan Zhang, Jiaoqi Shao, Xiaomin Ouyang, Daniel Nata Nugraha, Bing Luo
Mobile Application Model Aggregation Cross Device Federated Learning Hardware Aware Training

February 5, 2024

Embedding Hardware Approximations in Discrete Genetic-based Training for Printed MLPs
Florentia Afentaki, Michael Hefenbrock, Georgios Zervakis, Mehdi B. Tahoori
Discrete Environment Approximate Computing Hardware Aware Training Evolutionary Learning Hardware Approximation

December 11, 2023

Measurement-driven neural-network training for integrated magnetic tunnel junction arrays
William A. Borders, Advait Madhavan, Matthew W. Daniels, Vasileia Georgiou, Martin Lueker-Boden, Tiffany S. Santos, Patrick M. Braganca, Mark D. Stiles, Jabez J. McClelland, Brian D. Hoskins
Neural Network Magnetic Tunnel Junction Memory Stability Hardware Aware Training Measurement Based Approach

September 19, 2023

Logic Design of Neural Networks for High-Throughput and Low-Power Applications
Kangwei Xu, Grace Li Zhang, Ulf Schlichtmann, Bing Li
Neural Network Low Power High Throughput Logic Design Multiply Accumulate Hardware Aware Training

September 7, 2023

Improving Resnet-9 Generalization Trained on Small Datasets
Omar Mohamed Awad, Habib Hajimolahoseini, Michael Lim, Gurpreet Gosal, Walid Ahmed, Yang Liu, Gordon Deng
Image Classification Small Datasets Label Smoothing Neural Network Generalization Cifar 10 Hardware Aware Training Sharpness Measure

May 29, 2023

Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars
Philippe Drolet, Raphaël Dawant, Victor Yon, Pierre-Antoine Mouny, Matthieu Valdenaire, Javier Arias Zapata, Pierre Gliech, Sean U. N. Wood, Serge Ecoffey, Fabien Alibart, Yann Beilliard, Dominique Drouin
Native Robustness Resistive Random Access Memory Crossbar Array Hardware Aware Training Transferable Neural Passive Physical Equivalent

April 20, 2023

Backpropagation-free Training of Deep Physical Neural Networks
Ali Momeni, Babak Rahmani, Matthieu Mallejac, Philipp Del Hougne, Romain Fleury
Deep Learning Backpropagation Free Physical Neural Network Hardware Aware Training Digital Back Propagation

March 14, 2023

Gradient-descent hardware-aware training and deployment for mixed-signal Neuromorphic processors
Uğurcan Çakal, Maryada, Chenxi Wu, Ilkay Ulusoy, Dylan R. Muir
Spiking Neural Network Real World Deployment Hardware Aware Training Mixed Signal Neuromorphic

February 16, 2023

Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators
Malte J. Rasch, Charles Mackin, Manuel Le Gallo, An Chen, Andrea Fasoli, Frederic Odermatt, Ning Li, S. R. Nandakumar, Pritish Narayanan, Hsinyu Tsai, Geoffrey W. Burr, Abu Sebastian, Vijay Narayanan
Memory Computing DNN Accuracy Inference Workload Analog in Memory Computing Hardware Aware Training Memory Accelerator

January 18, 2023

Tailor: Altering Skip Connections for Resource-Efficient Inference
Olivia Weng, Gabriel Marcano, Vladimir Loncar, Alireza Khodamoradi, Nojan Sheybani, Andres Meza, Farinaz Koushanfar, Kristof Denolf, Javier Mauricio Duarte, Ryan Kastner
Deep Neural Network Co Design Dataflow Architecture Skip Connection Hardware Aware Training

October 8, 2022

GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation
Ondrej Sykora, Phitchaya Mangpo Phothilimthana, Charith Mendis, Amir Yazdanbakhsh
Graph Neural STONE Deterioration Pattern Hardware Aware Training Block Size

Hardware Aware Training

Papers

Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach

Genetic Motifs as a Blueprint for Mismatch-Tolerant Neuromorphic Computing

Efficient and Reliable Vector Similarity Search Using Asymmetric Encoding with NAND-Flash for Many-Class Few-Shot Learning

Hardware-aware training of models with synaptic delays for digital event-driven neuromorphic processors

FedKit: Enabling Cross-Platform Federated Learning for Android and iOS

Embedding Hardware Approximations in Discrete Genetic-based Training for Printed MLPs

Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

Logic Design of Neural Networks for High-Throughput and Low-Power Applications

Improving Resnet-9 Generalization Trained on Small Datasets

Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars

Backpropagation-free Training of Deep Physical Neural Networks

Gradient-descent hardware-aware training and deployment for mixed-signal Neuromorphic processors

Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators

Tailor: Altering Skip Connections for Resource-Efficient Inference

GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation