DNN Architecture

Deep neural network (DNN) architecture research focuses on designing efficient and effective network structures for various tasks, aiming to improve accuracy, reduce computational cost, and enhance robustness. Current research emphasizes areas like explainable AI (using methods such as prototype-based explanations and SHAP values), hardware-aware design (optimizing architectures for specific hardware constraints and incorporating techniques like pruning and quantization), and improving the resilience of DNNs to adversarial attacks and hardware faults. These advancements are crucial for deploying DNNs in resource-constrained environments and safety-critical applications, impacting fields ranging from computer vision and natural language processing to edge computing and medical imaging.

Papers

October 22, 2024

September 11, 2024

Deep Neural Network-Based Sign Language Recognition: A Comprehensive Approach Using Transfer Learning with Explainability
A. E. M Ridwan, Mushfiqul Islam Chowdhury, Mekhala Mariam Mary, Md Tahmid Chowdhury Abir
Deep Neural Network Transfer Learning High Explainability Sign Language Sign Language Recognition Deeper Network DNN Architecture Systematic Approach Bangla Sign Language

August 8, 2024

Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes
Bhushan Atote, Victor Sanchez
Deep Neural Network Explainable Artificial Intelligence DNN Architecture Prototypical Part Network Prototype Based Explanation

July 1, 2024

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network
Hyunseok Oh, Youngki Lee
DNN Architecture Neuronal Dynamic Discrete Dynamical System Spiking Neural Network Conversion

June 10, 2024

ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs
Seyedhamidreza Mousavi, Mohammad Hasan Ahmadilivani, Jaan Raik, Maksim Jenihhin, Masoud Daneshtalab
Deep Neural Network Activation Function Tiny Refinement Elicit Resilience DNN Architecture Adaptive Clipping Progressive Training

June 8, 2024

Exploring Adversarial Robustness of Deep State Space Models
Biqing Qi, Yang Luo, Junqi Gao, Pengfei Li, Kai Tian, Zhiyuan Ma, Bowen Zhou
Adversarial Training Adversarial Robustness Adversarial Perturbation DNN Architecture Deep State Space Model

May 23, 2024

What Variables Affect Out-of-Distribution Generalization in Pretrained Models?
Md Yousuf Harun, Kyungbok Lee, Jhair Gallardo, Giri Krishnan, Christopher Kanan
Pre Trained DNN Model DNN Architecture Pre Trained Deep Neural Network

March 1, 2024

Resilience of Entropy Model in Distributed Neural Networks
Milin Zhang, Mohammad Abdi, Shahriar Rifat, Francesco Restuccia
Neural Network Adversarial Attack Tiny Refinement Elicit Resilience DNN Architecture Entropy Model Entropy Coding Entropy Sampling

February 19, 2024

CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware
Souvik Kundu, Anthony Sarah, Vinay Joshi, Om J Omer, Sreenivas Subramoney
Neural Architecture Search Large Scale Joint Optimization DNN Architecture Computing in Memory System Configuration

February 9, 2024

Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
Bradley T. Baker, Barak A. Pearlmutter, Robyn Miller, Vince D. Calhoun, Sergey M. Plis
LeArning Abstract Deep Neural Network Product Design Neural Collapse DNN Architecture Low Rank Structure Network Architecture Nonlinear Activation Rank Collapse

December 23, 2023

Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
Konstantinos Balaskas, Andreas Karatzas, Christos Sad, Kostas Siozios, Iraklis Anagnostopoulos, Georgios Zervakis, Jörg Henkel
Deep Neural Network DNN Accelerator Mixed Precision Quantization DNN Architecture Energy Efficient Inference DNN Compression Hierarchical Pruning Pruning Quantization

November 21, 2023

Investigating Weight-Perturbed Deep Neural Networks With Application in Iris Presentation Attack Detection
Renu Sharma, Redwan Sony, Arun Ross
Deep Neural Network Application Proficiency DNN Architecture SE SPP DenseNet Iris Image Iris Presentation Attack Detection

November 17, 2023

Archtree: on-the-fly tree-structured exploration for latency-aware pruning of deep neural networks
Rémi Ouazan Reboul, Edouard Yvinec, Arnaud Dapogny, Kevin Bailly
Deep Neural Network Structured Pruning DNN Architecture Pruning Framework Rapidly Exploring Random Tree DNN Pruning

November 15, 2023

DLAS: An Exploration and Assessment of the Deep Learning Acceleration Stack
Perry Gibson, José Cano, Elliot J. Crowley, Amos Storkey, Michael O'Boyle
Deep Neural Network Environment Exploration Direct Assessment Deep Learning Application DNN Architecture Deep Learning Acceleration DNN Acceleration

October 17, 2023

From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport
Quentin Bouniot, Ievgen Redko, Anton Mallasto, Charlotte Laclau, Karol Arndt, Oliver Struckmeier, Markus Heinonen, Ville Kyrki, Samuel Kaski
Deep Neural Network Transformer Megatron Decepticons Optimal Transport DNN Architecture Non Linearity Alexnet PCN16

October 2, 2023

Naming Practices of Pre-Trained Models in Hugging Face
Wenxin Jiang, Chingwo Cheung, Mingyu Kim, Heesoo Kim, George K. Thiruvathukal, James C. Davis
Pre Trained Model Human Face DNN Architecture Model Reuse Semantic Anomaly Model Discovery Naming Variation

September 29, 2023

Adversarial Machine Learning in Latent Representations of Neural Networks
Milin Zhang, Mohammad Abdi, Francesco Restuccia
Neural Network Adversarial Attack Adversarial Robustness Adversarial Learning Latent Representation Adversarial Behavior DNN Architecture

August 28, 2023

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
Samuel Horvath, Stefanos Laskaridis, Shashank Rajput, Hongyi Wang
Low Rank DNN Architecture Simple Structure Low Rank Structure Learning Decomposition Low Rank Compression

June 21, 2023

Adaptive DNN Surgery for Selfish Inference Acceleration with On-demand Edge Resource
Xiang Yang, Dezhi Chen, Qi Qi, Jingyu Wang, Haifeng Sun, Jianxin Liao, Song Guo
DNN Architecture DNN Repair Inference Acceleration Decentralized Deep Learning Edge Resource

DNN Architecture

Papers

Detecting Adversarial Examples

Resource-Efficient Sensor Fusion via System-Wide Dynamic Gated Neural Networks

Deep Neural Network-Based Sign Language Recognition: A Comprehensive Approach Using Transfer Learning with Explainability

Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network

ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs

Exploring Adversarial Robustness of Deep State Space Models

What Variables Affect Out-of-Distribution Generalization in Pretrained Models?

Resilience of Entropy Model in Distributed Neural Networks

CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware

Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse

Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization

Investigating Weight-Perturbed Deep Neural Networks With Application in Iris Presentation Attack Detection

Archtree: on-the-fly tree-structured exploration for latency-aware pruning of deep neural networks

DLAS: An Exploration and Assessment of the Deep Learning Acceleration Stack

From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport

Naming Practices of Pre-Trained Models in Hugging Face

Adversarial Machine Learning in Latent Representations of Neural Networks

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition

Adaptive DNN Surgery for Selfish Inference Acceleration with On-demand Edge Resource