Efficient Neural Network Architecture

Efficient neural network architecture research focuses on designing models that minimize computational cost while maintaining or improving performance. Current efforts concentrate on automated architecture search methods, leveraging techniques like evolutionary algorithms and large language models to explore vast design spaces and optimize for various constraints (e.g., power consumption, model size, speed). These advancements are crucial for deploying deep learning on resource-limited devices (edge computing) and accelerating training/inference across diverse applications, ranging from image classification and object detection to natural language processing and medical image analysis. The resulting efficient architectures significantly impact both the scalability and accessibility of AI.

Papers

November 3, 2024

Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision
Xiangzhong Luo, Di Liu, Hao Kong, Shuo Huai, Hui Chen, Guochu Xiong, Weichen Liu
Convolutional Neural Network Deep Neural Network Comprehensive Survey Embedded System Efficient Deep Efficient Neural Network Architecture

August 21, 2024

Design Principle Transfer in Neural Architecture Search via Large Language Models
Xun Zhou, Liang Feng, Xingyu Wu, Zhichao Lu, Kay Chen Tan
Neural Architecture Search Design Principle Architecture Search Efficient Neural Network Architecture

March 5, 2024

G-EvoNAS: Evolutionary Neural Architecture Search Based on Network Growth
Juan Zou, Weiwei Jiang, Yizhang Xia, Yuan Liu, Zhanglu Hou
Neural Architecture Search Supervised ImageNet Neural Network Architecture Evolutionary Computation Network Growth Efficient Neural Network Architecture

February 28, 2024

LeMo-NADe: Multi-Parameter Neural Architecture Discovery with LLMs
Md Hafizur Rahman, Prabuddha Chakraborty
Medical LLM Neural Architecture Search Complex Deep Efficient Neural Network Architecture Neural Architecture Search Space

January 2, 2024

Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edge
Alessio Carpegna, Alessandro Savino, Stefano Di Carlo
New Framework Spiking Neural Network Scientific Inference Extreme Edge Efficient Neural Network Architecture

September 22, 2023

Investigating Efficient Deep Learning Architectures For Side-Channel Attacks on AES
Yohaï-Eliel Berreby, Laurent Sauvage
Side Channel Efficient Neural Network Architecture

September 20, 2023

Grassroots Operator Search for Model Edge Adaptation
Hadjer Benmeziane, Kaoutar El Maghraoui, Hamza Ouarnoughi, Smail Niar
Deep Learning Hardware Aware Neural Architecture Search Efficient Neural Network Architecture Intensive Operator

May 28, 2023

LowDINO -- A Low Parameter Self Supervised Learning Model
Sai Krishna Prathapaneni, Shvejan Shashank, Srikar Reddy K
Convolutional Neural Network Deep Learning Model Self Supervised Neural Network Architecture High Performing 1D ConvNet Efficient Neural Network Architecture

May 18, 2023

FLIGHT Mode On: A Feather-Light Network for Low-Light Image Enhancement
Mustafa Ozcan, Hamza Ergezer, Mustafa Ayazaoglu
Low Light Image Enhancement lightWeight Network Illumination Condition Low Light Condition Illumination Correction Efficient Neural Network Architecture

April 10, 2023

hist2RNA: An efficient deep learning architecture to predict gene expression from breast cancer histopathology images
Raktim Kumar Mondol, Ewan K. A. Millar, Peter H Graham, Lois Browne, Arcot Sowmya, Erik Meijering
Cancer Genome Atlas Efficient Neural Network Architecture miRNA Description Breast Cancer Histopathology Image

October 15, 2022

Deep Differentiable Logic Gate Networks
Felix Petersen, Christian Borgelt, Hilde Kuehne, Oliver Deussen
Differentiable Logic Efficient Neural Network Architecture Logic Gate

September 19, 2022

Learning To Rank Diversely At Airbnb
Malay Haldar, Mustafa Abdool, Liwei He, Dillon Davis, Huiji Gao, Sanjeev Katariya
LeArning Abstract Stable Rank Efficient Neural Network Architecture Airbnb Rental Booking Probability

August 15, 2022

Faster Attention Is What You Need: A Fast Self-Attention Neural Network Backbone Architecture for the Edge via Double-Condensing Attention Condensers
Alexander Wong, Mohammad Javad Shafiee, Saad Abbasi, Saeejith Nair, Mahmoud Famouri
Self Attention Extreme Edge Efficient Attention Efficient Neural Network Architecture Attention Condenser

July 18, 2022

GANDALF: Gated Adaptive Network for Deep Automated Learning of Features
Manu Joseph, Harsh Raj
Feature Wise Efficient Neural Network Architecture Gated Recurrent Automated Deep Learning

July 15, 2022

ESFPNet: efficient deep learning architecture for real-time lesion segmentation in autofluorescence bronchoscopic video
Qi Chang, Danish Ahmad, Jennifer Toth, Rebecca Bascom, William E. Higgins
Lesion Segmentation Efficient Neural Network Architecture Video Bronchoscopy

April 25, 2022

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Han Cai, Ji Lin, Yujun Lin, Zhijian Liu, Haotian Tang, Hanrui Wang, Ligeng Zhu, Song Han
Deep Learning Deep Neural Network Financial Application Neural Architecture Search NCD Method System Description Mobile Device Device Training Efficient Neural Network Architecture Efficient Deep Learning

April 9, 2022

Searching for Efficient Neural Architectures for On-Device ML on Edge TPUs
Berkin Akin, Suyog Gupta, Yun Long, Anton Spiridonov, Zhuo Wang, Marie White, Hao Xu, Ping Zhou, Yanqi Zhou
Neural Architecture Search Machine Learning Accelerator Tensor Processing Unit Device Machine Learning Efficient Neural Network Architecture Edge TPU