Multi Exit

Multi-exit networks aim to improve the efficiency of deep learning models by allowing predictions to be made at multiple points within the network, rather than solely at the final layer. Current research focuses on optimizing the placement and design of these "exits," exploring various training strategies (e.g., knowledge distillation, sample weighting) and model architectures to balance accuracy and computational cost. This approach holds significant promise for resource-constrained applications, such as mobile and edge computing, and is driving advancements in both model design and efficient hardware acceleration.

Papers

October 12, 2024

DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach
Daniel Gallo Fernández, Rǎzvan-Andrei Matişan, Alejandro Monroy Muñoz, Ana-Maria Vasilcoiu, Janusz Partyka, Tin Hadži Veljković, Metod Jazbec
Neural Network Diffusion Model Denoising Network Multi Exit Backbone Architecture

July 19, 2024

Deep Feature Surgery: Towards Accurate and Efficient Multi-Exit Networks
Cheng Gong, Yao Chen, Qiuyang Luo, Ye Lu, Tao Li, Yuzhi Zhang, Yufei Sun, Le Zhang
Deep Feature ImageNet Dataset Multi Scale Feature Multi Exit

July 15, 2024

Enhancing Building Safety Design for Active Shooter Incidents: Exploration of Building Exit Parameters using Reinforcement Learning-Based Simulations
Ruying Liu, Wanjing Wu, Burcin Becerik-Gerber, Gale M. Lucas
Positive Reinforcement Hurricane Evacuation Multi Exit Student Simulation System Safety Active Shooter

June 20, 2024

May 21, 2024

Multimodal Adaptive Inference for Document Image Classification with Anytime Early Exiting
Omar Hamed, Souhail Bakkali, Marie-Francine Moens, Matthew Blaschko, Jordy Van Landeghem
Large Language Model Visually Rich Document Early eXit Multi Exit Document Image Classification

November 1, 2023

NEO-KD: Knowledge-Distillation-Based Adversarial Training for Robust Multi-Exit Neural Networks
Seokil Ham, Jungwuk Park, Dong-Jun Han, Jaekyun Moon
Adversarial Attack Knowledge Distillation Adversarial Example Adversarial Training Multi Exit Adversarial Knowledge Distillation Near Earth Object

August 13, 2023

When Monte-Carlo Dropout Meets Multi-Exit: Optimizing Bayesian Neural Networks on FPGA
Hongxiang Fan, Hao Chen, Liam Castelli, Zhiqiang Que, He Li, Kenneth Long, Wayne Luk
Bayesian Neural Network Field Programmable Gate Array Multi Exit Bayes Net Monte Carlo Dropout

May 23, 2023

Model Stealing Attack against Multi-Exit Networks
Li Pan, Lv Peizhuo, Chen Kai, Cai Yuling, Xiang Fan, Zhang Shengzhi
Model Stealing Attack Multi Exit

May 20, 2023

Dynamic Transformers Provide a False Sense of Efficiency
Yiming Chen, Simin Chen, Zexin Li, Wei Yang, Cong Liu, Robby T. Tan, Haizhou Li
Adversarial Attack High Efficiency Inference Efficiency False Sense Early Exit Adversarial Data Collection Multi Exit Dynamic Transformer

January 15, 2023

Adaptive Deep Neural Network Inference Optimization with EENet
Fatih Ilhan, Ka-Ho Chow, Sihao Hu, Tiansheng Huang, Selim Tekin, Wenqi Wei, Yanzhao Wu, Myungjin Lee, Ramana Kompella, Hugo Latapie, Gaowen Liu, Ling Liu
Deep Neural Network Supervised ImageNet DNN Model DNN Inference Deep Learning Inference Multi Exit

December 22, 2022

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing
Tian Dong, Ziyuan Zhang, Han Qiu, Tianwei Zhang, Hewu Li, Terry Wang
Edge Computing Human Mind Backdoor Detection Human Heart Stealthy Backdoor Attack Multi Exit Dynamic Deep

December 3, 2022

Understanding the Robustness of Multi-Exit Models under Common Corruptions
Akshay Mehra, Skyler Seto, Navdeep Jaitly, Barry-John Theobald
Deep Neural Network Native Robustness Distribution Shift Early Exit Common Corruption Multi Exit

September 24, 2022

Joint Speech Activity and Overlap Detection with Multi-Exit Architecture
Ziqing Du, Kai Liu, Xucheng Wan, Huan Zhou
Overlap Detection Voice Activity Detection Traditional Neural Network Speech Application Multi Exit Overlapped Speech Detection Joint Task

September 20, 2022

Unsupervised Early Exit in DNNs with Multiple Exits
Hari Narayan N U, Manjesh K. Hanawal, Avinash Bhardwaj
Deep Neural Network Early eXit Early Exit Multi Exit

September 17, 2022

Learning to Weight Samples for Dynamic Early-exiting Networks
Yizeng Han, Yifan Pu, Zihang Lai, Chaofei Wang, Shiji Song, Junfen Cao, Wenhui Huang, Chao Deng, Gao Huang
LeArning Abstract Inference Efficiency Early eXit Multi Exit Early Exit Network Weight Prediction

August 23, 2022

Auditing Membership Leakages of Multi-Exit Networks
Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, Yang Zhang
Membership Inference Attack Attack Performance Multi Exit Membership Privacy Hybrid Attack

April 28, 2022

A Closer Look at Branch Classifiers of Multi-exit Architectures
Shaohui Lin, Bo Ji, Rongrong Ji, Angela Yao
Glance Annotation Strong Branching Multi Exit Branch Prediction