cs.CCDec 4, 2025

Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators

Authors: Alaa Zniber, Arne Symons, Ouassim Karrakchou, Marian Verhelst, Mounir Ghogho

Organizations: TICLab, International University of Rabat, Morocco · MICAS, KU Leuven, Belgium

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The deployment of Early-Exiting Neural Networks (EENNs) on edge accelerators requires optimizing not only the network architecture but also its hardware deployment. Exit configuration, quantization, and hardware workload mapping interact in non-trivial ways, influencing memory traffic, accelerator utilization, and ultimately the energy-latency trade-off. This work presents a hardware-aware co-design framework for EENNs that jointly optimizes exit configuration, quantization-aware training, and multi-core hardware mapping within a unified NAS process. Leveraging analytical design space exploration, the framework identifies efficient workload mappings for each candidate architecture while providing accurate latency and energy estimates during the search. We further formulate EENN deployment as a constrained multi-objective optimization problem balancing predictive accuracy, energy-latency product, exit overhead, and dynamic inference efficiency. Experimental results on CIFAR-10 demonstrate that the proposed framework achieves over a 50% reduction in energy-latency product compared with static baselines under 8-bit quantization. These results demonstrate that jointly optimizing architecture and deployment is essential for realizing the full efficiency potential of dynamic inference on heterogeneous edge accelerators.

Figures & tables

Fig. 1: An example early exiting network with 3 backbone blocks and 3 exits.
 Exit 1 (D)Exit 2 (F)Exit 3 (I)Exit 4 (K)  
Cum. Params30.92271.946354.6341.439.498  
Cum. MACs24.515.58448.752.640118.307.840195.377.152  
 AccuracyExit RatioAccuracyExit RatioAccuracyExit RatioAccuracyExit RatioACC_avgET_avg
FP32+3298.4834.2194.7314.2293.8028.2263.6823.3588.504921
INT8+899.1025.4996.8615.3195.7031.1466.3628.0688.51467
INT8+498.9724.1897.6915.1295.6029.3169.4831.3988.53489
TABLE I: Accuracy and exit ratio for different quantization configurations. FP/INT X+YX+Y means XX -bit precision for the backbone and YY -bit precision for the exit point classifiers. Cum. Params denotes cumulative number of parameters, Cum. MACs denotes cumulative multiply-accumulate operations.
Fig. 2: The cumulative (left) and average (right) energy-delay product ET at different stages for differently quantized models. The ET_avg is drastically reduced thanks to the early exiting at early stages.
Fig. 3: Family of 4-exit models in INT8+8 configuration with identical backbone and exits topologies. Each model has different exit points, identified by the exit indices (cf., Table 1 in Appendix A).
Fig. 4: The cumulative (left) and average (right) energy-delay product ET at different stages for different mounting points with identical backbone and exit architecture. The ET varies due to exit ratio differences and tensor dimension mismatches with the accelerator dataflows.
Fig. 5: Difference in ETavg\mathrm{ET}_{\mathrm{avg}} relative to the homogeneous EEEE configuration versus ACCavg\mathrm{ACC}_{\mathrm{avg}} for five representative EENN architectures evaluated on eight quad-core accelerator configurations of identical compute and memory budgets, composed of Edge TPU-like (E), Meta-prototype-like (M), and Eyeriss-like (Y) cores. The MMMM configuration is omitted for clarity due to its substantially higher ETavg\mathrm{ET}_{\mathrm{avg}} .
Fig. 6: Overview of the deployment-aware co-design framework, integrating quantization-aware training and analytical hardware cost estimation via the Stream [ 17 ] design space exploration platform to evaluate energy and latency at different exit configurations.
Fig. 7: The development of the search with respect to accuracy and EENN ET reduction (i.e. ratio of ET_avg and the respective static architecture ET, without early exits). More exploration tends to yield architectures with high accuracy and high ET reduction.
MethodPrecisionAccuracy (%)MAC Reduction (%)
EDANAS [ 12 ]FP3281.1036.79
NACHOS [ 13 ]FP3272.6558.99
OursINT888.0456.46
TABLE II: Comparison of efficient models in terms of accuracy and MAC operations from previous EENN-specific NAS studies.
Fig. 8: Distribution over EENN models in terms of Accuracy (in %) and Average ET (in J x cycles) across NAS iterations.
Fig. 9: Subset of efficient architectures (where ET_avg<1000\text{ET\_{avg}}<1000 ) from the NAS showing the Pareto front (gold line) with varying number of exits.
OperatorRepetitionExit IndexChannelsStrides
conv2d1-321
bottleneck1-161
bottleneck2A, B241
bottleneck2C, D321
bottleneck2E, F642
bottleneck2G, H961
TABLE III: MobileNetV2 with Early Exits Mounting Points

Explore similar work

CardsList

  1. A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits

    Apr 16, 2026Nekane Fernandez, Ivan Valdes, Steven Van Vaerenbergh +2Edge DevicesNeural Network Inference

    Nekane Fernandez, Ivan Valdes, Idoia de la Iglesia and Julen Arratibel are with the Dept. of Connected and Distributed Intelligence, Ikerlan Technology Research Centre, Arrasate, Spain · Nekane Fernandez and Steven Van Vaerenbergh are with the Dept. of Mathematics, Statistics and Computation, Universidad de Cantabria, Santander, Spain

  2. MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model

    Aug 7, 2026Zijun Jiang, Yangdi LyuMixed-Precision QuantizationArtificial Intelligence Systems

    Microelectronics Thrust, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China

  3. A Reconfigurable Multiplier Architecture for Error-Resilient Applications in RISC-V Core

    May 9, 2026Pragun Jaswal, L. Hemanth Krishna, B. SrinivasuRisc-VEdge Devices

    Indian Institute of Technology Mandi, Mandi–175005, India