cs.LGNov 27, 2025

AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices

Authors: Mengyang Liu, Chenyu Lu, Haodong Tian, Fang Dong, Ruiting Zhou, Wei Wang, Dian Shen, Guangtong Li, 

Organizations: Southeast University · Hong Kong University of Science and Technology · Tsinghua University · University of Macau

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

On-device machine learning (ML) has become a fundamental component of emerging mobile applications. Adaptive model deployment delivers efficient inference for heterogeneous device capabilities and performance requirements through customizing neural architectures. SuperNet-based approaches offer a promising solution by generating numerous model variants from a pre-trained ML model. However, applying SuperNet in existing frameworks requires tedious model-aware development and time-consuming hardware-aware profiling, which limits their practical adoption. We present AutoTailor, the first framework to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices. Unlike manual SuperNet construction, AutoTailor employs a computation graph-guided compilation approach to automatically transform user-provided ML models into SuperNets. To support efficient specialization, AutoTailor incorporates learning-free latency and accuracy predictors, enabling low-cost yet accurate performance prediction. Our extensive evaluations demonstrate that AutoTailor reduces the lines of code for SuperNet construction by 10.8--34.5×\times, reduces predictor profiling time by 2.2--11.5×\times, and achieves up to 15.60% absolute accuracy improvement and 60.03% latency reduction compared to state-of-the-art approaches across diverse models and devices. AutoTailor's source code is publicly available at https://github.com/liu-mengyang/AutoTailor.

Figures & tables

Figure 1. Latency-accuracy tradeoff of adaptive ResNet-50 on the Exynos 1380 SoC’s big cores using the ImageNet1k dataset across different methods. SuperNet consistently achieves state-of-the-art performance. Static uses ResNet-18 and ResNet-34 as smaller model variants.
Figure 2. An illustration of adaptive model deployment and different model variant generation methods. A four-layer DNN is used as a running example, where each color denotes a distinct layer.
Figure 3. Comparison between block scaling and SuperNet-based model variant generation.
Figure 4. An illustration of the SuperNet workflow.
Figure 5. Illustration of the graph structural consistency property and comparison of SuperNet development methods.
ModelResNet50MobileNetv3
Number of variants2×10132\times 10^{13}7×10227\times 10^{22}
Latency profiling (s/variant)0.2∼1.750.2\sim 1.750.4∼3.850.4\sim 3.85
Accuracy profiling (s/variant)31∼4731\sim 4731∼4331\sim 43
Table 1. Profiling costs for deploying SuperNets. Latency profiled on Samsung A54 big cores and accuracy profiled on an NVIDIA RTX 3070 GPU with a batch size of 128.
Figure 6. Comparison of predictor building methods.
Figure 7. Existing predictor building methods rely on time-consuming profiling for collecting training data.
Figure 8. Illustration of cross-SubNet architectural similarity in different models.
Figure 9. System architecture.
Figure 10. TailorIR overview.
Figure 11. Optimizations in TailorIR module development.
Figure 12. An example of space pruning.
Modelsnormal+ space pruning+ TailorIR
(GPU hours)(GPU hours)(CPU minutes)
ResNet501.25E+0910.811.8
MobileNetv28720000.03822.2
ConvNext8.08978E+150.02594.8
LeViT1.02E+4157.4446.5
Table 2. Extraction cost of unique operators. In the normal setting, the cost is estimated by multiplying the number of profiling runs by the average cost of a single profiling run.
Figure 13. Accuracy of top-ranked SubNets across FLOPs budgets. Top- kk denotes the minimum measured accuracy among the kk highest-ranked candidates in each search.
Figure 14. Median accuracy error of additive and pairwise predictors across FLOPs bins, with 100 SubNets per bin.
Figure 15. Performance comparison of hardware-aware adaptive model deployment.
Figure 16. Performance comparison of hardware-agnostic adaptive model deployment.
Original DNN# of Modifications# of Variants
ResNet50 He et al. (2016)1002×10132\times 10^{13}
MobileNetv2 Sandler et al. (2018)858×10108\times 10^{10}
MobileNetv3 Howard et al. (2019)1607×10227\times 10^{22}
ProxylessNet Cai et al. (2019)1666×10236\times 10^{23}
ConvNext-Tiny Liu et al. (2022)1632×10232\times 10^{23}
ViT-Base Dosovitskiy et al. (2021)1031×10121\times 10^{12}
Table 3. Illustration of SuperNets in this paper.
Figure 17. Overhead analysis of ResNet50 adaptive model on phone big cores.
Figure 18. Comparison of LoC in constructing SuperNets.
Figure 19. Analysis of TailorIR Modules.
Figure 20. Profiling cost of (a) latency predictors for LeViT operators and (b) accuracy predictors across models.
Figure 21. Model-level latency prediction: (a) fusion ablation; (b,c) measured and predicted latency for 1000 SubNets per setting on Samsung A54. Dashed lines mark ±10%\pm 10\% of measured latency.
Figure 22. Ablation study on computation-quality trade-offs. Baselines: MLP in (a,c) and MLP+LitePred in (b,d). R\mathrm{R} denotes the restricted design space.
Figure 23. Computation–quality trade-offs for object detection and language modeling.
More in this topic

Edge ML

Edge machine learning runs and adapts models on nearby devices to reduce latency, bandwidth use, and exposure of sensitive data.

See all 129 papers

Explore similar work

CardsList

  1. Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator

    Jul 20, 2026Mateusz Piechocki, Alessandro Capotondi, Marek KraftOn-Device TrainingEdge ML

    Poznan University of Technology, Institute of Robotics and Machine Intelligence, Poznan, Poland · University of Modena and Reggio Emilia, The High-Performance Real-Time Laboratory, Modena, Italy

  2. ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device

    May 5, 2026Mergen Nachin, Digant Desai, Sicheng Stephen Jia +36Heterogeneous ComputingEdge ML

    Meta · Work done while at Meta.

  3. GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

    Jun 4, 2026Oleeviya Babu Poikarayil, Cédric Schockaert, Abdulrahman Nahhas +3Efficient Neural Network InferenceLarge Language Model-Guided NAS

    Paul Wurth S.A., Luxembourg · Otto-von-Guericke University, Magdeburg, Germany · Technical University of Munich, Germany