cs.LGNov 27, 2025

AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices

Authors: Mengyang Liu, Chenyu Lu, Haodong Tian, Fang Dong, Ruiting Zhou, Wei Wang, Dian Shen, Guangtong Li, 

Organizations: Southeast University · Hong Kong University of Science and Technology · Tsinghua University · University of Macau

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

On-device machine learning (ML) has become a fundamental component of emerging mobile applications. Adaptive model deployment delivers efficient inference for heterogeneous device capabilities and performance requirements through customizing neural architectures. SuperNet-based approaches offer a promising solution by generating numerous model variants from a pre-trained ML model. However, applying SuperNet in existing frameworks requires tedious model-aware development and time-consuming hardware-aware profiling, which limits their practical adoption. We present AutoTailor, the first framework to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices. Unlike manual SuperNet construction, AutoTailor employs a computation graph-guided compilation approach to automatically transform user-provided ML models into SuperNets. To support efficient specialization, AutoTailor incorporates learning-free latency and accuracy predictors, enabling low-cost yet accurate performance prediction. Our extensive evaluations demonstrate that AutoTailor reduces the lines of code for SuperNet construction by 10.8--34.5×\times, reduces predictor profiling time by 2.2--11.5×\times, and achieves up to 15.60% absolute accuracy improvement and 60.03% latency reduction compared to state-of-the-art approaches across diverse models and devices. AutoTailor's source code is publicly available at https://github.com/liu-mengyang/AutoTailor.

Figures & tables

Figure 1. Latency-accuracy tradeoff of adaptive ResNet-50 on the Exynos 1380 SoC’s big cores using the ImageNet1k dataset across different methods. SuperNet consistently achieves state-of-the-art performance. Static uses ResNet-18 and ResNet-34 as smaller model variants.
Figure 2. An illustration of adaptive model deployment and different model variant generation methods. A four-layer DNN is used as a running example, where each color denotes a distinct layer.
Figure 3. Comparison between block scaling and SuperNet-based model variant generation.
Figure 4. An illustration of the SuperNet workflow.
Figure 5. Illustration of the graph structural consistency property and comparison of SuperNet development methods.
ModelResNet50MobileNetv3
Number of variants2×10132\times 10^{13}7×10227\times 10^{22}
Latency profiling (s/variant)0.2∼1.750.2\sim 1.750.4∼3.850.4\sim 3.85
Accuracy profiling (s/variant)31∼4731\sim 4731∼4331\sim 43
Table 1. Profiling costs for deploying SuperNets. Latency profiled on Samsung A54 big cores and accuracy profiled on an NVIDIA RTX 3070 GPU with a batch size of 128.
Figure 6. Comparison of predictor building methods.
Figure 7. Existing predictor building methods rely on time-consuming profiling for collecting training data.
Figure 8. Illustration of cross-SubNet architectural similarity in different models.
Figure 9. System architecture.
Figure 10. TailorIR overview.
Figure 11. Optimizations in TailorIR module development.
Figure 12. An example of space pruning.
Modelsnormal+ space pruning+ TailorIR
(GPU hours)(GPU hours)(CPU minutes)
ResNet501.25E+0910.811.8
MobileNetv28720000.03822.2
ConvNext8.08978E+150.02594.8
LeViT1.02E+4157.4446.5
Table 2. Extraction cost of unique operators. In the normal setting, the cost is estimated by multiplying the number of profiling runs by the average cost of a single profiling run.
Figure 13. Accuracy of top-ranked SubNets across FLOPs budgets. Top- kk denotes the minimum measured accuracy among the kk highest-ranked candidates in each search.
Figure 14. Median accuracy error of additive and pairwise predictors across FLOPs bins, with 100 SubNets per bin.
Figure 15. Performance comparison of hardware-aware adaptive model deployment.
Figure 16. Performance comparison of hardware-agnostic adaptive model deployment.
Original DNN# of Modifications# of Variants
ResNet50 He et al. (2016)1002×10132\times 10^{13}
MobileNetv2 Sandler et al. (2018)858×10108\times 10^{10}
MobileNetv3 Howard et al. (2019)1607×10227\times 10^{22}
ProxylessNet Cai et al. (2019)1666×10236\times 10^{23}
ConvNext-Tiny Liu et al. (2022)1632×10232\times 10^{23}
ViT-Base Dosovitskiy et al. (2021)1031×10121\times 10^{12}
Table 3. Illustration of SuperNets in this paper.
Figure 17. Overhead analysis of ResNet50 adaptive model on phone big cores.
Figure 18. Comparison of LoC in constructing SuperNets.
Figure 19. Analysis of TailorIR Modules.
Figure 20. Profiling cost of (a) latency predictors for LeViT operators and (b) accuracy predictors across models.
Figure 21. Model-level latency prediction: (a) fusion ablation; (b,c) measured and predicted latency for 1000 SubNets per setting on Samsung A54. Dashed lines mark ±10%\pm 10\% of measured latency.
Figure 22. Ablation study on computation-quality trade-offs. Baselines: MLP in (a,c) and MLP+LitePred in (b,d). R\mathrm{R} denotes the restricted design space.
Figure 23. Computation–quality trade-offs for object detection and language modeling.
More in this topic

Edge ML

Edge machine learning runs and adapts models on nearby devices to reduce latency, bandwidth use, and exposure of sensitive data.

See all 129 papers

Explore similar work

CardsList
Jul 20, 2026

Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator

On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks. This work proposes a heterogeneous adaptation pipeline that repurposes a commercial edge AI inference accelerator, Hailo-8L, for frozen-backbone feature extraction during on-device training. The computational graph is partitioned so that the pre-trained backbone is quantized to INT8 and run on the accelerator, while only a lightweight FP32 classification head is fine-tuned on the host CPU, enabling frequent, energy-efficient in-field updates with most weights remaining fixed. Across multiple architectures and datasets, this pipeline achieves up to 15.4x faster wall-clock training time compared to a Raspberry Pi 5 CPU baseline, offers competitive throughput in favorable settings, and consistently reduces energy per sample. Post-training quantization restoration is shown to be crucial for preserving the quality of accelerator-generated features and mitigating accuracy loss in quantization-sensitive architectures. Overall, the results demonstrate a practical approach to efficient on-device adaptation using inference-oriented edge accelerators. The implementation is available at https://github.com/MatPiech/accelerator-training.

Mateusz Piechocki, Alessandro Capotondi, Marek Kraft

Poznan University of Technology, Institute of Robotics and Machine Intelligence, Poznan, Poland · University of Modena and Reggio Emilia, The High-Performance Real-Time Laboratory, Modena, Italy

On-Device TrainingEdge ML
AskPDF
May 5, 2026

ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device

Local execution of AI on edge devices is important for low latency and offline operation. However, deploying models on diverse hardware remains fragmented, often requiring model conversion or complete reimplementation outside the PyTorch ecosystem where the model was originally authored. We introduce ExecuTorch, a unified PyTorch-native deployment framework for edge AI. ExecuTorch enables seamless deployment of machine learning models across heterogeneous compute environments. It scales from embedded microcontrollers to complex system-on-chips (SoCs) with dedicated accelerators, powering devices ranging from wearables and smartphones to large compute clusters. ExecuTorch preserves PyTorch semantics while allowing customization, support for optimizations like quantization, and pluggable execution "backends". These features together enable fast experimentation, allowing researchers to validate deployment behavior entirely within PyTorch, bridging the gap between research and production.

Mergen Nachin, Digant Desai, Sicheng Stephen Jia +36

Meta · Work done while at Meta.

Heterogeneous ComputingEdge MLOn-Device InferenceEdge AI
AskPDF
Jun 4, 2026

GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise. Traditional Automated Machine Learning (AutoML) systems typically rely on static, predefined search spaces, limiting their ability to adapt to diverse data characteristics. We present GenAutoML, an agentic framework that leverages Large Language Models (LLMs) as neural architects to bridge natural-language requirements and executable PyTorch implementations. The framework incorporates a Sandboxed Reflection Loop for autonomous code refinement and a Signature-Aware Runtime that enforces architectural consistency and execution safety. To improve robustness under non-stationary conditions, we further introduce a Dynamic Reversible Instance Normalization (Dyn-RevIN) wrapper. Experiments on the ETTh1, ETTm1, and Weather benchmarks demonstrate that GenAutoML can dynamically generate task-specific neural architectures tailored to dataset characteristics. Among the generated models, WaveInterferenceNet achieves inference latency below 0.01 ms per sample while maintaining competitive predictive performance. By emphasizing computational efficiency, architectural adaptability, and stable optimization behavior, GenAutoML enables the creation of ultra-lightweight neural networks suitable for resource-constrained and latency-sensitive Edge AI deployments.

Oleeviya Babu Poikarayil, Cédric Schockaert, Abdulrahman Nahhas +3

Paul Wurth S.A., Luxembourg · Otto-von-Guericke University, Magdeburg, Germany · Technical University of Munich, Germany

Efficient Neural Network InferenceLarge Language Model-Guided NASTime Series ForecastingEdge MLNeural Architecture SearchAutomated Machine Learning+1
AskPDF