EdgeCraft: Automated Model Crafting for Edge IoT

Genglin Wang; Kaiwei Liu; Liekang Zeng; Wangsong Yin; Shangcheng Jin; Guoliang Xing; Zhenyu Yan

cs.LGSep 28, 2026

EdgeCraft: Automated Model Crafting for Edge IoT

Authors: Genglin Wang, Kaiwei Liu, Liekang Zeng, Wangsong Yin, Shangcheng Jin, Guoliang Xing, Zhenyu Yan

Organizations: Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · School of Computer Science, Peking University, Beijing, China

Abstract

Machine learning (ML) increasingly powers Internet of Things (IoT) applications at the edge. Yet producing a deployable edge ML artifact for a specific scenario requires navigating a huge search space spanning data representation, model design, training on domain-specific data, and runtime customization. This workflow is fragmented and difficult to scale across diverse edge applications. We present EdgeCraft, an LLM-driven system that turns high-level intent into deployable edge ML artifacts. Building such a system raises two challenges: (1) How can an LLM be guided to find high-quality solutions that meet dynamic SLOs for task quality, latency, and energy? (2) How can trustworthy target-device verification be obtained at low cost? EdgeCraft addresses these challenges with two designs. (1) A constraint-aware synthesis tree explores alternative candidates and uses measured SLO gaps to guide each improvement. (2) A multi-fidelity verifier progressively combines low-cost checks with full target-device verification to reduce verification cost while preserving reliable verification results. It also records verified failures for reuse, avoiding repeated device work. To support concurrency, EdgeCraft provides a multi-tenant runtime that runs cloud training and target-device verification in parallel while isolating requests. Across 50 public tasks, EdgeCraft exceeds the task-specific Reference in best-observed quality on 40 tasks and finds an SLO-feasible artifact on 45, with the two outcomes overlapping on 38 tasks. Moreover, EdgeCraft achieves competitive performance on our self-collected SEN dataset, suggesting its generalizability to real-world IoT sensing tasks.

Figures & tables

Figure 1 . Qualitative comparison of EdgeCraft with existing edge ML development paradigms in (a) automation and coverage, and (b) required human effort.

Edge/Mobile ML App.	Service-Level Optimization Objective
Camera surveillance ( Liu et al., 2018 )	Max. AUC, acceptable latency
Visual inspection ( Bergmann et al., 2021 ; Bergmann et al., 2019 )	Max. recall/AUROC, acceptable latency
Drone-based inspection ( Zhu et al., 2021 )	Max. mAP, low latency, bounded energy
Wearable HAR ( He et al., 2026 )	Max. accuracy, low latency, bounded energy
Always-on keyword spotting ( Warden, 2018 )	Max. accuracy, very-low latency
AIOps anomaly detection ( Sana et al., 2025 )	Max. F1/AUROC, very-low latency

Table 1 . Representative service-level optimization objectives for edge ML applications, reflecting different priorities among task quality, latency, and energy.

Figure 2 . (a) Task quality, latency, and power vary across model–runtime configurations. Marker shape and size denote runtime and peak memory, while color denotes measured power from yellow (low) to red (high). (b–c) Measured p95 latency and power vary substantially across device–runtime combinations. (d) The HAR design changes with the dataset, device, and SLOs; hatching marks SLO violations and red boxes mark winners.

Panel of Figure 3 — Figure 3 . Motivation studies of LLM-driven synthesis. (a) LLM-driven synthesis fails before functional validation on the target device. (b) LLM-guided search identifies promising candidates more efficiently.

Figure 5 . An illustrative synthesis tree on MotionSense. The left panel shows the tree structure and its verification results; the right traces successive mutations.

Figure 6 . Calibrated device measurements prune a candidate before full training.

Figure 7 . A reproduced device–runtime failure becomes a verified compatibility rule reused by a later tenant.

Figure 8 . Normalized task quality gap of EdgeCraft and reference models on 50 edge/mobile datasets.

Figure 9 . Agentic development across 50 tasks: (a) stage-wise task retention and (b) cumulative SLO-feasible tasks versus per-task time.

Device	Abbr.	Compute Resources	Runtime
Jetson AGX Orin	Orin	8-core Arm, Ampere GPU 32GB	P, O, T
Jetson Xavier NX	NX	6-core Arm, Volta GPU 8GB	P, O, T
Jetson TX2	TX2	4-core Arm, Pascal GPU 4GB	P, O, T
Raspberry Pi 5	Pi5	4-core Cortex-A76 8GB	P, O, L
Desktop (CPU)	PC	Intel Ultra 5 235	P, O, L

Table 2 . Evaluated hardware platforms. Runtime abbr.: P (PyTorch), O (ONNX), T (TensorRT), and L (LiteRT).

Figure 10 . EdgeCraft and AutoML/NAS baselines on their supported datasets.

Figure 11 . Multi-tenant performance at 2–8 requests/hour: (a) mean queueing delay and (b) p95 completion time (mean

\pm

s.d.).

Figure 12 . Under the same budget, Constraint-Aware leads in (a) feasible-quality gain and (b) SLO-feasible incumbent-improvement rate;

\times

\leq 0.05\%

Figure 13 . Multi-Fidelity cuts verification time by 17.6–39.7% across the four runtimes in (a) and total exploration time by 25.7% in (b). In all four runtimes, it selects the same artifact as full verification.

Figure 14 . Verified Failure Reuse skips at least one candidate in each of the nine evaluated tasks in (a) and reduces required device evaluations from 101 to 44 (56.4%) across the full evaluation in (b).

Figure 15 . Search and SLO sensitivity: (a) feasible-quality gain with budget; (b) feasible-task and task-normalized quality retention for the fixed

1.0\times

cohort.

Method	Sample-level $\uparrow$	Cross-subject $\uparrow$	Avg. $\uparrow$
Expert+RF	$0.8241\!\pm\!0.0010$	$0.5477\!\pm\!0.0048$	0.6859
General+RF	$0.8945\!\pm\!0.0024$	$0.5280\!\pm\!0.0037$	0.7113
Kats+RF	$0.7803\!\pm\!0.0008$	$0.5595\!\pm\!0.0013$	0.6699
Catch22+RF	$0.5744\!\pm\!0.0053$	$0.5322\!\pm\!0.0004$	0.5533
MiniRocket	$0.7883\!\pm\!0.0006$	$0.5697\!\pm\!0.0044$	0.6790
NormWear	$0.8414\!\pm\!0.0168$	$0.5293\!\pm\!0.0170$	0.6854

Table 3 . SEN AUROC and Pi 5 deployment results.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 4 — Table 4 . A comprehensive benchmark of edge-oriented on-device AI tasks. For each frozen Reference, the table reports task quality (Q), p95 latency in ms (L), and energy per inference in mJ (E). Energy constrains the search only for the 12 tasks with an application-level energy SLO; “–” denotes tasks without one.

Measure	Evaluation and outcome
Multi-Fidelity verification
Verification cost	16 fixed candidates across four runtimes: per-runtime time falls by 17.6–39.7%, and total time falls from 69.13 to 51.35 minutes (25.7%), with the same selected artifact.
Applicability	Of 180 candidates, 120 yield matched fast/full measurements; 69 have prior calibration at decision time.
Decision safety	60 early rejections across the evaluated SLO settings; full verification agrees with all 60.
Verified Failure Reuse
Coverage	134 candidates from 15 datasets, five modalities, four devices, and three runtimes; all 50 rules are reused, matching 57 candidates.

Appendix

Table 5 . Evaluation scope and aggregate outcomes for the two verification mechanisms.

Explore similar work

EdgeFM: Efficient Edge Inference for Vision-Language Models
Apr 30, 2026Mengling Deng, Yuanpeng Chen, Sheng Yang +12Fast Inference LLM Inference Optimization
Go Further. AI · School of Data Science, Fudan University · RUYi Dynamics Co., Ltd +1
E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments
Jun 2, 2026Truong-Thanh Le, Amir Taherkordi, Hoang-Loc La +3LLM Inference Optimization Edge Devices
Department of Informatics University of Oslo Oslo, Norway · Department of Computer Science UiT The Arctic University of Norway Tromsø, Norway
Agentic Performance at the Edge: Insights from Benchmarking
May 11, 2026Shiqiang Wang, Herbert WoisetschlägerAgentic Edge
University of Exeter · Exeter, United Kingdom · Technical University of Munich +1

Apr 30, 2026, cs.CVJ/K move · Enter open · S save

EdgeFM: Efficient Edge Inference for Vision-Language Models

Mengling Deng, Yuanpeng Chen, Sheng Yang, Wei Tao+11

Go Further. AI · School of Data Science, Fudan University · RUYi Dynamics Co., Ltd+1

Fast Inference LLM Inference Optimization Edge Twitter-Fmnerg Benchmarks

Open paper AskPDF arXiv

Vision-language models (VLMs) have demonstrated strong applicability in edge industrial applications, yet their deployment remains severely constrained by requirements for deterministic low latency and stable execution under resource limitations. Existing frameworks either rely on bloated general-purpose designs or force developers into opaque, hardware-specific closed-source ecosystems, leading to hardware lock-in limitation and poor cross-platform adaptability. Observing that modern AI agents can efficiently search and tune configurations to generate highly optimized low-level kernels for standard LLM operators, we propose EdgeFM, a lightweight, agent-driven VLM/LLM inference framework tailored for cross-platform industrial edge deployment. EdgeFM removes non-essential features to reduce single-request latency, and encapsulates agent-tuned kernel optimizations as a modular library of reusable skills. By allowing direct invocation of these skills rather than waiting for closed-source implementations, it effectively closes the performance gap long dominated by proprietary toolchains. The framework natively supports mainstream platforms including x86 and NVIDIA Orin SoCs, and represents the first end-to-end VLA deployment on the domestic Horizon Journey platform, enhancing cross-platform portability. In most cases, it yields clearly better inference performance than conventional vendor-specific toolchains, achieving up to 1.49 times speedup over TensorRT-Edge-LLM on the NVIDIA Orin platform. Experimental results show that EdgeFM delivers favorable end-to-end inference performance, providing an open-source, production-grade solution for diverse edge industrial scenarios.

Abstract

Explore similar work

EdgeFM: Efficient Edge Inference for Vision-Language Models

E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments

Agentic Performance at the Edge: Insights from Benchmarking