cs.AINov 10, 2026

Benchmarking Vendor-Guided Optimizations on Edge NPUs: From NPU Only to End-to-End Performance. Extended technical report

Authors: Youngjin Jo, Jihwan Kim, Minseok Choi, Kwanghoon Choi, Jusung Kang, Jaein Kim

Organizations: Chonnam National University, Gwangju, 61186, Korea · Electronics and Telecommunications Research Institute, Gwangju, 61012, Korea

Abstract

Edge NPU performance depends not only on hardware capabilities, but also on model transformation and the overall deployment pipeline. We propose a benchmarking methodology that separates these effects by comparing three configurations: Baseline, Vendor Guided NPU Only (VG-NPU), and Vendor Guided End-to-End (VG-E2E), where VG-NPU preserves common model I/O and host-side processing while VG-E2E follows the vendor-native deployment pipeline. We apply this methodology to four edge NPUs and three vision workloads. VG-NPU improves both accuracy and Runtime-call latency in 10 of 12 NPU--workload combinations, whereas VG-E2E reduces System-level latency in all combinations but improves accuracy in only four. Moreover, NPU-level performance rankings do not always match End-to-End rankings, and VG-E2E provides an additional 10.4--70.7% reduction in System-level latency over VG-NPU. These results demonstrate the importance of evaluating vendor-guided optimization separately at the controlled model-transformation and full deployment levels when comparing heterogeneous commercial Edge NPU platforms in practice.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Empowering Hybrid Attention Models on NPUs

    Sep 26, 2026Yinyuan Zhang, Daliang Xu, Xiaolong Huang +4NPU AccelerationLLM Inference Acceleration

  2. Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms

    Oct 14, 2026Junlin Liao, Harshit Kondle, Di Liu +1Efficient VLM InferenceVLM Inference

  3. EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends

    Oct 17, 2026Yuhua Zheng, Xueyi Chen, Xinze Zou +3Large Language Model-Guided OptimizationEdge Computing