Edge Inference

Ask about this topic

Overview

Edge inference studies how to run AI models close to where data is collected, often on devices with limited memory, power, and computing capacity. Common approaches include quantization, pruning and distillation, hardware-aware model and accelerator design, and adaptive methods such as early exits, model routing, and device-edge splitting. Work spans tasks from object detection and segmentation to language and multimodal models, with evaluation on benchmarks such as COCO and MVTec as well as phones, Raspberry Pi, Jetson, microcontrollers, and FPGAs. A central challenge is balancing accuracy, latency, energy, and robustness: proxy measures such as FLOPs may not predict real-device performance, which also varies with hardware, workload contention, and sustained operation.

Latest papers

CardsList

228 papers24 papers in the last four weeks, down 4% on the four weeks before. 0.1% of all new papers.

  1. DLRMamba: Distilling Low-Rank Mamba for Edge Multispectral Fusion Object Detection

    Mar 6, 2026Qianqian Zhang, Leon Tabaro, Ahmed M. Abdelmoniem +1Efficient Object DetectionState Space Models

    National Space Science Center, Chinese Academy of Sciences, Beijing, 101499, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, 100049, China · School of Electronic Engineering and Computer Science, Queen Mary University of London, London, E1 4NS, UK +1

  2. ButterflyViT: 354×\times Expert Compression for Edge Vision Transformers

    Mar 6, 2026Aryan KarmoreEfficient ViTsVision Transformer

    Indian Institute of Information Technology, Nagpur

  3. EdgeDAM: Real-time Object Tracking for Mobile Devices

    Mar 5, 2026Syed Muhammad Raza, Syed Murtaza Hussain Abidi, Khawar Islam +2Visual Object TrackingEdge Inference

    Neubility Inc., Seoul, Republic of Korea · Kumoh National Institute of Technology, South Korea · The University of Melbourne, Australia +1

  4. Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States

    Mar 3, 2026Jierui Zhang, Jianhao Huang, Kaibin HuangAdaptive InferenceEdge Inference

    Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong

  5. Orchestrating Multimodal DNN Workloads in Wireless Neural Processing

    Mar 2, 2026Sai Xu, Kai-Kit Wong, Yanan Du +1Efficient Multimodal InferenceEdge Inference

    Department of Electronic and Electrical Engineering, University College London, WC1E 7JE, London, UK · Department of Electronic Engineering, Kyung Hee University, Yongin-si, Gyeonggi-do 17104, Republic of Korea · School of Electrical and Electronic Engineering, the University of Sheffield, Sheffield S10 2TN, UK +1

  6. SHIELD8-UAV: Sequential 8-bit Hardware Implementation of a Precision-Aware 1D-F-CNN for Low-Energy UAV Acoustic Detection and Temporal Tracking

    Mar 1, 2026Susmita Ghanta, Karan Nathwani, Rohit ChaurasiyaFPGA-Based Neural Network AccelerationEdge Inference

    Dept. of Electrical Engineering, Indian Institute of Technology Jammu, Jagti 181221, India

  7. ShiftLUT: Spatial Shift Enhanced Look-Up Tables for Efficient Image Restoration

    Mar 1, 2026Xiaolong Zeng, Yitong Yu, Shiyao Xiong +4Image RestorationEdge Inference

    Tsinghua University, Beijing · Kuaishou Technology, Beijing

  8. MatchED: Crisp Edge Detection Using End-to-End, Matching-based Supervision

    Feb 24, 2026Bedrettin Cetinkaya, Sinan Kalkan, Emre AkbasEdge DetectionEdge Inference

    Dept. of Computer Engineering and METU ROMER Robotics Center, Ankara, Turkey

  9. Latent Replay Detection: Memory-Efficient Continual Object Detection on Microcontrollers via Task-Adaptive Compression

    Feb 24, 2026Bibin WilsonContinual Object DetectionObject Detection

    Independent Researcher

  10. Pip-Stereo: Progressive Iterations Pruner for Iterative Optimization based Stereo Matching

    Feb 24, 2026Jintu Zheng, Qizhe Liu, HuangXin Xu +1Stereo MatchingStereo Depth Estimation

    ARIDGE XPENG

  11. Metasurfaces-Integrated Wireless Neural Networks for Lightweight Over-The-Air Edge Inference

    Feb 22, 2026Kyriakos Stylianopoulos, Mario Edoardo Pandolfo, Paolo Di Lorenzo +1Wireless CommunicationsEdge Inference

    Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, 16122 Athens, Greece · National Inter-University Consortium for Telecommunications (CNIT), Parma, Italy · DIAG Department, Sapienza University of Rome, via Ariosto 25, Rome, Italy +1

  12. DART: Input-Difficulty-AwaRe Adaptive Threshold for Early-Exit DNNs

    Feb 17, 2026Parth Patne, Mahdi Taheri, Christian Herglotz +3Efficient Neural Network InferenceEarly-Exit Neural Networks

    Brandenburg Technical University, Cottbus, Germany · Tallinn University of Technology, Tallinn, Estonia · Leibniz Institute for High Performance Microelectronics, Frankfurt Oder, Germany

  13. MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios

    Feb 12, 2026Shuhuai Li, Jianghao Lin, Dongdong Ge +1Mixture-of-Experts InferenceEdge Inference

    Shanghai University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Standford University, California, United States

  14. FlashMoE: Reducing SSD I/O Bottlenecks via ML-Based Cache Replacement for Mixture-of-Experts Inference on Edge Devices

    Jan 22, 2026Byeongju Kim, Jungwan Lee, Donghyeon Han +2Edge InferenceMixture-of-Experts Inference

  15. Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators

    Dec 4, 2025Alaa Zniber, Arne Symons, Ouassim Karrakchou +2Efficient Neural Network InferenceDynamic Neural Networks

    TICLab, International University of Rabat, Morocco · MICAS, KU Leuven, Belgium

  16. LeanTTA: A Backpropagation-Free and Stateless Approach to Quantized Test-Time Adaptation on Edge Devices

    Mar 20, 2025Cynthia Dong, Hong Jia, Young D. Kwon +2Test-Time AdaptationEdge Inference

  17. Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies

    Feb 26, 2025Shaibal Saha, Lanyu XuEfficient ViTsNeural Network Compression

    Department of Computer Science and Engineering, Oakland University, , Rochester Hills, 48309, Michigan, USA

  18. A Comprehensive Evaluation of Deep Learning Object Detection Models on Heterogeneous Edge Devices

    Sep 25, 2024Daghash K. Alqahtani, Muhammad Aamir Cheema, Maria A. Rodriguez +1Energy-Efficient MLEdge Inference

    DisNet Lab, School of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, Australia · Monash University, Melbourne, VIC, Australia