On-Device Inference

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 117

All topics

Embedded Arena: Iterative Optimization via Hardware Feedback
Jun 15, 2026Zhihan Zhang, Alexander Le Metzger, Jiuyang Lyu +10Large Language Model-Guided Optimization Tiny ML
University of Washington · University of California San Diego · Northeastern University
Enabling Real-Time Point-of-Care Ultrasound Segmentation: A GPU-Free Deployment in Resource-Limited Settings
Jun 13, 2026Weihao GaoUltrasound Image Segmentation Ultrasound Imaging
School of Computer Science and Artificial Intelligence, Guangdong University of Education
WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition
Jun 11, 2026Maximilian Burzer, Tobias King, Till Riedel +2Efficient Neural Network Inference Human Activity Recognition
Karlsruhe Institute of Technology, Karlsruhe, Germany · IPAI Foundation gGmbH, Heilbronn, Germany
Stereo Vision-Based Fall Prediction and Detection using Human Pose Estimation on the AMD Kria K26 SOM
Jun 10, 2026Shreyas Narasimhiah Ramesh, P. D. Rathika, Mahasweta Sarkar +3Fall Detection On-Device Inference
Electrical and Computer Engineering, San Diego State University, 5500 Campanile Drive, San Diego, 92182-1309, California, USA · Department of Robotics and Automation Engineering, PSG College of Technology, Peelamedu, Coimbatore, 641004, Tamil Nadu, India · Psychology, San Diego State University, 5500 Campanile Drive, San Diego, 92182-4611, California, USA +1
Local Is Not a Sufficient Privacy Boundary: Governing OS-Integrated On-Device AI
Jun 8, 2026Jonghyun Chung, Sanket BadhePrivacy Auditing AI Accountability
Google
Fast and Robust On-Device Speaker Diarization: Relative Minimum Cluster Size for Stride-Accelerated Pipelines
Jun 7, 2026Fumiaki YamaguchiSpeaker Diarization On-Device Inference
Independent Researcher
OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision
Jun 5, 2026Pietro Bonazzi, Julian Moosmann, Ahmet Celik +2Event-Based Vision Wearable Sensing
Department of Information Technology and Electrical Engineering, ETH Zürich, 8092 Zürich, Switzerland
CFRNet: Cycle-Consistent Fixed-Point Training for Real-Time Blind Face Restoration on Consumer Embedded NPUs
Jun 5, 2026Fuchen Li, Xinyang Wang, Yahui Zhang +4Image Restoration On-Device Inference
University of Florida, Gainesville, FL, USA · University of Southampton, Southampton, UK · Chongqing University, Chongqing, China +2
TinyML-Driven Cybersecurity for Autonomous Spacecraft: Latency-Accuracy Analysis for SPARTA RF and Cyber Threat Detection
Jun 4, 2026Van Le, Trevor Tran, Tan LeAutonomous Cyber Defense Cybersecurity
Virginia Tech, Blacksburg, VA 24061, USA · Hampton University, Hampton, VA 23669, USA
BayaHAR: Lightweight Bayesian Few-Shot User Adaptation for On-Device Personalized Human Activity Recognition
Jun 3, 2026Maximilian Burzer, Till Riedel, Michael Beigl +1Human Activity Recognition Domain Adaptation
Karlsruhe Institute of Technology Karlsruhe, Germany · IPAI Foundation gGmbH Heilbronn, Germany
Feasibility of Time-Domain DNN-Based Speech Enhancement on Embedded FPGA for Hearing Aid
Jun 2, 2026Feyisayo Olalere, Umut Altin, Kiki van der Heijden +1Efficient Neural Network Inference Speech Processing
Radboud University, Donders Institute for Brain, Cognition, and Behaviour, The Netherlands · Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, USA.
ArrythML: An Autoencoder-Based TinyML Approach for On-Device Arrhythmia Detection on Resource-Constrained Embedded Systems
Jun 1, 2026Nagarajan S, Kurian PolachanElectrocardiogram Classification Reconstruction-Based Anomaly Detection
International Institute of Information Technology, Bangalore, India
Exploiting In-Sensor Computing for Energy-Efficient Earth Observation
May 31, 2026Luigi Capogrosso, Pietro Bonazzi, Loris Hoxhaj +1Remote Sensing Image Understanding Energy-Efficient ML
Interdisciplinary Transformation University of Austria · ETH Zurich · University of Verona
On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making
May 29, 2026Joonhee Lee, Hyunseung Shin, Hyunmi Kim +2Efficient Inference Large Language Model-Based Robot Planning
School of Integrated Technology, College of Computing, Yonsei University · Department of Hyperscale AI SoC Research Section, ETRI · 3EECS - Electrical and Computer Engineering, University of Michigan
BitTP: The Lightweight Trajectory Prediction Model with BitLLM for Edge-Devices
May 28, 2026Mincheol Kang, Hyunjin Lim, Bomin Kang +1Multi-Agent Trajectory Prediction Weight-Only Quantization
1KAIST, Republic of Korea · 2DGIST, Republic of Korea
Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models
May 28, 2026Kai Bian, Xucheng Guo, Bin Chen +4Medical Imaging Efficient VLM Inference
The University of Auckland, New Zealand · Shandong University, China · The University of Melbourne, Australia
BlazeEdit: Generalist Image Editing on Mobile Devices with Image-to-Image Diffusion Models
May 27, 2026Fei Deng, Yanwu Xu, Zhipeng Bao +4Diffusion Model Acceleration Diffusion-Based Image Editing
Google
MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration
May 26, 2026Runxi Huang, Liyu Zhang, Shengzhong Liu +1Mobile GUI Automation Efficient VLM Inference
Hong Kong University of Science and Technology · Shanghai Jiao Tong University
AIGaitor: Privacy-preserving and cloud-free motion analysis for everyone, using edge computing
May 20, 2026Lauhitya Reddy, Trisha M. Kesar, Hyeokhyen KwonMarkerless Motion Capture Edge Computing
Department of Biomedical Informatics, Emory University, Atlanta, Georgia, United States of America · Department of Rehabilitation Medicine, Emory University, Atlanta, Georgia, United States of America · The Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia, United States of America
Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds
May 20, 2026Lakshani Manamperi, Disumi Pathirana, Thiwanka Pathirana +2Efficient Neural Network Inference Edge Computing
Department of Computer Science and Engineering, University of Moratuwa, Moratuwa, Sri Lanka.
On-Device Continual Learning with Dual-Stage Buffer and Dynamic Loss for Point-of-Care Pneumonia Diagnosis
May 19, 2026Danu KimMedical Diagnosis Continual Learning
Korea International School, Jeju Campus, Jeju-do 63644, Republic of Korea
Spiker-LL: An Energy-Efficient FPGA Accelerator Enabling Adaptive Local Learning in Spiking Neural Networks
May 18, 2026Alessio Caviglia, Filippo Marostica, Alessandro Savino +1On-Device Training Energy-Efficient ML
Control and Computer Engineering Department, Politecnico di Torino, Turin, Italy
Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices
May 15, 2026Alexander Gräfe, Ding Huo, Vincent de Bakker +3Efficient Transformer Inference Transformer Inference
1RWTH Aachen University · 2TU Darmstadt
ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices
May 15, 2026Kunpeng Du, Haizhen Xie, Sen Lu +11Diffusion Transformer Diffusion-Based Image Generation
Huawei Technologies
Bridging the Rural Healthcare Gap: A Cascaded Edge-Cloud Architecture for Automated Retinal Screening
May 13, 2026Nishi Doshi, Shrey ShahDiabetic Retinopathy Grading Edge-Cloud Computing
University of Southern California · Los Angeles, USA
Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning
May 13, 2026Marco Angioli, Kevin Johansson, Antonello Rosato +2Hyperdimensional Computing Contextual Bandits
Sapienza University of Rome · Orebro University · Linkoping University +1
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
May 11, 2026Chenyang Song, Weilin Zhao, Xu Han +3Sparse Mixture-of-Experts Mixture-of-Experts Inference
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China
Split CNN Inference on Networked Microcontrollers
May 10, 2026Junyu Lu, Shashwath Suresh, Hao Liu +2Efficient Neural Network Inference Edge Inference
Delft University of Technology, Delft, The Netherlands
HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learning
May 8, 2026Jiseok Youn, You Rim Choi, Goodsol Lee +3Meta-Learning OOD Generalization
Department of ECE and INMC, Seoul National University, Seoul, South Korea · Graduate School of Data Science, Seoul National University, Seoul, South Korea · Department of Computer Science, University of Colorado Boulder, CO, USA
On-Orbit Real-Time Wildfire Detection Under On-Board Constraints
May 7, 2026Matthias Rötzer, Veronika Pörtge, Martin Ickerott +6Efficient Neural Network Inference Wildfire Detection
OroraTech GmbH, St.-Martin-Str. 112, 81669 Munich, Germany · Ludwig-Maximilians-Universität München, Munich, Germany

Jun 15, 2026, cs.ARJ/K move · Enter open · S save

Embedded Arena: Iterative Optimization via Hardware Feedback

Zhihan Zhang, Alexander Le Metzger, Jiuyang Lyu, Chun-Cheng Chang+9

University of Washington · University of California San Diego · Northeastern University

Large Language Model-Guided Optimization Tiny ML Hardware-Software Co-Design Edge ML Neural Network Compression On-Device Inference

Open paper AskPDF arXiv

Embedded devices from wildlife monitoring stations to clinical wearables require local AI inference due to latency, communication, or privacy constraints. Optimizing models for heterogeneous microcontrollers (MCUs) requires simultaneously satisfying hard physical constraints on memory, power, and temperature while preserving accuracy, a multidimensional optimization that is today performed manually by experts. We ask whether an LLM agent can autonomously navigate this complex, multi-turn pipeline guided by real hardware feedback, and introduce a hardware-in-the-loop agent arena in which the agent iteratively refines both model and firmware -- compiling, flashing, and measuring on real hardware -- to enable closed-loop optimization. Frontier models, including Claude Opus 4.7 and Gemini 3.1 Pro, fail entirely without hardware feedback (0% deployment success), whereas our hardware-in-the-loop formulation achieves the first successful deployment within three iterations and can surpass human expert results within seven. This agentic co-optimization achieves 250x compression for vision models with <3.3% accuracy loss and 400x for audio with <6% Feature Error Rate loss, enabling battery-free operation on a commercial MCU via solar harvesting. We demonstrate practical impact in two real-world systems: an elk-detection camera trap (96.7% accuracy) and a phonetic-transcription wearable (8.44% FER) for child development research.

Momentum

Related topics

Latest papers 117