Power-Efficient Inference

Follow this topic Ask about this topic

Latest papers

14 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference
Oct 6, 2026Zou Qingyun, Bin Gao, Wenju Zhao +3Energy-Efficient ML VLM Quantization
National University of Singapore · Huazhong University of Science and Technology
Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators
Jul 24, 2026Afzal Ahmad, Gaoyu Mao, Shoubo Hu +4Structured Sparsity FPGA-Based Neural Network Acceleration
The Hong Kong University of Science and Technology · Huawei Noah’s Ark Lab · The Hong Kong University of Science and Technology (Guangzhou)
Mitigating Compiler Fusion-Induced Power Bursts in Mobile NPU Inference as the Battery Depletes
Jul 17, 2026Ryoga Yuzawa, Masayoshi TomizukaEnergy-Efficient ML Power-Efficient Inference
UC Berkeley · Sony · Visiting researcher at University of California, Berkeley.
EnerInfer: Energy-Aware On-Device LLM Inference
Jun 22, 2026Bohua Zou, Nian Liu, Binqi Sun +6On-Device Language Model Inference LLM Inference Acceleration
Technical University of Munich · Shanghai Jiao Tong University · Huawei Hilbert Research Center (Dresden) +1
Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study
Jun 19, 2026Alfarizy Alfarizy, Hung Truong Thanh Nguyen, René Richard +2LLM Inference Efficiency On-Device Language Model Inference
Analytics Everywhere Lab, University of New Brunswick, Fredericton, NB, Canada · TSAI Lab, University of New Brunswick, Fredericton, NB, Canada
TFZ-Tree: An Ultra-Lightweight Waveform Classification Framework for Resource-Constrained Devices
May 15, 2026Hao Wang, Kuang Zhang, Yonggang Chi +3Efficient Inference Internet of Things
Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference
May 1, 2026Yuxuan Gao, Megan Wang, Yi Ling YuLLM Evaluation Energy-Efficient ML
University of Pennsylvania · OpenMesh AI · Columbia University
LLM-Guided Runtime Parameter Optimization for Energy-Efficient Model Inference
Apr 29, 2026Katelyn Crumpacker, Dimitrios NikolopoulosLLM Inference Efficiency Energy-Efficient ML
Virginia Polytechnic and State University
KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
Apr 17, 2026Yichao Yuan, Mosharaf Chowdhury, Nishil TalatiEfficient Inference LLM Inference Serving
University of Illinois, Urbana Champaign Urbana, IL, USA · University of Michigan Ann Arbor, MI, USA
CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiency
Apr 8, 2026Timothy K Johnsen, Marco LevoratoPower-Efficient Inference Adaptive Computation
Computational Science joint program with San Diego State University and The University of California, Irvine in CA, USA · Computer Science department at the University of California, Irvine in CA, USA
Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models
Mar 24, 2026Mohammad Saleh Vahdatpour, Yanqing ZhangPower-Efficient Inference Energy-Efficient ML
Department of Computer Science Georgia State University Atlanta, GA, USA
Covariance-Guided Resource Adaptive Learning for Efficient Edge Inference
Mar 15, 2026Ahmad N. L. Nabhaan, Zaki Sukma, Rakandhiya D. Rachmanto +4Power-Efficient Inference Inference-Time Optimization
Universitas Amikom Yogyakarta, Department of Informatics · University of Georgia, School of Computing · Universitas Gadjah Mada, Department of Computer Science and Electronics +1
Hardware Efficient Approximate Convolution with Tunable Error Tolerance for CNNs
Mar 10, 2026Vishal Shashidhar, Anupam Kumari, Roy P PailyEfficient Neural Network Inference Neural Network Hardware Acceleration
Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati
Inference Energy and Latency in AI-Mediated Education: A Learning-per-Watt Analysis of Edge and Cloud Models
Mar 4, 2026Kushal KhemaniPower-Efficient Inference On-Device Language Model Inference
Billabong High International School, Hadapsar, Pune, 411028, Maharashtra, India.

Oct 6, 2026J/K move · Enter open · S save

ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference

Zou Qingyun, Bin Gao, Wenju Zhao, Weng-Fai Wong+2

National University of Singapore · Huazhong University of Science and Technology

Energy-Efficient ML VLM Quantization Efficient VLA Model Inference Power-Efficient Inference

Open paper AskPDF arXiv

Vision-language-action (VLA) policies repeatedly invoke inference to control robots, making graphics processing unit (GPU) energy a recurring cost of task execution. Reducing energy per inference call, however, may not reduce energy per successful task if numerical errors increase failures or slower inference prolongs execution. We therefore target GPU energy per successful task while preserving task success and keeping the inference-latency increase within 10%. Our approach builds on two observations: quantization sensitivity varies across action classes, model layers, and weights versus activations; and numerical precision changes the workload, shifting favorable GPU operating points. We introduce ActTune, an action-aware framework that connects layer-wise precision allocation with workload-dependent GPU operating-point selection over requested frequency--power-cap pairs. A lightweight decision tree learns its splits and leaf precision configurations directly from configuration action errors, then selects precision before each policy call. The controller forecasts the next workload and applies the selected GPU operating point asynchronously using a lookup table calibrated under a latency budget. A shared resident quantized weight bank enables configuration switching without weight reconstruction or additional policy evaluations. On LIBERO, a benchmark for lifelong robot learning, ActTune improves mean task success by up to 2.3% relative to state of the art. Relative to the original BF16 implementations, it delivers up to

2.02\times

faster inference and, with GPU operating-point adaptation, reduces energy per successful task by up to 76.8%.