Energy-Aware Open-Vocabulary Detection Scheduling for Low-Power Robotic Autonomy

Abdul Basit; Muhammad Shafique

cs.RONov 5, 2026

Energy-Aware Open-Vocabulary Detection Scheduling for Low-Power Robotic Autonomy

Authors: Abdul Basit, Muhammad Shafique

Organizations: eBRAIN Lab, New York University Abu Dhabi, Abu Dhabi, UAE

Abstract

Open-vocabulary object detection (OVD) lets robots search for run-time object queries, but fixed-rate inference spends embedded compute on low-value frames during idle motion, map revisits, or stable viewpoints. This paper presents an energy-aware OVD scheduler for low-power mobile robots. The scheduler uses odometry, map change, and detector confidence to switch among low-, mid-, and high-power YOLO-World-S modes on a Jetson Orin Nano. We evaluate the energy-quality trade-off on an AgileX LIMO using 40 real navigation trials: two target objects, five placements per target-policy pair, and four perception policies. Across bottle and cup trials, the scheduler records 8/10 mission success compared with 7/10 for each fixed-rate baseline, while reducing mean Jetson input power (a Jetson-side power-draw metric) by 5.2% relative to an 8 Hz high-rate baseline and lowering achieved OVD call rate by 34.0%. It does not minimize mission-window Jetson input energy because longer missions can erase instantaneous power savings. A separate 60 s stationary microbenchmark isolates the perception workload: the scheduler lowers mean power from 8.045 W to 7.301 W and energy from 484.7 J to 438.1 J relative to the 8 Hz baseline. These results support robot-state-aware OVD scheduling as a promising power-management mechanism with competitive task reliability, while showing that mission-window Jetson input energy must be read together with task progress and runtime.

Figures & tables

Work	OVD	Robot	Adapt.	$R/E$	Task
OWL-ViT/GLIP/GDINO [ 2 , 3 , 4 ]	✓	$\times$	$\times$	$\times$	$\times$
Detic/YOLO-World [ 5 , 6 ]	✓	$\times$	$\times$	$\blacktriangle$	$\times$
VLMaps/LM-Nav/ROVER [ 7 , 10 , 11 ]	$\blacktriangle$	✓	$\times$	$\times$	$\blacktriangle$
OV scene graphs [ 8 , 9 ]	✓	$\blacktriangle$	$\times$	$\times$	$\blacktriangle$
Active/NBV perception [ 12 , 13 ]	$\times$	✓	✓	$\blacktriangle$	✓
NoScope/Chameleon [ 14 , 15 ]	$\times$	$\times$	✓	$\blacktriangle$	$\times$

TABLE I: Qualitative coverage of representative related work.

Fig. 1: Proposed energy-aware OVD runtime. Robot sensing and mapping produce motion, map-change, confidence, and target-distance signals. The tri-state scheduler maps these signals to LOW/MID/HIGH detector modes, while the logging pipeline records Jetson input power, mission-window energy, inference count, success, TTD, and mode occupancy.

Parameter	Value	Purpose
$r_{s}$	20 Hz	Scheduler update rate
$f_{L}/f_{M}/f_{H}$	1.5/4/8 Hz	Requested OVD rates
$\ell_{\Omega}$	4.0 m	Local grid-window side
$v_{\mathrm{idle}}$	0.03 m/s	Idle-motion threshold
$\tau_{H}$	0.02	Map-change threshold
$T_{\mathrm{low}}$	2.0 s	Stable interval before LOW

TABLE II: Scheduler parameters used in the reported experiments.

Fig. 2: Real-robot arena with fixed LIMO start pose and bottle/cup target placements reused across policies.

Policy	Succ.	$\bar{P}_{J}$	$E_{\mathrm{m}}$	Run	TTD succ	Hz	L/M/H
		(W)	(J)	(s)	(s)		(%)
Base-8Hz	7/10	9.35	866.9	92.8	25.9	4.74	0/0/100
Base-5Hz	7/10	8.92	1019.8	114.2	53.6	3.39	0/100/0
Base-2Hz	7/10	8.38	1155.0	140.4	0.9	1.48	100/0/0
Proposed	8/10	8.86	1114.0	125.9	10.9	3.12	25/60/15

TABLE III: Real-robot navigation summary over bottle and cup trials (ten trials per policy). low-, mid-, and high-power modes.

Fig. 3: Policy dashboard over the 40 navigation trials. The proposed scheduler has the highest aggregate success count and lower mean Jetson input power than Base-8Hz, while Jetson input energy is dominated by runtime.

Fig. 4: Mission runtime versus Jetson input energy for all 40 trials. Circles are successes, crosses failures, and open diamonds condition means; dashed lines show constant-power contours.

Fig. 5: Compact proposed-scheduler trace from the 60 s scripted stationary run. Colored regions show state, orange ticks show detector calls, the blue curve is Jetson input power, and the red dashed line is mean power.

Fig. 6: Representative adaptive navigation trace showing Jetson input power/energy, scheduler state and detector calls, detector confidence with the 0.25 threshold, and path length.

Fig. 7: Qualitative video-frame progression for representative bottle and cup searches. Blue R marks the robot and red T marks the target; LOW/MID/HIGH labels are qualitative stage estimates, not synchronized scheduler labels.

Policy	$\bar{P}_{J}$	$E$	Hz	#Inf.	L/M/H
	(W)	(J)			(%)
Base-8Hz	8.045	484.7	6.67	402	0/0/100
Base-5Hz	7.578	456.6	4.40	265	0/100/0
Base-2Hz	6.847	410.8	1.90	114	100/0/0
Proposed	7.301	438.1	3.37	202	50/38/13

TABLE IV: Stationary microbenchmark over the power-integration window.

Fig. 8: Object-level success and state occupancy, mapped to LOW/MID/HIGH power modes. In the top plot, solid bars denote bottle and hatched bars denote cup.

Power-Efficient Inference

See all 17 papers

May 7, 2026

XiYOLO: Energy-Aware Object Detection via Iterative Architecture Search and Scaling

Object detection on heterogeneous edge devices must satisfy strict energy, latency, and memory constraints while still providing reliable perception for downstream autonomy. Existing energy-aware NAS methods often target limited deployment settings, while real energy remains difficult to optimize because it is highly device-dependent and costly to measure. We address these challenges with an energy-adaptive framework that combines an energy-aware XiResOFA search space, a two-stage energy estimator, and iterative search to identify a single energy-efficient base architecture. We then apply compound scaling to transform this base design into the XiYOLO family across deployment budgets, enabling interpretable accuracy-energy tradeoffs under sparse hardware measurements. Experiments on PascalVOC, COCO, and real-device deployment show that XiYOLO achieves a stronger energy-accuracy tradeoff than YOLO baselines. On PascalVOC, the medium XiYOLO model reaches 86.15 mAP50 while reducing energy relative to YOLOv12m by 20.6% on GPU and 35.9% on NPU. On COCO, XiYOLO reduces energy relative to YOLOv12 by up to 53.7% on GPU and 51.6% on NPU at the small scale. The proposed two-stage estimator also improves sample efficiency over a joint predictor under few-shot adaptation with only 2-20 target-device samples.

Tony Tran, Richie R. Suganda, Bin Hu

Department of Research Computing University of Houston Houston, TX 77479 · Department of Electrical and Computer Engineering University of Houston Houston, TX 77479 · Department of Engineering Technology University of Houston Houston, TX 77479

Energy-Efficient ML Edge Inference Efficient Object Detection Hardware-Aware NAS Object Detection Neural Architecture Search

Ask PDF

May 10, 2026

ORICF -- Open Robotics Inference and Control Framework

Recent advances in artificial intelligence (AI) have enabled effective perception and language models for robots, but their deployment remains computationally expensive, increasing latency and energy use. This work presents the Open Robotics Inference and Control Framework (ORICF), a modular, declarative, and model-agnostic platform for composing multimodal robotic inference pipelines. ORICF integrates input/output (I/O) adapters, pluggable inference back ends, and post-processing logic, while lightweight YAML specifications allow models, hardware targets, and data channels to be changed without code modification. The framework also supports edge offloading, i.e., executing inference on nearby external computers instead of onboard the robot. ORICF is evaluated on a mobile robot that answers spoken queries about people detected in its camera stream by combining automatic speech recognition (ASR), a large language model (LLM), and a convolutional neural network (CNN) detector through Robot Operating System 2 (ROS2). Compared with onboard execution, ORICF-based edge deployment reduces robot-side compute utilization by up to 83.16% and estimated energy consumption by 65.8%, while preserving modularity and reproducibility.

Andrés Meseguer Valenzuela, Luís Miguel Bartolín Arnau

Instituto Tecnológico de Informática (ITI), Paterna, Spain

Efficient Multimodal Inference Edge Inference

Ask PDF

Mar 30, 2026

BlankSkip: Early-exit Object Detection onboard Nano-drones

Deploying tiny computer vision Deep Neural Networks (DNNs) on-board nano-sized drones is key for achieving autonomy, but is complicated by the extremely tight constraints of their computational platforms (approximately 10 MiB memory, 1 W power budget). Early-exit adaptive DNNs that dial down the computational effort for "easy-to-process" input frames represent a promising way to reduce the average inference latency. However, while this approach is extensively studied for classification, its application to dense tasks like object detection (OD) is not straightforward. In this paper, we propose BlankSkip, an adaptive network for on-device OD that leverages a simple auxiliary classification task for early exit, i.e., identifying frames with no objects of interest. With experiments using a real-world nano-drone platform, the Bitcraze Crazyflie 2.1, we achieve up to 24% average throughput improvement with a limited 0.015 mean Average Precision (mAP) drop compared to a static MobileNet-SSD detector, on a state-of-the-art nano-drones OD dataset.

Carlo Marra, Beatrice Alessandra Motetti, Alessio Burrello +3

†Politecnico di Torino, Turin, Italy

Efficient Object Detection Object Detection Edge Inference Aerial Object Detection

Ask PDF

Abstract

Power-Efficient Inference

Explore similar work

XiYOLO: Energy-Aware Object Detection via Iterative Architecture Search and Scaling

ORICF -- Open Robotics Inference and Control Framework

BlankSkip: Early-exit Object Detection onboard Nano-drones