Albireo: Adaptive, Energy-Efficient Inference Framework for Video Object Detection on the Edge

Amir Taherin; José Cano; Bin Ren; Yanzhi Wang; David Kaeli

cs.CVAug 29, 2026

Albireo: Adaptive, Energy-Efficient Inference Framework for Video Object Detection on the Edge

Authors: Amir Taherin, José Cano, Bin Ren, Yanzhi Wang, David Kaeli

Organizations: Northeastern University Boston, MA, USA · University of Glasgow Glasgow, UK · William & Mary Williamsburg, VA, USA

Abstract

Video object detection on edge devices runs computationally expensive detectors over long frame streams, causing high energy consumption and sustained GPU utilization. Although consecutive frames are highly redundant, naive frame skipping is content-blind: it skips during critical moments such as object entry, occlusion recovery, and abrupt motion, degrading detection quality. We present Albireo, a detector-agnostic, codec-free adaptive inference framework that wraps off-the-shelf detectors and decides when detector invocation can be safely skipped based on scene content and per-object temporal state, requiring no detector modification or retraining. Albireo maintains a 10-dimensional Kalman filter (KF) per active object and invokes the detector only when prediction uncertainty exceeds a threshold; on skipped frames, boxes are predicted from the KF state at near-zero GPU cost. A KF-based rescue mechanism preserves confirmed objects through brief detector misses to prevent output fragmentation, while a lightweight empty-scene screen avoids detector calls on objectless frames. We evaluate Albireo on the BDD100K MOT validation split with three architecturally distinct detectors (YOLO11x, YOLO26x, RF-DETR-Large) on two NVIDIA Jetson platforms (AGX Thor, AGX Orin). Across all configurations, Albireo keeps AP@50 within +/-1.2 pp of per-frame inference while reducing total energy by 12.1-17.6%. On YOLO26x, it improves AP@50 by +0.8 pp while reducing energy by 17.6% (Thor) and 14.4% (Orin) and per-frame energy-delay product by 24.9% and 26.1%, respectively. Thus, the default operating point improves accuracy, energy, and latency together. In contrast, FixedSkip-2, a fixed-interval baseline with a 50% skip rate, loses 8.6 pp AP@50. Source code, evaluation pipeline, and per-clip results are available at https://github.com/amirtaherin/albireo

Figures & tables

Method	DA	NDR	EM	CF	EE	POA
DFF [ 25 ]	×	×	✓	✓	×	×
SDOF-Tracker [ 22 ]	×	✓	×	✓	×	×
Tran et al. [ 38 ]	✓	×	✓	×	×	×
Skip-Conv [ 20 ]	×	×	✓	✓	×	×
PASS [ 44 ]	×	×	✓	✓	✓	×
Statues [ 3 ]	✓	✓	×	✓	✓	×

TABLE I: Deployment properties of frame-skipping and adaptive inference methods for video object detection.

State	Name	Frame output / computation
E	ERD-Empty	ERD only; detector skipped
P	KF-Predict	Predicted boxes; detector skipped
E+I	ERD-triggered Infer.	ERD then detector
I	Inference	Detector, match, KF update
AI	Augmented Inference	Detector, match/update, rescue output

TABLE II: Frame states used by Albireo . Each frame is assigned exactly one state.

Panel of Figure 3 — Fig. 1: Frame-state transitions across consecutive frames. Edges show the events that move execution between ERD skipping, prediction-only, normal inference, and rescue-augmented inference states. AI follows the same next-frame routing as I; self-loops are omitted.

Plat.	System	AP@50	Energy	ms/fr	EDP	Skip
			(J)		(J $\cdot$ ms/fr)	(%)
Thor	Vanilla	0.609	811.5	50.9	41,305	0.0
Thor	Albireo	0.617	668.6	46.4	31,023	19.3
$\Delta$		+0.8 pp	$-$ 17.6%	$-$ 8.8%	$-$ 24.9%	—
Orin	Vanilla	0.609	994.0	111.5	110,831	0.0
Orin	Albireo	0.617	851.2	96.2	81,885	19.3

TABLE III: YOLO26x headline results on BDD100K MOT. Values are means over 200 validation clips.

Fig. 2: AP@50 versus per-frame energy-delay product (EDP) for YOLO26x on the BDD100K MOT validation split. Albireo ’s threshold sweep (

u=3\!\times\!10^{-4}

1\!\times\!10^{-3}

) forms the upper Pareto frontier among the evaluated systems in the

\mathrm{AP@50}\geq 0.50

operating region on both Jetson platforms.

Plat.	Detector	$\Delta$ AP@50	$\Delta$ mAP	$\Delta$ Energy	Skip
		(pp)	(pp)		(%)
Thor	YOLO11x	$-0.5$	$-4.7$	$-14.4\%$	21.3
Thor	YOLO26x	$+0.8$	$-3.7$	$-17.6\%$	19.3
Thor	RF-DETR-Large	$-1.1$	$-4.8$	$-16.3\%$	22.2
Orin	YOLO11x	$-0.5$	$-4.7$	$-16.4\%$	21.3
Orin	YOLO26x	$+0.8$	$-3.7$	$-14.4\%$	19.3

TABLE IV: Default Albireo relative to Vanilla across detectors and platforms. Values are changes from per-frame inference, averaged over 200 BDD100K MOT validation clips.

Fig. 3: Per-clip energy savings of default Albireo over Vanilla across detector families and Jetson platforms.

Board	Mode	Vanilla (J/clip)	Albireo (J/clip)	$\Delta$ Energy
Orin	15 W	1417	1166	$-17.7\%$
Orin	30 W	1094	917	$-16.2\%$
Orin	50 W	847	714	$-15.7\%$
Thor	70 W	920	776	$-15.7\%$
Thor	90 W	920	779	$-15.3\%$
Thor	120 W	779	654	$-16.1\%$

TABLE V: Per-clip energy under each nvpmodel power mode (YOLO26x, mean over 200 clips per mode). The AP@50 gain (

+0.8

pp) and skip rate (

19.3\%

) are identical in every mode.

Fig. 4: Frame-state distribution of Albireo across three detectors on the BDD100K MOT validation split (200 clips). Skip frames (E + P) account for

19

–

22\%

; the augmented-inference rate (AI) of

43

–

49\%

measures the rescue mechanism’s contribution.

Variant	KF	Lifecycle	AP@50	mAP	Skip	ms/fr
					(%)
Kill-only	8D CA	kill	0.578	0.358	21.5	41.8
+ERD	8D CA	kill + ERD	0.572	0.355	25.8	42.9
+10D KF	10D CA	kill + ERD	0.577	0.407	26.3	42.5
Albireo	10D CA	rescue + ERD	0.617	0.433	19.3	46.4

TABLE VI: Component ablation on BDD100K MOT using YOLO26x on Thor. Values are means over 200 clips.

Fig. 5: Mapped per-rail energy per clip, grouped as GPU, CPU, and IO/MEM, for Vanilla and Albireo across three detectors on Thor and Orin. IO/MEM denotes memory-related and non-GPU/non-CPU I/O rails exposed by platform telemetry. Grouped rails are used for trend analysis and do not necessarily sum to the top-level board energy reported in Table III .

Fig. 6: Per-clip skip-rate components versus AP@50 delta (Vanilla

\rightarrow

Albireo ). Across the full P-frame range, AP@50 deltas cluster around zero or positive; large negative deltas concentrate on high-E clips where the absolute ground-truth count is low and AP is more sensitive to individual missed detections.

Fig. 7: Per-clip power traces (Total, GPU, CPU, IO/MEM) for two representative BDD100K clips on Thor with YOLO26x. Left: dense urban traffic (

0\%

96\%

AI), where detector invocations dominate, and rescue preserves AP@50. Right: sparse highway (

69\%

4\%

AI), where long ERD-only stretches sharply reduce GPU power.

Edge Inference

Edge inference runs trained AI models on nearby devices such as phones, cameras, robots, and embedded computers under tight resource and latency limits.

See all 228 papers

May 7, 2026

XiYOLO: Energy-Aware Object Detection via Iterative Architecture Search and Scaling

Object detection on heterogeneous edge devices must satisfy strict energy, latency, and memory constraints while still providing reliable perception for downstream autonomy. Existing energy-aware NAS methods often target limited deployment settings, while real energy remains difficult to optimize because it is highly device-dependent and costly to measure. We address these challenges with an energy-adaptive framework that combines an energy-aware XiResOFA search space, a two-stage energy estimator, and iterative search to identify a single energy-efficient base architecture. We then apply compound scaling to transform this base design into the XiYOLO family across deployment budgets, enabling interpretable accuracy-energy tradeoffs under sparse hardware measurements. Experiments on PascalVOC, COCO, and real-device deployment show that XiYOLO achieves a stronger energy-accuracy tradeoff than YOLO baselines. On PascalVOC, the medium XiYOLO model reaches 86.15 mAP50 while reducing energy relative to YOLOv12m by 20.6% on GPU and 35.9% on NPU. On COCO, XiYOLO reduces energy relative to YOLOv12 by up to 53.7% on GPU and 51.6% on NPU at the small scale. The proposed two-stage estimator also improves sample efficiency over a joint predictor under few-shot adaptation with only 2-20 target-device samples.

Tony Tran, Richie R. Suganda, Bin Hu

Department of Research Computing University of Houston Houston, TX 77479 · Department of Electrical and Computer Engineering University of Houston Houston, TX 77479 · Department of Engineering Technology University of Houston Houston, TX 77479

Energy-Efficient ML Edge Inference Efficient Object Detection Hardware-Aware NAS Object Detection Neural Architecture Search

Ask PDF

Jun 1, 2026

Tiny Collaborative Inference for Occlusion-Robust Object Detection

Edge AI nodes for search and rescue are increasingly expected to run computer vision locally, yet ultra-low-end hardware imposes hard constraints on memory, compute, and inter-device communication. This work addresses occlusion-robust object detection on devices with less than 1 MB SRAM by combining an MCUNet backbone, a YOLOv2 detection head, and Lite quantisation. Two collaborative inference strategies are evaluated: feature-level fusion, concatenating intermediate feature maps, and decision-level fusion via Weighted Boxes Fusion (WBF). WBF outperforms feature-level fusion under all tested occlusion conditions, yielding gains of up to +0.2736 mAP in asymmetric scenarios. Extending fusion to three views improves accuracy further (up to +0.3827 mAP) at modest communication overhead (~1.3 KB per exchange). Hardware experiments progress from a host-assisted USB-relay baseline to a Wi-Fi peer-to-peer deployment on two Coral Dev Board Micro units, where WBF executes on-device with negligible communication energy relative to inference. In a 301.9 s autonomous session of 108 frames, fused output is produced on 61 frames versus 47 for a single board - a coverage gain of +29.8%. A decentralised federated learning feasibility note is included but not treated as a primary result, as performance remains limited under non-iid data. The results support decision-level fusion as a viable option for improving occlusion robustness in small-scale edge object detection, including host-free multi-board operation on ultra-low-end hardware.

Chieh-Tung Cheng, Mustafa Aslanov, Eiman Kanjo

Imperial College London, United Kingdom · Nottingham Trent University, United Kingdom

Edge Inference Efficient Object Detection Object Detection

Ask PDF

May 14, 2026

MR2-ByteTrack: CNN and Transformer-based Video Object Detection for AI-augmented Embedded Vision Sensor Nodes

Modern smart vision sensors need on-device intelligence to process video streams, as cloud computing is often impractical due to bandwidth, latency, and privacy constraints. However, these sensory systems typically rely on ultra-low-power microcontrollers (MCUs) with limited memory and compute, making conventional video object detection methods, which require feature storage or multi-frame buffering, unfeasible. To address this challenge, we introduce Multi-Resolution Rescored ByteTrack (MR2-ByteTrack), a Video Object Detection (VOD) method tailored for MCU-based embedded vision nodes. MR2-ByteTrack reduces computational cost by alternating between full- and low-resolution inference, while linking detections across frames via ByteTrack and correcting misclassifications through the Rescore algorithm, which applies probability union rules to aggregate detection confidence scores across frames. We apply our approach to both a CNN-based detector and a Transformer-based model, demonstrating its generality across architectures with fundamentally different spatial processing. Experiments on ImageNetVID demonstrate that MR2-ByteTrack maintains accuracy, achieving mAP scores of up to 49.0 for the CNN-based models and 48.7 for the Transformer, while reducing multiply-accumulate operations by as much as 53% for the CNNs and 32% for the Transformer. When deployed on GAP9, an ultra-low-power RISC-V multicore MCU, our method yields up to 55% energy savings compared to processing only full-resolution images, enabling the first real-time Transformer-based VOD on an MCU-class embedded vision node. Code available at https://github.com/Bomps4/Multi_Resolution_Rescored_ByteTrack/tree/IEEE_Access

Luca Bompani, Manuele Rusci, Luca Benini +2

Electrical, Electronic and Information Engineering (DEI), University of Bologna, Italy. · Department of Electrical Engineering (ESAT), KU Leuven, Belgium. · Integrated Systems Laboratory (IIS), ETH Zürich, Switzerland. +1

Edge Inference Efficient Object Detection Video Object Detection Real-Time Object Detection

Ask PDF

Abstract

Edge Inference

Explore similar work

XiYOLO: Energy-Aware Object Detection via Iterative Architecture Search and Scaling

Tiny Collaborative Inference for Occlusion-Robust Object Detection

MR2-ByteTrack: CNN and Transformer-based Video Object Detection for AI-augmented Embedded Vision Sensor Nodes