Coding What Matters: A Semantic-Aware Memory Interface for Energy-Efficient Perception in Autonomous Vehicles

Haohua Que; Handong Yao

cs.ROAug 29, 2026

Coding What Matters: A Semantic-Aware Memory Interface for Energy-Efficient Perception in Autonomous Vehicles

Authors: Haohua Que, Handong Yao

Organizations: College of Engineering, University of Georgia, Athens, GA 30602 USA

Abstract

Autonomous vehicles stream high-resolution surround-camera frames into memory before perception runs. This sensor-to-memory path consumes energy when cells store ones and adjacent bytes toggle on the data bus, so its cost follows bit-1 density and switching activity rather than pixel semantics. We present MotiMem-Omega, a semantic-aware memory-interface coder that lowers this cost while preserving perception predictions. Its semantic importance field protects traffic participants, especially vulnerable road users, while assigning lower fidelity to sky and empty background. Cross-dataset bit-sensitivity sweeps determine class weights, with a safety floor for pedestrians, cyclists, and motorcyclists. Each image block then selects a precision tier by minimizing a joint energy-distortion cost. When ego pose is available, a motion-compensated prior carries protected regions between frames. We estimate interface-energy reduction from the two measured proxies using a coefficient-swept memory-energy model. Across 29 detectors on 12 driving datasets, 5 occupancy models, and 5 segmentation networks, MotiMem-Omega retains about 90% of detection mean average precision, 91% of vulnerable-road-user recall, over 98% of occupancy accuracy, and the strongest segmentation retention among energy-reducing methods. It reduces front-camera bit-1 density by 52%, corresponding to a modeled memory-interface energy reduction near 36%, with a lower end of 27% under the literature coefficient sweep. It also gives higher retention than the baseline and energy-matched truncation at the same or lower bit-1 density, whereas image codecs preserve accuracy without reducing memory-interface energy.

Figures & tables

Fig. 1: Detection accuracy versus memory-interface energy across the 12 driving datasets. The vertical axis is mAP retention (each of the 29 detectors measured against its own original-frame detections, then averaged over the 12 datasets); the horizontal axis is normalized bit-1 density $\rho_{1}$ , our memory-interface energy proxy, on the same 12-dataset front-camera basis as Table II . MotiMem- $\Omega$ gives the strongest accuracy–energy trade-off among the energy-reducing methods, retaining about $90\%$ of mAP at $\rho_{1}\!\approx\!0.48$ , where energy-matched truncation (uniform 4-bit, global $k{=}4$ ) and the baseline give up $7$ to $10$ points of accuracy at the same or higher energy, while image codecs lie at $\rho_{1}\!\geq\!1$ with no memory saving.

Fig. 2: Overview of the MotiMem- $\Omega$ encoding pipeline: $t{-}1$ detections $\rightarrow$ semantic importance field $\rightarrow$ tiered hierarchical coding ( $+$ sky prior) $\rightarrow$ DRAM/SRAM frame buffer $\rightarrow$ decode $\rightarrow$ perception; the field is projected $t{-}1\!\rightarrow\!t$ by ego-compensated motion.

Symbol	Meaning
$\mathbf{I}_{t},\tilde{\mathbf{I}}_{t}$	original / decoded frame at time $t$
$\Phi,\Psi$	MotiMem- $\Omega$ encoder / decoder
$\mathcal{D}_{t-1}$	detections on the previous frame
$\rho_{1},\alpha$	bit-1 density / switching activity
$E(\mathbf{b})$	memory-interface energy proxy, Eq. ( 2 )
$\rho(\cdot),\varepsilon$	retention / tolerated accuracy loss

TABLE I: Primary notation.

	Per-dataset mAP retention [ 77 ]													All-class [ 77 ]			VRU				Energy [ 6 ]		Image qual.
Method	nuSc [ 14 ]	A2D2 [ 21 ]	AV2 [ 16 ]	BDD [ 20 ]	CADC [ 19 ]	CaR [ 22 ]	JODD [ 25 ]	KUAS [ 24 ]	ONCE [ 17 ]	PnSt [ 18 ]	USyd [ 23 ]	Way [ 15 ]	Avg	R	A50	A75	R	A50	A75	mAP	$\rho_{1}$	$\alpha$	SSIM [ 78 ]	PSNR	LPIPS [ 79 ]
Original	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.000	$\infty$	.000
$\Omega$	.90	.95	.89	.81	.82	.90	.94	.90	.92	.93	.91	.91	.90	.91	.92	.92	.91	.93	.93	.91	.48	.52	.953	25.0	.168
base [ 4 ]	.83	.91	.87	.66	.81	.79	.88	.80	.87	.88	.82	.83	.83	.84	.87	.86	.79	.84	.83	.79	.63	.35	.940	30.5	.242
global-k4	.77	.89	.84	.62	.79	.79	.85	.74	.84	.84	.78	.80	.80	.81	.86	.84	.74	.82	.80	.74	.58	.28	.931	29.8	.263
uniform-4b	.77	.89	.84	.62	.79	.79	.85	.74	.84	.84	.78	.80	.80	.81	.86	.84	.74	.82	.80	.74	.48	.26	.929	29.2	.264

TABLE II: Comprehensive detection comparison: every encoding method (rows) on every metric, averaged over the 29 detectors. Rows:

\Omega

= MotiMem-

\Omega

, base = conference MotiMem [ 4 ] , then the naive truncations (global-k4, uniform-4b) and the image codecs JPEG [ 36 ] , WebP [ 38 ] , PNG [ 39 ] , JPEG2000 [ 37 ] , AVIF [ 40 ] , HEIF [ 41 ] . Left block: mAP retention per dataset

+

average; right blocks: aggregate all-class and VRU recall (R), AP@IoU

0.5/0.75

(A50/A75), COCO mAP, energy proxies

\rho_{1},\alpha

(both normalized so the original frame is

1.00

), and SSIM/PSNR/LPIPS. All figures are retention vs. the original-frame predictions (

\uparrow

;

\rho_{1},\alpha

, LPIPS

\downarrow

). MotiMem-

\Omega

(bold) gives the highest average mAP among the energy-reducing methods while using low human-visible fidelity (low PSNR), indicating that the bit budget is concentrated on task-relevant content; codecs match retention only at

\rho_{1}{\approx}1

			All classes				VRU				Energy
Dataset	Location		R	A50	A75	mAP	R	A50	A75	mAP	$\rho_{1}$	$\alpha$
nuSc [ 14 ]	US, Boston	$\Omega$	.92	.92	.92	.90	.91	.95	.94	.93	.57	.50
		base	.84	.88	.86	.83	.76	.82	.81	.77	.63	.38
		nv	.79	.84	.82	.77	.70	.80	.79	.73	.58	.29
A2D2 [ 21 ]	Germany	$\Omega$	.96	.96	.96	.95	.93	.95	.95	.94	.46	.51
		base	.92	.94	.93	.91	.85	.90	.89	.87	.63	.38

TABLE III: Per-dataset detection retention and energy for the three key methods (12 datasets, each averaged over 29 detectors). base = MotiMem [ 4 ] , nv = naive global-k4; R = recall, A50/A75 = AP@IoU

0.5/0.75

(middle block: VRU);

\rho_{1}

\alpha

are the energy proxies, both normalized to the original frame (per-dataset values scaled to the Table VI means). The 12 sets span five continents and day/night/snow/desert.

\Omega

(bold) has the strongest overall retention profile among the energy-reducing methods and the lowest

\rho_{1}

, with one near-tie (CADC A75,

.85

.86

Fig. 3: Cross-dataset operating-point sweep used to choose the fixed configuration ( $12$ datasets $\times$ $29$ detectors). Each panel varies one design variable and plots retention against the energy proxies; stars mark the selected points. (a) Object precision $k_{\mathrm{obj}}$ : all usable values $k{\leq}7$ are near-lossless ( $\pm 1$ -bit flip, flat ${\sim}98\%$ retention), and $k{=}7$ is both the maximum precision and an odd- $k$ low- $\rho_{1}$ point. (b) Background precision $k_{\mathrm{bg}}$ : retention saturates after $k{=}5$ and $\rho_{1}$ jumps at $k{=}6$ , so $k_{\mathrm{bg}}{=}5$ . (c) Sky precision $k_{\mathrm{sky}}$ : sky moves from untruncated $k{=}5$ to $k_{\mathrm{sky}}{=}2$ , returning energy at a small accuracy cost. (d) Block-mode weight $\lambda$ : the combined energy is minimized at $\lambda{=}1$ . (e) RoI dilation: VRU recall saturates at $20$ px.

Fig. 4: Qualitative detection over the 12 datasets (rows) $\times$ seven encoding methods (columns), with native YOLO26 boxes on each decoded frame. MotiMem- $\Omega$ (red) keeps detections that the conference base MotiMem [ 4 ] (orange) and the energy-matched naive truncations (global-k4, uniform-4b) drop: the base has no motion-compensated closed loop, so an object that moved out of its $t{-}1$ RoI is truncated and lost, whereas $\Omega$ ’s tiered allocation keeps it. Image codecs alter the bitstream but not the predictions. Snow (CADC), night (BDD), and clear scenes are covered.

Architecture	$\Omega$	base	g-k4	unif	JPEG
$\rho_{1}$	.57	.63	.58	.48	.96
DeepLabV3 [ 31 ]	.934	.899	.885	.886	.904
FCN [ 32 ]	.889	.861	.827	.827	.862
LRASPP [ 33 ]	.836	.789	.769	.770	.817
SegFormer [ 34 ]	.715	.626	.625	.625	.713
Mask2Former [ 35 ]	.758	.702	.701	.701	.754

TABLE IV: Segmentation mIoU retention (vs. original-frame map) under every encoding method, on five architectures (nuScenes; base = MotiMem [ 4 ] , g-k4 = global-

k

4, unif = uniform-4bit;

\rho_{1}

row = bit-1 density).

\Omega

(bold) leads on every architecture; JPEG only matches it at

\rho_{1}{\approx}1

(no saving). The lead also holds across 12 datasets (DeepLabV3

0.906

vs.

0.858

+4.8

pp; range

+1.2

+10.7

Fig. 5: Semantic segmentation under the seven encoding methods (columns) across the five architectures of Table IV (rows; DeepLabV3 / FCN / LRASPP are Pascal-VOC, SegFormer / Mask2Former are Cityscapes), on one nuScenes frame. Each cell blends the model’s seg map over the decoded frame; mislabelled pixels (class changed vs. the original-frame map) are highlighted in red , with the error-% (excluding the deliberately-truncated sky/background) printed. The decoded frames are near-identical by design (MotiMem preserves visual quality), so the visible signal is the downstream seg-map error, not the pixels. MotiMem- $\Omega$ (the second column) reproduces the original labelling with almost no error across all five models.

Model	Metric	$\Omega$	base	g-k4	unif	JPEG
	$\rho_{1}$	.59	.63	.59	.45	1.04
GaussRender [ 29 ]	mIoU	99.7	94.8	92.8	92.8	97.7
DAOcc [ 28 ]	mIoU	99.9	97.8	97.5	97.5	99.6
OPUS [ 26 ]	mIoU	99.6	96.8	95.7	95.6	99.1
GaussianFormer-2 [ 30 ]	mIoU	98.2	96.9	96.3	96.4	99.2
SparseOcc [ 27 ]	RayIoU	99.8	96.3	95.6	95.5	99.1

TABLE V: 3D occupancy retention (% of each model’s native mIoU/RayIoU vs. original) under every encoding method, on five SOTA models (base = MotiMem [ 4 ] , g-k4 = global-

k

4, unif = uniform-4bit;

\rho_{1}

row = bit-1 density).

\Omega

(bold) retains the most among energy-saving methods; base and naive lose

2

–

7

pp, while JPEG matches only at

\rho_{1}{>}1

(no saving).

Fig. 6: 3D occupancy on the five SOTA models (rows): BEV (top) $+$ 3D (bottom) under the original input vs. MotiMem- $\Omega$ (red), the conference base MotiMem [ 4 ] (orange), the non-semantic truncations, and JPEG; changed voxels are highlighted in light blue. Each column title gives the retention (% of the model’s native mIoU/RayIoU) and the bit-1 density $\rho_{1}$ . $\Omega$ alone is both high-retention ( $98.2$ – $99.9\%$ ) and low-energy ( $\rho_{1}{=}0.6$ ), whereas JPEG matches retention but at $\rho_{1}{>}1$ (no memory saving).

Task	Method	$\rho_{1}$	$\alpha$	$\Delta E$ (%)
Det/Seg	MotiMem- $\Omega$	0.480	0.516	35.5 [27.7, 43.3]
	naive global-k4	0.580	0.277	37.0 [26.1, 46.3]
	JPEG q50	0.957	1.493	$-10.4$ [ $-16.0$ , $-2.3$ ]
Occ	MotiMem- $\Omega$	0.593	0.526	30.2 [23.1, 37.0]
	naive global-k4	0.589	0.247	37.3 [26.0, 46.9]
	JPEG q50	1.041	1.508	$-14.6$ [ $-19.9$ , $-6.9$ ]

TABLE VI: Derived memory-interface energy. From the measured

\rho_{1}

and

\alpha

ratios, the model of Eq. ( 2 ) yields a reduction

\Delta E

with a coefficient-agnostic range.

Variant	mAP	VRU	$\rho_{1}$	$\alpha$
MotiMem- $\Omega$ (full)	0.900	0.910	0.480	0.516
no Gray	0.904	0.916	0.517	0.522
sky at bg- $k$	0.932	0.921	0.514	0.636
ego-CV	0.903	0.915	0.482	0.547
2D-vel.	0.903	0.915	0.480	0.544
static	0.903	0.915	0.482	0.547

TABLE VII: Detection design sensitivity. Top rows perturb energy-related design choices; last rows swap the temporal prior. Metrics are normalized to the original frame.

Energy-Efficient ML

Energy-efficient machine learning reduces the energy and environmental costs of training and inference while preserving useful model performance.

See all 257 papers

MotiMem: Motion-Aware Approximate Memory for Energy-Efficient Neural Perception in Autonomous Vehicles
Mar 28, 2026Haohua Que, Mingkai Liu, Jiayue Xie +5Energy-Efficient ML Memory-Efficient Inference
University of Georgia · Peking University · Tsinghua University +1
RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Prediction
Sep 23, 2026Xiaokai Bai, Lei Yang, Songkai Wang +3Autonomous Driving Perception 3D Occupancy Prediction
College of Information Science and Electronic Engineering, Zhejiang University · School of Mechanical and Aerospace Engineering, Nanyang Technological University · School of Automotive Studies, Tongji University
Driving on Memory
Aug 31, 2026Christian Löwens, Thorben Funke, Alexandru Paul ConduracheAutonomous Driving Benchmarks End-to-End Autonomous Driving
Bosch Research · University of Lübeck · Automated Driving, Bosch

Mar 28, 2026J/K move · Enter open · S save

MotiMem: Motion-Aware Approximate Memory for Energy-Efficient Neural Perception in Autonomous Vehicles

Haohua Que, Mingkai Liu, Jiayue Xie, Haojia Gao+4

University of Georgia · Peking University · Tsinghua University+1

Energy-Efficient ML Memory-Efficient Inference Autonomous Driving Perception

Open paper AskPDF arXiv

High-resolution sensors are critical for robust autonomous perception but impose a severe memory wall on battery-constrained electric vehicles. In these systems, data movement energy often outweighs computation. Traditional image compression is ill-suited as it is semantically blind and optimizes for storage rather than bus switching activity. We propose MotiMem, a hardware-software co-designed interface. Exploiting temporal coherence,MotiMem uses lightweight 2D Motion Propagation to dynamically identify Regions of Interest (RoI). Complementing this, a Hybrid Sparsity-Aware Coding scheme leverages adaptive inversion and truncation to induce bitlevel sparsity. Extensive experiments across nuScenes, Waymo, and KITTI with 16 detection models demonstrate that MotiMem reduces memory-interface dynamic energy by approximately 43 percent while retaining approximately 93 percent of the object detection accuracy, establishing a new Pareto frontier significantly superior to standard codecs like JPEG and WebP.

Abstract

Energy-Efficient ML

Explore similar work

MotiMem: Motion-Aware Approximate Memory for Energy-Efficient Neural Perception in Autonomous Vehicles

RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Prediction

Driving on Memory