High-resolution sensors are critical for robust autonomous perception but impose a severe memory wall on battery-constrained electric vehicles. In these systems, data movement energy often outweighs computation. Traditional image compression is ill-suited as it is semantically blind and optimizes for storage rather than bus switching activity. We propose MotiMem, a hardware-software co-designed interface. Exploiting temporal coherence,MotiMem uses lightweight 2D Motion Propagation to dynamically identify Regions of Interest (RoI). Complementing this, a Hybrid Sparsity-Aware Coding scheme leverages adaptive inversion and truncation to induce bitlevel sparsity. Extensive experiments across nuScenes, Waymo, and KITTI with 16 detection models demonstrate that MotiMem reduces memory-interface dynamic energy by approximately 43 percent while retaining approximately 93 percent of the object detection accuracy, establishing a new Pareto frontier significantly superior to standard codecs like JPEG and WebP.
Figures & tables
Fig. 1 : Pareto Efficiency Analysis (Accuracy vs. Energy Proxy) across nuScenes, Waymo, and KITTI Datasets [ 1 , 2 , 3 , 4 ] . MotiMem (marked with a red star ) establishes a superior Pareto frontier compared to standard codecs (JPEG, WebP) which fall into the inefficient zone due to high entropy. Evaluated across three diverse datasets and 16 detection models , MotiMem achieves the optimal trade-off by reducing the normalized bit-1 density (a direct proxy for dynamic energy) by ≈ 43% while maintaining ≈ 93% of the original uncompressed mAP. Crucially, MotiMem significantly outperforms the semantically blind “Global k=4” ablation (blue square), validating that semantic-aware allocation is superior to uniform quantization for energy-constrained perception.
Fig. 2 : The MotiMem closed-loop interface for neural perception. MotiMem sits between the camera stream and the memory hierarchy. It forms a closed loop: detections at time t are fed back as compact metadata to predict the RoI mask for time t+1 . Guided by the RoI mask, MotiMem applies RoI-guided hybrid coding to shape the stored bitstream toward lower activity (fewer 1s/toggles) along the sensor-to-memory path, while preserving perception accuracy.
Fig. 3 : RoI prediction from temporal coherence. Detections at frame t−1 are propagated to frame t using a lightweight 2D motion prior, inflated by margin δ , and rasterized to a compact block-level RoI mask Mt (e.g., 16×16 blocks).
Fig. 4 : RoI-guided hybrid coding. Pixels are routed into two paths based on Mt(p) and parameter k . RoI: Compute and embed an inversion flag f in the LSB, selectively invert top- k MSBs if f=1 . Background: Truncate to top- k MSBs, apply the same inversion, and embed f in the LSB. Both paths preserve B -bit width, reducing bit-1 density and transitions for lower memory energy.
Fig. 5 : Design Space Exploration: Perception vs. Energy Trade-off varying bit-width k . The plot sweeps the retained parameter k from 1 to 7. The blue line (Accuracy/Confidence) saturates around k=4 , while the green line (Energy Cost) continues to rise linearly. This identifies k=4 as the algorithmic ”Sweet Spot,” providing maximum perceptual gain for the minimum necessary energy expenditure.
Fig. 6 : Qualitative comparison of detection results across three autonomous driving datasets (nuScenes, Waymo, KITTI) under different encoding methods. All images are processed by YOLO26m detector. MotiMem preserves detection accuracy comparable to the Original while significantly reducing energy consumption (43% reduction). JPEG Q10 exhibits severe compression artifacts leading to missed detections. Standard codecs (JPEG Q50, WebP Q50) maintain visual quality but offer no energy benefit.
Method
nuScenes
Waymo
KITTI
Average
mAP
Eng.
SSIM
PSNR
LPIPS
mAP
Eng.
SSIM
PSNR
LPIPS
mAP
Eng.
SSIM
PSNR
LPIPS
mAP
Eng.
SSIM
PSNR
LPIPS
Lossless Baselines
Original (8-bit)
1.00
1.00
1.00
∞
.000
1.00
1.00
1.00
∞
.000
1.00
1.00
1.00
∞
.000
1.00
1.00
1.00
∞
.000
JPEG2000
1.00
1.02
1.00
∞
.000
1.00
.94
1.00
∞
.000
1.00
1.02
1.00
∞
.000
1.00
1.00
1.00
∞
.000
MotiMem Variants (Ours)
MotiMem ⋆
.91
.63
.92
30.6
.215
.92
.51
.93
31.8
.198
.95
.59
.91
29.4
.228
.93
.57
.92
30.6
.214
TABLE I : Comprehensive comparison of encoding methods across three autonomous driving datasets. All metrics are averaged over 16 detection models. MotiMem achieves the optimal trade-off : highest mAP retention among energy-efficient methods while consuming only 57% of the original energy.
Model
MotiMem
Glo. k=4
Uni. 4b
JPEG
WebP
YOLOv5n
91.9
90.4
90.5
91.7
89.8
YOLOv8n
92.0
90.6
90.6
91.6
89.7
YOLOv8s
93.0
91.7
91.7
92.7
90.7
YOLOv8m
93.5
91.9
91.9
91.8
90.0
YOLOv8l
93.5
92.1
92.2
91.3
90.1
YOLOv8x
93.7
92.3
92.3
91.3
89.9
TABLE II : Relative mAP retention (%) across 16 detection models. Results averaged over 3 datasets. MotiMem achieves highest retention on all models.
Autonomous vehicles stream high-resolution surround-camera frames into memory before perception runs. This sensor-to-memory path consumes energy when cells store ones and adjacent bytes toggle on the data bus, so its cost follows bit-1 density and switching activity rather than pixel semantics. We present MotiMem-Omega, a semantic-aware memory-interface coder that lowers this cost while preserving perception predictions. Its semantic importance field protects traffic participants, especially vulnerable road users, while assigning lower fidelity to sky and empty background. Cross-dataset bit-sensitivity sweeps determine class weights, with a safety floor for pedestrians, cyclists, and motorcyclists. Each image block then selects a precision tier by minimizing a joint energy-distortion cost. When ego pose is available, a motion-compensated prior carries protected regions between frames. We estimate interface-energy reduction from the two measured proxies using a coefficient-swept memory-energy model. Across 29 detectors on 12 driving datasets, 5 occupancy models, and 5 segmentation networks, MotiMem-Omega retains about 90% of detection mean average precision, 91% of vulnerable-road-user recall, over 98% of occupancy accuracy, and the strongest segmentation retention among energy-reducing methods. It reduces front-camera bit-1 density by 52%, corresponding to a modeled memory-interface energy reduction near 36%, with a lower end of 27% under the literature coefficient sweep. It also gives higher retention than the baseline and energy-matched truncation at the same or lower bit-1 density, whereas image codecs preserve accuracy without reducing memory-interface energy.
Haohua Que, Handong Yao
College of Engineering, University of Georgia, Athens, GA 30602 USA
This paper presents FluxMem, a training-free framework for efficient streaming video understanding. FluxMem adaptively compresses redundant visual memory through a hierarchical, two-stage design: (1) a Temporal Adjacency Selection (TAS) module removes redundant visual tokens across adjacent frames, and (2) a Spatial Domain Consolidation (SDC) module further merges spatially repetitive regions within each frame into compact representations. To adapt effectively to dynamic scenes, we introduce a self-adaptive token compression mechanism in both TAS and SDC, which automatically determines the compression rate based on intrinsic scene statistics rather than manual tuning. Extensive experiments demonstrate that FluxMem achieves new state-of-the-art results on existing online video benchmarks, reaching 76.4 on StreamingBench and 67.2 on OVO-Bench under real-time settings, while reducing latency by 69.9% and peak GPU memory by 34.5% on OVO-Bench. Furthermore, it maintains strong offline performance, achieving 73.1 on MLVU while using 65% fewer visual tokens.
Yiweng Xie, Bo He, Junke Wang +3
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · Shanghai Key Laboratory of Multimodal Embodied AI +1
Vision agent memory has shown remarkable effectiveness in streaming video understanding. However, storing such memory for videos incurs substantial memory overhead, leading to high costs in both storage and computation. To address this issue, we propose StreamMeCo, an efficient Stream Agent Memory Compression framework. Specifically, based on the connectivity of the memory graph, StreamMeCo introduces edge-free minmax sampling for the isolated nodes and an edge-aware weight pruning for connected nodes, evicting the redundant memory nodes while maintaining the accuracy. In addition, we introduce a time-decay memory retrieval mechanism to further eliminate the performance degradation caused by memory compression. Extensive experiments on three challenging benchmark datasets (M3-Bench-robot, M3-Bench-web and Video-MME-Long) demonstrate that under 70% memory graph compression, StreamMeCo achieves a 1.87* speedup in memory retrieval while delivering an average accuracy improvement of 1.0%. Our code is available at https://github.com/Celina-love-sweet/StreamMeCo.
Junxi Wang, Te Sun, Jiayi Zhu +6
Shanghai Jiao Tong University · Fudan University · Shanghai AI Laboratory +2