Lossy Event Compression: From Event Stream Distortion to Task Performance
Authors: Zahra Rezaee, Catarina Brites, João Ascenso
Organizations: Instituto Superior Técnico, University of Lisbon, Instituto de Telecomunicações, Lisbon, Portugal · Instituto Universitário de Lisboa (ISCTE-IUL), Instituto de Telecomunicações, Lisbon, Portugal
Event cameras generate asynchronous, sparse data streams with microsecond temporal resolution, but in moderate-to-high motion scenes they can produce as many as hundreds of millions of events per second, creating significant bandwidth and storage challenges. Lossy compression is therefore essential for practical deployment, yet existing event stream distortion metrics fail to reliably predict compression-induced degradation at the task level, forcing codec optimization to rely on expensive task-specific evaluations. To address this gap, this paper introduces two fundamentally different event compression pipelines: i) an aggregation-based pipeline that converts the event stream into polarity-based histogram frames for compression with the conventional image codec JPEG 2000, and ii) a frame-free point cloud-based pipeline that codes events natively as 3D points using the octree-based codec G-PCC. Both pipelines are then assessed within a unified task-driven evaluation framework that relates event stream distortion to downstream application performance across four representative tasks: i) video reconstruction, ii) object detection, iii) optical flow estimation, and a delay-sensitive task iv) asynchronous feature tracking under a reference-relative protocol. Building on this framework, five classification-based distortion metrics are applied to event compression for the first time, to the best of the authors' knowledge, and benchmarked against existing event stream metrics. Experimental results demonstrate that the proposed metrics reliably predict compression-induced task degradation across different coding frameworks. This demonstrates that event stream distortion assessment can be an efficient alternative to repeated task-specific evaluation, providing direct guidance for the development and optimization of future event data coding solutions.
Figures & tables
Fig. 1 : Some applications of event data: object detection for autonomous driving [ 2 ] (left), gesture recognition for human-computer interaction [ 3 ] (middle), and optical flow estimation for motion analysis [ 4 ] (right).
Fig. 3 : Point cloud-based event stream compression pipeline using G-PCC.
Fig. 4 : Rate-utility curves for the aggregation-based codec (JPEG 2000-based) across the three downstream tasks, evaluated over the temporal aggregation intervals and operating points defined in Section III-B under the RR protocol. Each curve is obtained only with the operating points of the convex hull.
Fig. 5 : Rate-utility curves for the point cloud-based codec (G-PCC-based) across the three downstream tasks, evaluated over the operating points defined in Section IV-B under the RR protocol. Each curve corresponds to one sequence.
Fig. 6 : Illustration of the impact of three different image coding schemes on a sparse event count decoded histogram (ECD boxes_6dof, frame 0, negative polarity, 98.9% zeros, 483 events). JPEG 2000 operates at 0.2 bpp, while JPEG XL and AVIF operate at their minimum distortion level.
Fig. 7 : Rate-distortion curves for the video reconstruction task (HyperE2VID) on the ECD dataset, averaged over the seven ECD sequences. Shaded regions indicate ±1 standard deviation across sequences.
Fig. 8 : HyperE2VID reconstruction at four lossy bitrates and lossless coding mode on ECD boxes_6dof at t=4.03 s (20 ms aggregation interval). All images correspond to the same time instant with identical grayscale scaling for direct codecs comparison.
TABLE I : SROCC and PLCC correlation between event stream distortion metrics and task degradation (15 ms window), evaluated separately per codec and over the pooled lossy operating points of both codecs. Bold indicates reliable prediction, defined as correlation ≥ 0.800.
Fig. 9 : Video reconstruction (HyperE2VID, ECD): task score (PSNR against the raw events reference) versus each event stream distortion metric, over the pooled lossy operating points of both codecs (G-PCC- and JPEG 2000-based). Points are colored by sequence and shaped by codec; the black curve is the cubic fit described in Section VI-B . The first five plots correspond to the proposed metrics and the remaining six to the benchmark metrics, following the same grouping as Tables I(a) – I(f) .
Fig. 10 : Rate-utility curves for asynchronous feature tracking (HASTE, ECD), evaluated over the operating points defined in Sections III-B and IV-B under the RR protocol. Each curve corresponds to one sequence. As in Fig. 4 , each curve of the aggregation-based codec shows the convex hull of the operating points.
TABLE II : SROCC and PLCC correlation for asynchronous feature tracking (15 ms window), reported separately per codec and over the pooled lossy operating points of both codecs. Bold indicates reliable prediction, defined as correlation ≥ 0.800.
Fig. 11 : SROCC between Cohen’s Kappa and the task score as a function of the metric temporal window tw , for the point cloud-based codec (top) and the aggregation-based codec (bottom); red marks feature tracking and blue video reconstruction. Both tasks use the same six ECD sequences, operating points and decoded event streams, so the downstream task is the only variable within each plot. Stars mark each curve’s maximum; the dotted line marks the 15 ms setting defined in Section VI .
Event cameras promise low latency and high dynamic range, yet their sparse output challenges integration into standard robotic pipelines. We introduce \nameframew (Efficient Event Camera Volume System), a novel framework that models event streams as continuous-time Dirac impulse trains, enabling artifact-free compression through direct transform evaluation at event timestamps. Our key innovation combines density-driven adaptive selection among DCT, DTFT, and DWT transforms with transform-specific coefficient pruning strategies tailored to each domain's sparsity characteristics. The framework eliminates temporal binning artifacts while automatically adapting compression strategies based on real-time event density analysis. On EHPT-XC and MVSEC datasets, our framework achieves superior reconstruction fidelity with DTFT delivering the lowest earth mover distance. In downstream segmentation tasks, EECVS demonstrates robust generalization. Notably, our approach demonstrates exceptional cross-dataset generalization: when evaluated with EventSAM segmentation, EECVS achieves mean IoU 0.87 on MVSEC versus 0.44 for voxel grids at 24 channels, while remaining competitive on EHPT-XC. Our ROS2 implementation provides real-time deployment with DCT processing achieving 1.5 ms latency and 2.7X higher throughput than alternative transforms, establishing the first adaptive event compression framework that maintains both computational efficiency and superior generalization across diverse robotic scenarios.
Event cameras have emerged as a high-bandwidth, low-latency sensing modality for safety-critical perception in automated driving systems (ADS), offering microsecond temporal resolution, 120-140 dB dynamic range, and intrinsic absence of motion blur. However, no task-agnostic quality metric currently operates directly on the asynchronous event stream: state-of-the-art proxies require a downstream task (e.g., detection accuracy, tracking error) to assess stream integrity, which is incompatible with the certification requirements of ISO 21448 (SOTIF) and ISO/PAS 8800:2024. The recent BiasBench benchmark (CVPR 2025) explicitly identifies this gap. This work proposes a unified algebraic framework that lifts the Pearson Correlation Coefficient (PCC), historically used in two prior works for redundancy filtering and ROI selection on frame-based images, to the three standard event representations: Time Surface, Event Frame, and Voxel Grid. The framework yields three metrics: (i) r-TS for stream integrity monitoring against an ego-motion-predicted Time Surface, (ii) r2-EF for adaptive ROI selection requiring only integer comparisons, and (iii) r-VG for temporal redundancy gating. A structural isomorphism is established between the contrast-threshold mechanism of the event camera (|Delta L| >= C) and the PCC-based change criterion, the three lifted metrics are formalized, and pipeline latency and information loss are analyzed symmetrically against the raw stream. Illustrative behavior of each metric is demonstrated on a procedural-synthetic event stream, generated by direct simulation of the emission model rather than drawn from any real or video-derived dataset, including a tunnel-dip integrity-anomaly scenario in which r_C drops from 0.93 (coherent flow) to below 0 (alarm). An explicit epistemic convention ([ESTABLISHED], [SOLID], [HYPOTH.], [OPEN]) delineates the status of every contribution.
Neural video codecs derive motion and temporal contexts mainly from RGB frames, leaving room for cross-modal guidance from complementary temporal observations. Event streams can provide such observations by recording brightness changes between frames. In this work, we propose an Event-guided Neural Video Codec (ENVC) that uses events shared by the encoder and decoder to improve RGB compression efficiency. For motion coding, ENVC forms an event-guided motion prior and codes the remaining motion residual. For frame coding, an event-conditioned predictor supplies multi-scale features for gated temporal context refinement. To support training and evaluation on standard video datasets, we synthesize paired RGB-event data and assess its predictive utility through comparisons with real events. Across six benchmarks, ENVC achieves average BD-rate savings of 39.13% using PSNR-RGB and 67.63% using LPIPS relative to DCMVC. Further analyses show that our gains persist on large-motion sequences and that ENVC effectively learns to integrate event information. These results demonstrate the potential of events as a complementary modality for reducing the RGB coding rate. Our model and code are available at https://github.com/kjungwoo03/ENVC.
Jiyun Kong, Jungwoo Kim, Enes Eray Demirtas +2
Department of Artificial Intelligence, Yonsei University, Republic of Korea · School of Integrated Technology, Yonsei University, Republic of Korea · Multimedia Signal Processing Group, EPFL, Switzerland