STeP-C: Real-Time Neural Compression for Distributed Fiber-Optic Sensing at Scale
Organizations: NEC Laboratories America
Abstract
Distributed fiber-optic sensing (DFOS) technologies such as Distributed Acoustic Sensing (DAS) generate continuous, high-resolution spatio-temporal data at rates exceeding hundreds of megabytes per second, creating severe challenges for real-time storage and transmission. While neural compression methods can achieve strong rate-distortion performance on sensing data, prior approaches either require per-segment retraining incompatible with streaming, or have only been validated offline. None have been demonstrated end-to-end on a continuously streaming DAS acquisition system at scale. We present STeP-C (Spatio-Temporal Predictive Compression), a neural compression framework that decouples spatial encoding, temporal prediction, and entropy-optimized residual coding, trained end-to-end with a rate-distortion objective motivated by conditional entropy reduction. Critically, we identify entropy coding as the dominant bottleneck for streaming deployment. We address it through GPU-side symbol computation, cached probability tables, and parallel GIL-free arithmetic coding, together with FP16 inference and batched entropy coding, enabling a full real-time pipeline built around a GPU ring buffer. We validate this pipeline on a live DAS testbed, achieving a 98.85% file size reduction at 31.04 dB full-band PSNR and an average throughput of 535 MB/s, exceeding the 324 MB/s requirement for 1 kHz, 81 km DAS streams. We further show that STeP-C substantially outperforms conventional codecs at practical streaming bitrates, can be applied to unseen channel ranges within the same acquisition system without retraining, and largely maintains downstream detector consistency, with 96.62% high-confidence class agreement and 100% Jaccard similarity on detected event types.
Figures & tables
| Method | General-purpose reconstruction | No per-segment training | Entropy coding | Real-time processing | Downstream task validation |
| Dong et al. [ 7 ] | ✓ | ✓ | ✓ | ||
| Wang et al. [ 25 ] | ✓ | ✓ | |||
| Ni et al. [ 21 ] | ✓ | ✓ | ✓ | ||
| Chen et al. [ 4 ] | ✓ | ||||
| Shen et al. [ 23 ] | ✓ | ✓ | ✓ | ||
| Wang et al. [ 26 ] | ✓ | ✓ |
| no predictor | 3 | 5 | 7 | 10 | |
| PSNR | 33.15 | 33.29 | 33.32 | 33.24 | 33.16 |
| BPS | 0.281 | 0.207 | 0.196 | 0.189 | 0.184 |
| Metric | Value |
| Prediction consistency | |
| Overall top-1 class agreement | 83.01% |
| Overall top-5 class agreement | 98.72% |
| High-confidence top-1 agreement | 96.62% |
| Probability score correlation | 0.851 |
| Mean probability deviation | 0.079 |
Entropy Coding
Entropy coding assigns short codes to likely symbols and longer codes to rare ones, reducing the bits needed to store or transmit data.
See all 30 papers