Organizations: Coovally, Barcelona, Spain · Technical University of Vienna, Austria · Peking University, China · Beijing University of Posts and Telecommunications, China · ICREA, Barcelona, Spain
Abstract
Earth-observation satellites capture more imagery than intermittent ground contacts can transmit. Onboard systems threshold a cloud detector, discard frames or tiles, and compress the survivors with a fixed codec. On expert-labeled imagery, these rules remove more than one-fifth of clear pixels, primarily through detector false positives. We train a neural codec with a clear-probability-weighted reconstruction loss, reallocating coded bytes from clouds to clear ground without requiring or transmitting a cloud map onboard. Each capture is encoded into a resumable base layer and a dependent refinement layer, while clear content is estimated from features produced by the encoder. At each contact, we causally rank arrived layers using estimated clear content, unfinished bytes, deadline slack, and aggregate deadline pressure. The scheduler serves base and computational deadlines, bounds stored residual bytes, and resumes interrupted packets. We evaluate the onboard-to-downlink pipeline using real entropy-coded bytes, orbit-derived interruptible contact capacities, and measured service time and energy on resource-constrained embedded accelerators. Clear-weighted codecs require up to 47.8% fewer bytes than learned-compression baselines at matched clear-region quality. The optimized encoder consumes less time and energy than one pass of the cloud detector used by the frame-discard rules. Relative to fixed two-stage service on the same streams, our scheduler more than doubles deadline-full clear-content delivery for the interrupted combined cohort, reaches 83.6% of a certified clairvoyant upper bound, and exceeds replayed reference orders in deadline-usable delivery.
In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective. At such extreme compression levels, codecs trained with acoustic reconstruction losses tend to allocate bits to perceptual detail, leading to substantial degradation in word error rate (WER). This paper proposes ClariCodec, a neural speech codec operating at 200 bit per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility. Specifically, the encoder is fine-tuned using WER-driven rewards while the acoustic reconstruction pipeline remains frozen. Even without RL, ClariCodec achieves 3.68% WER on the LibriSpeech test-clean set at 200 bps, already competitive with codecs operating at higher bitrates. Further RL fine-tuning reduces WER to 3.20% on test-clean and 8.93% on test-other, corresponding to a 13% relative reduction while preserving perceptual quality.
In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective. At such extreme compression levels, codecs trained with acoustic reconstruction losses tend to allocate bits to perceptual detail, leading to substantial degradation in word error rate (WER). This paper proposes ClariCodec, a neural speech codec operating at 300 bit per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility. Specifically, the encoder is fine-tuned using WER-driven rewards while the acoustic reconstruction pipeline remains frozen. Even without RL, ClariCodec achieves 4.64% WER on the LibriSpeech test-clean set at 300 bps, already competitive with codecs operating at higher bitrates. Further RL fine-tuning reduces WER to 3.55% on test-clean and 10.4% on test-other, corresponding to a 23% relative reduction while preserving perceptual quality.
Modern satellite edge systems, including those performing remote sensing tasks such object detection and tracking, are characterized by severely limited bandwidth and intermittent connections, making continuous data transmission to the cloud impractical. Existing edge-cloud systems, however, either require heavy pre-processing before analysis, for instance, full decompression of imagery data, or transmit all compressed data regardless of relevance. To address these challenges, we design DynaFilter, a dynamic filtering technique that enables satellite edge devices to perform selective region-of-interest (RoI) inference directly in the compressed-domain, without full decompression. Our key insight is that low-level compression syntax, specifically DC coefficients/AC energy in JPEG images and motion vectors in video streams, exhibits strong correlations with high-level semantic queries. By establishing a precise mapping between cloud query semantics and multimodal compressed-domain features, DynaFilter enables the edge to identify and transmit only relevant data associated to RoIs. Extensive evaluations show that DynaFilter reduces the total volume of pixel data for decoding and subsequent inference by 1.6x-7.1x for images, and achieves 92.0% bandwidth savings for video streams compared to state-of-the-art baselines. Furthermore, it decreases energy consumption by 43.1-88.6% on target devices and achieves a 1.6x-3.0x speedup in inference latency.