cs.LGSep 19, 2026

Neural Residual Modeling for Scientific Data Compression under Guaranteed Error Bounds

Authors: Surya Majumder, Liangji Zhu, Sanjay Ranka, Anand Rangarajan

Organizations: Department of Computer & Information Science & Engineering University of Florida, Gainesville, FL, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Lossy compression of scientific simulation data increasingly relies on learned, latent-space architectures such as Residual Vector Quantization (RVQ), which iteratively quantize a base representation and its residuals to progressively reduce reconstruction error. While effective, RVQ performs this residual modeling entirely in latent space, leaving the pixel-space error structure of the reconstruction largely unaddressed. In this work, we propose a post-processing pipeline that augments an RVQ-based compressor with a U-Net trained to predict and correct pixel-space residuals between the original volume and its RVQ reconstruction. We show that these residuals are spatially structured rather than driven by local intensity or gradient features, motivating the need for a deep spatial model rather than simple statistical correction. The U-Net-corrected reconstruction is then passed through a Guaranteed Autoencoder (GAE) stage, which projects the remaining residual onto a per-block PCA basis to enforce a user-specified block-wise error bound. To the best of our knowledge, this is the first pipeline to combine latent-space RVQ, explicit pixel-space residual correction via a deep spatial post-processing network, and GAE-based error-bound guarantees within a single framework for scientific data compression. We evaluate our approach on S3D, JHTDB and E3SM datasets, demonstrating consistent improvements in NRMSE, compression ratio] over RVQ-only and standard residual-correction baselines, while maintaining strict error guarantees required for scientific data fidelity.

Figures & tables

Figure 1: Overview of the proposed three-stage compression pipeline. The input volume is tiled into spatial blocks and passed through an RVQ-based VAE (Stage 1) to obtain a pixel-space reconstruction x~\tilde{x} . A pixel-space U-Net (Stage 2) predicts the residual r^=x−x~\hat{r}=x-\tilde{x} and produces a refined reconstruction x~refined=x~+r^\tilde{x}_{\text{refined}}=\tilde{x}+\hat{r} . A Guaranteed Autoencoder (Stage 3) then projects the remaining residual onto a per-block PCA basis to enforce a user-specified NRMSE bound τ\tau , producing the final certified output xGx^{G} .
Figure 2: Correlation between RVQ pixel-space residual magnitude and local structural features on the JHTDB dataset. Scatter plots show residual magnitude against gradient magnitude (column 2), Laplacian response (column 3), local variance (column 4), and temporal difference (column 5), each computed per-pixel. The low correlation across all four features indicates that no simple pixel-wise or gradient-wise statistic explains where RVQ reconstruction fails, motivating a deep spatial model capable of learning the more complex underlying pattern. This is further highlighted by the high correlation between the true and predicted residuals from the U-Net (column 1) showing that the U-Net is able to capture some spatial features resulting in proper residual predictions.
Figure 3: Final NRMSE vs. Compression Ratio on the E3SM dataset. RVQ (CB: 4, Size: 64) is the Stage 1 baseline. U-Net Residual (L1) with 0.462 MB and 7.349 MB model sizes correspond to the small ( b=8b=8 ) and large ( b=32b=32 ) U-Net variants respectively. The U-Net correction consistently shifts the curve downward, achieving lower NRMSE at the same compression ratio across the full operating range.
Figure 4: Final NRMSE vs. Compression Ratio on the JHTDB dataset. The turbulence dataset presents the most structured and high-frequency residuals among our three benchmarks. Both U-Net variants improve over RVQ alone, with the larger model providing a more consistent gain especially at high compression ratios.
Figure 5: Final NRMSE vs. Compression Ratio on the S3D dataset. The smaller U-Net variant improves over the RVQ baseline with the larger model achieving a more substantial gain in NRMSE. But we take a huge hit in the compression ratios. This highlights a case where the U-Net model size matters since we are working with a comparatively smaller dataset.
Figure 6: Ablation study of U-Net loss functions on the S3D dataset. All loss variants substantially outperform the RVQ-only baseline. Robust losses like ℓ1\ell_{1} , Smooth L1, Huber, Log-Cosh, and Pseudo-Huber, cluster tightly together and outperform MSE, suggesting that residual-robust loss functions are more appropriate for the heavy-tailed residual distribution typical of learned scientific data compressors.
Figure 7: Qualitative visualization for a sample representative JHTDB tile and frame. Top row (left to right): original field, RVQ reconstruction, predicted residual r^\hat{r} , refined reconstruction x~refined=x~+r^\tilde{x}_{\text{refined}}=\tilde{x}+\hat{r} . Bottom row: absolute error map ∣| RVQ −- Original ∣| , absolute error map ∣| Refined −- Original ∣| , predicted residual histogram, and true vs. predicted residual overlay. The U-Net substantially reduces the absolute error (visible as reduced intensity in the error map), and the predicted residual distribution closely matches the true residual distribution, confirming that the U-Net has learned a useful approximation to the pixel-space reconstruction error.
MethodNRMSECR
SZ3 only9.956×10−49.956\times 10^{-4}29.52
SZ3 + U-Net (small)6.163×10−46.163\times 10^{-4}12.99
Table 2: Comparison of SZ3 alone vs. SZ3 + U-Net (small) on the S3D dataset at target NRMSE =1×10−3=1\times 10^{-3} . The U-Net model overhead is included in the total bitstream for the SZ3 + U-Net case.
More in this topic

Residual Learning

Residual learning studies methods that learn incremental updates or error corrections, helping deep networks and hybrid models refine predictions.

See all 158 papers

Explore similar work

CardsList
Jun 3, 2026

Residual Modeling for High-Fidelity Learned Compression of Scientific Data

Lossy compression is essential for massive spatiotemporal data from scientific simulations. Learned compressors can achieve high compression ratios at moderate accuracy targets, but their aggregate reconstruction losses do not guarantee accuracy for each block. Existing Guaranteed Autoencoder (GAE) methods add a per-block residual correction by retaining SVD/PCA-style coefficients until the target is met. This works at moderate tolerances, but in the high-fidelity regime with block-level NRMSE from 10^-6 to 10^-4, the number of retained coefficients grows quickly and the correction stream dominates the total rate. We propose a residual-centric view: the learned residual is structurally different from the original scientific field and should be coded with a representation designed for that residual. We introduce two residual coders. LBRC is a deterministic, training-free pipeline that adaptively quantizes the learned residual to the target NRMSE and losslessly encodes the resulting integer residual using 3D Lorenzo differencing, zigzag mapping, bit-plane coding, and entropy coding. NGLR adds a causal neural predictor that outputs a normalized bias for an integer-rounded Lorenzo prediction in the same deterministic integer pipeline, reducing the entropy of the remaining residual code while preserving deterministic decoding. The predictor weights are serialized and counted in the bitstream. Across E3SM, JHTDB, and ERA5 at block-level NRMSE targets from 10^-6 to 10^-4, LBRC improves compression ratio over GAE by 30-60% and is broadly competitive with SZ. NGLR adds a further 10-40% over LBRC and outperforms SZ in the evaluated high-fidelity regime. These results show that residual representations tailored to learned-compressor residuals can preserve the advantage of learned compression when global residual correction becomes rate-dominant.

Liangji Zhu, Sanjay Ranka, Anand Rangarajan

Department of Computer & Information Science & Engineering University of Florida Gainesville, FL, USA

Predictive Coding
AskPDF
Jun 12, 2026

Can Deep Neural Networks Improve Compression of Very Large Scientific Data?

Error-bounded lossy compression is a fundamental technique for managing the rapidly growing volumes of scientific data produced by modern simulations and observational instruments. Most state-of-the-art-compressors follow a prediction-residual paradigm, where compression effectiveness depends on the quality of the predictor: more accurate predictions generate smaller residuals that are easier to compress. This observation raises a question: can modern machine learning models serve as superior predictors for scientific data compression? Answering this question directly is challenging because developing compression-specific ML predictors requires substantial resources. Instead, we leverage the climate domain where highly accurate pretrained weather forecasting foundation models already exist, making them an ideal testbed. We present a framework that integrates spatial and temporal deep learning models into a conventional error-bounded compression pipeline. The framework supports auto-regressive forecasting models and avoids error accumulation. Using ERA5 climate data as a representative large-scale scientific dataset, we evaluate three distinct ML predictors: a VAEformer-based codec (CRA5), a graph neural network forecaster (GraphCast), and a vision-transformer forecaster (Aurora), against the state-of-the-art compressor SZ3.1 under identical quantization and entropy-coding backends. Our evaluation over approximately 1.7 TB of data reveals a surprising result: although ML predictors generate more accurate predictions and can improve reconstruction quality by up to 91% while achieving up to 9.6x higher compression ratios for highly predictable variables, they do not improve overall dataset-level compression ratio. We show that prediction accuracy alone is insufficient: the spatial structure of the resulting residuals plays a decisive role in entropy coding efficiency.

Muhannad Alhumaidi, Guozhong Li, Spiros Skiadopoulos +1

King Abdullah University of Science and Technology · Thuwal, Saudi Arabia · University of the Peloponnese +1

Predictive Coding
AskPDF
Jul 20, 2026

EVOLVE: Efficient Learned Volume Compression with Variable-Rate Encoding on a Cross-Domain Database

Large-scale scientific simulations generate volumetric data at rates that far outpace advances in storage and network bandwidth, making effective lossy compression increasingly critical. However, conventional compressors often struggle to preserve fine structural details at high compression ratios (CRs), and implicit neural representations (INRs) require costly per-volume optimization and produce models with fixed CRs. To respond, we present EVOLVE, an autoencoder (AE)-based volume-compression framework that targets high CRs for offline compression, with three key contributions. First, we construct a large-scale cross-domain database of 6,376 volumes from 21 scientific simulations, curated via perceptual hashing to ensure diversity, enabling the optimized model to extract features that generalize across volumes within the covered scientific simulation domains. Second, we reexamine the design space of AE-based compressors and incorporate several macro- and micro-designs into a vanilla AE to develop EVOLVE, which substantially improves the expressive power and compression capability. Third, we develop a learnable gain mechanism with a three-stage training strategy to enable variable-rate encoding, allowing a single model to support continuous CR adjustment at inference time. Experiments on multiple unseen scientific simulation datasets demonstrate that EVOLVE achieves substantially higher CRs than conventional compressors at comparable reconstruction quality, while delivering compression speeds that are orders of magnitude faster than INR-based methods, highlighting its promise as a strong alternative for compressing scientific data. The code, model weights, and results are available on our project page at https://evolve-vis.github.io.

Kaiyuan Tang, Maizhe Yang, Chaoli Wang

Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA

Autoencoders
AskPDF