The persistent storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs) have reached the petabyte-to-exabyte scale. Transient simulations modeling Navier-Stokes equations, magnetohydrodynamics, plasma physics, or binary black hole mergers generate data volumes that are prohibitive for modern high-performance computing (HPC) infrastructures. To address this bottleneck, we introduce ANTIC (Adaptive Neural Temporal in situ Compressor), an end-to-end in situ compression pipeline. ANTIC consists of an adaptive temporal selector tailored to high-dimensional physics that identifies and filters informative snapshots at simulation time, combined with a spatial neural compression module based on continual fine-tuning that learns residual updates between adjacent snapshots using neural fields. By operating in a single streaming pass, ANTIC enables a combined compression of temporal and spatial components and effectively alleviates the need for explicit on-disk storage of entire time-evolved trajectories. Experimental results demonstrate how storage reductions of several orders of magnitude relate to physics accuracy.
Figures & tables
Figure 1: Figure 1: Schematic Overview of ANTIC . ANTIC facilitates high-fidelity neural compression via a dual-stage asynchronous architecture. Physics-aware Temporal Selector serves as a physics-aware filter that identifies salient physical transients. Spatial Neural Compression encodes the selected snapshots into compact NN weights, stored in the persistent storage. ANTIC enables a combined in-situ compression resulting in multi-fold reduction in data volume while preserving the temporal coherence of the entire simulation.
Figure 2: Workflow of ANTIC . A new snapshot at time t+1 is passed over by the simulator. The Metric extracts physics of interest for the new snapshot ϕt+1 , which is passed to the Regulator. Based on ϕt+1 , the regulator truncates the context queue and adds ϕt+1 . Finally, the truncated context is passed to the gate along with ϕt+1 to determine whether or not to compress the new snapshot.
Method
2D Kolmogorov flows (16.8 GiB)
3D BSSN simulations (4.2 TiB)
TR( ↑ ) (%)
PA ( ✓ / ✗ )
SC( ↑ ) (×)
TC( ↑ ) (×)
TR( ↑ ) (×)
PA ( ✓ / ✗ )
SC( ↑ ) (×)
TC( ↑ ) (×)
sparse sampling + ZFP [tol= 1e-1 ]
20%
✗
13×
65×
20%
✗
27×
141×
Dense sampling + ZFP [tol= 1e-1 ]
100%
✓
13×
13×
100%
✓
27×
28×
PATS + ZFP [tol= 1e-1 ]
37%
✓
13×
120×
55%
✓
27×
52×
Sparse sampling + FT
20%
✗
12×
60×
20%
✗
471×
2457×
Sparse sampling + LoRA
20%
✗
47× [r :32 ]
235×
20%
✗
3744× [r :16 ]
18720×
Table 1: Performance of ANTIC compared to baselines on 2D Kolmogorov flows and 3D BSSN simulation trajectories . We compare (i) Uniform sparse sampling (every 5th step), (ii) Dense sampling (every timestep), combined with (i) traditional compression ( ZFP ), (ii) CFT, or (iii) CFT via LoRA fine-tuning. For comparison to ZFP we match reconstruction quality to ANTIC and compare for compression ratio. TR (Temporal Retention) denotes the percentage of the simulation duration preserved and Physics-awareness (PA) denotes whether or not the temporal selector uses a PDE specific metric. SC (Spatial Compression) and TC (Total Compression) represent the memory reduction factors. ANTIC consistently outperforms ZFP and different temporal sampling schemes. In Sec. F , we report the performance of neural compression (CFT, CFT + LoRA) against well-known scientific computing classical compressors like SZ3 , MGARD , Blosc-2 zstd , clearly indicating the extreme compression obtained via neural fields for 3D high-resolution tensor-valued BSSN evolution simulations.
Figure 3: Qualitative results for Physics-aware Adaptive Temporal Selection. (Left) The enstrophy flux (top) identifies turbulent regions in 2D Kolomogorov flows, hence is well suited as metric for our PATS. As a consequence, PATS densely samples at the start of the trajectory that exhibits strong chaotic behavior, whereas sampling more sparsely in non-turbulent regions. (Right) The Weyl scalar (top) isolates the non-linear gravitational wave (GW), resulting in dense sampling around the merger region and sparse sampling otherwise.
Figure 5
Method
Mean Rel. ℓ2 (5 snapshots)
Mean MAE (5 snapshots)
Mem. /snapshot
Scratch
1.7e-3±8.5e-5
0.20±0.026
1.50 MiB ( 12× )
Full FT
6.1e-4±8.0e-6
0.061±0.011
1.50 MiB ( 12× )
LoRA 8
6.9e-2±6.3e-3
0.63±0.09
98 KiB (188×)
LoRA 16
2.1e-2±2.5e-3
0.13±0.02
196 KiB (94×)
LoRA 32
6.8e-4±7.0e-5
0.063±0.011
392 KiB ( 47× )
LoRA 64
\pagecolorbackground4.3e-4±2.4e-5
\pagecolorbackground0.041±0.006
784 KiB ( 23× )
Table 2: Accuracy metrics vs memory footprints for 2D Kolmogorov flows. Different spatial compression schemes evaluated across five sequential snapshots in a highly-turbulent regime. The last column reports the memory compression per snapshot relative to the 17 MiB ground truth. Continual fine-tuning using LoRA establishes an accuracy-memory pareto front.
Method
Mean Rel. ℓ2 (5 snapshots)
Mem. / snapshot
Time / snapshot
Scratch
6.2e-4±4.2e-5
1.52 MiB ( 471× )
∼162 (s)
Full FT
4.6e-4±7.6e-5
1.52 MiB ( 471× )
∼162 (s)
LoRA 8
6.0e-4±4.1e-5
98KiB(7489×)
∼131 (s)
LoRA 16
5.7e-4±3.9e-5
196 KiB ( 3744× )
∼133 (s)
LoRA 32
5.3e-4±3.7e-5
392 KiB ( 1872× )
∼137 (s)
LoRA 64
4.6e-4±3.2e-5
784 KiB ( 936× )
∼149 (s)
Table 3: Accuracy metrics vs memory footprints for 3D BBH merger simulations. Different spatial compression schemes evaluated across five sequential snapshots within the black hole merger regime. For completenees, we report wallclock-time per snapshot in the last column for the different training schemes.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8
Hardware
Method
Wallclock (s)
Epochs
MSE
H200
BSSN Solver (GPU)
∼0.25−0.27
–
–
Cold Start
∼183−187
35
∼3×10−8
CFT (Ours)
∼21−24
3
∼3×10−8
B300
BSSN Solver (GPU)
∼0.27−0.44
–
–
Cold Start
∼52−54
35
∼3×10−8
CFT (Ours)
∼4.5−4.7
3
∼3×10−8
Appendix
Table 4: Hardware Benchmark: Continual Fine-Tuning (CFT) vs. Solver vs. Cold-Start. Comparison of wall-clock convergence times across NVIDIA H200 and B300 (Blackwell) architectures for the 3D BSSN system. CFT updates perturbative changes between snapshots, achieving an order-of-magnitude speedup over cold-Starts while maintaining a comparable MSE of 3×10−8 . The BSSN solver is a bit slower on B300s due to skinny batched GEMM — millions of tiny einsum-based contraction operations that are lowered to dot_general on cuBLAS for which below the Tensor Core tile threshold, rendering the solver memory-bandwidth-bound rather than compute-bound.
Compressor
2D Kolmogorov (16.8 MiB/Snapshot)
3D BSSN (0.7 GiB/Snapshot)
CR ( ↑ )
Time ( ↓ )
Rel ℓ2 ( ↓ )
CR ( ↑ )
Time ( ↓ )
Rel ℓ2 ( ↓ )
ZFP (tol=1e-03)
8×
0.06 (s)
1.02 e-4
4×
2.82 (s)
1.27 e-4
MGARD [CPU] (abs=1e-03)
19×
0.37 (s)
4.02 e-4
5×
38.67 (s)
7.97 e-4
SZ3 (abs=1e-03)
154×
0.07 (s)
8.31 e-4
16×
3.42 (s)
1.02 e-4
BLOSC-2 zstd (lossless)
1.7×
1.38 (s)
0.0 e+00
1.3×
54.48 (s)
0.0 e+00
BLOSC-2 zstd (lossy trunc=10)
5×
1.03 (s)
4.23 e-4
2.8×
37.82 (s)
3.93 e-4
Appendix
Table 5: Performance Benchmarks of Traditional vs. Neural Compression . Comparison of Compression Ratio (CR), Wall-clock Time, and Relative ℓ2 Error across 2D Kolmogorov flows vorticity field ( 20482 ) in the turbulent regime and large-scale 3D BSSN evolved variables ( ∼0.7 GiB/snapshot, (2133,18) shaped array) in the premerger phase. While traditional compressors (SZ3, ZFP, MGARD and Blosc-2) maintain low latency for small-scale 2D data, our NC with (CFT) particularly the LoRA variant achieves an unprecedented 3745× compression ratio for tensor-valued BSSN variables. This represents a two-order-of-magnitude improvement over state-of-the-art error-bounded compressors (SZ3) while maintaining comparable reconstruction fidelity ( ∼10−4 ), effectively enabling the storage of high-resolution multi-scal simulations with a minimal memory footprint on the persistent storages. The neural field compression numbers reported are conducted on the B300 GPU.
Figure 6: Enstrophy flux reconstruction for 2D Kolmogorov flow. ( Left ) Enstrophy flux extracted from ground truth snapshots (solid blue) and ANTIC reconstructions (dashed orange) over the peak turbulent phase τ∈[0,15] (timesteps [0,400] ). ANTIC faithfully reproduces the temporal evolution of the enstrophy flux, including sharp transient features associated with turbulent intermittency. ( Right ) Pointwise absolute error over the same interval, remaining uniformly small with no systematic drift or error accumulation, confirming the temporal stability of the CFT-based continual learning scheme over long-horizon trajectories.
Figure 7: CFT reconstruction fidelity over long-horizon 2D Kolmogorov flow trajectories. Relative ℓ2 error of ANTIC reconstructions plotted against timestep for the full 2D Kolmogorov flow simulation. The trajectory spans two dynamically distinct regimes: a peak turbulence phase over timesteps [0,100] , characterized by rapid vorticity production, strong enstrophy cascade, and sharp small-scale features that place maximal demand on the neural field parametrization, followed by a gradual decay phase over timesteps [100,400] as the flow relaxes towards a quasi-steady turbulent state. Despite the elevated reconstruction difficulty during the high-turbulence regime — where inter-snapshot variability Δu:=∥u(x,t+Δt)−u(x,t)∥ is largest — the CFT-based continual learning scheme maintains a stable relative ℓ2 error <4×10−3 in the highest turbulence region and <2×10−3 , with no systematic degradation across the trajectory. This confirms that perturbative weight editing between subsequent neural field parametrizations remains well-conditioned even under large dynamical variability, demonstrating the robustness of ANTIC for in-situ compression over long-horizon simulation runs.
Quantity
Metric
Value
Enstrophy Flux
Mean Absolute Error
1.830×10−4
Max Absolute Error
2.870×10−4
Vorticity Field
Mean Relative ℓ2 Error
1.587×10−3
Max Relative ℓ2 Error
3.669×10−3
Appendix
Table 6: Reconstruction Fidelity Metrics. Error analysis for Enstrophy Flux (scalar) and the Vorticity Field (2D grid). The low maximum relative L2 error ( 0.37% ) demonstrates that PATS successfully captures the most physically informative transients in the Kolmogorov flow.
Figure 8: Vorticity field reconstruction quality across dynamical regimes. Each row corresponds to a representative timestep t∈{10,50,100,220,370} , spanning the peak turbulence phase and its subsequent decay, for the 2D Kolmogorov flow simulation at spatial resolution 20482 . First column: Ground truth vorticity field ω(x,t) , exhibiting the characteristic fine-scale vorticity filaments, coherent structures, and inter-scale energy transfer that define the turbulent cascade. Second column: ANTIC neural field reconstructions at the corresponding timesteps, obtained by querying the continually fine-tuned implicit parametrization. Third column: Pointwise relative ℓ2 error between the ground truth and reconstructed vorticity fields, confirming that reconstruction fidelity is maintained across both the high-turbulence regime - where vorticity gradients are steepest and small-scale structures are most pronounced — and the quasi-steady decay phase (after timestep t>150 ), with no systematic spatial concentration of errors around coherent vortical structures.
Figure 9: Weyl scalar magnitude reconstruction during the BBH merger phase. The gravitational wave signal ∣Ψ4(t/M)∣ extracted from ground truth snapshots (solid) and ANTIC neural field reconstructions (dashed) at extraction radius r=14.3M , over the merger phase t/M∈[136,210] . This interval encompasses the peak gravitational wave emission, ringdown onset, and the most dynamically complex regime of the spacetime evolution, placing maximal demand on the fidelity of the neural field parametrization. ANTIC accurately reproduces the amplitude and phase evolution of the Weyl scalar throughout, including the characteristic amplitude peak at merger.
Figure 10: Absolute reconstruction error in Weyl scalar magnitude. Pointwise absolute error ∣Ψ4model(t/M)∣−∣Ψ4GT(t/M)∣ at extraction radius r=14.3M over t/M∈[136,210] . Errors remain uniformly small across the full merger phase, with no systematic growth near the amplitude peak, confirming that ANTIC preserves the gravitational wave content of the spacetime to high fidelity under the CFT-based continual learning scheme.
Figure 11: Lapse function reconstruction quality across the BBH merger trajectory. Each row corresponds to a representative timestep t/M∈{136.30,…,209.22} , spanning the inspiral-to-merger transition and post-merger ringdown phase of the 3D BBH simulation at spatial resolution 2133 . First column: Ground truth lapse function α(x,t) evolved under the BSSN framework (Sec. B.2 ), exhibiting the characteristic collapse toward zero in the black hole interior regions as the merger proceeds. Second column: ANTIC neural field reconstructions at the corresponding timesteps, obtained by querying the continually fine-tuned implicit parametrization. Third column: Pointwise relative ℓ2 error between the ground truth and reconstructed lapse fields, confirming that ANTIC faithfully captures both the smooth exterior spacetime geometry and the sharp gradients near the apparent horizons, with no systematic spatial concentration of errors in the dynamically critical merger region.
Category
Metric
NeF (Ours)
Ground Truth
∣Ψ4∣ Range
Minimum Value
2.825×10−5
3.381×10−5
Maximum Value
6.267×10−4
6.741×10−4
Metric
Quantity
Value
Error
Mean Absolute Error
1.835×10−5
Max Absolute Error (at t/M=170.49 )
4.838×10−5
Appendix
Table 7: BSSN Reconstruction Performance. Error analysis for the Weyl Scalar magnitude ∣Ψ4∣ during the Binary Black Hole (BBH) merger and initial post-merger regime ( t/M∈[136.30,209.22] ). The results compare 146 snapshots across the peak gravitational wave emission phase.
Table 8: Temporal selector retention analysis across resolutions for 2D Kolmogorov flows. Comparison of snapshot retention rates across temporal selectors and spatial resolutions {2562,5122,10242,20482} for a trajectory of 1000 snapshots. Retention is computed as (selectedframes/1000)×××100% . The enstrophy-based PATS selector retains 37.5%–42.9% of snapshots consistently across all resolutions, reflecting a physically meaningful and resolution-stable selection that concentrates retained frames in high-turbulence transient regimes while discarding redundant snapshots during quasi-static evolution. In contrast, the physics-agnostic selectors exhibit two distinct failure modes: under-retention , where JSD (0.30% across all resolutions), Spectral (2.0%–2.7%), and Momentum (2.8%–5.5%) selectors discard the vast majority of snapshots indiscriminately, failing to identify dynamically critical transients; and over-retention , where MI retains all 1000 snapshots at every resolution (100%), providing no compression of the temporal dimension whatsoever. The Residual selector exhibits resolution-dependent instability, retaining 31.4%–39.4% at coarser resolutions but collapsing to full retention (100%) at 20482 , indicating sensitivity to the spectral content of high-resolution fields that renders it unreliable for in-situ deployment. Collectively, these results demonstrate that physics-agnostic selectors fail to generalize across resolutions and dynamical regimes, whereas PATS provides stable, physically grounded temporal compression that adapts to the underlying simulation dynamics.
Figure 18: Solver latency vs. neural compression scaling for 2D Kolmogorov flows. Wall-clock time per snapshot plotted against spatial resolution N2 for the traditional Navier-Stokes solver and ANTIC neural compression (NC) on a single NVIDIA H200 GPU. Solver latency scales super-linearly with resolution, consistent with O(N2) mesh-point complexity under CFL constraints, while NC training time grows sub-linearly due to the resolution-invariant capacity of the underlying implicit neural field. The narrowing NC-to-solver ratio.
Resolution ( N2 )
Total Pixels
Solver ( dt )
NC (per snapshot)
NC/Solver Ratio
2562
65,536
3.4 ms
24.90 s ± 1 ms
∼ 7323 ×
5122
262,144
7.3 ms
28.37 s ± 4 ms
∼ 3886 ×
10242
1,048,576
28 ms
42.02 s ± 16 ms
∼ 1500 ×
20482
4,194,304
0.2 s
62.19 s ± 22 ms
∼ 310 ×
28002
7,840,000
1.3 s
75.67 s ± 41 ms
∼ 58 ×
Appendix
Table 9: Hardware Benchmarks: Solver Latency vs. Neural Compression (NC). Wall-clock training time per snapshot ( dt ) for 2D Navier-Stokes Kolmogorov flows. All benchmarks were performed on a single NVIDIA H200 CUDA GPU. The scaling ratio ( NC/Solver ) demonstrates that while NC has a higher absolute latency, its complexity scales sub-linearly relative to the exponential growth of the traditional solver.
Figure 19: Temporal retention comparison between dynamic vs binary regulator. (Left) illustrates the comparative efficiency of our two regulation regimes. The Dynamic Regulator (Left) achieves a retention of 37% by mapping physical saliency to a multi-step stride set {1,…,5} . Intermediate jumps (sizes 2, 3, and 4) allow the selector to smoothly transition between stationary and transient regimes, isolating the high-enstrophy activity with minimal redundancy. (Right) Conversely, the Binary Regulator utilizes a ”Bang-Bang” strategy, restricted to either dense ( Δτ=1 ) or maximum-stride ( Δτ=5 ) sampling. While it successfully captures the high-enstrophy transients, the lack of intermediate quantization leads to over-sampling in moderate-activity regions, resulting in a higher temporal retention of 53% .
Large-scale scientific simulations generate volumetric data at rates that far outpace advances in storage and network bandwidth, making effective lossy compression increasingly critical. However, conventional compressors often struggle to preserve fine structural details at high compression ratios (CRs), and implicit neural representations (INRs) require costly per-volume optimization and produce models with fixed CRs. To respond, we present EVOLVE, an autoencoder (AE)-based volume-compression framework that targets high CRs for offline compression, with three key contributions. First, we construct a large-scale cross-domain database of 6,376 volumes from 21 scientific simulations, curated via perceptual hashing to ensure diversity, enabling the optimized model to extract features that generalize across volumes within the covered scientific simulation domains. Second, we reexamine the design space of AE-based compressors and incorporate several macro- and micro-designs into a vanilla AE to develop EVOLVE, which substantially improves the expressive power and compression capability. Third, we develop a learnable gain mechanism with a three-stage training strategy to enable variable-rate encoding, allowing a single model to support continuous CR adjustment at inference time. Experiments on multiple unseen scientific simulation datasets demonstrate that EVOLVE achieves substantially higher CRs than conventional compressors at comparable reconstruction quality, while delivering compression speeds that are orders of magnitude faster than INR-based methods, highlighting its promise as a strong alternative for compressing scientific data. The code, model weights, and results are available on our project page at https://evolve-vis.github.io.
Kaiyuan Tang, Maizhe Yang, Chaoli Wang
Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA
High-resolution turbulence modeling is essential for scientific computing, but remains constrained by the cost of direct numerical simulation and the scarcity of full-resolution data. Existing scientific compressors reduce storage but typically operate on per-frame representations, whereas learned compressors yield compact latents that are often resolution-dependent and weakly aligned with the physics of turbulence. This raises the need for a compression framework that reduces data size, preserves physical diagnostics, and transfers from low-resolution training fields to high-resolution test fields without retraining. In this paper, we propose Physics-Preserving Latent Compression (PPLC), a patch-local latent compressor for three-dimensional turbulence. Motivated by inertial-range scale similarity, PPLC treats fixed-size patches as transferable units and applies a shared variational autoencoder independently of the global grid size. It combines exact mean preservation, zero-mean fluctuation encoding, an invertible Haar wavelet front-end, shift-consistency regularization, and overlap-aware reconstruction. Instantiated on forced isotropic turbulence, PPLC is trained only on stride-downsampled 256^3 fields and transfers zero-shot to 1024^3 fields. Experiments show that PPLC improves the balance between reconstruction accuracy and physical fidelity over classical and learned baselines, keeping diagnostics such as dissipation, enstrophy, energy spectra, and incompressibility closer to the ground truth. Beyond turbulence compression, PPLC offers a general strategy for physics-preserving latent representations that support data-efficient scientific surrogate modeling.
Yilong Dai, Yiming Sun, Yiheng Chen +4
The University of Alabama · University of Pittsburgh · University of Maryland, College Park
Lossless compression can reduce the storage and movement of model weights without changing their floating-point values, but repeated statistical analysis and code construction add computational overhead. We study whether the statistical structure of exponents can be prepared once and reused. For this, we introduce NeuralZip, which groups chunks with similar exponent distributions, shares Huffman codes, and selectively represents recurring exponent tuples using packed exponents, thereby achieving additional moderate compression ratios. A setup chooses these representations before subsequent encodings, while every encoding still processes the current tensor values. In floating-point model checkpoints, post-setup compression is 1.81-21.33× faster than the baselines and achieves exact bit-to-bit reconstruction. We show that this setup can be precomputed and transferred from another compatible architecture, preserving similar compression ratios and avoiding the need to amortize setup costs. Therefore, compression adaptation is transferable and reusable. Training checkpoints demonstrate continued reuse as the weights evolve. Finally, GPU experiments reduce active memory usage by up to 27.5% while reproducing the logits exactly.
Martín Bravo, Samuel Horváth, Gonzalo Navarro +1
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Computer Science, University of Chile, Santiago, Chile · National Center for Artificial Intelligence (CENIA), Santiago, Chile