Reconstructing continuous physical fields from sparse measurements is central to scientific monitoring, inverse modeling, and digital-twin construction. Generative reconstruction has recently emerged as a promising paradigm for this task by learning data-driven physical priors that complete plausible full fields from limited observations. However, existing methods largely assume fixed, batch conditioning, whereas real sensing systems often produce structured streams: probes scan local regions, instruments observe moving fields of view, and communication constraints may leave entire frames missing. We propose TRACE, a retrospective streaming generative reconstruction framework for physical fields under structured sensing. TRACE performs approximate Bayesian inference in a learned continuous-coordinate latent space, converting sparse off-grid measurements into generative latent evidence, fusing it with a state-space temporal prior through Kalman-style filtering, and refining under-observed past frames via retrospective smoothing. Experiments on active matter, ocean sound-speed fields, and supernova simulations show that TRACE matches or surpasses frame-wise generative reconstructors, offline spatiotemporal methods, and streaming data-assimilation baselines in reconstruction quality under temporally sparse and spatially localized sensing protocols.
Figures & tables
Figure 1: Overview of TRACE. Top: offline pretraining of the continuous-coordinate field representation and latent diffusion prior. Bottom: online streaming inference, which fuses per-frame generative evidence with a state-space prior via Kalman filtering and refines it via retrospective smoothing.
Active Matter
Ocean
Supernova
(a) Temporally Sparse Observations
Miss
Blk-5
Blk-10
Miss
Blk-5
Blk-10
Miss
Blk-5
Blk-10
LRTFR
0.413
0.258
0.461
0.205
0.163
0.177
0.561
0.430
0.503
MMGN
0.313
0.217
0.360
0.097
0.080
0.092
0.318
0.318
0.320
SDIFT
0.490
0.391
0.530
0.158
0.127
0.145
0.436
0.398
0.424
DBF
0.960
0.814
0.873
0.809
0.627
0.709
0.855
0.809
0.845
Table 1: Field-domain RMSE under structured sensing ( ρ=1% ). (a) Temporally sparse: Miss (gap n=3 , one active frame in four) and Blk- L (contiguous blackout of L∈{5,10} frames). (b) Spatially localized: moving window at local density ρloc=15% , with S-curve single-pass and circular sweeps of one and three loops. Lower is better; bold is best per column; “—” marks cells a per-frame method cannot reconstruct.
Figure 2: Moving-window trajectories on Active Matter ( 256×256 domain, 96×96 window). The centre path (colour-coded by time) and shaded squares mark representative window positions; circular revisits accumulate coverage over time.
Figure 3: Qualitative reconstruction on Active Matter under a one-lap circular moving window.
Figure 4: Regime-structured analysis of TRACE.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
AM
Ocean
SN
Tucker rank R
(1,48,48)
(5,20,20)
(32,32,32)
Latent dim. d
2304
2000
32768
SIREN ω
20
10
20
Appendix
Table 2: Per-dataset architecture configuration: Tucker multilinear rank R , vectorised-core latent dimension d=∏kRk , and SIREN basis frequency ω .
Dataset
Regime
ζ
(α,ℓ,σf)
AM
Control
0.001
(0.5,1.0,2.0)
Temporally Sparse
0.001
(0.5,5.0,1.0)
Spatially Localized
0.01
(0.5,5.0,1.0)
Ocean
Control
0.06
(0.5,2.0,1.0)
Temporally Sparse
0.06
(0.5,5.0,2.0)
Spatially Localized
0.2
(0.5,5.0,2.0)
Appendix
Table 3: Inference hyperparameters per dataset and observation regime. Shared across all runs: S=20 , N=100 , σobs=0.05 , β=1 , diagonal fusion.
Full-availability RMSE
Ocean
Active Matter
Supernova
DBF (matched ρ=3% )
0.346
0.473
0.681
TRACE-Smoother (ours)
0.040
0.066
0.298
ratio
8.8×
7.1×
2.3×
Appendix
Table 4: DBF at its matched-density upper bound ( ρ=3% for both training and testing, no missing frames) versus our framework evaluated at the same ρ=3% .
Active Matter
Ocean
Supernova
1%
3%
1%
3%
1%
3%
LRTFR
0.156
0.090
0.161
0.114
0.406
0.418
MMGN
0.162
0.148
0.070
0.047
0.318
0.313
SDIFT
0.313
0.220
0.112
0.078
0.379
0.342
DBF
0.747
0.473
0.527
0.346
0.771
0.681
TRACE-Frame
0.078
0.066
0.045
0.040
0.322
0.309
Appendix
Table 5: Control regime (frame-rich: every frame observes the full domain at density ρ ), field-domain RMSE. Lower is better; bold is best per column.
Active Matter
Ocean
Supernova
1%
3%
1%
3%
1%
3%
LRTFR
0.235
0.178
0.175
0.136
0.460
0.424
MMGN
0.187
0.178
0.073
0.055
0.317
0.313
SDIFT
0.416
0.403
0.154
0.147
0.409
0.370
DBF
0.907
0.755
0.713
0.587
0.829
0.771
TRACE-Frame
—
—
—
—
—
—
Appendix
Table 6: Missing pattern with gap n=1 (every other frame active), field-domain RMSE at ρ∈{1%,3%} . Bold is best per column.
Active Matter
Ocean
Supernova
Miss
Blk-5
Blk-10
Miss
Blk-5
Blk-10
Miss
Blk-5
Blk-10
LRTFR
0.377
0.221
0.442
0.177
0.128
0.142
0.450
0.425
0.445
MMGN
0.307
0.205
0.352
0.084
0.060
0.075
0.314
0.313
0.316
SDIFT
0.478
0.366
0.526
0.148
0.110
0.139
0.399
0.356
0.397
DBF
0.901
0.617
0.742
0.747
0.502
0.627
0.829
0.747
0.812
TRACE-Frame
—
—
—
—
—
—
—
—
—
Appendix
Table 7: Temporally sparse observations at ρ=3% , field-domain RMSE. Columns follow main-text Table 1 ( Miss : gap n=3 ; Blk- L : blackout length L ). Bold is best per column.
Active Matter
Ocean
Supernova
DPS ( S=1 )
0.091
0.055
0.365
TRACE-Frame
0.078
0.045
0.322
TRACE-Smoother
0.078
0.044
0.310
Appendix
Table 8: Effect of the transition prior under full observation (Control regime, ρ=1% ), field-domain RMSE. “DPS ( S=1 )” is vanilla single-sample DPS.
Fusion
Time/frame
Outcome
Diagonal (ours)
1.7 ms
RMSE 0.078
Full-covariance
≳20 s
OOM at d>5000
Appendix
Table 9: Per-frame fusion time (Active Matter, Control regime, ρ=1% , d=2304 ; CPU, excluding DPS sampling).
S
1
5
20
50
RMSE
0.091
0.083
0.081
0.080
Appendix
Table 10: Sample-count S ablation for TRACE-Frame on Active Matter (Control regime, ρ=1% , 10 trajectories), field-domain RMSE.
Figure 5: Hyperparameter sensitivity on Active Matter ( ρ=1% , full TRACE-Smoother), one curve per Miss level n∈{0,1,3,5} . (a) RMSE is strongly ℓ -sensitive and the optimal ℓ shifts with n , saturating at ℓ≈5 for all sparse regimes. (b) RMSE is nearly α -insensitive (note the compressed vertical range); the dotted line is the untempered Kalman update ( α=1 ). Other hyperparameters held fixed ( σf=1 , diagonal fusion).
σ
TRACE-Frame
TRACE-Filter
TRACE-Smoother
0.0
0.078
0.078
0.078
0.1
0.097
0.097
0.098
0.3
0.195
0.192
0.184
Appendix
Table 11: Observation-noise robustness (Active Matter, Control regime, ρ=1% ), field-domain RMSE. σ is the additive Gaussian sensor noise. σ=0 : 28 trajectories; σ>0 : 3 trajectories.
Method
Miss-3
Blk-5
Blk-10
Active Matter ( ρ=1% )
LRTFR
0.371/0.220/0.422
0.218/0.156/0.451
0.368/0.160/0.660
MMGN
0.313/0.156/0.366
0.217/0.164/0.418
0.360/0.164/0.635
SDIFT
0.458/0.363/0.490
0.352/0.280/0.626
0.472/0.303/0.710
DBF
0.930/0.826/0.965
0.789/0.744/0.957
0.843/0.749/0.975
TRACE-Filter
0.429/0.084/0.544
0.211/0.103/0.619
0.385/0.103/0.780
Appendix
Table 12: Per-frame error decomposition at ρ=1% , stacked by dataset (Active Matter, Ocean, Supernova). Each cell reports total / observed-frame / missing-frame RMSE under Miss-3, Blk-5, and Blk-10 (same symbols as Table 1 ). Bold marks the best total per column within each dataset block.
nominal 0.90
nominal 0.95
ECE
TRACE-Smoother
0.80
0.86
0.058
TRACE-Frame ens.
0.56
0.62
0.220
Appendix
Table 13: Reliability on Active Matter (Control regime, 28 trajectories): empirical coverage at two nominal credible levels, and expected calibration error (ECE) over ten levels. Closer to the nominal level is better.
ρ
TRACE-Filter
TRACE-Smoother
0.5%
0.104
0.099
1%
0.093
0.089
2%
0.082
0.079
10%
0.072
0.070
Appendix
Table 14: Observation-rate sweep (Active Matter, Control regime, σobs=0 , 28 trajectories, DPS reverse steps N=50 , guidance ζ∝1/ρ ). Field-domain RMSE; lower is better. This sweep uses a reduced 50 -step sampler and is therefore not directly comparable to the default 100 -step results of Table 5 .
Loops
1
2
3
4
6
AM, TRACE-Smoother
0.537
0.498
0.476
0.475
0.468
AM, MMGN
0.782
0.770
0.777
0.771
0.768
Ocean, TRACE-Smoother
0.106
0.097
0.088
0.083
0.084
Ocean, MMGN
0.277
0.283
0.271
0.284
0.286
SN, TRACE-Smoother
0.334
0.320
0.318
0.314
0.311
SN, MMGN
0.321
0.321
0.321
0.321
0.321
Appendix
Table 15: Coverage accumulation: RMSE versus number of revisits (loops) under the circular moving window. “TRACE-Smoother” is the full model (RTS smoothing); “MMGN” is the strongest offline per-frame baseline.
Component
Time
DPS reverse ODE ( N=100 , S=20 )
≈3.5 s / frame
SS-GP filter (forward)
0.64 ms / frame
RTS smoothing (backward)
2.35 ms / frame
FTM decode
1.26 ms / frame
diagonal fusion ( d=32768 )
4.1 ms / frame
Appendix
Table 16: Per-frame inference cost on Active Matter ( d=2304 ), measured on a single NVIDIA RTX 5090 GPU (the closed-form fusion step of Table 9 is timed on CPU).
Figure 6: Active Matter, Miss pattern ( ρ=1% ); five frames mixing observed and missing.
Figure 7: Ocean, Miss pattern ( ρ=1% , gap n=3 ): TRACE-Smoother recovers coherent spatial structures throughout missing intervals; TRACE-Filter drifts on gap frames; offline baselines produce temporally inconsistent transitions. Columns: ground truth, observation, TRACE-Smoother, TRACE-Filter, SDIFT, MMGN, DBF, LRTFR.
Figure 12: Active Matter, moving window (circular, one lap, ρloc=15% ); nine frames span one sweep. TRACE-Smoother accumulates previously-swept regions into a coherent global field, while the per-frame baselines collapse outside the current window.
Figure 15: Failure case: Active Matter moving window where per-frame evidence is too weak for the generative prior; TRACE-Filter propagates the error and TRACE-Smoother cannot recover.
Reconstructing continuous physical fields from sparse measurements is a central inverse problem, but data-driven generative models can produce states that violate governing dynamics. We introduce a physics-informed generative solver that separates stable prior learning from inference-time enforcement of conservation laws. Martingale-Regularized Score Matching regularizes score pretraining with a Score Fokker-Planck constraint, yielding a dynamically stable prior. Physics-Informed Implicit Score Sampling then guides denoising trajectories by gradients of physical residuals, projecting samples toward admissible manifolds without retraining. In acoustics, the method co-generates pressure and particle velocity from sparse sensors, enabling dense virtual arrays that suppress spatial aliasing. The same framework generalizes to real-world ERA5 meteorological fields under extreme sparsity. Together, this work establishes a rigorous and generalizable paradigm for solving high-dimensional inverse problems, bridging the gap between generative artificial intelligence and first-principles science.
Ziyuan Zhu, Keyu Hu, Zhifei Chen +10
School of Advanced Manufacturing Engineering, Nanjing University, Suzhou, 215163, China. · National Laboratory of Solid State Microstructures, Nanjing University, Nanjing, 210093, China. · Suzhou Acoustics Industry Technology Research Institute Co., Ltd., Suzhou, 215513, China. +8
Inferring the evolution of high-dimensional and multi-modal (e.g., spatio-temporal) physical fields from irregular sparse measurements in real time is a fundamental challenge in science and engineering. Existing approaches, including diffusion-based generative models and functional tensor methods, typically operate in offline settings, depend on full temporal observations, or incur substantial inference cost. We propose StreamPhy, an end-to-end framework that enables efficient and accurate streaming inference of full-field physical dynamics from incoming irregular sparse measurements. The framework integrates a data-adaptive observation encoder that is robust to arbitrary observation patterns, a structured state-space model that supports memory-efficient online updates across irregular time intervals, and an expressive Functional Tensor Feature-wise Linear Modulation (FT-FiLM) decoder for continuous-field generation. We prove that FT-FiLM is more expressive than the functional Tucker model, admitting a richer function class for handling complex dynamics. Experiments on three representative physical systems under challenging sampling patterns show that StreamPhy consistently outperforms state-of-the-art baselines, with at least 48% improvement in accuracy and up to 20--100X faster inference than diffusion-based methods.
Panqi Chen, Yifan Sun, Shikai Fang +2
College of Information Science and Electronic Engineering, Zhejiang University · School of EECS,Oregon State University
Reconstructing PDE-governed fields from sparse and irregular measurements is challenging due to their ill-posed nature. Deterministic surrogates are trained on dense fields that struggle with limited measurements and uncertainty quantification. Generative models, by learning distributions over spatiotemporal fields, can better handle sparsity and uncertainty. However, existing generative approaches enforce data consistency and PDE constraints simultaneously via sampling-time gradient guidance, resulting in slow and unstable inference. To this end, we propose PerFlow, a Physics-embedded rectified Flow for efficient sparse reconstruction and uncertainty quantification of spatiotemporal dynamics. PerFlow decouples observation conditioning from physics enforcement, performing guidance-free conditioning by feeding observations into rectified-flow dynamics while embedding hard physics via a constraint-preserving projection (e.g., incompressibility or conservation). Theoretically, we establish invariance guarantees to ensure that trajectories remain on the physics-consistent manifold throughout sampling. Experiments on various PDE systems demonstrate competitive reconstruction accuracy with sound physics consistency, while enabling efficient conditional sampling (e.g., 50 steps) and up to 320x faster inference than 2000-step guided diffusion baselines.
Hao Zhou, Rui Zhang, Han Wan +1
Gaoling School of Artificial Intelligence, Renmin University of China