Learning Field Reconstruction from Incomplete Data by Globally Correcting Local Estimates
Organizations: The Chinese University of Hong Kong, Shenzhen · Shanghai Artificial Intelligence Laboratory
Abstract
Reconstructing physical fields from training samples that are always incomplete requires learning spatial structure from fragmented observations. Existing context--query work establishes how held-out observations provide valid training targets, but this does not make the complete-field distribution identifiable when every training field is incomplete. With finite data, weak evidence of sharp transitions and localized variations can further favor averaged predictions that attenuate local detail. A structural prior is therefore needed to favor plausible completions; local spatial relationships offer one grounded in the observations. We propose a locally constructed, globally revisable estimator that explicitly learns local field estimates and subsequently corrects them using full-domain observations. A shared coordinate-conditioned predictor learns from incomplete patches, allowing relatively well-observed neighborhoods to provide direct supervision of local structure. Its overlapping predictions are reconciled into an observation-conditioned consensus field. A full-domain estimator retains the original observations and learns a residual correction around this frozen field estimate, allowing locally constructed structure to be revised by broader evidence. The local estimate serves as both an explicit input, accompanied by its discrepancies with the observations, and a prediction starting point that the global model can revise. On three real-world ocean datasets with authentic observation gaps, our estimator achieves the lowest MSE and highest PSNR on withheld source-supported values, reducing MSE by 28.9%--34.5% against the strongest external baseline.
Figures & tables
| Method | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |||
|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| DINEOF | ||||||
| AmbientDiff | ||||||
| DINDiff | ||||||
| MissDiff | ||||||
| MSM | ||||||
| Estimator | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |||
|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| Local Prior | ||||||
| Full Direct | ||||||
| Low-rank2Global | ||||||
| Global2Global | ||||||
| Prior Direct | ||||||
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Original setting | Principal mechanism | Use in this paper |
|---|---|---|---|
| Evaluated baselines | |||
| DINEOF [ Beckers and Rixen, 2003 , Alvera-Azcárate et al., 2005 ] | Incomplete ocean fields | Low-rank EOF reconstruction | Ocean baseline; low-rank-prior control |
| AmbientDiff [ Daras et al., 2023 ] | Corrupted images | Further corruption; prediction supervised on available measurements | Ocean and PDE baseline |
| DINDiff [ Barth et al., 2024 ] | Satellite ocean chlorophyll | Noise prediction on additionally withheld observations | Ocean and PDE baseline |
| MissDiff [ Ouyang et al., 2023 ] | Incomplete tabular data | Masked denoising score matching | Ocean and PDE baseline |
| MSM [ Park et al., 2026 ] | Natural and medical imaging | Partial measurement scores; stochastic sampling | Ocean and PDE baseline |
| Method | Original training setting | Construction and revision | Distinction from our estimator |
|---|---|---|---|
| GLCIC [ Iizuka et al., 2017 ] | Artificially masked complete images | A completion network trained with global and local discriminators | Local–global consistency is imposed through training losses, rather than a local-value consensus followed by a learned full-domain corrector. |
| Contextual Attention [ Yu et al., 2018 ] | Artificially masked complete images | Coarse completion followed by contextual-attention refinement | Uses a coarse-to-fine image-completion pipeline rather than a reference learned from incomplete spatial patches. |
| EdgeConnect [ Nazeri et al., 2019 ] | Images and derived edge targets | Predicted edges condition subsequent image completion | The intermediate representation is an edge map, not a field-valued estimate used as a residual anchor. |
| Patch Diffusion [ Wang et al., 2023 ] | Clean images with coordinate-conditioned, multiscale training | Learns scores from patch and full-image training views | Patch learning serves diffusion-model training rather than held-out-query prediction followed by a separate full-domain corrector. |
| PaDIS [ Hu et al., 2024 ] | Clean image patches for prior learning | Patch scores define a prior used by an inverse-problem solver | Global measurement constraints enter the solver; there is no separately trained full-domain residual corrector around an observed-query-trained local consensus. |
| MultiDiffusion [ Bar-Tal et al., 2023 ] | A pretrained text-to-image diffusion model | Reconciles regional diffusion updates during sampling | Fusion operates on sampling updates, rather than constructing a frozen clean-value reference for a learned corrector. |
| Setting | Value |
|---|---|
| Anchor retention | 0.7 |
| Guidance scale | 80 ; 32 for the first two updates |
| Sampling steps | 20 |
| Reverse solver / schedule | Guided BFN ODE; uniform time grid |
| Sampling seed | 42 ; run-level random-number state |
| Empty-support handling | Initial proposal plus at most 8 retries; post-loop minimum-size repair |
| Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |
|---|---|---|---|
| Product ID | OCEANCOLOUR_BLK_BGC_L3_MY_009_153 | OCEANCOLOUR_BAL_BGC_L3_MY_009_133 | MULTIOBS_GLO_PHY_SSS_L3_MYNRT_015_014 |
| DOI | 10.48670/moi-00303 | 10.48670/moi-00296 | 10.48670/mds-00368 |
| Variable | CHL | NANO | Sea_Surface_Salinity |
| Spatial subset | Full Black Sea domain; approximately – E, – N | – E, – N | W– E, S– N |
| Native grid | Regional grid; approximately 1-km spacing | Regional grid; approximately 1-km spacing | Global cylindrical EASE-Grid 2.0 |
| Setting | Available training support | Training context rule | Training query | Validation/test input | Scoring support |
|---|---|---|---|---|---|
| Black Sea CHL | All source-supported values | Guided-mask intersection within the current support | Remaining available values | Cached support from a same-split authentic-mask overlay | Withheld source-supported values, |
| Baltic Sea NANO | All source-supported values | Guided-mask intersection within the current support | Remaining available values | Cached support from a same-split authentic-mask overlay | Withheld source-supported values, |
| Global Ocean SSS | All source-supported values | Empirical-track intersection within the current support | Remaining available values | Cached support from a same-split authentic-mask overlay | Withheld source-supported values, |
| Pixel | Fixed mask; approximately | Retention probability ; approximately of the field | Approximately of the field | Fixed mask; approximately | |
| Block | Three fixed blocks | Two available blocks | One available block | Three blocks | Remaining six blocks |
| Block | Eight fixed blocks | Seven available blocks | One available block | Eight blocks | Remaining one block |
| Method | Shallow Water Pixel | Shallow Water Block | Navier–Stokes Pixel | |||
|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| AmbientDiff | ||||||
| DINDiff | ||||||
| MissDiff | ||||||
| MSM | ||||||
| IDCD | ||||||
| Method | Shallow Water Block | Navier–Stokes Block | ||
|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| AmbientDiff | ||||
| DINDiff | ||||
| MissDiff | ||||
| MSM | ||||
| IDCD | ||||
| Aspect | MSM [ Park et al., 2026 ] | Ours |
|---|---|---|
| Training observations | Partial measurements, including noisy or subsampled measurement settings | Incomplete physical-field observations |
| Supervised coordinates | Denoising targets lie on the measurement support supplied in noisy form | Query values are withheld from the network input and supervise local and full-domain predictions |
| Learned object | Partial-measurement denoisers and associated scores | Local clean-value predictions and a full-domain residual corrector |
| Aggregation object | Lifted partial scores or corresponding denoising estimates | Overlapping spatial-patch predictions reconciled into a field-valued consensus |
| Role of full-domain information | Propagated through sampling and observation-consistency updates | Supplied to a separately trained corrector together with the local reference and discrepancy features |
| Reconstruction procedure | Iterative stochastic sampling for generation or conditional reconstruction | A local-to-global reconstruction path per selected context at near-zero diffusion time; final fields are averaged when context ensembling is used |
| Method | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |||
|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| DINEOF | ||||||
| AmbientDiff | ||||||
| DINDiff | ||||||
| MissDiff | ||||||
| MSM | ||||||
| Variant | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |||
|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | |
| A: Local coordinate conditioning | ||||||
| With local coordinates | ||||||
| No local coordinates | ||||||
| B: Explicit prior–evidence representation | ||||||
| Full state | ||||||
| View family / fusion | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS |
|---|---|---|---|
| MSE ( ) | MSE ( ) | MSE ( ) | |
| A: identical overlap views; fusion weights vary | |||
| Confidence-weighted mean | |||
| Gaussian-weighted mean | |||
| Uncertainty-weighted fusion | |||
| Uniform mean (ours) | |||
| Patch / stride | Views per field | Black Sea CHL | Baltic Sea NANO | Global Ocean SSS | |||
|---|---|---|---|---|---|---|---|
| MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | MSE ( ) | PSNR (dB) | ||
| / 16 | 9 | ||||||
| / 8 | 49 | ||||||
| Item | Setting |
|---|---|
| Resolution | |
| Black Sea / Baltic bank | : , stride 32; four windows per field |
| Global SSS bank | : , stride 16; 16 windows per field |
| Eligibility | At least eight scored coordinates, |
| Black Sea / Baltic local views | patches, stride 16; nine views |
| Global SSS local views | patches, stride 8; 49 views |
| Dataset | Mean [95% CI] | Median | Max | |
|---|---|---|---|---|
| Baltic Sea NANO | [-1pt] | |||
| Black Sea CHL | [-1pt] | |||
| Global Ocean SSS | [-1pt] |
| block MSE | ||||
|---|---|---|---|---|
| Dataset | Remote Near | Remote Mid | Remote Far | Local Near |
| Baltic Sea NANO | [-1pt] | [-1pt] | [-1pt] | [-1pt] |
| Black Sea CHL | [-1pt] | [-1pt] | [-1pt] | [-1pt] |
| Global Ocean SSS | [-1pt] | [-1pt] | [-1pt] | [-1pt] |
| Model | Inference | MSE ( ) | PSNR (dB) | Median latency (ms/field) | P95 latency (ms/field) |
|---|---|---|---|---|---|
| A: Black Sea CHL | |||||
| Full Direct | Full-mask | ||||
| Full Direct | Context | ||||
| Full Direct | Context | ||||
| Full Direct | Context | ||||
| Full Direct | Context | ||||
| Mask statistic | Black Sea CHL | SW Pixel 30% |
|---|---|---|
| Full-mask observations / valid domain | 50.72% | 29.96% |
| Subcontext observations / valid domain | 27.70% | 20.97% |
| Supplied observations retained per subcontext | 51.23% | 70.00% |
| Supplied observations covered by the first 4 contexts | 72.93% | 99.19% |
| Supplied observations covered by the first 8 contexts | 78.05% | 99.994% |
| Mean nearest-observation distance: full-mask | 3.09 pixels | 1.15 pixels |