cs.LGSep 28, 2026

EnJoi: Ensemble Joint Score Filter for Generative Data Assimilation

Authors: Julien Moreau, Marc Lelarge

Organizations: Inria Paris, DI ENS PSL Research University Renault Group

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Data Assimilation (DA) aims to recover the full state of a dynamical system that is only partially observed. A solution is to use Score-based models to generate physically consistent trajectories that agree with the observations. These Autoregressive Diffusion models are trained by conditioning on the previous state; however, they do not take into account the uncertainty of their past predictions. We propose a new diffusion-based assimilation algorithm that dynamically balances the confidence in the current state and the new observations. Crucially, we choose to learn the distribution of the joint state containing both the past and future. This allows us to use a modified version of En4DVar, a classical DA algorithm that relies on the covariance of an ensemble of particles. Experiments on fluid and traffic flow simulations show improved reconstruction performance, especially in situations where observations are sparse and non-homogeneous.

Figures & tables

Figure 1: Three assimilation scenarios on 1D Kuramoto Sivashinsky data. Left: Scenario 1 (Warmup) , the initial condition is perturbed and the filter must correct itself with the observations. Center: Scenario 2 (Noise) , from perfect initial state, the filter should assimilate noisy measurements. Right: Scenario 3 (Forecast) , the model must predict the state without observations. Our proposed EnJoi Score Filter offers the best compromise. We note that our method isn’t the most efficient to forecast from perfect initial conditions, but we argue that this is not an issue in the assimilation setting.
Figure 2: Kuramoto Sivashinsky assimilation experiment with non-homogneous observation patterns. The two left columns show the mean of an ensemble of 10 particles.
Obs. shape H(k)H(k)BernoulliSineExponential
Obs. noise std. σobs\sigma_{\text{obs}}0.10.520.50.5
Cond. P.F. (s) (g=MMPS)0.568 ±\pm 0.0001.042 ±\pm 0.1331.029 ±\pm 0.0940.976 ±\pm 0.1440.921 ±\pm 0.126
Joint AR (s) (g=SDA)0.491 ±\pm 0.1360.619 ±\pm 0.0920.694 ±\pm 0.0310.598 ±\pm 0.0580.591 ±\pm 0.051
Cond. AR (s) (g=SDA)0.457 ±\pm 0.0960.606 ±\pm 0.1280.576 ±\pm 0.0870.574 ±\pm 0.0860.571 ±\pm 0.078
Joint AR (v) (g=scaled)0.088 ±\pm 0.0100.093 ±\pm 0.0080.200 ±\pm 0.0170.516 ±\pm 0.0360.213 ±\pm 0.026
Joint P.F. (v) (g=scaled)0.013 ±\pm 0.0010.063 ±\pm 0.0030.239 ±\pm 0.0140.546 ±\pm 0.1290.120 ±\pm 0.019
Table 1: Kuramoto Sivashinsky CRPS ( ↓\downarrow ) average ±3\pm 3 standard errors on the mean. We denote volatile models with (v) and regular steady models with (s), the guidance method is (g)
Figure 4
AlgorithmObserved grid points (out of 64x64)
 (4,4)(4,4)(8,8)(8,8)(16,16)(16,16)
Joint AR (v) (g=scaled)0.31 ±\pm 0.040.247 ±\pm 0.0280.083 ±\pm 0.007
Cond. P.F. (s) (g=MMPS)0.32 ±\pm 0.040.245 ±\pm 0.0820.043 ±\pm 0.009
Cond. AR (s) (g=SDA)0.23 ±\pm 0.030.186 ±\pm 0.0240.074 ±\pm 0.018
Joint AR (v) (g=SDA)0.18 ±\pm 0.020.043 ±\pm 0.0020.036 ±\pm 0.001
EnJoi (s) (ours)0.12 ±\pm 0.020.047 ±\pm 0.0020.029 ±\pm 0.001
Table 2: Kolmogorov CRPS ( ↓\downarrow ) average ±3\pm 3 standard errors on the mean.
AlgorithmPenetration rate
 2.5%5%
Joint AR (s) (g=scaled)0.21 ±\pm 0.140.21 ±\pm 0.14
Joint P.F. (s) (g=MMPS)0.23 ±\pm 0.050.18 ±\pm 0.05
EnJoi (s) (ours)0.15 ±\pm 0.060.11 ±\pm 0.03
Table 3: ARZ traffic CRPS ( ↓\downarrow ) average ±\pm 3 standard errors on the mean
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

 Learned scoreObservation guidancePast guidance
Sampling Method ∇log⁡p=\quad\nabla\log p=sθ(.,t)s_{\theta}(.,t)+\frac{1}{2}\nabla\|{\color[rgb]{0,0,0}\mathbf{y}}-H{\color[rgb]{0,0,0}\mathbf{\tilde{x}}_{0}}\|^{2}_{R}+\frac{1}{2}\nabla\|{\color[rgb]{0,0,0}\mathbf{x}}^{k}-{\color[rgb]{0,0,0}\mathbf{\tilde{x}}^{k}_{0}}\|^{2}_{P}
Rozet and Louppe (2023)jointon all timestepsno
Savary et al. (2026)conditionalon k+1k+1no
Shysheya et al. (2024)jointon k+1k+1constant P\mathbf{P}
Oursjointon k+1k+1empirical Pk\mathbf{P}^{k} of particles
Appendix
Table 4: Comparison between diffusion trajectory sampling methods.
 KSKolmogorovARZ
 P4T4P4T4
Method    
Cond. AR0:34–0:35–
Joint AR0:56–1:230:15
Joint PF–0:54––
Joint PF (N=128)–––2:48
Appendix
Table 5: Assimilation time per single trajectory (h:mm). The KS and Kolmogorov experiments are run on an Nvidia P4 GPU and the ARZ experiment on an Nvidia T4 GPU. All methods use 10 particles, except the ARZ particle filter which uses 128.
Figure 5: KS assimilation experiment with sine pattern observation. The left column displays one sample from the ensemble filters.
Figure 6: ARZ assimilation experiment. The left column displays one sample from the ensemble filters.
More in this topic

Data Assimilation

Data assimilation combines observations with dynamical models to estimate hidden system states and their uncertainty over time.

See all 58 papers

Explore similar work

CardsList

  1. Flow Matching for Efficient and Scalable Data Assimilation

    Aug 18, 2025Taos Transue, Bohan Chen, So Takao +1Flow MatchingKalman Filtering

    Department of Mathematics and Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, 84112 · The Computing and Mathematical Sciences Department, California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125

  2. DAWIS: Data Assimilation with Windowed Inverse Sampling via Multitask Interpolants

    Oct 2, 2026Erik Wikingsson, Martin Andrae, Tomas Landelius +1Data AssimilationFlow-Based Generative Modeling

    Linköping University, Sweden · Swedish Meteorological and Hydrological Institute

  3. ForcingDAS: Unified and Robust Data Assimilation via Diffusion Forcing

    May 14, 2026Yixuan Jia, Siyi Chen, Yida Pan +9Data Assimilation

    University of Michigan, Ann Arbor · Michigan State University · Los Alamos National Laboratory