NeuralDMD: Interpretable Neural Representation of Dynamics from Sparse and Noisy Measurements

Ali SaraerToosi; Renbo Tu; Esther Y. H. Lin; Kamyar Azizzadenesheli; Aviad Levis

cs.CVJul 3, 2025

NeuralDMD: Interpretable Neural Representation of Dynamics from Sparse and Noisy Measurements

Authors: Ali SaraerToosi, Renbo Tu, Esther Y. H. Lin, Kamyar Azizzadenesheli, Aviad Levis

Organizations: University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · NVIDIA Corporation, USA

Abstract

Many challenges in scientific imaging involve solving ill-posed inverse problems, where the goal is to recover spatio-temporal fields from indirect, noisy, and highly sparse measurements - often without access to ground truth data or reliable simulators. To address this challenging scenario, we present NeuralDMD, an interpretable, untrained (per-instance) reconstruction framework that combines neural implicit representations with Dynamic Mode Decomposition (DMD) to reconstruct continuous spatio-temporal dynamics directly from measurements. NeuralDMD parameterizes DMD modes as continuous neural fields, and imposes a low-rank linear dynamics prior with spectral time evolution to enforce temporal continuity. This formulation enables both forecasting under sparsity, and yields interpretable modes and spectra. We find that NeuralDMD outperforms baselines on a wide variety of tasks: from weather data assimilation from sparse station observations to interferometric (Fourier domain) observations of Sagittarius A, the black hole at the center of our galaxy. Moreover, NeuralDMD remains stable when extrapolating into the future. While this framework is most naturally suited to linear dynamics, we show that it can be applied to nonlinear regimes, though with extrapolation performance that degrades with increasing nonlinearity. Together, these results show that NeuralDMD enables interpretable reconstruction and forecasting of spatio-temporal dynamics directly from sparse and indirect measurements without relying on numerical simulators or training data.

Figures & tables

Figure 1: NeuralDMD reconstructs spatio-temporal dynamics from sparse observations in both indirect and direct sensing regimes, including sparse Fourier (top) and image-domain (bottom) measurements. It models the field as a low-rank linear combination of spatial modes with simple spectral time evolution, yielding an interpretable decomposition that supports forecasting. We demonstrate this on black hole imaging (top) and atmospheric data assimilation (bottom).

Figure 2: (a) Block diagram of the NeuralDMD architecture. The model comprises three MLPs defining the spatio-temporal dynamics: $\Theta_{w}$ , mapping positionally encoded spatial coordinates $(x,y)$ to mode values $w_{j}(x,y)$ ; $\Theta_{\Omega}$ , producing complex spectral components (decay rate $\alpha$ , frequency $\omega$ ) from a learnable latent vector; and $\Theta_{b}$ , mapping a learnable latent vector to the initial state $b_{j}$ for temporal evolution. (b), (c) The learned modes and coefficients are combined to produce the predicted dynamics $\hat{I}(x,y,t)$ . For the black hole setting, predictions are Fourier transformed ( $\hat{F}$ ) and sampled at measured baseline coordinates ( $\mathcal{S}(\hat{\mathcal{F}})$ ). The loss is computed between the observed visibilities $V$ and the corresponding model prediction. For image-domain experiments (weather data assimilation) the loss is instead computed directly on the sparse measurements.

Method	Samples per frame	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeuralDMD (ours)	$\sim 500$	2.3 $\pm$ 0.1	19.1 $\pm$ 0.8	0.53 $\pm$ 0.03	0.59 $\pm$ 0.02
NeuralDMD (ours)	$\sim 5000$	1.5 $\pm$ 0.1	24.7 $\pm$ 0.5	0.70 $\pm$ 0.01	0.32 $\pm$ 0.01
3D-Var	$\sim 500$	-	-	-	-
3D-Var	$\sim 5000$	3.6 $\pm$ 0.8	18 $\pm$ 4	0.48 $\pm$ 0.09	0.5 $\pm$ 0.1
Vanilla MLP	$\sim 500$	2.7 $\pm$ 0.2	17 $\pm$ 1	0.49 $\pm$ 0.04	0.65 $\pm$ 0.03
Vanilla MLP	$\sim 5000$	2.3 $\pm$ 0.2	21.0 $\pm$ 0.6	0.56 $\pm$ 0.01	0.6 $\pm$ 0.2

Table 1: Quantitative evaluation: Weather data assimilation (ERA5). Quantitative comparison (mean

\pm

std over frames) for dense (

\sim 5000

) and sparse (

\sim 500

) observations per frame. We report results at both sparsity levels to highlight performance in a highly undersampled, operationally realistic (

\sim 500

) and in a denser regime (

\sim 5000

), where 3D-Var is applicable. NeuralDMD achieves the best overall performance at both sparsity levels.

Figure 3: Qualitative evaluation: Weather data assimilation (ERA5). Rows show $t{=}5$ hrs, $t{=}100$ hrs, and extrapolation, with $\sim 500$ samples per-frame. 3D-Var is accurate near initialization but degrades over time and cannot extrapolate. NeuralDMD better preserves storm structure and forecasts more reliably than INR baselines, which lack explicit dynamics.

	CORAL ID	NeuralDMD ID	CORAL OOD	NeuralDMD OOD
RMSE $\downarrow$	0.02 $\pm$ 0.01	(3.6 $\pm$ 0.8)e-3	0.05 $\pm$ 0.02	0.014 $\pm$ 0.004
PSNR $\uparrow$	39 $\pm$ 1	56 $\pm$ 1.3	30 $\pm$ 0.3	43 $\pm$ 1
SSIM $\uparrow$	0.89 $\pm$ 0.0	0.9994 $\pm$ 0.0002	0.69 $\pm$ 0.05	0.997 $\pm$ 0.001
LPIPS $\downarrow$	0.012 $\pm$ 0.001	0.0002 $\pm$ 0.0001	0.020 $\pm$ 0.001	0.002 $\pm$ 0.001

Table 2: In distribution (ID) and out of distribution (OOD) quantitative performance of CORAL vs. NeuralDMD . Sparsity rates of

10\%

and

40\%

of the total image pixels are used for fitting for ID and OOD cases respectively.

Panel of Figure 4 — Figure 4: Spatio-temporal reconstruction from sparse EHT 2017 visibilities. (a) Reconstructions of a GRMHD sequence from sparse observations during fitting and extrapolation (forecasting). Values denote image-space MSE (lower is better). NeuralDMD produces lower error and better temporal consistency than StarWarps , resolve, and the spatio-temporal INR (vanilla MLP); The bottom row shows the average optical flow, with $Q$ denoting cosine similarity to the ground truth flow (higher is better). (b) Per-frame normalized image-space MSE. Gray region denotes the extrapolation window, where NeuralDMD remains stable in contrast to the vanilla MLP diverges. (c) Robustness under increased visibility noise and reduced Fourier sampling. Fractional noise is relative to the data amplitude. A $100\times 100$ Fourier grid is used; $20$ Fourier components correspond $0.2\%$ sparsity.

Method	$L_{\text{low}}\,\|\,L_{\text{high}}$	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SIREN	3	0.072 $\pm$ 0.011	23.0 $\pm$ 1.2	0.58 $\pm$ 0.032	0.29 $\pm$ 0.026
SIREN	10	0.075 $\pm$ 0.0058	23.0 $\pm$ 0.68	0.56 $\pm$ 0.019	0.32 $\pm$ 0.011
MFN	3	0.13 $\pm$ 0.0095	18.0 $\pm$ 0.68	0.54 $\pm$ 0.0087	0.30 $\pm$ 0.0036
MFN	10	0.15 $\pm$ 0.025	17.0 $\pm$ 1.6	0.53 $\pm$ 0.020	0.32 $\pm$ 0.022
FFM	256	0.093 $\pm$ 0.0037	21.0 $\pm$ 0.33	0.42 $\pm$ 0.0061	0.66 $\pm$ 0.018
FFM	1024	0.094 $\pm$ 0.0047	21.0 $\pm$ 0.42	0.47 $\pm$ 0.031	0.59 $\pm$ 0.017

Table 3: EHT 2017 ablations. Reconstruction metrics for different coordinate-network backbones, positional-encoding bandwidths

L

, and the direct optimization baseline for

(\Omega,b)

Figure 5: top: PSNR vs. normalized time for Navier-Stokes cylinder flow simulations at Re $=5200$ and Re $=200$ . The white and gray regions denotes the observation and forecasting regions respectively. bottom: Ground truth and NeuralDMD reconstructions at $t_{\text{norm}}=0.5,~1.05,~1.60$ . Error grows more rapidly for the highly non-linear case of Re $=5200$ than Re $=200$ (the same fact is reflected in the PSNR plot).

Figure S1 : GRMHD modes and eigenvalues of NeuralDMD reconstruction from ngEHT+ coverage. The top panel shows both the real (red) and imaginary (blue) parts of the eigenvalues. The imaginary part of the eigenvalues, $\omega=\text{Im}(\Omega)$ , indicates the oscillatory behavior and the real part, $\alpha=\text{Re}(\Omega)$ , indicates the decay rate. Note that the higher order modes have highly decaying behavior $\alpha<-0.05$ , indicating that the reconstruction quality and interpretability of the dynamical system does not depend on these modes. None of the modes depicted here are highly decaying.

Figure S2 : zeroth-order modes of the GRMHD experiment as compared to the average frames, for ngEHT and ngEHT+ coverages. Note that there is but small differences between $w_{0}$ and the average.

Figure S3 : NeuralDMD reconstruction error with varying number of modes. Note that the only high-RMSE case is

r=2

which means that using only 2 modes is not enough for high-fidelity reconstruction.

Figure S4 : CORAL and NeuralDMD reconstructions of ID, OOD, and far OOD 2d wave equation test cases. NeuralDMD outperforms CORAL on all cases, most notably when the test case is far outside of the distribution CORAL is trained on. The left two frames for CORAL and NeuralDMD at times $t=0,8$ are reconstructed from 10 $\%$ uniformly sampled pixels for ID case and $40\%$ samples for OOD and far OOD case. The right two frames $t=16,24$ are then extrapolated with no observational data.

Figure S5 : GRMHD reconstruction from sparse interferometric data with and without additional gain corruptions.

Figure S6 : NeuralDMD reconstruction results as compared to additional INR baselines along with NeuralDMD but with direct optimization on $\Omega,b$ parameters. NeuralDMD outperforms all baselines.

Metric	Vanilla MLP	SIREN	MFN	FFM	Direct $\Omega/b$	NeuralDMD
$\chi^{2}$	0.92	0.94	0.86	0.7	1.24	1.09

Table S1 : Comparison of

\chi^{2}

values across different model architectures for black hole EHT 2017 experiment.

Method	Freq	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SIREN	1	0.1 $\pm$ 0.02	20 $\pm$ 1	0.5 $\pm$ 0.02	0.4 $\pm$ 0.03
	3	0.07 $\pm$ 0.01	20 $\pm$ 1	0.6 $\pm$ 0.03	0.3 $\pm$ 0.03
	10	0.08 $\pm$ 0.006	20 $\pm$ 0.7	0.6 $\pm$ 0.02	0.3 $\pm$ 0.01
MFN	1	0.1 $\pm$ 0.007	20 $\pm$ 0.5	0.5 $\pm$ 0.01	0.3 $\pm$ 0.006
	3	0.1 $\pm$ 0.01	20 $\pm$ 0.7	0.5 $\pm$ 0.009	0.3 $\pm$ 0.004
	10	0.2 $\pm$ 0.03	20 $\pm$ 2	0.5 $\pm$ 0.02	0.3 $\pm$ 0.02

Table S2 : Ablation study across architectures and frequency choices.

Figure S7 : Top 11 Zernicke polynomials used for pretraining NeuralDMD . The “disk” initialization for neural representation is equivalent to the first Zernicke polynomial used for mode 1 of NeuralDMD .

Figure S8 : This figure demonstrates that the dynamics observed by the EHT, when blurred with a $25\mu$ as beam, are well approximated by a linear dynamical system. The inset displays the frame at $1.28$ hours after the initial time: without blurring (blur $=0$ column), highly nonlinear structures are present and not captured by NeuralDMD , while under blurring, these features are smoothed out. The nonlinear structure is outlined and shown in the zoomed-in panel. The curves show the per-frame normalized $\mathrm{L}_{2}$ error between the NeuralDMD reconstructions and the ground truth, with error decreasing as blurring increases and the nonlinear features are progressively suppressed.

Figure S9 : Reconstruction of a GRMHD sequence from ngEHT - synthetic interferometric data. Rows (top to bottom): ground truth, NeuralDMD , neural representation, and StarWarps . The left two frames are data - constrained reconstructions (“data-fitting”), the right two are model extrapolations. Numbers at each frame’s top - left give the normalized L2 error versus ground truth (lower is better). NeuralDMD shows the lowest errors and best visual match across both observed and predicted frames. Since StarWarps directly fits pixels to the observation data, it cannot extrapolate The final column presents mean optical flow; $Q$ is the cosine similarity with the ground - truth flow (higher is better).

Figure S10 : Reconstruction results on a GRMHD with interferometric observations synthesized for the ngEHT+ array. From left to right, the columns show the ground truth, NeuralDMD , neural representation, StarWarps , NeuralDMD reconstruction from ngEHT array. In each row, the first two frames are reconstructed from the observations, i.e., constrained by data for fitting the model. The second two frames are extrapolation results obtained by advancing the recovered dynamical (or neural) model forward in time to unobserved frames. The value at the top left corner of each frame shows the normalized L2-error of that reconstruction as compared to the ground truth (lower is better).

Figure S11 : Reconstruction error for NeuralDMD (blue) and the neural representation (orange) for ngEHT coverage. Solid curves give total error, dashed curves the dynamics - only error (mean image removed), and reconstructed frames are shown at selected points. Data-fitting relies on observations from a $1.6$ hour window; the grey band marks the extrapolation time window. The neural representation error exhibits rapidly increasing error beyond the observational window, whereas NeuralDMD maintains stable reconstructions – preserving both the average structure and dynamic behavior – over extended time horizons.

Figure S12 : EHT2017 coverage cylinder plots. The top row panels show the cylinder plots along the ring at the radius of peak emission. The second row shows the average frames where the selected ring used for the cylinder plot is depicted. The position angle (PA) is measured clockwise from the direction of East ( $E$ ) and NorthWest ( $NW$ ), is highlighted in both for orientation. The white dashed line shows an analytical counter-clockwise Keplerian angular velocity, while the magenta denotes a sub-Keplerian (slower) angular velocity extracted from an image feature. Note that we do not show these lines for neural representation, and StarWarps as the poor reconstruction quality prohibits precise angular velocity measurements.

Figure S13 : ngEHT coverage cylinder plots. The top row panels show the cylinder plots along the ring at the radius of peak emission. The second row shows the average frames where the selected ring used for the cylinder plot is depicted. The position angle (PA) is measured clockwise from the direction of East ( $E$ ) and NorthWest ( $NW$ ), is highlighted in both for orientation. The white dashed line shows an analytical counter-clockwise Keplerian angular velocity, while the magenta denotes a sub-Keplerian (slower) angular velocity extracted from an image feature. Note that we do not show these lines for neural representation, and StarWarps as the poor reconstruction quality prohibits precise angular velocity measurements.

Figure S14 : ngEHT+ coverage cylinder plots. First row shows the cylinder plots at the radius of the maximum emission. The second row shows the average frames where the selected ring is depicted. The white dashed line shows Keplerian angular velocity, while the magenta denotes a sub-Keplerian angular velocity of the point of maximum emission for the ground truth due to its smaller slope. We do not show these lines for StarWarps as the reconstructions are not fine enough to find the anuglar velocity of these features. The reconstructions are from ngEHT+ coverage unless otherwise stated.

Figure S15 : Mean optical flows of ground truth, and ngEHT+ coverage reconstruction results of NeuralDMD , neural representation, StarWarps, and ngEHT coverage reconstruction result of NeuralDMD . The value

Q

shows the cosine similarity (higher is better) of the reconstructions with the ground truth. Note that

Q

does not capture the noise reconstruction; hence, even though

Q

for neural representation is close to NeuralDMD , it hallucinates dynamics, resulting in a poor reconstruction quality.

Figure S16 : NeuralDMD and neural representation (Neural Rep.) imaging performance under different amounts of noise. NeuralDMD consistently outperforms neural representation. The percentage shows the amount of fractional noise added to the data. The two right-most columns are extrapolation after the data-fitting window.

Figure S17 : NeuralDMD and optDMD imaging performance under different sparsity settings. NeuralDMD outperforms TV-regularized optDMD (while there are some numerical issues for some of the first frames). The number in the format $num$ pts is the number of Fourier frequencies used in data fitting for each frame. The two right-most columns are extrapolation after the data-fitting window.

Figure S18 : A comparison of NeuralDMD with the data assimilation baseline 3D-Var . The results are shown on sparsely sampled ERA5 datasets of wind velocity magnitudes. [ First row ]: Results for $5$ hours from the initial frame [ Second row ] $60$ hours from the initial frame with $10\%$ of data given as sparse observations, [ Third Row ] $60$ hours from the initial frame with $1\%$ of the data given. [ Fourth row ] demonstrates NeuralDMD ’s ability to extrapolate by forecasting 5 hours beyond the observation window. From left to right: sparsely sampled observations, 3D-Var , and NeuralDMD reconstructions, and ground truth.

Figure S19 : NeuralDMD and neural representation (Neural Rep.) imaging performance on an edge-on GRMHD simulation. This test is more challenging than a face-on GRMHD since the rotation dynamics of the accretion moves in and out of the image plane as the line of observation is almost parallel to the image plane. Whereas, in the face-on case, the majority of the dynamics is well-captured within the image plane.

Figure S20 : Orbiting hot spot modes (real part) and eigenvalues. The top panel shows both the real (red) and imaginary (blue) parts of the eigenvalues. The imaginary part of the eigenvalues, $\omega=\text{Im}(\Omega)$ , indicates the oscillatory behavior and the real part, $\alpha=\text{Re}(\Omega)$ , indicates the decay rate. The gray-shaded region highlights higher-order modes with fast decay ( $\alpha<-0.05$ ) which do not impact the reconstruction. The bottom panel showcases four of the recovered NeuralDMD modes, illustrating the multi-scale nature of the modal decomposition.

Figure S21 : Frequency coverage of EHT2017 , ngEHT , and the enhanced ngEHT+ . The scatter plots overlay the observed frequencies collected during the entire time window of 3.2 hours, color-coded by time. Note that the uv sampling smoothly changes as the Earth rotates throughout the acquisition period.

Station	x (m)	y (m)	z (m)
PV	5,088,967.900	-301,681.600	3,825,015.800
SMT	-1,828,796.200	-5,054,406.800	3,427,865.200
SMA	-5,464,523.400	-2,493,147.080	2,150,611.750
LMT	-768,713.964	-5,988,541.798	2,063,275.947
ALMA	2,225,061.164	-5,440,057.370	-2,481,681.150
APEX	2,225,039.530	-5,441,197.630	-2,479,303.360

Table S3 : Selected station coordinates for EHT2017 , ngEHT , and ngEHT+ .

Spatiotemporal Forecasting

Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.

See all 410 papers

Aug 22, 2026

Symbolic Neural ODEs: Learning interpretable models from time-series data

We present a machine learning framework for identifying sparse, interpretable models of dynamical systems directly from time-series data. Our approach parameterizes the underlying vector field using a neural architecture and trains it by minimizing a multi-step prediction loss over a finite horizon. To ensure numerical tractability, we optimize a mean absolute error objective averaged across prediction steps, and progressively increase the horizon during training. A key feature of this formulation is that it enforces consistency under repeated composition of the learned dynamics. As a result, the identified models exhibit significantly improved stability compared with approaches based on one-step regression of the vector field. When combined with sparsity-promoting regularization, this leads to parsimonious models that generalize beyond the training data. We demonstrate accurate recovery of systems exhibiting a wide range of behaviors, including stable and unstable fixed points, periodic orbits, and chaotic attractors. For chaotic systems, while long-term trajectory prediction is inherently limited by sensitivity to initial conditions, we show that multi-step training yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents. Moreover, we establish theoretical bounds linking trajectory error to statistical accuracy, providing a step toward a principled explanation for this behavior.

Nibodh Boddupalli, Jeff Moehlis

Dept. of Mechanical Engineering, University of California, Santa Barbara, CA 93106

Nonlinear System Identification System Identification Neural ODEs Sparse Identification of Nonlinear Dynamics Ordinary Differential Equations Chaotic Time-Series Prediction

Ask PDF

Mar 11, 2026

Factorized Neural Implicit DMD for Parametric Dynamics

A data-driven, model-free approach to modeling the temporal evolution of physical systems mitigates the need for explicit knowledge of the governing equations. Even when physical priors such as partial differential equations are available, such systems often reside in high-dimensional state spaces and exhibit nonlinear dynamics, making traditional numerical solvers computationally expensive and ill-suited for real-time analysis and control. Consider the problem of learning a parametric flow of a dynamical system: with an initial field and a set of physical parameters, we aim to predict the system's evolution over time in a way that supports long-horizon rollouts, generalization to unseen parameters, and spectral analysis. We propose a physics-coded neural field parameterization of the Koopman operator's spectral decomposition. Unlike a physics-constrained neural field, which fits a single solution surface, and neural operators, which directly approximate the solution operator at fixed time horizons, our model learns a factorized flow operator that decouples spatial modes and temporal evolution. This structure exposes underlying eigenvalues, modes, and stability of the underlying physical process to enable stable long-term rollouts, interpolation across parameter spaces, and spectral analysis. We demonstrate the efficacy of our method on a range of dynamics problems, showcasing its ability to accurately predict complex spatiotemporal phenomena while providing insights into the system's dynamic behavior.

Siyuan Chen, Zhecheng Wang, Yixin Chen +4

Department of Computer Science, University of British Columbia, Vancouver, Canada · Department of Computer Science, University of Toronto, Toronto, Canada

Dynamical Systems Neural Surrogate Modeling Dynamic Mode Decomposition Koopman Operator Learning Implicit Neural Representations

Ask PDF

Jan 28, 2026

Leveraging Differentiable PDE Solvers for Semi-Neural Spatial Reconstruction From Sparse Measurements

Generating dense physical fields from sparse measurements is a fundamental question in sampling, signal processing, and many other applications. State-of-the-art approaches to this problem either rely on spatial statistics that ignore the governing physics, integrate the physics into a multiple-objective optimization process, or require examples of the complete, fully-resolved simulation state during training, which are frequently unavailable outside of synthetic benchmarks. Here, we present a novel alternative that leverages recent advances in the integration of numerical simulators with data-driven models. Namely, we propose a hybrid modeling pipeline that couples Radial Basis Function (RBF) reconstruction with a Neural Network (NN) correction and a Partial Differential Equation (PDE) solver, so that the numerical simulator itself is embedded directly in the training loop of the learned component. Notably, the NN is trained without assuming availability of examples of the fully-resolved simulation state. This is made possible by implementing the PDE solver so that it is end-to-end differentiable, allowing gradients to be backpropagated through the simulation step during training. This grey-box methodology is evaluated on three standard benchmarks from fluid mechanics, where it achieves superior results over statistical and machine-learning-based reconstruction methods.

Ofek Aloni, Barak Fishbain

Department of Mathematics, Technion, Israel · Department of Civil and Environmental Engineering, Technion, Israel

Differentiable Physics Physics-Informed ML Computational Fluid Dynamics

Ask PDF

Abstract

Spatiotemporal Forecasting

Explore similar work

Symbolic Neural ODEs: Learning interpretable models from time-series data

Factorized Neural Implicit DMD for Parametric Dynamics

Leveraging Differentiable PDE Solvers for Semi-Neural Spatial Reconstruction From Sparse Measurements