cs.CVNov 4, 2026

Dynamical State-Space Diffusion for Spatiotemporal Prediction

Authors: Woojun Lee, Hankyeol Kim, Pilsung Kang

Organizations: Department of Industrial Engineering, Seoul National University

Abstract

Diffusion models for temporal prediction typically corrupt each future frame with independent noise, leaving temporal dependence entirely to the denoiser. We propose Dynamical State-Space Diffusion (DSSD), which learns the temporal structure of the corruption process itself. DSSD builds a temporal kernel from the damped oscillatory modes of a stable diagonal state-space model, whose learned decay rates and frequencies let the corruption capture multiple timescales and oscillatory correlations, while the denoiser architecture remains unchanged. Running this model on a finite clip, however, requires the state before the first frame, which is unobserved; starting from a zero state leaves the early frames with less noise than the diffusion schedule prescribes. We avoid this by using the Cholesky factor of the model's stationary covariance as the temporal operator. We prove that this operator is causal, exactly invertible, keeps each frame's noise variance equal to that of standard diffusion, and reduces to the identity at the clean end of the process, and that it is exactly the Kalman innovations form of the state-space model. The denoiser is trained to predict the temporally correlated noise, and sampling removes and reapplies the temporal mixing at each step. On scientific-field and robot-video datasets, DSSD outperforms diffusion baselines that share its denoiser, and the evolution of its learned modes during training shows that the temporal operator adapts to each dataset.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Diffusion-Driven State Space Models

    Jun 19, 2026Jack Ruder, Michael WojnowiczLatent Dynamics ModelingLatent Diffusion Models

  2. Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting

    Jun 12, 2026Chen Su, Yuanhe Tian, Yan SongTime Series ForecastingProbabilistic Time Series Forecasting

  3. Latent Matters: Learning Deep State-Space Models

    Feb 26, 2026Alexej Klushyn, Richard Kurle, Maximilian Soelch +2State Space ModelsLatent Dynamics Modeling