Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control

Yinan Huang; Shitij Govil; Bo Dai; Pan Li

cs.LGOct 7, 2026

Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control

Authors: Yinan Huang, Shitij Govil, Bo Dai, Pan Li

Organizations: Georgia Institute of Technology

Abstract

Many scientific forecasting tasks require updating a distribution over future trajectories as new observations arrive. Conventional diffusion and flow models generate each forecast from Gaussian noise, often at the cost of many sampling steps. Warm-start methods reuse earlier predictions to reduce this cost, but their models are not trained to perform the forecast update itself, which can compromise quality under few-step sampling. In this work, we introduce Seq-Flow, a conditional flow model whose ODE transports samples from the previous forecast distribution to the updated one. Because successive forecasts often differ only modestly, this transport starts from an informative distribution and can produce accurate updates with few flow evaluations. Recursive reuse also creates a challenge: errors in one forecast become errors in the initial states of subsequent flows. We address this with self-rollout training, in which a moving average copy of the model generates forecasts that initialize later training updates. Unlike self-forcing methods, which reuse generated outputs as conditioning context, Seq-Flow reuses them as the source of the next flow. Experiments On particle-accelerator beam spill forecasting show Seq-Flow reduces CRPS by 65% under a few-NFE sampling budget, while remaining competitive with strong baselines on fluid-dynamics forecasting tasks. Although trained on self-rollouts of at most four updates, Seq-Flow remains accurate over more than 400 consecutive updates. Our code is available at https://github.com/Graph-COM/Seq-Flow.

Figures & tables

Figure 1: Seq-Flow with self-rollout training. Seq-Flow tracks the forecast distribution by modeling the forecast update. It uses a renoised sample from the previous forecast as the source distribution for the next flow, enabling efficient few-step updates. Self-rollout training exposes the model to its own forecast sources, teaching it to correct errors that would otherwise compound across updates.

Method	NFE	Synthetic	Beam Spill		Burgers’ Equation		RealPDE-FSI
Method	NFE	$\mathcal{W}_{1}$ dist. $\downarrow$	CRPS $\downarrow$	RMSE $\downarrow$	CRPS $\downarrow$	RMSE $\downarrow$	CRPS $\downarrow$	RMSE $\downarrow$
AR Diffusion	$H\times 3$	$1.270$	$0.363$	$1.001$	$0.0589$	$0.1183$	$0.0150$	$0.0260$
Self-forced AR Diffusion	$H\times 3$	$2.765$	$0.055$	$0.209$	$0.0143$	$0.0395$	$0.0031$	$0.0062$
Full-trajectory Diffusion	3	$2.260$	$0.043$	$0.179$	$0.0139$	$0.0366$	$0.0030$	$0.0063$
AR Flow	$H\times 3$	$1.203$	$0.446$	$1.092$	$0.0822$	$0.1401$	$0.0130$	$0.0208$
Self-forced AR Flow	$H\times 3$	$2.872$	$0.059$	$0.230$	$0.0198$	$0.0356$	$0.0039$	$0.0081$

Table 1: Results on the synthetic dataset, Beam Spill, Burgers’ Equation, and RealPDE-FSI. Prediction horizon

H=5

for the synthetic task and

10

for the remaining tasks.

Figure 2: Performance-efficiency trade-offs . X axis: sampling steps (NFE), Y axis: task performance. For clarity, each figure shows only a subset of the top-performing baselines. Left: synthetic task. Right: beam spill forecasting.

Figure 3: Seq-Flow with controlled error accumulation over forecast steps . Each figure shows performance over forecast step (window id). Left: synthetic task. Right: beam spill forecasting.

Figure 4: Short self-rollout steps

T_{0}

leads to robust long-run forecasting . Each figure shows performance per forecast step with different

T_{0}

. Left: synthetic task. Right: beam spill forecasting.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	State shape	Training	Held-out	$L$	$H$
Synthetic	$1$	50,000	5,000	100	5
Beam Spill	$1$	40,000	5,000	430	10
Burgers	$32$	90,000	10,000	100	10
RealPDE-FSI	$2\times 32\times 32$	39	6	100	10

Appendix

Table 2: Dataset settings.

L

: episode length;

H

: forecast horizon. Counts are trajectories; field shapes are channels

\times

grid.

Transformer		Spatiotemporal U-Net
Synthetic, Beam Spill, Burgers		RealPDE-FSI
Hidden width	128	Base width	64
Layers	12	Channel multipliers	$(1,2,4)$
Feed-forward width	512	Attention heads	4
Attention heads	4	Head dimension	32

Appendix

Table 3: Backbone configurations. U-Net channel multipliers scale the base width.

Setting	Synthetic	Beam Spill	Burgers	RealPDE-FSI
Learning rate	$5\times 10^{-4}$	$5\times 10^{-4}$	$5\times 10^{-4}$	$2\times 10^{-4}$
Effective batch size	4096	512	64	32
Weight decay	$10^{-4}$	$10^{-4}$	$10^{-4}$	$10^{-4}$
AdamW $(\beta_{1},\beta_{2})$	$(0.9,0.999)$	$(0.9,0.999)$	$(0.9,0.999)$	$(0.9,0.999)$
Warmup steps	10,000	10,000	10,000	1,000
Checkpoint step	12k	36k	36k	10k

Appendix

Table 4: Seq-Flow settings.

C_{0}/C_{\max}

: initial/maximum context length. Ablations vary NFE or rollout length.

Explore similar work

Variational Streaming Flow: Probabilistic Forecasting in Physical Time
Oct 1, 2026Hans Hao-Hsun Hsu, Minseon Gwak, Soon Hoe Lim +2Dynamical Systems Flow Matching
Georgia Institute of Technology · Lawrence Berkeley National Lab · International Computer Science Institute +2
FlowForge: A Staged Local Rollout Engine for Flow-Field Prediction
Apr 21, 2026Xiaowen Zhang, Ziming Zhou, Fengnian Zhao +1Efficient Inference PDE Surrogate Modeling
Shanghai Jiao Tong University · University of Michigan
Towards Scalable One-Step Generative Modeling for Autoregressive Dynamical System Forecasting
May 7, 2026Tianyue Yang, Xiao XuePDE Surrogate Modeling One-Step Generative Modeling
The Center for Computational Science University College London

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Variational Streaming Flow: Probabilistic Forecasting in Physical Time

Hans Hao-Hsun Hsu, Minseon Gwak, Soon Hoe Lim, Pan Li+1

Georgia Institute of Technology · Lawrence Berkeley National Lab · International Computer Science Institute+2

Dynamical Systems Flow Matching Latent Dynamics Modeling Probabilistic Forecasting

Open paper AskPDF arXiv

Probabilistic forecasting is important for predicting complex dynamical systems because intrinsic randomness and incomplete observations can cause the same observed state to evolve into multiple plausible futures. While flow matching is a flexible approach for probabilistic forecasting, it is computationally expensive. Streaming flow (SF) reformulates this approach to model temporal evolution efficiently by learning a continuous velocity field directly in physical time. However, SF learns a deterministic velocity field. Thus, it provides only a single future trajectory for a given fixed initial state and observation history. To overcome this limitation, we introduce Variational Streaming Flow (VSF). Our approach learns a latent distribution that is conditioned on the dynamics of interest. In turn, this enables probabilistic forecasting. Importantly, we retain the computational efficiency of SF by generating in physical time. Across deterministic and stochastic dynamical systems, VSF demonstrates superior predictive accuracy and distributional fidelity. We demonstrate the advantage for both long-horizon rollouts exceeding 1,000 steps, and settings with bifurcating dynamics. Moreover, VSF can be integrated into existing Joint-Embedding Predictive Architecture (JEPA)-based world models as a plug-and-play predictor to improve temporal dynamics and goal-directed success rate in navigation, motion planning, and manipulation.

Abstract

Explore similar work

Variational Streaming Flow: Probabilistic Forecasting in Physical Time

FlowForge: A Staged Local Rollout Engine for Flow-Field Prediction

Towards Scalable One-Step Generative Modeling for Autoregressive Dynamical System Forecasting