FlowTSFM: Turning Encoder Depth into Quantile Transport

Bahaeddine Abdessalem; Shifeng Xie; Zehao Xiao; Youssef Attia El Hili; Ambroise Odonnat; Jianfeng Zhang; Lujia Pan; Keli Zhang

cs.LGSep 12, 2026

FlowTSFM: Turning Encoder Depth into Quantile Transport

Authors: Bahaeddine Abdessalem, Shifeng Xie, Zehao Xiao, Youssef Attia El Hili, Ambroise Odonnat, Jianfeng Zhang, Lujia Pan, Keli Zhang,

Organizations: Huawei Noah’s Ark Lab, Paris, France · École Polytechnique, France · LIPADE, Université Paris Cité, Paris, France · Centre de Recherche en Informatique, Mines Paris, PSL University, France ·

Abstract

Encoder-based time series foundation models (TSFMs) typically rely on deep stacks of independently parameterized Transformer layers, where only the final forecast is supervised and intermediate representations have no explicit predictive role. We introduce FlowTSFM, an encoder architecture that interprets depth as a recurrent transport process: a single Transformer block is iteratively applied with shared parameters, while a quantile-flow objective supervises intermediate states along a prescribed trajectory from a prior distribution toward the final forecast. The objective combines pinball forecasting loss with path-level position matching. With only 38.8M parameters, FlowTSFM achieves competitive performance on GIFT-Eval and TIME, remaining within 1.8-4.6% MASE of stronger baselines while using approximately $3\times$ fewer parameters than a 12-layer Chronos-2 model (119.5M). Beyond accuracy, we introduce CosMean, a scale-free diagnostic measuring whether recurrent updates consistently align toward the final prediction. Under a matched intermediate-state probing protocol, FlowTSFM achieves a CosMean score of 0.919 compared with 0.350 for Chronos-2, suggesting that recurrent parameter sharing combined with path supervision is associated with substantially more structured predictive trajectories at a favorable accuracy-efficiency trade-off.

Figures & tables

Figure 1: Depth-wise quantile evolution on a held-out Electricity-H series. Columns correspond to increasing normalized depth

t=\frac{k}{K}

. In each panel, the solid line is the median forecast, the darker and lighter bands are the

30

–

70\%

and

10

–

90\%

predictive intervals, and the dashed line is the ground truth. FlowTSFM (top row) already forms a coherent forecast at early exits and refines it smoothly across depth. Chronos-2 (bottom row), probed with the same frozen output head, shows weaker early forecast structure and larger late-stage corrections.

Model	GIFT MASE $\downarrow$	GIFT CRPS $\downarrow$	TIME MASE $\downarrow$	TIME CRPS $\downarrow$	Paramaters
FlowTSFM (base)	0.729	0.500	0.694	0.580	38.8M
Sundial (base)	0.750	0.559	0.758	0.663	128.3M
Tirex	0.716	0.488	0.683	0.573	35.3M
Moirai-2	0.728	0.516	0.703	0.588	11.4M
Timesfm 2.0	0.758	0.550	0.718	0.620	498.8M
Chronos-2	0.697	0.485	0.662	0.556	119.5M

Table 1: Seasonal-Naive-normalized forecasting accuracy on the two standard benchmarks. GIFT-Eval contains 97 configurations and TIME contains 98 tasks. Lower is better.

Model	Dataset macro $\uparrow$	Path weighted $\uparrow$
FlowTSFM	0.919	0.933
Chronos-2	0.350	0.317

Table 2: Mean cosine alignment between local trajectory velocities and the endpoint chord on GIFT-Eval. Dataset-macro averages weight each of the 55 datasets equally, whereas path-weighted averages weight every valid test trajectory equally. Higher is better.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Detailed FlowTSFM computation. (a) Input construction runs once; optional pre-recurrent layers are outside the loop. (b) Attention and MLP contributions form one shared field; only the Euler block updates the recurrent state. (c) The shared decoder also reads $h^{0}$ , so $q^{0}$ is learned and anchored to $r$ . Dashed arrows denote conditioning or training-only supervision.

Setting	Base accuracy run
Context length / patch size	8,192 / 16
$d_{\text{model}}$ / attention heads	1,024 / 16
Quantiles / recurrent calls	99 / 12
Register tokens	4
Parameters	38,845,536
Real / kernel-synthetic data	50% / 50%

Appendix

Table 3: Reported FlowTSFM configuration for the base accuracy run.

Time-Series Foundation Models

Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.

See all 301 papers

QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
Jul 2, 2026Shah Nawaz Haider, Steve Austin, Arnab Barua +2Selective SSMs Time Series Forecasting
Department of Computer Science and Engineering, University of Science and Technology Chittagong · Department of Electrical and Electronic Engineering, University of Science and Technology Chittagong · Faculty of Science, Engineering and Technology, University of Science and Technology Chittagong
Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models
Mar 10, 2026Anurag MishraTime-Series Foundation Models Transformer Interpretability
Rochester Institute of Technology
TempusBench: An Evaluation Framework for Time-Series Forecasting
Apr 13, 2026Denizalp Goktas, Gerardo Riaño-Briceño, Omkar Tekawade +14Time Series Forecasting Forecasting Benchmarks
Simulacrum New York City, NY, USA

Jul 2, 2026J/K move · Enter open · S save

QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting

Shah Nawaz Haider, Steve Austin, Arnab Barua, Sarowar Morshed Shawon+1

Department of Computer Science and Engineering, University of Science and Technology Chittagong · Department of Electrical and Electronic Engineering, University of Science and Technology Chittagong · Faculty of Science, Engineering and Technology, University of Science and Technology Chittagong

Selective SSMs Time Series Forecasting Time-Series Foundation Models Probabilistic Time Series Forecasting

Open paper AskPDF arXiv

Time-series forecasting supports decisions in finance, en-ergy, transportation, public health, and industrial monitoring. Recent foundation models improve transfer across forecast-ing tasks, but many depend on centralized data and Trans-former attention, which restricts their use for long, high-di-mensional, and privacy-sensitive signals. This paper presents QuantFlow, a probabilistic forecasting framework that com-bines inverted sequence embedding, bidirectional Mamba state-space decoders, quantile regression, and federated learning. Each variable is embedded over the complete ob-servation window, processed in forward and reverse direc-tions, and projected to five conditional quantiles. TSMixup expands temporal diversity through Dirichlet-weighted inter-polation while preserving sequence structure. Experiments cover cryptocurrency, traffic, electricity, Electricity Trans-former Temperature, influenza, and weather data. QuantFlow obtains mean squared errors of 0.2834 on ETTm1 and 0.2218 on Weather, and a 20-client non-IID deployment retains use-ful accuracy after three communication rounds without cen-tralizing raw records. The results indicate that selective state-space modelling is a promising basis for scalable, uncer-tainty-aware, and privacy-conscious time-series prediction, while also revealing limitations on irregular epidemiological signals and long-horizon generalization.

Abstract

Time-Series Foundation Models

Explore similar work

QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting

Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models

TempusBench: An Evaluation Framework for Time-Series Forecasting