cs.LGNov 10, 2026

The Unused Input: The Initial State as a Conditioning Interface for Recurrent TSFMs

Authors: Bernhard Voggenberger, Christian Ganhör, Levente Zólyomi, Sebastian Böck, Sepp Hochreiter

Organizations: NXAI GmbH, Linz, Austria · Institute for Machine Learning, Johannes Kepler University, Linz, Austria

Abstract

Time series foundation models (TSFMs) forecast unseen series without task-specific training. Increasingly they are built on recurrent backbones, whose constant per-step cost makes long-context inference cheap, and which, unlike transformers, carry a state. A recurrent backbone computes its prediction from two inputs: the observed context window and the initial recurrent state. In practice only the first is utilised and the initial state is unused by convention. We ask whether the initial state can be utilised in recurrent TSFMs. Using the xLSTM-based zero-shot forecaster TiRex as a case study, we freeze the backbone and train only a small head that writes a context vector into the initial state, so that any change in accuracy is attributable to the injected state rather than to adapted weights. A spectral summary of the input window, learned once and applied unchanged to every series, lowers mean GiftEval MASE by 0.175%, with every seed improving, which neither a parameter-matched LoRA adapter nor a learnable static state reproduces. An embedding of text attached to each series, evaluated on six MoTime datasets, reveals two roles for the state. As a per-dataset adaptation surface, a static state lowers MASE by 0.04-6.96% of the zero-shot baseline (8-70% of the gain of full fine-tuning). As a channel for exogenous information, the text-conditioned state lowers it by a further 0.15-0.41% of the zero-shot baseline on four of six datasets and, combined with a LoRA adapter, exceeds full fine-tuning, which has no access to the text, on one of them. In TiRex, the effect concentrates in the mLSTM blocks and is strongest at short context. The results are preliminary and the improvements small, but they point to the initial state as a direction for further research.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Latent Inference-Time Guidance of Time Series Foundation Models

    Sep 29, 2026Chloé Hashimoto-Cullen, Amaury Durand, Laurent Bozzi +3Time Series ForecastingInference-Time Guidance

  2. Universal Redundancies in Time Series Foundation Models

    Feb 2, 2026Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai +1Transformer InterpretabilityKernel Regression

  3. Align-RAG: Alignment Is All You Need for TSFM In-Context Learning

    Aug 6, 2026Mohammad Asadi, Soheil Hor, Bardiya Akhbari +6Zero-Shot Time Series ForecastingRetrieval-Augmented ICL