KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting

Jonas Petersen; Pratik Jawahar; Riccardo Maggioni; Gian-Alessandro Lombardi; Matteo Cozzi; Marek Gajewski; Federico Martelli; Camilla Mazzoleni

cs.LGOct 8, 2026

KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting

Authors: Jonas Petersen, Pratik Jawahar, Riccardo Maggioni, Gian-Alessandro Lombardi, Matteo Cozzi, Marek Gajewski, Federico Martelli, Camilla Mazzoleni,

Organizations: ETH Zurich · Forgis AG · University of Vienna

Abstract

Forecasting rare, high-cost events in multivariate time-series systems is hard because their precursors are seldom observed, so a purely data-driven model has little to learn from. The engineers who run these systems, however, hold knowledge that precedes such events, information absent from the scarce labels. We call this a semantic prior and inject it into a self-supervised event forecaster through two switchable mechanisms, a covariate mechanism and an attention-bias mechanism, each the identity at its null, so one architecture serves every prior type with no per-dataset architecture search. A prior helps when it supplies information the observed window does not already contain: in a short context an informative prior beats no prior in-domain (pooled h-AUROC over the no-prior baseline: a periodicity clock +14%, p<0.001; a conservation residual +47% at a 0.25 label fraction), and a working prior transfers out-of-domain (+53%), substituting for feature transfer, while priors the window already recovers stay inert and the attention-bias mechanism shows no measurable gain. KITE is a switchable, attributable route to contextual conditioning, and a substrate for scaling toward cross-domain, pretrained time-series foundation world models that carry their governing structure explicitly.

Figures & tables

Figure 1: Two-stage KITE architecture. Stage 1 (pretraining): a funnel encoder with causal-graph attention bias ingests the context; static/past-known covariates enter the encoder while future-known covariates enter the horizon-conditioned predictor, and a shared-weight target encoder (not EMA) supplies the latent-prediction target. The causal-graph bias acts only in Stage-1 pretraining (the encoder is frozen at Stage 2); its graph is per-dataset , and in the reported runs its gate stays at the null

\beta=0

(Section 3.3 , Appendix G ). Stage 2 (finetuning): a compact head over the frozen encoder consumes a past/future by variable-type input grid and emits a monotone event-probability surface

p(t,\Delta t)

Dataset (estimand)	Prior (#)	KITE (prior)	No prior	Chronos-2	PatchTST	iTransformer
Short-context, in-domain (Claim A)
Occupancy (silent-onset)	Periodicity clock (#5)	0.866	0.762	$0.733{\pm}0.003$	$0.702{\pm}0.037$	$0.754{\pm}0.025$
BATADAL (ongoing)	Conservation resid. (#10)	0.877	0.136	$0.947$ (det.)	$0.732{\pm}0.073$	$0.672{\pm}0.055$
BATADAL (overall, $0.25$ lab.)	Conservation (#10)	$0.644$	0.437	$0.642$ (det.)	$0.656{\pm}0.024$	$0.534{\pm}0.036$
SYNTH-ROUTE-v2 (onset)	Structural descr. (#1, synth.)	$0.892$	0.674	$0.909{\pm}0.007$	$0.710{\pm}0.065$	$0.916{\pm}0.011$
Out-of-domain transfer (Claim B)

Table 1: Semantic priors against external baselines (h-AUROC, mean

\pm

sd over

5

seeds unless noted; higher is better; “det.” = training-free deterministic detector). No prior is the prior-free control under the same recipe (zeroed on the BATADAL rows, absent elsewhere; the two agree on Occupancy,

0.767

vs.

0.762

); the external forecasters have no semantic-prior pathway. The estimand differs by row and is comparable only within a row (stratum-restricted h-AUROC on the onset/ongoing rows, overall on the

0.25

-label row), so No prior values below

0.5

are not errors: on the hard stratum the prior-free backbone anti-ranks the rare-event windows. In the transfer row, N/A marks supervised baselines, which have no transfer mode; Chronos-2 is scored zero-shot on the same

2507

windows (

225

onset), window-bootstrap

95\%

[0.718,0.822]

. The two KITE transfer cells share one Stage-1 encoder; the prior arm scores

0.922

whether it was pretrained on the target, the source, or not at all.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Causal-graph attention bias. A causal graph over variables (

v_{2},v_{3}

driven by

v_{1}

; a) gives the driver-adjacency

A

(b). Since a token is a (variable, time-patch) pair,

A

is tiled over all causally-allowed token pairs and added to the query–key logit as

\beta A

(c; future keys masked). At

\beta{=}0

the bias is the identity.

Dataset	Prior categories	Role
Occupancy	Periodicity (#5), event mechanism (#12)	periodicity clock (flagship)
BATADAL	Conservation (#10), control status (#7)	conservation residual
ETTm1	Identity (#1), periodicity (#5), smoothness (#6)	static priors inert; full-context boundary
Weather	Units (#2), noise (#8), causal edges (#9)	negative control
Beijing air-quality	Bounds/thresholds (#3)	bounds prior
SKAB	Trend (#4)	trend prior

Appendix

Table 2: Dataset–prior alignment. Category numbers refer to Table 3 .

Figure 3: An informative prior beats no prior (h-AUROC; chance

0.5

, rank-based; each bar pairs the prior against the prior-free control of Table 1 ). (a) In-domain, short context: each prior clears its null on the stratum where it applies (clock on silent-onset forecasting ; conservation residual on ongoing-violation detection and at a

0.25

label fraction; a structural descriptor on synthetic SYNTH-ROUTE-v2). (b) Out-of-domain transfer, frozen encoder (BATADAL

\to

SYNTH-CON): a working prior transfers and substitutes for feature transfer, while a static-identity prior does not.

#	Category	Detailed mechanism	Example	Status
1	Identity	Static covariate to encoder	High-voltage active load channel (ETTm1)	E
2	Units/scale	Static covariate to encoder	Temperature in ∘ C, $O(10)$ magnitude (Weather)	N
3	Bounds/thresholds	Covariate; opt. past-only dist.-to-thr.	Air-quality particulate warning $>250\,\mu g/m^{3}$ (BeijingAQ)	C
4	Trend	Static covariate to encoder	Thermocouple monotone, $\tau=+0.97$ (SKAB)	C
5	Periodicity	Future-known covariate to predictor; $P$ static	Daily load cycle $\to$ sin/cos, period $1440$ (ETTm1)	E
6	Smoothness	Static covariate to encoder	Oil temperature slowly varying (ETTm1)	N

Appendix

Table 3: Extended taxonomy with the finer per-category mechanism and implementation status. E : implemented and evaluated here; I : mechanism implemented; N : tested and found inert; C : conceptual scaffold. Static per-variable priors (#1–#4, #6–#8, #12 via static injection) were found inert in the static-injection ablations; #1 is evaluated via a structural per-variable descriptor on synthetic data.

Figure 4: Covariate mechanism. Static and past-known priors (ETTm1 identity; Weather units and smoothness) enter the encoder through the zero-initialized identity-at-null pathway; the future-known periodicity clock (

P=1440

) enters the predictor at the forecast horizon.

Figure 5: KITE tokenization. A

T\times C

window is cut into

N=T/P

temporal patches; a shared

\mathrm{Linear}(P\!\to\!d)

maps each (variable, patch) to a token, yielding a

(C\times N)

grid fed to the encoder

f_{\theta}

. Temporal patches are ordered (RoPE); variables form an unordered set with no per-variable parameter, making KITE permutation-invariant and cardinality-free (

C_{\text{test}}\neq C_{\text{train}}

Configuration	Change	Occupancy (silent-onset)	BATADAL (ongoing)	BATADAL ( $0.25$ labels)
shipped (re-run of Table 1 )	$\Delta$	$+0.096$	$+0.736$	$+0.205$
no per-variable slot	$\Delta$	$+0.092$	$+0.510$	$+0.192$
	cost	$-0.011$ ( $p=0.158$ )	$-0.031$ ( $p=0.537$ )	$-0.007$ ( $p=0.664$ )
scalar-gated fusion, $\tanh(g)$	$\Delta$	$+0.041$	$+0.166$	$+0.099$
	cost	$-0.055$ ( $p=0.015$ )	$-0.570$ ( $p<0.001$ )	$-0.106$ ( $p=0.061$ )
future-known covariate also at pretraining	$\Delta$	$+0.089$	$+0.745$	$+0.195$

Appendix

Table 4: Prior-versus-null

\Delta

h-AUROC under each idealised change, against the identity-at-null (zeroed-covariate) control. The shipped row is this study’s paired re-run of the configuration behind Table 1 , reproducing each flagship effect up to seed noise. Cost is the paired per-seed difference in the prior arm’s h-AUROC from that shipped configuration (so it need not equal the change in

\Delta

, whose null arm can also move).

Row (estimand)	Scorer	Uses	h-AUROC
Occupancy (silent-onset)	no prior (KITE backbone)	sensors	$0.762$
	clock only, time-of-day sine	prior	$0.721$
	clock only, logistic fit on the calendar	prior	$0.078$
	KITE, clock as encoder input channels	sensors + prior	$0.797$
	KITE, clock to the predictor at the horizon	sensors + prior	$\mathbf{0.866}$
BATADAL (ongoing)	no prior (zeroed)	sensors	$0.136$

Appendix

Table 5: Prior-only and generic-routing baselines (h-AUROC). Prior-only scorers use the prior and nothing else: no sensors, no encoder, no pretraining. “Input channels” feeds the clock’s past phase to the encoder as extra variables, the way any covariate-capable forecaster would take it. Occupancy KITE arms are means over

10

seeds, BATADAL over

5

Rare-Event Prediction

See all 11 papers

SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting
Oct 2, 2026Mingtian Tan, Palash Goyal, Mihir Parmar +6Time Series Forecasting Exogenous Time Series Forecasting
Google Cloud AI Research · University of Virginia
World2TS: Real-World Event Context for Time-Series Forecasting
Oct 10, 2026Zhiding Liu, Jiqian Yang, Mingyue Cheng +2Time Series Forecasting Forecasting Benchmarks
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, China · Shenzhen International Graduate School, Tsinghua University, China
Foundation Models for Event Prediction Across Heterogeneous Event Logs
Oct 13, 2026Fabian Spaeh, Sean Jaffe, Shandian Zhe +1Next-Event Prediction Predictive Process Monitoring
Celonis

Oct 2, 2026J/K move · Enter open · S save

SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting

Mingtian Tan, Palash Goyal, Mihir Parmar, Sarkar Snigdha Sarathi Das+5

Google Cloud AI Research · University of Virginia

Time Series Forecasting Exogenous Time Series Forecasting Robust Time Series Forecasting Retrieval-Augmented Time Series Forecasting LLM-Based Time Series Forecasting Non-Stationary Time Series Forecasting

Open paper AskPDF arXiv

Real-world time series are frequently driven by exogenous events and structural shifts, rendering conventional forecasting based solely on historical numerical observations insufficient. While language models can retrieve external news, standard retrieval-augmented approaches struggle with high noise, missing signals, and an inability to reason causally about event impacts. We propose SEER (Self-Evolving Event Reasoning and Retrieval), a closed-loop framework that dynamically optimizes event conditioning for time series forecasting. SEER translates prediction errors into two decoupled feedback mechanisms: (i) a reflective retrieval memory that refines subsequent search queries and filters spurious noise, and (ii) a persistent causal knowledge base that distills transferable domain dynamics. SEER enforces strict chronological boundaries across both event retrieval and reflection, preventing look-ahead bias and data leakage. Across six volatile time-series benchmarks, SEER consistently outperforms state-of-the-art time series foundation models and language model baselines.

Abstract

Rare-Event Prediction

Explore similar work

SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting

World2TS: Real-World Event Context for Time-Series Forecasting

Foundation Models for Event Prediction Across Heterogeneous Event Logs