LaSTR: Language-Driven Time-Series Segment Retrieval

Kota Dohi; Harsh Purohit; Tomoya Nishida; Takashi Endo; Yusuke Ohtsubo; Koichiro Yawata; Koki Takeshita; Tatsuya Sasaki

cs.CLFeb 28, 2026

LaSTR: Language-Driven Time-Series Segment Retrieval

Authors: Kota Dohi, Harsh Purohit, Tomoya Nishida, Takashi Endo, Yusuke Ohtsubo, Koichiro Yawata, Koki Takeshita, Tatsuya Sasaki,

Organizations: Research and Development Group, Hitachi, Ltd.

Abstract

Effectively searching time-series data is essential for system analysis, but existing methods often require expert-designed similarity criteria or rely on global, series-level descriptions. We study language-driven segment retrieval: given a natural language query, the goal is to retrieve relevant local segments from large time-series repositories. We build large-scale segment--caption training data by applying TV2-based segmentation to LOTSA windows and generating segment descriptions with GPT-5.2, and then train a Conformer-based contrastive retriever in a shared text--time-series embedding space. On a held-out test split, we evaluate single-positive retrieval together with caption-side consistency (SBERT and VLM-as-a-judge) under multiple candidate pool sizes. Across all settings, LaSTR outperforms random and CLIP baselines, yielding improved ranking quality and stronger semantic agreement between retrieved segments and query intent.

Figures & tables

Fig. 1: Example VLM input and corresponding outputs for segment captioning. The four captions are generated in the same order as the segment indices shown in the plot.

Pool	Method	Recall@1	Recall@5	Recall@10	mAP
100	Random	0.000	0.000	0.000	0.009
100	CLIP	0.010	0.020	0.030	0.027
100	LaSTR	0.240	0.710	0.840	0.446
1000	Random	0.000	0.000	0.000	0.001
1000	CLIP	0.000	0.000	0.010	0.003
1000	LaSTR	0.050	0.220	0.310	0.145

TABLE I: Retrieval performance evaluated in single-positive segment retrieval setting with different pool sizes.

Pool	Method	Mean SBERT @10	VLM Score (1–5)@10	VLM Precision (binary)@10
100	Random	0.496	3.550	0.351
100	CLIP	0.505	3.631	0.433
100	LaSTR	0.625	4.037	0.719
1000	Random	0.489	3.521	0.327
1000	CLIP	0.505	3.690	0.437
1000	LaSTR	0.639	4.122	0.804

TABLE II: SBERT and VLM-as-a-judge scores (Top-10) across candidate pool sizes.

Fig. 2: Distribution of per-query mean VLM scores over top-10 retrieved segments (test split, pool size $=10000$ , 5-point scale).

Panel of Figure 5 — Fig. 3: Qualitative retrieval examples on the test split. For each query, the figure shows the query caption and the corresponding rank-1 retrieved time-series window at pool size 10,000. Segment boundaries are overlaid, and the retrieved segment is highlighted to indicate the matched region.

Temporal IR

Temporal information retrieval finds information relevant to when events occurred or facts were valid, not just what they describe.

See all 65 papers

TS-Haystack: A Multi-Task Retrieval Benchmark for Long-Context Time-Series Reasoning
Feb 15, 2026Nicolas Zumarraga, Thomas Kaar, Ning Wang +12Time Series Reasoning Time Series QA
1Agentic Systems Lab, ETH Zurich · 2Stanford University · 3Traffic Engineering Group, Institute for Transport Planning and Systems, ETH Zurich +4
Lapras: Latent Reasoning for Time Series Language Models
Oct 8, 2026Yuliang Chen, Yu Yvonne Wu, Patrick Langer +11Time Series Language Modeling Temporal Reasoning in Language Models
Dartmouth College · Aionic Labs · Agentic Systems Lab, ETH Zurich +2
LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics
Apr 19, 2026Yueyang Ding, HaoPeng Zhang, Rui Dai +4Vision-Language Models Time Series Reasoning
Amap, Alibaba Group

Feb 15, 2026J/K move · Enter open · S save

TS-Haystack: A Multi-Task Retrieval Benchmark for Long-Context Time-Series Reasoning

Nicolas Zumarraga, Thomas Kaar, Ning Wang, William Tennien+11

1Agentic Systems Lab, ETH Zurich · 2Stanford University · 3Traffic Engineering Group, Institute for Transport Planning and Systems, ETH Zurich+4

Time Series Reasoning Time Series QA Temporal Reasoning in Language Models Temporal IR Agentic Retrieval Time-Series Foundation Models+2

Open paper AskPDF arXiv

Time Series Language Models (TSLMs) promise reasoning over real-world temporal data, but their ability to retrieve and reason over long time-series remains largely untested. We introduce TS-Haystack, a multi-domain retrieval benchmark with ten event-grounded question-answering tasks over contexts from 100 seconds to 24 hours, spanning direct retrieval, temporal reasoning, multi-step reasoning, and contextual anomaly detection. Existing TSLMs exhibit severe long-context degradation: accuracy declines with context length, direct-tokenization models run out of memory beyond 100 seconds on high-rate signals, and time-interval-grounded tasks collapse toward near-zero accuracy when increasing the time-series lengths, aligning with existing literature on text and multi-modal long context retrieval. An agentic retrieval framework using specialized time-series classifier tools matches or outperforms SoTA TSLMs on 9 of 10 tasks, highlighting agentic retrieval as a promising approach for long-context TSLMs.

Abstract

Temporal IR

Explore similar work

TS-Haystack: A Multi-Task Retrieval Benchmark for Long-Context Time-Series Reasoning

Lapras: Latent Reasoning for Time Series Language Models

LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics