Existing time series language models predominantly follow an offline, one-shot paradigm: they process fixed-length inputs independently and do not retain past states. This static approach fundamentally misaligns with the continuous nature of real-world streaming applications and fails to maintain the historical context necessary for cross-window reasoning. To bridge this gap, we introduce Continuous & Live Time Series Understanding (CL-TSU), a novel paradigm that shifts the focus toward maintaining a persistent, stateful understanding over open-ended data streams. To systematically evaluate this paradigm, we present LiveTS-Suite, the first comprehensive data suite tailored for CL-TSU, supporting data generation, model training, and systematic evaluation across 18 tasks spanning six core capabilities. It comprises over 5K streams totaling 2.2B timesteps, annotated with 15K specific response events. We further design an evaluation protocol that jointly assesses response timeliness, content accuracy, and computational efficiency. Building on this foundation, we propose TimeOmni-Live, the first live time series understanding model. It integrates a hybrid time series encoding for natively temporal input, a stream-persistent hybrid memory mechanism to effectively preserve exact historical evidence and long-term task states, and an elastic stream-inference synchronization strategy to reduce processing lag. Evaluations show that TimeOmni-Live establishes a strong baseline for CL-TSU, outperforming frontier LLMs (90.0 vs. 79.3 for Opus 5) with only 4B parameters and achieving 206× lower p95 latency than its backbone. Deployment on a Jetson edge device further shows its ability to support real-time continuous temporal understanding locally at the edge.
Figures & tables
Figure 1: Overview of LiveTS-Suite . The 18 tasks span six capabilities, organized by stream length (vertical) and memory demand (horizontal). Each task illustrates the instruction, the open-ended stream, and example responses. (a) Dataset statistics summarize stream counts by capability and Real Test data sources (see Appendix A.2 for details). (b) Data pipeline details the four-stage data construction process used to generate large-scale training and evaluation data for CL-TSU.
Figure 2: TimeOmni-Live architecture. (a) The continuous Read-Decide-Repeat inference loop. (b) Hybrid time series encoding combining continuous waveform embeddings with explicit textual statistics. (c) Stream-persistent hybrid memory retaining both text states and selected waveforms, utilizing early KV-cache forking to prevent stalling. (d) Elastic stream-inference synchronization that buffers chunks during response generation and realigns via fast single-token silence decisions.
Method
Base LLMs
C1: Instantaneous [-1pt] Conditions
C2: Sequential [-1pt] Conditions
C3: Event [-1pt] Tracking
C4: Event [-1pt] Reference
C5: Online [-1pt] Instructions
C6: Ultra-Long [-1pt] Reliability
Overall
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
Proprietary Models
Opus 5
98.0
98.0
96.1
95.0
86.6
71.4
74.4
62.1
49.0
56.7
72.8
28.3
79.3
66.5
Sonnet 5
100.0
100.0
88.3
90.6
39.4
33.9
75.6
41.4
50.7
50.4
71.5
35.3
70.0
57.7
Open-Source Models
DeepSeek-V4.1-Flash
100.0
100.0
87.8
89.0
64.9
62.1
0.0
4.5
0.0
0.0
62.7
41.5
55.4
50.9
Table 1: Joint Score S on in-distribution (ID) and out-of-distribution Real Testbeds, with results aggregated from 18 tasks into six capabilities. Detailed per-task Timing F1 Fj and Content Accuracy Cj are provided in Tables and 7 of Appendix C . Red : the best, Blue : the 2nd best.
Variant
C1: Instantaneous [-1pt] Conditions
C2: Sequential [-1pt] Conditions
C3: Event [-1pt] Tracking
C4: Event [-1pt] Reference
C5: Online [-1pt] Instructions
C6: Ultra-Long [-1pt] Reliability
Overall
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
(a) Hybrid Time Series Encoding
TimeOmni-Live (trained for 2,500 steps)
83.1
84.9
89.3
77.2
97.9
45.3
93.8
53.9
76.8
54.8
76.3
43.9
85.6
59.0
Time Series Embeddings Only
11.7
15.0
76.5
20.5
98.7
19.2
72.3
25.0
74.1
21.6
48.9
13.2
66.0
18.7
Statistical Values Only
85.6
74.9
95.5
73.4
84.5
38.8
76.0
56.8
83.3
59.4
65.1
47.1
81.6
57.8
(b) Stream-Persistent Hybrid Memory
Table 2: Encoding and memory ablations. Encoding variants are each trained separately for exactly 2,500 steps; memory variants use the main checkpoint without retraining. Bold : the best.
Figure 3: Backbone scores on the 180-stream test subset, with 95% confidence bands.
Figure 4: Generalization evaluations. (a) Instruction paraphrasing on 180 paired ID test streams. (b) Numerical shift on the entire ID and NS test sets, where ΔS is the Joint Score change. (c) Third-party evaluation on KPI using mean per-series Event-F1 PA. Details are provided in Appendix C.4 .
Figure 5: Scheduling ablations.
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Screenshot of the annotation review interface used in Stage 4 (Expert Annotation Audit).
Task
Train
Validation
ID Test
Real Test
Total
Events
Length (timesteps)
MAX
AVG
T1
164
20
50
50
284
780
36,864
30,325
T2
153
19
50
50 †
272
674
36,864
30,850
T3
171
20
50
50
291
589
36,864
30,764
T4
167
20
50
50
287
608
36,864
30,647
T5
174
19
50
50
293
377
36,864
30,362
Appendix
Table 4: Stream counts and lengths in LiveTS-Suite . MAX and AVG pool the four splits; AVG is rounded to the nearest timestep. Events counts annotated response occasions (alarms, scheduled reports, and query answers); each response counts once regardless of its fields. † For T2, controlled generation provides precise joint-condition labels for 50 held-out synthetic Real Test streams with independently sampled condition combinations and thresholds.
Setting
Main checkpoint
Encoding ablations
Seed
0
1337
Initial training
Up to 5,000 steps; selected step 2,500
Fixed 1,000 steps
Continuation
Fixed 1,500 steps
Fixed 1,500 steps
Continuation sampling
T18 share 0.5
Original task sampling
Channel dropout
0.25 in training and continuation
0 in initial training; 0.25 for hybrid in continuation; 0 for single-channel inputs
Warm-up
100 steps per optimization run
100 steps per optimization run
Appendix
Table 5: Qwen3.5-4B training recipes. Encoding ablations correspond to Table 2 (a); memory ablations in Table 2 (b) reuse the main checkpoint.
Figure 7: Code-generation convergence on ID and Real Test. Each stage uses k training streams per task; later stages show diminishing gains with some fluctuations. Horizontal lines mark the trained TimeOmni-Live reference.
Table 6: ID Test: per-task main results. Each model reports Timing F1 ( Fj ), Content Accuracy ( Cj ), and Joint Score ( Sj=100FjCj ). For readability, Fj and Cj are displayed as percentages; Sj is on a 0–100 scale. Scores use unrounded values and the same evaluation results as Table 1 . Overall averages each metric across all 18 tasks; its Joint Score is the mean of task scores, not the product of the aggregate metrics. TimeOmni-Live is repeated for comparison.
Table 7: Real Test: per-task main results. Each model reports Timing F1 ( Fj ), Content Accuracy ( Cj ), and Joint Score ( Sj=100FjCj ). For readability, Fj and Cj are displayed as percentages; Sj is on a 0–100 scale. Scores use unrounded values and the same evaluation results as Table 1 . Overall averages each metric across all 18 tasks; its Joint Score is the mean of task scores, not the product of the aggregate metrics. TimeOmni-Live is repeated for comparison.
Table 8: Per-task encoding ablations from Table 2 (a). Fj and Cj are percentages; Sj=100FjCj . Panels report ID and Real Test; shading marks TimeOmni-Live . Overall averages task scores. Bold : the best.
Table 9: Per-task memory ablations from Table 2 (b). Fj and Cj are percentages; Sj=100FjCj . Panels report ID and Real Test; shading marks TimeOmni-Live . Overall averages task scores. Bold : the best.
Backbone
C1: Instantaneous Conditions
C2: Sequential Conditions
C3: Event Tracking
C4: Event Reference
C5: Online Instructions
C6: Ultra-Long Reliability
Overall
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
ID
Real
Qwen2.5-3B (base)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Qwen2.5-3B-Instruct
83.2
84.8
86.3
61.4
74.3
33.1
86.8
43.9
65.5
36.1
72.8
37.3
77.5
47.8
Qwen2.5-Omni-3B (Thinker)
85.3
81.8
92.3
69.8
82.6
25.8
96.2
49.0
58.3
31.1
82.1
39.9
82.4
48.4
Qwen3-4B
86.0
75.4
90.0
71.9
91.6
33.1
91.1
58.4
84.3
55.1
79.1
38.5
86.6
54.1
Qwen3.5-4B
88.1
84.9
93.2
77.1
95.6
45.0
93.8
52.2
79.4
52.8
88.7
39.5
89.8
57.4
Appendix
Table 10: Backbone comparison by capability on 180 uniformly sampled ID/Real streams (11.2% of full evaluation; Joint Score S ). Bold : best.
Figure 8: Backbone convergence. Training loss (left) and validation response-decision F1 (right) during initial training. Validation uses teacher forcing and is distinct from streaming Timing F1.
Method
C1: Instantaneous Conditions
C2: Sequential Conditions
C3: Event Tracking
C4: Event Reference
C5: Online Instructions
C6: Ultra-Long Reliability
Overall
ID
IP
ID
IP
ID
IP
ID
IP
ID
IP
ID
IP
ID
IP
Proprietary Models
Opus 5
95.0
83.7
95.9
89.6
84.2
44.8
69.2
31.3
50.2
55.0
72.5
49.9
78.1
60.4
Sonnet 5
100.0
77.4
88.1
62.3
41.8
1.0
72.2
43.6
54.7
23.8
74.5
35.0
71.3
39.2
Open-Source Models
DeepSeek-V4.1-Flash
100.0
53.7
92.1
59.3
65.3
20.8
0.0
0.5
0.0
0.0
58.4
18.2
55.4
26.7
Appendix
Table 11: Instruction paraphrasing (IP; Joint Score S ) by capability on paired ID streams before (ID) and after instruction paraphrasing (IP). Overall averages all 18 tasks. Bold : best.
Method
C1: Instantaneous Conditions
C2: Sequential Conditions
C3: Event Tracking
C4: Event Reference
C5: Online Instructions
C6: Ultra-Long Reliability
Overall
ID
NS
ID
NS
ID
NS
ID
NS
ID
NS
ID
NS
ID
NS
Proprietary Models
Opus 5
98.0
97.0
96.1
98.0
86.6
89.2
74.4
77.1
49.0
44.2
72.8
67.0
79.3
78.2
Sonnet 5
100.0
100.0
88.3
78.5
39.4
39.6
75.6
78.6
50.7
51.3
71.5
63.7
70.0
66.6
Open-Source Models
DeepSeek-V4.1-Flash
100.0
100.0
87.8
84.4
64.9
65.5
0.0
0.6
0.0
0.0
62.7
60.5
55.4
54.3
Appendix
Table 12: Numerical shifts (NS; Joint Score S ) by capability on ID and numerical-shift (NS) testbeds. Code-executing models and TimeOmni-Live use all 889 ID and 850 NS streams; the two direct streaming baselines use 90 streams per split due to evaluation cost. Overall averages all 18 tasks. Bold : best.
Setting
Method
Precision ↑
Recall ↑
Event-F1 PA ↑
Zero-shot
LLMAD ( Liu et al., 2025 )
0.033
0.940
0.053
SigLLM-Prompter ( Alnegheimish et al., 2024 )
0.007
0.498
0.014
TimeOmni-Live
0.340
0.197
0.200
KPI-trained
TFAD ( Zhang et al., 2022 )
0.690
0.670
0.564
AnomalyTransformer ( Xu et al., 2022 )
0.482
0.395
0.282
TimeOmni-Live
0.602
0.568
0.539
Appendix
Table 13: KPI evaluation ( ↑ ; per-series means). Published baselines from Gu et al. (2025) .
Model
Group 1
Group 2
Group 3
Pooled (54 streams)
S↑
Tokens ↓
S↑
Tokens ↓
S↑
Tokens ↓
S↑
Tokens ↓
H100
Qwen3.5-4B (streaming)
0.14
8,691,413
0.05
8,640,881
5.75
8,353,534
2.0
8,558,122 (151.4×)
TS-Reasoner-7B (streaming)
0.37
592,390
0.16
588,440
0.46
602,393
0.3
594,623 (10.5×)
TimeOmni-Live ( c=15 )
60.32
56,450
61.85
56,312
44.24
56,837
53.8
56,541 (1×)
Jetson AGX Orin
Appendix
Table 14: Efficiency by group and pooled over all 54 streams. Each group contains 18 streams replayed at 1 chunk/s. Pooled values match Table 3 . Parentheses give ratios to H100 TimeOmni-Live ( c=15 , 1× ), computed before rounding. Tokens are input tokens per 1k chunks.
Figure 9: Streaming backlog at 1 chunk/s. Each panel shows the maximum number of arrived but unfinished chunks within each one-second interval, including the chunk being processed. Configurations are shown separately to avoid overlapping traces; note the different vertical scales.
c
Missed updates ↓
p95 latency (s) ↓
10
117/480
2.246 (3.8×)
15
59/480
0.649 (1.1×)
20
14/480
0.592 (1×)
Appendix
Table 15: Overlap sensitivity on Jetson.
Capability C1: Instantaneous Conditions
Task
T1: Single-Condition Triggering
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Alert me whenever the reading rises above 3.3. One alarm per crossing: report the chunk where the reading first goes over the line, not the chunks that stay over. Each time, report exactly: ”peaked at <value>”, where <value> is the highest reading in that chunk.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 44. Reported content: peaked at 5.0391.
Appendix
Table 16: Task 1 (Single-Condition Triggering): Example with ground-truth response and scoring criteria.
Capability C1: Instantaneous Conditions
Task
T2: Multi-Condition Triggering
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Notify me exactly when the two conditions coincide: the average level is above 1.37, and the fluctuation is above 1.57. Report every chunk in which both conditions hold. Report both readings in one message, exactly: ”average at <reading>, fluctuation at <reading>”, where the first <reading> is that chunk’s mean and the second is that chunk’s standard deviation.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 52. Reported content: average at 2.02, fluctuation at 1.88.
Appendix
Table 17: Task 2 (Multi-Condition Triggering): Example with ground-truth response and scoring criteria.
Capability C2: Sequential Conditions
Task
T3: Consecutive Exceedance
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). I only care if the level remains above 5.1 for 4 chunks straight – then raise an alarm. Speak at the chunk that completes the run, once per run; the run has to break and build up again before the next alarm. When you call it, report exactly: ”peaked at <value>”, where <value> is the highest reading across those consecutive chunks.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 20. Reported content: peaked at 7.8192.
Appendix
Table 18: Task 3 (Consecutive Exceedance): Example with ground-truth response and scoring criteria.
Capability C2: Sequential Conditions
Task
T4: Trend Recognition
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Speak up when the value has clearly increased for 4 straight chunks. ”Clearly” means each chunk mean moving more than 2.257 from the previous chunk. Speak at the chunk that completes the run, once per run; a chunk that fails to move clearly resets the count. Report exactly: ”moving <up—down>, covered <amount>”, where <amount> is the total change of the chunk mean across those consecutive chunks.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Table 19: Task 4 (Trend Recognition): Example with ground-truth response and scoring criteria.
Capability C2: Sequential Conditions
Task
T5: Mean Shift
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Tell me the moment the signal moves clearly off its recent norm and keeps that for 4 chunks. ”Clearly” means the chunk mean moving more than 0.319 away from the median of the 8 chunks just before the shift began, and staying that far away for 4 chunks in a row. Speak at the last of those 4 chunks, once; after an alarm, stay silent until a chunk comes along that no longer counts as clearly shifted from the median of the 8 chunks before it. Report exactly: ”settled <higher—lower> by <amount>”, where <amount> is the average chunk mean after the change minus the average chunk mean before it.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 48. Reported content: settled higher by 1.1149.
Appendix
Table 20: Task 5 (Mean Shift): Example with ground-truth response and scoring criteria.
Capability C2: Sequential Conditions
Task
T6: Event Density
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Raise an alarm once 3 spikes have occurred within the last 6 chunks. A spike means the signal shooting past 5.8; smaller wiggles are not spikes. Speak at the chunk where the count inside the window first reaches 3; after that, stay silent until the count inside the window drops below 3 again. Report exactly: ”spanned <n> chunks”, where <n> is the chunk index of the last spike minus that of the first, among the 3 spikes that set off the alarm.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Table 21: Task 6 (Event Density): Example with ground-truth response and scoring criteria.
Capability C3: Event Tracking
Task
T7: Single-Type Tracking
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Ignore the first 3 upward spikes. Starting with the 4th, report every one. A fault means the signal shooting past 3.1; smaller wiggles are not faults. Report each as: ”fault #<n> — <g> chunks since the last”, where <n> counts every fault since the start (silent ones included) and <g> is this fault’s chunk index minus the previous fault’s.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 401. Reported content: fault #4 — 279 chunks since the last.
Appendix
Table 22: Task 7 (Single-Type Tracking): Example with ground-truth response and scoring criteria.
Capability C3: Event Tracking
Task
T8: Multi-Type Tracking
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Keep an eye on the sine-shaped bursts for me. 2 of them? Fine. The 3rd and beyond – speak up every time. A burst means a shaped pulse dropping below -6.2; the background’s own wiggles never count. Report each as: ”sine #<n> — square at <count>, triangle at <count>”, where <n> is how many sine bursts have occurred so far including this one (so the 3rd sine is ”sine #3”), and each <count> is how many bursts of that other shape have occurred so far, up to and including this chunk.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 232. Reported content: sine #3 — square at 0, triangle at 2.
Appendix
Table 23: Task 8 (Multi-Type Tracking): Example with ground-truth response and scoring criteria.
Capability C3: Event Tracking
Task
T9: Ordered Event Tracking
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Only this sequence matters: first a sine, then a triangle within 24 chunks. Speak at the triangle. Lone bursts and late follow-ups are fine. A burst means a shaped pulse reaching past 5.9; the background’s own wiggles never count. Call it exactly as: ”<g> chunks apart”, where <g> is the chunk index of the triangle minus the chunk index of the sine that came before it.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Table 24: Task 9 (Ordered Event Tracking): Example with ground-truth response and scoring criteria.
Capability C4: Event Reference
Task
T10: Shape Reference
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Right at the start you’ll see one burst – keep its picture in mind. I want to know every time that exact look returns. The rest, stay quiet. A burst means a shaped pulse dropping below -6.0; the background’s own wiggles never count. Report each return exactly as: ”<g> chunks after the opener”, where <g> is the chunk index of this burst minus the chunk index of the opening burst.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 211. Reported content: 209 chunks after the opener.
Appendix
Table 25: Task 10 (Shape Reference): Example with ground-truth response and scoring criteria.
Capability C4: Event Reference
Task
T11: Numerical Reference
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Track the highest peak so far. A newcomer that tops it gets reported with both values, new and old. The rest, silence. A burst only counts if it reaches past 5.3; anything tamer never enters the record book. The first burst that reaches past 5.3 sets the initial record and is not reported; from then on, report only bursts that beat the current record. Use exactly: ”new record <value>, previous <value>”, where the first is the peak (for a below-the-line watch: the lowest point) of this burst and the second is that of the previous record.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 314. Reported content: new record 15.1, previous 11.
Appendix
Table 26: Task 11 (Numerical Reference): Example with ground-truth response and scoring criteria.
Capability C5: Online Instructions
Task
T12: On-Demand Recall
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Just keep watch and keep count of the bursts – I’ll ask you along the way. Answer my questions with the number; otherwise stay quiet. A burst means a shaped pulse dropping below -3.7; the background’s own wiggles never count. Answer with just the number, or just the shape name, and nothing else.
Question at chunk 268: What’s the count on triangle-shaped bursts up to now? Answer with just the number.
Question at chunk 520: How many square-shaped bursts so far? Answer with just the number.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Appendix
Table 27: Task 12 (On-Demand Recall): Example with ground-truth response and scoring criteria.
Capability C5: Online Instructions
Task
T13: Rule Replacement
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Count the faults quietly. The 5th and every one after it gets reported. A fault means the signal shooting past 6.8; smaller wiggles are not faults. Report each as: ”fault #<n> — <g> chunks since the last”, where <n> counts every fault since the start (silent ones included) and <g> is this fault’s chunk index minus the previous fault’s.
Instruction update at chunk 108: New rule: report from the 7th one on. Forget the 5th.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Appendix
Table 28: Task 13 (Rule Replacement): Example with ground-truth response and scoring criteria.
Capability C5: Online Instructions
Task
T14: Rule Addition
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). The line to watch is 10.3. Any burst going past it gets reported with its peak. Rule: whenever a burst’s highest reading rises above 10.3, say ”peaked at <reading>”, where <reading> is the highest reading in that chunk. A second rule will be added part-way through the stream; when both rules apply to the same burst, put both in one message, in either order.
Instruction update at chunk 76: One more rule – sine-shaped bursts, the 2nd and beyond (count from the beginning), report them as well. Rule: whenever a sine burst is the 2nd or later, counting from the very start, say ”sine #<n>”, where <n> is how many sine bursts have occurred so far including this one. When both rules apply to the same burst, put both in one message, in either order.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Appendix
Table 29: Task 14 (Rule Addition): Example with ground-truth response and scoring criteria.
Capability C6: Ultra-Long Reliability
Task
T15: Sustained Vigilance
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). The line is 3.4. Report every crossing, however long the quiet stretches get. One alarm per crossing: report the chunk where the reading first goes over the line, not the chunks that stay over. Each time, report exactly: ”peaked at <value>”, where <value> is the highest reading in that chunk.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 176. Reported content: peaked at 5.5813.
Appendix
Table 30: Task 15 (Sustained Vigilance): Example with ground-truth response and scoring criteria.
Capability C6: Ultra-Long Reliability
Task
T16: Long-Gap Reference
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). The very first thing on this line is a sample burst. Matching bursts after it: report each one. Non-matching: not a word. A burst means a shaped pulse dropping below -10.8; the background’s own wiggles never count. Report each return exactly as: ”<g> chunks after the opener”, where <g> is the chunk index of this burst minus the chunk index of the opening burst.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 3234. Reported content: 3232 chunks after the opener.
Appendix
Table 31: Task 16 (Long-Gap Reference): Example with ground-truth response and scoring criteria.
Capability C6: Ultra-Long Reliability
Task
T17: Sustained Reporting
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). Long shift: keep count of the bursts by shape. Every 500 chunks – at chunk 500, 1000, and so on – give me the running totals unprompted. I may also ask in between. Bursts themselves: no comment. Two kinds of replies. Scheduled report: when chunk 500, 1000, 1500, … arrives, say exactly ”sine <n>, square <n>, triangle <n>”, where each <n> is how many bursts of that shape arrived before this chunk (bursts in the current chunk are not counted yet). Answer to my question: reply with just the number; the number is the count of that shape that arrived before the chunk the question comes in.
Question at chunk 1916: How many sine-shaped bursts so far? Answer with just the number.
Question at chunk 2098: How many triangle-shaped bursts so far? Answer with just the number.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Appendix
Table 32: Task 17 (Sustained Reporting): Example with ground-truth response and scoring criteria.
Capability C6: Ultra-Long Reliability
Task
T18: Cumulative Evidence
User
The signal arrives one chunk at a time (512 samples per chunk; chunks are numbered from 0). This machine ages slowly. If the signal’s level walks away from where it began and the cumulative shift (skip anything under a quarter) tops 60, report it. A single report will do. Treat the first 100 chunks as normal: their average level is the starting level, and the spread (standard deviation) of their samples is one unit. From chunk 100 on, keep a running score: add this chunk’s mean minus the starting level, measured in those units, then subtract 0.25; if the score falls below zero, reset it to zero. Keep a second score the same way for downward moves, using the starting level minus this chunk’s mean. The moment either score exceeds 60, say so once and then stay silent. Report exactly: ”started near chunk <n>, moved <amount>”, where <n> is the chunk at which the winning score last rose from zero, and <amount> is how far this chunk’s mean is from the starting level, in the original reading units, always given as a positive number.
Input time series stream (shown in full; supplied sequentially in chunks of 512 samples):
Note: Only raw samples are provided as signal input. Shading, labels and guide lines are for illustration only and are not model inputs.
Ground Truth
Response time: chunk 988. Reported content: started near chunk 850, moved 1.4909.
Appendix
Table 33: Task 18 (Cumulative Evidence): Example with ground-truth response and scoring criteria.
Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.
Real-world time series evolve continuously, with meaningful changes potentially emerging at any moment. However, existing time-series language models (TSLMs) remain inherently static. They either receive complete sequences for offline processing or alternate between streaming input and response generation, which prevents processing of new observations during interaction. We introduce a new regime, Time-Series Interaction: a model continuously perceives incoming time-series observations and user intent, autonomously decides when to remain silent or respond, and continues processing new observations during response generation. To realize this, we develop TimeInteract with three key designs: a dual-view streaming TS encoder that captures local variations and historical dynamics, a response control mechanism that learns when to trigger a response, and a decoupled streaming inference mechanism that separates control from response generation to avoid blocking subsequent observations. We further formulate a hierarchy of interaction capabilities, progressing from Understanding to Adaptivity. Based on this hierarchy, we construct StreamTSI-34K, a large-scale streaming TS interaction dataset with 34,588 episodes and 77,505 responses across synthetic and real-world time series in single- and multi-turn settings. Across all four interaction levels, TimeInteract consistently outperforms existing LLMs, VLMs, and TSLMs, with gains of up to 23.92 points on challenging tasks. It also improves response triggering while achieving near-zero stream stall and up to 2.15× inference speedup.
The ongoing digitization has led to a proliferation of time-series data streams that monitor a variety of processes, from which valuable insights may be obtained. Further, the emergence of successful foundational language models begs the question of whether it is possible to achieve time-series models with the foundational properties of handling multiple tasks, while being sufficiently lightweight to allow real-time data stream processing. Existing foundational time-series models are often large and only effective in offline settings without stringent time and computational constraints, and where repeated model calibration is not needed. However, when applied to data streams, these models are ineffective due to their size and lack of support for continual calibration, which compromise their ability to deliver accurate real-time responses, their durability, and their deployability in hardware-limited settings. We propose TimeBlocks to enable versatile time-series processing by facilitating the efficient building of lightweight models suitable for multiple tasks under variable conditions. In particular, the method maintains a pool of interchangeable and modular model blocks that can be used to construct new time-series models. When presented with specific time-series data, a routing strategy iteratively selects the most suitable blocks to construct a lightweight and accurate model for the data. We equip TimeBlocks with a method called StreamCore to build a representative small subset of the data stream, which preserves a guaranteed approximation of the stream over time, enabling continual model calibration. An experimental study on multiple data sets and covering multiple tasks shows that TimeBlocks enables to build models capable of outperforming existing baselines.
David Campos, Bin Yang, Tung Kieu +3
Aalborg University, Denmark · University of Amsterdam Netherlands · East China Normal University China +1
Despite the remarkable progress of Video Large Language Models (Video-LLMs), current online architectures still struggle to simultaneously process continuous video streams, decide autonomously when to respond, and preserve long-horizon contextual memory. These obstacles undermine real-time responsiveness and cause severe forgetting throughout prolonged interactions. In this work, we introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long-horizon streams. The design of LiveStarPro rests on three complementary components. The first component is Streaming Verification Decoding (SVeD), an inference framework that identifies the appropriate response timing through single-pass perplexity verification, thereby eliminating the dependency on explicit silence tokens. The second component is Streaming Causal Attention Masks (SCAM), a training strategy that enforces incremental video-language alignment over variable-length streams. The third component is Tree-Structured Hierarchical Memory (TSHM), a recursive memory architecture that organizes evicted historical information into event chains and consequently enables efficient retrieval from effectively unbounded video streams. To facilitate a comprehensive evaluation under realistic online conditions, we further present OmniStarPro, a large-scale benchmark that spans 15 diverse real-world scenarios and that extends to hour-scale streams for the assessment of long-term recall. Extensive experiments demonstrate that LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness and an 18.2% reduction in timing error, while its streaming key-value cache further yields a 1.58x inference speedup over the same model without caching. The model and the code are publicly available at https://github.com/sotayang/LiveStarPro.