Time series anomaly detection plays a critical role in many dynamic systems. Despite its importance, previous approaches have primarily relied on unimodal numerical data, overlooking the importance of complementary information from other modalities. In this paper, we propose a novel multimodal time series anomaly detection model (MindTS) that focuses on addressing two key challenges: (1) how to achieve semantically consistent alignment across heterogeneous multimodal data, and (2) how to filter out redundant modality information to enhance cross-modal interaction effectively. To address the first challenge, we propose Fine-grained Time-text Semantic Alignment. It integrates exogenous and endogenous text information through cross-view text fusion and a multimodal alignment mechanism, achieving semantically consistent alignment between time and text modalities. For the second challenge, we introduce Content Condenser Reconstruction, which filters redundant information within the aligned text modality and performs cross-modal reconstruction to enable interaction. Extensive experiments on six real-world multimodal datasets demonstrate that the proposed MindTS achieves competitive or superior results compared to existing methods. The code is available at: https://github.com/decisionintelligence/MindTS.
Figures & tables
Figure 1: (a) LLM-based methods generate endogenous text from time series without incorporating exogenous information. (b) Exogenous-based methods incorporate text information by retrieving background knowledge from the web. The absence of connecting lines indicates that the two modalities are not aligned. (c) MindTS employs cross-view fusion to ensure semantic consistency between the exogenous text and the time series, enabling more precise alignment across modalities.
Figure 2: MindTS overview. Given an input time series X , we first apply instance normalization and patching, then encode the patches using a time encoder. (a) Each patch generates its corresponding endogenous text O . Along with the input exogenous text C , both views are encoded and (b) fused via cross-view fusion to obtain fused text representations Ztext . Time and text representations are then semantically aligned via a multimodal alignment layer. (c) To mitigate textual redundancy, the aligned text is compressed using a content condenser. Finally, (d) the condensed text Zcon is used to reconstruct the masked time series, enhancing cross-modal interaction.
Datasets
Metric
MindTS
DADA
LMixer
UTime
Timer
UniTS
G4TS
Modern
TsNet
DC
A.T.
Patch
TranAD
iTrans
PCA
IF
LODA
HBOS
Weather
Aff-F
82.66
69.01
73.68
76.46
75.46
76.17
72.56
81.06
80.58
42.80
49.22
77.17
77.81
75.37
64.91
54.06
52.55
47.70
V-PR
57.48
30.00
43.47
51.90
43.21
44.35
41.30
52.13
50.09
18.33
19.17
50.03
52.08
42.56
47.13
49.66
55.03
46.58
V-ROC
82.64
61.03
71.71
78.45
73.22
75.08
70.03
81.14
81.91
45.56
43.32
79.97
78.75
73.37
57.38
56.45
57.00
54.16
Energy
Aff-F
74.37
64.38
65.85
61.98
60.20
63.84
66.37
70.76
66.00
47.07
43.39
66.85
49.69
70.81
57.65
62.03
63.45
55.85
V-PR
50.36
34.18
30.35
32.88
29.46
31.04
31.68
36.60
38.61
22.57
19.69
34.41
33.80
35.82
44.30
46.03
48.63
42.57
V-ROC
74.44
54.37
53.04
49.97
46.03
51.15
53.10
65.05
59.47
45.93
31.56
58.31
56.37
63.06
53.07
53.61
55.90
51.50
Table 1: Results of MindTS compared with unimodal and LLM-based methods on six real-world datasets. These methods only use the time series in the dataset. MindTS leverages both endogenous and exogenous text together with time series. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Metric
MindTS
DADA*
LMixer*
UTime*
Timer*
UniTS*
G4TS*
Modern*
TsNet*
Patch*
TranAD*
iTrans*
Weather
Aff-F
82.66
69.73
76.30
73.93
75.37
76.38
76.82
81.50
80.09
77.05
77.73
75.36
V-PR
57.48
30.42
45.94
43.19
43.36
44.58
45.83
53.42
50.53
50.17
52.09
40.30
V-ROC
82.64
61.51
75.29
73.71
73.26
75.21
74.61
81.67
82.06
80.08
78.72
70.11
Energy
Aff-F
74.37
64.80
61.46
65.38
60.36
65.42
67.38
72.13
66.71
66.28
50.53
72.49
V-PR
50.36
34.38
30.91
30.44
29.57
31.34
31.83
37.44
38.88
34.66
33.74
36.21
V-ROC
74.44
55.63
49.06
49.85
46.39
51.89
53.52
66.37
59.80
58.47
56.38
65.62
Table 2: The notation with ∗ indicates the results of extending the baselines to their multimodal versions using the recent time series multimodal framework MM-TSFLib, where both time series and text data from datasets are utilized.
Figure 3: Ablation studies for MindTS, with the highest metrics highlighted in dark-colored bars.
Figure 4: Results of the sensitivity analysis. The vertical coordinate shows the Aff-F score, with higher scores representing better performance. The dark line represents the mean of 5 experiments, and the light area represents the range.
Figure 5: Visualization comparisons of anomaly scores from MindTS for all datasets.
Method
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
GPT2
82.71
57.81
82.97
73.38
43.78
68.40
84.95
52.83
93.96
91.21
55.88
90.15
81.39
48.97
80.96
85.56
60.24
80.81
BERT
80.48
57.84
82.67
72.99
43.07
66.65
84.75
54.25
92.14
87.05
53.75
91.61
84.63
43.95
78.85
82.30
53.68
76.37
LLAMA
81.34
56.93
82.39
75.64
49.21
74.58
84.83
52.99
92.86
90.81
53.24
85.17
80.17
45.76
82.33
85.48
59.24
81.90
DeepSeek
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Table 3: Comparison of different LLMs. Metrics include Aff-F, V-PR, and V-ROC for each dataset.
Method
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Condenser Variant
82.02
56.74
82.36
73.29
49.01
73.20
82.46
56.77
93.89
89.93
51.16
90.55
81.89
47.05
83.77
85.59
62.72
80.67
MindTS
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Table 4: Performance comparison of content condenser variants conditioned with and without unmasked time patches. The best results are highlighted in bold.
Method
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MindTS (w/o LSM )
81.59
55.88
81.48
73.26
48.53
73.18
83.40
54.19
91.58
88.49
49.20
88.31
82.57
49.25
83.64
86.29
63.23
81.11
MindTS
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Table 5: Ablation study on LSM (all results in %, best results are highlighted in bold).
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Dim
AR (%)
Avg Total Length
Timespan (start - end)
Description
Weather
4
17.10
12339
2012.7.17 - 2023.10.20
Temperature and humidity statistics and reports collected from government websites.
Energy
9
17.23
1622
1993.4.5 - 2024.4.29
Gasoline price statistics and energy reports are collected from the U.S. Energy Information Administration.
Environment
1
5.81
15981
1980.1.1 - 2023.9.30
Air Quality Index data and related reports collected from the U.S. Environmental Protection Agency and NBC.
KR
1
6.21
2655
2009.9.15 - 2020.4.1
Financial datasets include numerical stock data from Yahoo Finance and news information collected from various financial news websites such as NASDAQ, Bloomberg, and others.
EWJ
1
9.96
2658
2009.11.16 - 2020.6.9
MDT
1
11.17
2732
2009.8.6 - 2020.6.12
Appendix
Table 6: Statistics and descriptions of datasets used for multimodal time series anomaly detection. AR (%) denotes anomaly ratio.
Dataset
Dim
Prediction Length
Avg Total Length
Timespan (start - end)
Description
Agriculture
1
{6, 8, 10, 12}
496
1983-2024
Retail Broiler Composite.
Climate
5
{6, 8, 10, 12}
496
1983-2024
Drought Level.
SocialGood
1
{6, 8, 10, 12}
900
1950-2024
Unemployment Rate.
Appendix
Table 7: The statistics of evaluation datasets for the forecasting task.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
HBOS
64.47
54.12
54.16
60.80
51.06
51.50
56.42
50.79
51.03
75.16
61.80
61.41
71.82
61.02
62.07
60.26
54.86
55.30
LODA
69.67
56.88
57.00
59.54
56.22
55.90
58.44
50.54
50.69
73.74
60.27
59.99
71.40
60.60
61.65
66.69
55.56
55.98
IF
67.81
56.69
56.45
60.32
52.64
53.61
52.37
45.96
46.20
74.45
61.10
60.70
69.20
57.94
59.24
63.92
53.52
54.02
PCA
67.17
57.80
57.38
61.14
52.64
53.07
48.60
35.71
37.08
63.58
51.01
47.51
54.35
43.78
45.26
54.51
41.90
44.09
iTrans
41.22
74.45
73.37
65.60
64.25
63.06
82.35
70.98
73.81
83.22
74.41
76.12
76.37
68.29
72.16
79.32
67.65
71.87
Appendix
Table 8: Average A-R (AUC-ROC), R-A-R (R-AUC-ROC) and V-ROC (VUS-ROC) accuracy for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
ACC
ACC
ACC
ACC
ACC
ACC
HBOS
84.60
59.69
94.87
95.86
87.03
90.48
LODA
87.60
52.92
92.03
95.85
86.28
90.48
IF
85.62
55.38
84.87
94.54
86.84
88.10
PCA
62.72
59.08
49.27
82.11
72.18
65.02
iTrans
67.02
53.85
81.46
77.97
68.70
77.84
Appendix
Table 9: Average ACC (Accuracy) measures for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
HBOS
31.16
46.37
46.58
21.55
42.14
42.57
16.97
49.84
50.30
41.09
51.69
52.06
25.24
38.48
41.19
28.66
43.16
44.77
LODA
41.22
54.75
55.03
20.75
48.94
48.63
9.98
18.19
18.66
40.14
51.31
51.82
24.16
37.46
40.08
29.81
43.18
44.63
IF
35.44
49.65
49.66
21.17
45.19
46.03
6.18
8.28
8.94
32.21
43.07
43.31
22.86
34.74
37.81
22.41
33.63
35.33
PCA
25.02
47.47
47.13
21.69
43.89
44.30
5.67
16.05
17.87
10.18
18.99
22.13
10.99
16.49
19.37
12.29
19.53
22.93
iTrans
41.22
42.93
42.56
35.63
35.71
35.82
34.90
23.09
24.87
36.05
24.95
27.37
34.42
24.99
28.98
46.91
32.94
36.36
Appendix
Table 10: Average A-P (AUC-PR), R-A-P (R-AUC-PR) and V-PR (VUS-PR) accuracy measures for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
HBOS
58.61
33.89
42.94
24.14
62.50
34.83
92.31
12.90
22.64
73.91
51.52
60.71
38.89
52.83
44.80
64.52
32.79
43.48
LODA
73.97
42.42
53.92
22.29
69.64
33.77
26.21
20.43
22.96
76.19
48.48
59.26
36.84
65.73
43.41
62.86
36.07
45.83
IF
62.09
40.76
49.21
23.03
64.29
34.39
8.15
15.59
10.70
56.67
51.52
53.97
37.31
47.17
41.67
45.45
32.79
38.10
PCA
27.81
73.93
40.41
24.16
96.43
35.12
5.51
47.85
9.88
15.56
42.42
22.76
13.18
32.07
18.68
13.89
40.98
20.75
iTrans
30.48
72.51
42.92
22.67
67.86
34.21
19.67
70.96
30.81
18.84
78.79
30.41
22.44
66.04
33.49
30.00
73.33
42.65
Appendix
Table 11: Average P (Precision), R (Recall) and F1 (F1-score) accuracy measures for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
HBOS
22.51
36.88
27.96
57.14
1.51
2.94
10.50
92.11
18.85
48.15
72.55
57.88
58.18
53.72
55.86
34.62
89.91
49.99
LODA
27.78
70.30
39.82
67.86
20.64
31.65
19.39
31.55
24.02
44.44
77.38
56.46
58.18
47.86
52.52
35.77
86.74
50.65
IF
28.08
50.24
36.02
64.29
5.13
9.50
15.27
10.16
12.20
48.15
53.03
50.47
51.36
50.52
50.94
33.85
63.78
44.22
PCA
61.71
11.98
20.07
60.71
1.42
2.78
44.68
5.45
9.71
39.26
10.81
16.95
34.09
15.58
21.38
37.31
16.47
22.86
DulTF
93.99
12.08
21.41
95.00
6.34
11.89
54.37
4.43
8.19
74.81
9.35
16.62
64.09
42.20
50.89
68.85
37.61
48.65
Appendix
Table 12: Average R-R (Range-Recall), R-P (Range-Precision) and R-F (Range-F1-score) accuracy measures for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
HBOS
74.47
35.09
47.70
54.62
57.14
55.85
94.43
12.61
22.25
89.44
50.78
64.78
80.15
64.76
71.03
91.16
36.30
52.33
LODA
81.60
38.78
52.55
54.15
76.60
63.45
71.07
34.49
46.45
96.99
44.44
60.96
79.74
65.73
72.06
91.26
39.42
55.56
IF
76.25
41.88
76.25
54.50
71.96
62.03
51.31
42.52
46.51
86.20
58.05
69.38
79.18
58.89
67.55
87.66
38.75
53.74
PCA
59.14
71.93
64.91
54.88
60.71
57.65
46.75
68.64
55.62
61.94
44.44
51.75
53.93
48.48
51.06
56.98
52.51
54.66
DulTF
55.12
98.43
70.66
53.22
99.39
69.32
50.04
87.83
63.76
49.01
88.62
63.11
76.97
80.47
78.68
76.21
80.53
78.31
Appendix
Table 13: Average Aff-P (Affiliated-Precision), Aff-R (Affiliated-Recall) and Aff-F (Affiliated-F1score) accuracy measures for MindTS and all numerical-only unimodal methods. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
A-R
R-A-R
V-ROC
iTrans*
73.48
71.21
70.11
67.76
67.07
65.62
81.33
70.60
73.66
84.28
75.71
77.17
78.23
70.67
74.11
78.11
66.36
69.95
TranAD*
85.51
79.33
78.72
67.00
56.66
56.38
26.51
9.93
14.29
60.76
37.23
41.24
60.55
44.36
49.85
44.36
24.82
28.88
Patch*
82.18
80.59
80.08
66.89
61.57
58.47
94.17
91.14
90.87
82.28
72.94
74.80
78.68
69.48
72.21
84.55
75.65
77.72
TsNet*
81.28
83.22
82.06
71.53
61.97
59.80
91.91
87.70
88.14
85.92
78.32
78.94
82.52
74.40
75.91
82.05
68.57
73.57
Modern*
53.07
81.67
81.67
71.61
66.80
66.37
92.76
89.53
89.14
93.66
90.20
89.22
87.96
83.90
83.98
88.99
81.95
82.66
Appendix
Table 14: Average A-R (AUC-ROC), R-A-R (R-AUC-ROC) and V-ROC (VUS-ROC) accuracy for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
ACC
ACC
ACC
ACC
ACC
ACC
iTrans*
76.09
63.69
87.31
70.62
74.44
73.44
TranAD*
64.67
70.15
44.73
87.95
64.47
65.38
Patch*
74.59
44.92
90.22
84.37
85.53
81.32
TsNet*
82.01
66.15
93.25
94.35
84.02
76.92
Modern*
80.23
62.46
90.50
89.45
87.22
84.62
Appendix
Table 15: Average ACC (Accuracy) measures for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
A-P
R-A-P
V-PR
iTrans*
38.56
40.43
40.30
35.68
36.17
36.21
35.35
23.91
25.85
37.89
25.61
28.12
33.63
25.74
29.79
44.10
30.76
33.88
TranAD*
60.91
52.04
52.09
36.34
33.13
33.74
7.08
4.51
4.93
53.27
28.07
28.47
27.89
15.25
17.87
26.06
13.31
14.55
Patch*
53.69
49.95
50.17
34.66
35.51
34.66
58.65
45.40
45.52
53.60
35.57
36.37
47.92
33.45
36.22
54.52
39.92
41.85
TsNet*
48.29
51.00
50.53
42.47
38.47
38.88
63.82
50.39
50.39
67.58
53.03
51.73
55.04
41.95
43.28
60.61
41.55
52.30
Modern*
53.07
54.17
53.42
33.98
36.64
37.44
54.31
41.53
41.36
57.71
41.72
40.86
54.48
44.64
45.41
65.84
52.31
45.88
Appendix
Table 16: Average A-P (AUC-PR), R-A-P (R-AUC-PR) and V-PR (VUS-PR) accuracy measures for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
iTrans*
36.62
54.50
43.81
25.78
58.93
35.87
26.19
65.05
37.35
15.64
84.84
26.42
22.15
62.26
32.67
26.40
77.05
39.33
TranAD*
31.65
91.94
47.08
27.47
44.64
34.01
2.52
22.58
4.54
26.87
54.55
36.00
13.44
47.17
20.92
12.35
34.43
18.18
Patch*
37.70
74.41
50.04
21.40
82.14
33.95
34.84
78.49
48.26
24.49
72.73
36.64
36.36
60.38
45.39
33.60
68.85
45.16
TsNet*
47.77
55.92
51.53
29.55
69.64
41.49
44.79
69.35
54.43
53.19
75.76
62.50
35.45
73.58
47.85
29.56
77.05
42.73
Modern*
44.68
65.64
53.17
28.57
78.57
41.90
34.87
73.12
47.22
34.67
78.79
48.15
41.76
71.70
52.78
40.50
80.33
53.85
Appendix
Table 17: Average P (Precision), R (Recall) and F1 (F1-score) accuracy measures for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
R-R
R-P
R-F
iTrans*
57.06
28.34
37.87
60.48
21.28
31.48
63.78
24.31
35.20
84.20
10.59
18.81
70.00
20.27
31.43
78.46
25.60
38.60
TranAD*
87.59
22.86
36.26
39.29
11.04
17.24
21.13
3.54
6.06
50.37
28.75
36.61
51.36
15.69
24.04
36.53
19.41
25.35
Patch*
76.28
26.40
39.23
73.10
13.12
22.25
79.48
38.28
51.67
76.30
23.26
35.65
65.00
39.94
49.48
67.50
35.81
46.80
TsNet*
68.30
44.44
53.85
57.86
28.17
37.89
71.63
44.48
54.88
80.00
58.85
67.82
79.55
38.61
51.99
75.38
26.87
39.62
Modern*
77.18
37.38
50.37
77.57
24.80
37.58
75.37
37.65
50.22
81.23
42.62
55.91
80.00
47.45
59.57
80.96
41.71
55.05
Appendix
Table 18: Average R-R (Range-Recall), R-P (Range-Precision) and R-F (Range-F1-score) accuracy measures for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
Aff-P
Aff-R
Aff-F
iTrans*
65.34
89.01
75.36
61.39
88.49
72.49
68.74
85.01
76.02
66.33
95.80
78.39
67.35
93.65
78.23
65.66
94.48
77.47
TranAD*
65.06
96.53
77.73
55.68
46.25
50.53
46.37
85.21
61.38
72.94
72.07
72.50
56.74
88.12
69.03
58.33
67.33
63.60
Patch*
65.00
95.59
77.05
52.79
89.04
66.28
76.82
87.27
81.71
74.63
85.09
79.52
74.07
77.53
76.49
73.28
85.38
78.87
TsNet*
72.96
88.77
80.09
59.98
70.49
66.71
78.96
81.50
80.21
86.98
84.73
85.84
75.23
89.92
81.92
70.26
94.56
80.62
Modern*
71.93
94.02
81.50
61.88
87.67
72.13
77.34
85.83
81.36
80.39
89.87
84.87
77.77
86.27
81.88
75.68
88.72
81.68
Appendix
Table 19: Average Aff-P (Affiliated-Precision), Aff-R (Affiliated-Recall) and Aff-F (Affiliated-F1score) accuracy measures for MindTS and baselines within the MM-TSFLib. The best results are highlighted in bold, and the second-best results are underlined.
Figure 6: Visualization comparisons of anomaly scores between MindTS and GPT4TS for all datasets.
Datasets
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MindTS (q/kv reverse)
82.29
57.33
82.27
73.73
48.22
73.06
83.94
55.49
92.64
87.14
50.72
88.72
82.71
49.99
84.34
87.95
65.07
82.30
MindTS (self-attention)
81.69
56.69
82.46
72.47
48.17
71.18
85.79
56.54
93.41
87.27
52.48
90.31
82.96
46.28
83.59
83.82
57.67
77.80
MindTS (two-way)
79.43
57.17
82.70
72.57
44.64
70.42
84.47
53.17
93.17
87.39
54.54
91.77
77.76
46.92
78.28
81.12
57.62
75.23
MindTS
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Appendix
Table 20: Evaluation of cross-view text fusion variants.
Datasets
(a) w/o text
(b) w/o filtering
(c) LLM-based compression
(d) content condenser
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
KR
84.11
45.11
79.43
80.52
37.82
74.38
87.32
47.95
88.52
90.28
53.15
89.86
EWJ
81.87
45.22
79.32
78.79
38.47
80.03
80.26
42.90
78.84
83.89
50.42
84.12
MDT
84.00
58.32
81.68
81.79
51.40
75.43
84.31
58.74
81.13
89.19
65.44
83.02
Appendix
Table 21: Ablation on text redundancy filtering strategies. The best results are highlighted in bold.
Method
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MindTS(unimodel)
75.96
45.69
74.19
73.14
46.66
71.36
80.16
44.28
86.43
84.11
45.11
79.43
81.87
45.22
79.32
84.00
58.32
81.68
DADA
69.01
30.00
61.03
64.38
34.18
54.37
84.11
54.20
87.69
84.22
45.90
70.82
81.26
43.36
71.79
77.99
46.81
66.76
ModernTCN
81.06
52.13
81.14
70.76
36.60
65.05
81.07
42.26
89.78
84.42
39.95
88.87
81.57
44.75
83.88
80.81
52.18
82.30
Timer
75.46
43.21
73.22
60.20
29.46
46.03
84.19
51.42
92.10
89.55
51.41
75.99
78.06
33.17
67.72
78.51
38.38
60.28
MindTS
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Appendix
Table 22: The results of MindTS and MindTS(unimodel) across all datasets (all results in %, best results are highlighted in bold).
Method
Inference Time (s)
Memory Cost (GB)
MDT
KR
EWJ
MDT
KR
EWJ
MindTS
0.2302
0.1977
0.4130
14.69
14.62
14.41
ModernTCN ∗
0.1582
0.1383
0.3965
13.61
13.66
13.57
GPT4TS ∗
0.2676
0.2425
0.4716
13.85
13.89
13.80
LLMMixer ∗
0.2585
0.2104
0.4619
14.13
14.08
13.97
UniTime ∗
0.2537
0.2462
0.4760
14.20
14.15
14.06
Appendix
Table 23: Run times and memory costs on different datasets. Lower values represent better performance. The notation with ∗ denotes results obtained by extending the baselines using the recent time-series multimodal framework MM-TSFLib.
Method
MindTS
MindTS (noisy)
MindTS (irrelevant)
MindTS (incomplete)
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MDT
89.19
65.44
83.02
87.45
62.12
81.02
86.54
60.10
80.17
87.85
61.95
80.59
Energy
74.37
50.36
74.44
72.53
47.42
72.16
72.21
46.52
71.29
73.13
47.62
73.50
Appendix
Table 24: Results under different types of exogenous text quality variations (all results in %, best results are highlighted in bold).
Dataset
MindTS
s = 0.2
s = 0.4
s = 0.6
s = 0.8
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MDT
89.19
65.44
83.02
87.45
62.12
81.02
86.38
60.03
79.46
83.71
58.71
78.69
79.84
55.21
76.46
Energy
74.37
50.36
74.44
72.53
49.42
73.16
71.17
48.26
70.03
69.35
47.22
69.97
66.69
45.02
66.13
Appendix
Table 25: Results of the noisy method under different noise strengths (s).
Method
MindTS
MindTS (noisy + irrelevant)
MindTS (noisy + incomplete)
MindTS (irrelevant + incomplete)
MindTS (three types)
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MDT
89.19
65.44
83.02
84.91
57.29
78.23
85.28
55.60
78.34
84.81
58.34
77.68
81.64
52.17
75.28
Energy
74.37
50.36
74.44
70.33
44.71
70.18
71.89
45.86
69.96
69.74
45.26
70.51
66.05
42.08
66.88
Appendix
Table 26: Results under combined types of exogenous text quality variations.
Figure 7: Visualization comparisons between before and after alignment. The x-axis denotes time representations of each patch, while the y-axis represents text representations of each patch.
Figure 8: Visualization comparisons of the anomaly score between with and without alignment.
Figure 9: Prompt example.
Figure 10: Different template formulation.
Dataset
Metric
MindTS
MindTS-S(a)
MindTS-S(b)
MindTS-S(c)
MindTS-S(d)
MindTS-T
MindTS-TG
MDT
Aff-F
89.19
83.22
87.59
87.62
87.14
88.38
84.78
V-PR
65.44
57.27
64.05
64.49
64.33
64.65
58.49
V-ROC
83.02
78.72
82.75
82.13
82.90
83.19
80.75
Energy
Aff-F
74.37
66.65
73.74
74.03
73.85
74.21
69.31
V-PR
50.36
44.29
48.77
49.25
49.81
49.34
45.32
V-ROC
74.44
68.59
72.33
71.78
72.66
72.14
67.97
Appendix
Table 27: Ablation on endogenous prompt design across different variants.
Models
MindTS
Time-MMD
TaTS
Metrics
mse
mae
mse
mae
mse
mae
Agriculture
6
0.167
0.269
0.146
0.263
0.140
0.251
8
0.195
0.283
0.189
0.310
0.187
0.282
10
0.228
0.316
0.254
0.320
0.244
0.320
12
0.258
0.343
0.338
0.369
0.290
0.350
AVG
0.212
0.303
0.232
0.316
0.215
0.301
Appendix
Table 28: Multimodel time series forecasting results with forecasting horizons F∈{6,8,10,12} .
Method
Weather
Energy
Environment
KR
EWJ
MDT
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MindTS ( Htime as query)
79.35
56.21
80.13
72.69
47.52
71.47
82.67
52.45
92.87
87.93
48.27
89.02
82.33
48.98
81.48
87.85
64.33
81.26
MindTS
82.66
57.48
82.64
74.37
50.36
74.44
85.29
56.79
93.78
90.28
53.15
89.86
83.89
50.42
84.12
89.19
65.44
83.02
Appendix
Table 29: Ablation study on Htime as the query to fuse exogenous text, without endogenous text generation (all results in %, best results are highlighted in bold).
Figure 11: Scalability comparison under different data sizes.
Datasets
MindTS (auxiliary)
MindTS (standalone)
Metric
Aff-F
V-PR
V-ROC
Aff-F
V-PR
V-ROC
MDT
89.19
65.44
83.02
80.33
57.18
76.35
Energy
74.37
50.36
74.44
70.28
44.89
69.61
Appendix
Table 30: Multimodal alignment as a standalone objective (all results in %, best in Bold).
Datasets
MindTS
GPT4TS*
LLMMixer*
ModernTCN*
UniTime*
KR
80.83
81.17
84.35
19.72
77.98
EWJ
75.56
73.66
70.09
18.65
70.23
MDT
73.76
74.49
70.01
18.61
68.53
Appendix
Table 31: Training time (s) comparison with baselines.
Reconstruction errors in multivariate time-series anomaly detection may not reliably distinguish abnormal behavior from benign deviations. Language-derived semantics offer complementary context, but existing multimodal approaches may rely on time-associated paired textual information that is difficult to obtain consistently and is not provided by standard multivariate time-series anomaly detection benchmarks. This setting poses two challenges: (1) conditioning masked reconstruction on window-specific semantics without exposing exact numerical targets or anomaly-specific cues, and (2) using a window-independent concept of normality as a complementary semantic reference rather than an independent anomaly detector. We propose LLM-Enhanced Alignment and Reconstruction with Normality Guidance for Time Series (LEARN-TS), which uses a frozen language model to construct two role-separated semantic representations without requiring temporally paired external text. Window-specific observation semantics encode temporal and cross-variable context without exact numerical values to guide channel-shared patch-masked reconstruction. A fixed, dataset-agnostic normality prompt provides a window-independent semantic reference for aligning normal representations and estimating normality discrepancy. At inference, masking each temporal patch once yields timestamp-level reconstruction evidence, conditionally modulated by discrepancy from a separate unmasked view. Across four benchmarks, LEARN-TS achieves the highest mean performance in 13 of 16 dataset-metric comparisons. Controlled ablations examine observation conditioning, joint normality alignment and scoring, and reference content, showing dataset-dependent ranking benefits and modest average gains from semantic over random references.
Jahyeob Koo, Kio Yun, Byoungmo Koo +1
Department of Industrial and Management Engineering, Korea University Seoul, Republic of Korea
Recent advances in multimodal learning have motivated the integration of auxiliary modalities such as text or vision into time series (TS) forecasting. However, most existing methods provide limited gains, often improving performance only in specific datasets or relying on architecture-specific designs that limit generalization. In this paper, we show that multimodal models with naive fusion strategies (e.g., simple addition or concatenation) often underperform unimodal TS models, which we attribute to the uncontrolled integration of auxiliary modalities which may introduce irrelevant information. Motivated by this observation, we explore various constrained fusion methods designed to control such integration and find that they consistently outperform naive fusion methods. Furthermore, we propose Controlled Fusion Adapter (CFA), a simple plug-in method that enables controlled cross-modal interactions without modifying the TS backbone, integrating only relevant textual information aligned with TS dynamics. CFA employs low rank adapters to filter irrelevant textual information before fusing it into temporal representations. We conduct over 20K experiments across various datasets and TS/text models, demonstrating the effectiveness of the constrained fusion methods. Code is available at: https://github.com/seunghan96/cfa.
Time-series anomaly detection (TSAD) requires identifying both immediate Point Anomalies and long-range Context Anomalies. However, existing foundation models face a fundamental trade-off: 1D temporal models provide fine-grained pointwise localization but lack a global contextual perspective, while 2D vision-based models capture global patterns but suffer from information bottlenecks due to a lack of temporal alignment and coarse-grained pointwise detection. To resolve this dilemma, we propose VETime, the first TSAD framework that unifies temporal and visual modalities through fine-grained visual-temporal alignment and dynamic fusion. VETime introduces a Reversible Image Conversion and a Patch-Level Temporal Alignment module to establish a shared visual-temporal timeline, preserving discriminative details while maintaining temporal sensitivity. Furthermore, we design an Anomaly Window Contrastive Learning mechanism and a Task-Adaptive Multi-Modal Fusion to adaptively integrate the complementary perceptual strengths of both modalities. Extensive experiments demonstrate that VETime significantly outperforms state-of-the-art models in zero-shot scenarios, achieving superior localization precision with lower computational overhead than current vision-based approaches. Code available at: https://github.com/yyyangcoder/VETime .
Yingyuan Yang, Tian Lan, Yifei Gao +6
Department of Industrial Engineering, Tsinghua University, Beijing, China · Lab, Huawei Technologies Ltd, Beijing, China · Datadog AI Research.