Existing time series forecasting methods primarily rely on the numerical data itself. However, real-world time series exhibit complex patterns associated with multimodal information, making them difficult to predict with numerical data alone. While several multimodal time series forecasting methods have emerged, they either utilize text with limited supplementary information or focus merely on representation extraction, extracting minimal textual information for forecasting. To unlock the Value of Text, we propose VoT, a method with Event-driven Reasoning and Multi-level Alignment. Event-driven Reasoning combines the rich information in exogenous text with the powerful reasoning capabilities of LLMs for time series forecasting. To guide the LLMs in effective reasoning, we propose the Historical In-context Learning that retrieves and applies historical examples as in-context guidance. To maximize the utilization of text, we propose Multi-level Alignment. At the representation level, we utilize the Endogenous Text Alignment to integrate the endogenous text information with the time series. At the prediction level, we design the Adaptive Frequency Fusion to fuse the frequency components of event-driven prediction and numerical prediction to achieve complementary advantages. Experiments on real-world datasets across 10 domains demonstrate significant improvements over existing methods, validating the effectiveness of our approach in the utilization of text. The code is made available at https://github.com/decisionintelligence/VoT.
Figures & tables
Figure 1: Unemployment rate time series ( Liu et al., 2024a ) (1970-2020). While certain patterns ( pink ) exhibit predictable temporal regularities, abrupt changes ( green ) driven by external events necessitate the integration of textual information to complement numerical forecasting.
Sub-categories
VoT
GPT4TS
TimeLLM
TEST
CALF
CMIN
Maformer
DualTime
Time-MMD
TaTS
FNSPID
GPT4MTS
CiK
( 2025 )
( 2024b )
( 2024 )
( 2025a )
( 2023 )
( 2024 )
( 2024 )
( 2024a )
( 2026 )
( 2024 )
( 2024 )
( 2025 )
LLM Usage
Feature Extraction
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Reasoning
✓
×
×
×
×
×
×
×
×
×
×
×
×
Text Type
Endogenous
✓
✓
✓
✓
✓
×
×
×
×
×
×
×
✓
Exogenous
✓
×
×
×
×
✓
✓
✓
✓
✓
✓
✓
✓
Table 1: Comparison of multimodal time series forecasting methods with text incorporation
Figure 2: The architecture of VoT. The event-driven branch processes exogenous text through a three-step generative pipeline with the Historical In-Context Learning (HIC). The numerical branch aligns endogenous text with time series via the Endogenous Text Alignment (ETA). The Adaptive Frequency Fusion (AFF) combines both predictions across frequency bands with adaptive weights.
Figure 3: The processing procedure of the Historical In-Context Learning.
Figure 4: The processing procedure of the Endogenous Text Alignment (ETA).
Category
Time series-only
Text-enhanced
Models
VoT
PatchTST ( 2023 )
iTransformer ( 2024b )
RaFT ( 2025 )
PatchTST*
iTransformer*
RaFT*
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Agriculture
0.209
0.302
0.228
0.303
0.220
0.308
0.226
0.322
0.232
0.316
0.229
0.310
0.246
0.333
Climate
1.078
0.840
1.184
0.888
1.135
0.865
1.289
0.926
1.178
0.887
1.117
0.858
1.342
0.944
Economy
0.201
0.353
0.210
0.363
0.222
0.378
0.265
0.411
0.219
0.370
0.213
0.367
0.275
0.420
Energy
0.222
0.343
0.250
0.363
0.269
0.382
0.254
0.367
0.253
0.365
0.265
0.383
0.246
0.360
Table 2: Forecasting results of time series-only and text-enhanced methods, and our method VoT. The best results are highlighted in bold , and the second-best results are underlined .
Models
VoT
GPT4TS ( 2025 )
GPT4MTS ( 2024 )
TaTS ( 2026 )
Time-VLM ( 2025 )
CALF ( 2025a )
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Agriculture
0.209
0.302
0.220
0.294
0.225
0.298
0.215
0.301
0.238
0.303
0.250
0.315
Climate
1.078
0.840
1.184
0.891
1.182
0.889
1.180
0.887
1.195
0.899
1.286
0.922
Economy
0.201
0.353
0.217
0.371
0.208
0.363
0.215
0.368
0.229
0.384
0.207
0.357
Energy
0.222
0.343
0.260
0.376
0.262
0.380
0.255
0.368
0.260
0.374
0.244
0.365
Environment
0.268
0.380
0.322
0.393
0.323
0.400
0.319
0.396
0.320
0.398
0.325
0.387
Table 3: Forecasting results of multimodal methods and VoT. The best results are highlighted in bold , and the second-best results are underlined .
Figure 8
Climate
Energy
SocialGood
Variant
MSE
MAE
MSE
MAE
MSE
MAE
w/o Decomposition
1.120
0.859
0.254
0.368
0.892
0.440
w/o TS-Text CL
1.184
0.888
0.250
0.363
0.944
0.475
ETA
1.092
0.848
0.232
0.350
0.841
0.410
Table 5: Ablation Study on Components of the Event-based Temporal Alignment (ETA)
Figure 6: Frequency domain analysis of time series predictions (Social Good). (a) Original signals without frequency decomposition. (b)-(d) Frequency-filtered components: (b) Low-pass filtered signals, (c) High-pass filtered signals, and (d) Band-pass filtered signals. Ground Truth ( blue ), Time series-only prediction ( pink ), and event-driven prediction ( green )
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Method
Repository
Time Series Only
iTransformer
https://github.com/thuml/iTransformer
PatchTST
https://github.com/yuqinie98/PatchTST
RAFT
https://github.com/archon159/RAFT
Text-Enhanced (Time-MMD)
iTransformer*
Based on iTransformer with Time-MMD text integration
PatchTST*
Based on PatchTST with Time-MMD text integration
RAFT*
Based on RAFT with Time-MMD text integration
Appendix
Table 6: Implementation details and code repositories for baseline methods
Figure 7: Heat Map of MSE/MAE Values Based on Low-Frequency and High-Frequency Percentage Cutoffs. The x-axis represents the percentage cutoffs for the low-frequency range, varying from 10% to 40% of the low-frequency spectrum. The y-axis represents the percentage cutoffs for the high-frequency range, ranging from 10% to 40% of the high-frequency spectrum. The color intensity in the map indicates the magnitude of the Mean Squared Error (MSE) or Mean Absolute Error (MAE)
Branch / Module
Inference Time (s)
Event-driven Branch
5.9
Numerical Branch
0.002
AFF
0.0005
Appendix
Table 7: Inference Latency of Main Model Components
Dataset
Method
MSE
MAE
Climate
No Retrieval
1.151
0.877
Retrieve TS Only
1.137
0.866
Retrieve Summary Only
1.091
0.846
Full HIC
1.078
0.840
SocialGood
No Retrieval
0.845
0.410
Retrieve TS Only
0.853
0.421
Appendix
Table 8: Ablation Comparison of HIC with Simplified Retrieval Methods
Dataset
Retrieved Example by Similarity Rank
MSE
MAE
Energy
1st-most Similar Example
0.222
0.343
10th-most Similar Example
0.229
0.348
100th-most Similar Example
0.233
0.348
No Retrieval
0.238
0.363
SocialGood
1st-most Similar Example
0.804
0.389
10th-most Similar Example
0.807
0.408
Appendix
Table 9: Model Robustness to Imperfect Retrieved Examples
Dataset
Condition
MSE
MAE
Climate
Mask 10% (mask10)
1.099
0.850
Mask 20% (mask20)
1.109
0.854
Noise 10% (noise10)
1.098
0.850
Noise 20% (noise20)
1.103
0.854
Our Method (Original)
1.078
0.840
SocialGood
Mask 10% (mask10)
0.822
0.448
Appendix
Table 10: Robustness Test of the Model Against Noisy and Sparse Text Data
Dataset
Method
MSE
MAE
Climate
Gated-Residual
1.230
0.912
Cross-Attention
1.202
0.893
FiLM
1.211
0.897
ETA
1.092
0.848
Energy
Gated-Residual
0.248
0.360
Cross-Attention
0.254
0.369
Appendix
Table 11: Ablation Study on Temporal Alignment Methods
Figure 8: The visualization of the datasets
Dataset
Fusion Method
MSE
MAE
Climate
MLP
1.228
0.875
Cross-Attention
1.768
1.055
AFF
1.078
0.840
Energy
MLP
0.685
0.591
Cross-Attention
0.430
0.502
AFF
0.222
0.343
Appendix
Table 12: Performance Comparison of Fusion Methods on Multiple Datasets
Period
Branch
Energy
SocialGood
MSE
MAE
MSE
MAE
Event Part
Event-driven Branch
0.402
0.450
3.080
0.979
Numerical Branch
0.443
0.523
3.339
1.022
Fusion Result
0.362
0.460
3.084
0.969
Non-Event Part
Event-driven Branch
0.266
0.362
0.116
0.270
Numerical Branch
0.105
0.257
0.068
0.208
Appendix
Table 13: Prediction Performance Breakdown in Event and Non-Event Periods
Pred Length
Comparison
High Freq
Low Freq
8
Event-driven vs Ground Truth
0.155412
0.497763
Numerical vs Ground Truth
0.423805
0.458464
10
Event-driven vs Ground Truth
0.265945
0.402431
Numerical vs Ground Truth
0.349408
0.381150
12
Event-driven vs Ground Truth
0.273838
0.466686
Numerical vs Ground Truth
0.310766
0.436835
Appendix
Table 14: Pearson Correlation of Frequency Components (SocialGood)
Figure 9: Illustration of the LLM generation pipeline for event-driven prediction. The process flows from template generation through summarization to reasoning, enhanced by Historical In-Context Learning that provides error-informed examples as guidance for improved forecasting accuracy.
LLM
Energy
Social Good
MSE
MAE
MSE
MAE
DeepSeek-R1
0.222
0.343
0.804
0.389
GLM-4.5
0.229
0.337
0.829
0.411
Qwen3-8B
0.226
0.341
0.773
0.427
Appendix
Table 15: Ablation study on different LLMs
Category
TS-only
Text-enhanced
Multimodal
Models
VoT
numerical branch
iTransformer
PatchTST
Raft
iTransformer*
PatchTST*
Raft*
GPT4TS
GPT4MTS
TATS
TimeVLM
CALF
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Agriculture
6
0.131
0.245
0.146
0.256
0.128
0.254
0.151
0.254
0.154
0.283
0.153
0.265
0.146
0.263
0.157
0.282
0.135
0.242
0.161
0.257
0.140
0.251
0.143
0.245
0.142
0.250
8
0.182
0.286
0.188
0.287
0.200
0.296
0.197
0.287
0.194
0.297
0.195
0.296
0.189
0.310
0.237
0.344
0.198
0.284
0.207
0.288
0.187
0.282
0.215
0.287
0.195
0.285
10
0.235
0.319
0.221
0.317
0.254
0.331
0.256
0.320
0.252
0.324
0.263
0.326
0.254
0.320
0.262
0.328
0.258
0.313
0.230
0.305
0.244
0.320
0.271
0.320
0.350
0.370
12
0.288
0.355
0.301
0.356
0.300
0.352
0.309
0.351
0.301
0.383
0.305
0.352
0.338
0.369
0.326
0.379
0.291
0.338
0.301
0.342
0.290
0.350
0.322
0.359
0.314
0.355
Appendix
Table 16: Model Performance Comparison Across Different Categories
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Most existing time series forecasting methods rely solely on numerical observations, overlooking rich contextual information from auxiliary texts. Recent multimodal approaches attempt to incorporate textual signals, but they often treat text as static features or use large language models as forecasting backbones, limiting their ability to capture temporal dynamics and increasing computational cost. To address these challenges, we propose TAC-Time, a unified framework that transforms textual information into additional temporal channels. By modeling text features jointly with numerical sequences in a shared temporal backbone, TAC-Time preserves temporal continuity and periodic structures while remaining efficient and scalable. This formulation also enables systematic interpretability analyses. We show strong cross-modal dependencies through attention and frequency-domain analyses, and identify predictive textual signals whose correlation-aware alignment yields partial forecasting improvements. Extensive experiments on real-world multimodal benchmarks demonstrate that TAC-Time outperforms prior methods.
Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series' spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at https://github.com/hiepnh137/SpecTF.
Huu Hiep Nguyen, Minh Hoang Nguyen, Dung Nguyen +1
Applied Artificial Intelligence Initiative Deakin University Geelong, Australia
Multimodal time series forecasting has garnered significant attention for its potential to provide more accurate predictions than traditional single-modality models by leveraging rich information inherent in other modalities. However, due to fundamental challenges in modality alignment, existing methods often struggle to effectively incorporate multimodal data into predictions, particularly textual information that has a causal influence on time series fluctuations, such as emergency reports and policy announcements. In this paper, we reflect on the role of textual information in numerical forecasting and propose Time series transformers with Multimodal Mixture-of-Experts, TiMi, to unleash the causal reasoning capabilities of LLMs. Concretely, TiMi utilizes LLMs to generate inferences on future developments, which serve as guidance for time series forecasting. To seamlessly integrate both exogenous factors and time series into predictions, we introduce a Multimodal Mixture-of-Experts (MMoE) module as a lightweight plug-in to empower Transformer-based time series models for multimodal forecasting, eliminating the need for explicit representation-level alignment. Experimentally, our proposed TiMi demonstrates consistent state-of-the-art performance on sixteen real-world multimodal forecasting benchmarks, outperforming advanced baselines while offering both strong adaptability and interpretability.