WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
Organizations: Sun Yat-sen University · Xiaomi Corporation · National University of Singapore
Abstract
Time series foundation models (TSFMs) have recently achieved remarkable success in universal forecasting by leveraging large-scale pretraining on diverse time series data. Complementing this progress, incorporating frequency-domain information yields promising performance in enhancing the modeling of complex temporal patterns, such as periodicity and localized high-frequency dynamics, which are prevalent in real-world time series. To advance this direction, we propose a new perspective that integrates explicit frequency-domain representations into scalable foundation models, and introduce WaveMoE, a wavelet-enhanced mixture-of-experts foundation model for time series forecasting. WaveMoE adopts a dual-path architecture that jointly processes time series tokens and wavelet tokens aligned along a unified temporal axis, and coordinates them through a shared expert routing mechanism that enables consistent expert specialization while efficiently scaling model capacity. Preliminary experimental results on 16 diverse benchmark datasets indicate that WaveMoE has the potential to further improve forecasting performance by incorporating wavelet-domain corpora.
Figures & tables
| Layers | Heads | Experts | Routing Experts | Hidden Size | FFN Dim. | Patch Length | Top- Attention | Activated Params | Total Params |
| 12 | 12 | 8 | 2 | 384 | 1536 | 8 | 10 | 100M | 226M |
| Dataset | WaveMoE | Time-MoE | Timer | Chronos | Sundial | |||||
| MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETT1 | 0.2012 | 0.2578 | 0.1756 | 0.2498 | 0.9760 | 0.5020 | 0.4456 | 0.3540 | 0.2052 | 0.2403 |
| ETT2 | 0.0342 | 0.1226 | 0.0840 | 0.1809 | 0.0361 | 0.1250 | 0.0357 | 0.1200 | 0.0245 | 0.0958 |
| Exchange Rate | 0.0284 | 0.1050 | 0.0320 | 0.1168 | 0.0416 | 0.1158 | 0.0353 | 0.1119 | 0.0382 | 0.1139 |
| M1 Monthly | 0.0009 | 0.0232 | 0.0060 | 0.0704 | 0.0909 | 0.2219 | 0.0869 | 0.0192 | 0.1816 | 0.0371 |
| M1 Quarterly | 0.0012 | 0.0291 | 0.0052 | 0.0667 | 0.0073 | 0.1923 | 0.0279 | 0.0089 | 0.1168 | 0.0234 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | WaveMoE | Time-MoE | Timer | Chronos | Sundial | |||||
| MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| Australian Electricity | 0.1454 | 0.2438 | 0.1508 | 0.2480 | 1.8303 | 1.1054 | 0.6366 | 0.5632 | 0.1926 | 0.2805 |
| CIF 2016 12 | 0.0002 | 0.0023 | 0.0069 | 0.0735 | 0.1253 | 0.3015 | 0.0259 | 0.0159 | 0.2634 | 0.0581 |
| CIF 2016 6 | 0.0003 | 0.0040 | 0.0041 | 0.0602 | 0.0761 | 0.1561 | 0.0017 | 0.0033 | 0.0014 | 0.0061 |
| ERCOT | 0.2080 | 0.3152 | 0.2085 | 0.3188 | 1.4126 | 0.9399 | 0.4388 | 0.4874 | 0.2641 | 0.3553 |
| Tourism Monthly | 0.0002 | 0.0023 | 0.0106 | 0.0886 | 0.2783 | 0.3609 | 0.0459 | 0.0167 | 0.0492 | 0.0271 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 839 papers