Forecasting Benchmarks

Follow this topic Ask about this topic

Forecasting benchmarks test how well models predict future values or events in settings such as energy, weather, health, finance, and prediction markets. They compare classical statistical methods, machine-learning models, time-series foundation models, and language-model agents using fixed test sets, rolling or live evaluations, and zero-shot or fine-tuned setups. Stronger benchmarks use leakage controls, simple baselines, probabilistic scores, and tests across changing regimes, missing data, extreme events, or downstream decisions. A central challenge is that model rankings can change with the dataset, evaluation protocol, metric, or operating conditions, so low average error alone does not establish reliable or useful forecasts.

Latest papers

227 papers37 papers in the last four weeks, up 32% on the four weeks before. 0.2% of all new papers.

Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting
May 26, 2026Shuang Liang, Chaochuan Hou, Xu Yao +4Multivariate Time Series Forecasting Forecasting Benchmarks
Shanghai University of Finance and Economics Shanghai, China · Ant Group Shanghai, China · Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics Shanghai, China
StakeBench: Evaluating Language Understanding Grounded in Market Commitment
May 25, 2026Yunhua Pei, Jingyu Hu, Yiwei Shi +3LLM Evaluation Forecasting Benchmarks
University of Bristol, Bristol, UK
RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges
May 24, 2026Ruize Li, Zhibin Wen, Tao Han +5Benchmark Design Forecasting Benchmarks
The Hong Kong University of Science and Technology · Nanjing University · Southern University of Science and Technology +2
Forecasting Scientific Progress with Artificial Intelligence
May 21, 2026Sean Wu, Pan Lu, Yupeng Chen +7Event Time Prediction Scientific Discovery
University of Oxford · Stanford University · Allen Institute for AI +1
Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most
May 21, 2026Nick Merrill, Jaeho Lee, Ezra KargerLanguage Model Scaling Laws Forecasting Benchmarks
Forecasting Research Institute · UC Berkeley
SURGE: An Event-Centric Social Media Sentiment Time Series Benchmark with Interaction Structure
May 20, 2026Chen Su, Pengsen Cheng, Yuanhe Tian +1Social Media Analysis Multivariate Time Series Forecasting
University of Science and Technology of China · Sichuan University · Zhongguancun Academy
FLUXtrapolation: A benchmark on extrapolating ecosystem fluxes
May 19, 2026Anya Fries, Jacob A Nelson, Martin Jung +2Distribution Shift Robustness OOD Generalization
Seminar for Statistics, ETH Zürich Zürich, Switzerland · Max Planck Institute for Biogeochemistry Jena, Germany
GenTS: A Comprehensive Benchmark Library for Generative Time Series Models
May 18, 2026Chenxi Wang, Xiaorong Wang, Peiyang Li +1Time Series Generation Forecasting Benchmarks
The University of Hong Kong Hong Kong SAR, China · Fudan University Shanghai, China
FutureSim: Replaying World Events to Evaluate Adaptive Agents
May 14, 2026Shashwat Goel, Nikhil Chandak, Arvindh Arun +5Continual Test-Time Adaptation Forecasting Benchmarks
1ELLIS Institute Tübingen · 2Max Planck Institute for Intelligent Systems · 4Tübingen AI Center +3
SciPaths: Forecasting Pathways to Scientific Discovery
May 14, 2026Eric Chamoun, Yizhou Chi, Yulong Chen +4Scientific Reasoning Scientific Discovery
University of Cambridge, Cambridge, United Kingdom · Singapore University of Technology and Design, Singapore · The Alan Turing Institute, London, United Kingdom
Does Your Wildfire Prediction Model Actually Work, or Just Score Well?
May 14, 2026Yangshuang Xu, Yuyang Dai, Liling Chang +2Geospatial Foundation Models Wildfire Forecasting
Florida State University · Northeastern University
SpikeProphecy: A Large-Scale Benchmark for Autoregressive Neural Population Forecasting
May 13, 2026John R. Minnick, Jinghui Geng, Kamran Hussain +6Neural Processes Time Series Forecasting
Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, USA · UC Santa Cruz Genomics Institute, University of California, Santa Cruz, CA, USA · Department of Computer Science and Engineering, University of California, Santa Cruz, CA, USA +2
The Growing Pains of Frontier Models: When Leaderboards Stop Separating and What to Measure Next
May 13, 2026Adil AminLLM Evaluation Forecasting Benchmarks
ZEHEN Labs
From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning
May 12, 2026Eric S. Qiu, Danielle R. Thomas, Boyuan Guo +2Intelligent Tutoring Systems Time Series Forecasting
Cornell University · Ithaca, NY, USA · Carnegie Mellon University +1
ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks
May 12, 2026Zhizhen Zhang, Hyemin Gu, Benjamin J. Zhang +6Zero-Shot Time Series Forecasting Forecasting Benchmarks
University of Massachusetts Amherst · University of North Carolina · SRI International +1
EpiCastBench: Datasets and Benchmarks for Multivariate Epidemic Forecasting
May 12, 2026Madhurima Panja, Danny D'Agostino, Huitao Li +2Multivariate Time Series Forecasting Epidemic Forecasting
Sorbonne University Abu Dhabi · Duke-NUS Medical School, Singapore
Benchmarking Sensor-Fault Robustness in Forecasting
May 11, 2026Alexander Windmann, Philipp Wittenberg, Gianluca Manca +3Cyber-Physical Systems Time Series Forecasting
1Helmut Schmidt University · 2Ruhr University Bochum · 3ABB Corporate Research +2
Set Prediction for Next-Day Active Fire Forecasting
May 11, 2026Yuchen Bai, Georgios Athanasiou, Xin Yu +4Wildfire Forecasting Set Prediction
Max Planck Institute for Biogeochemistry, Jena, Germany · Orion Lab, National Technical University of Athens · University of Utah, Salt Lake City, USA +4
Yield Curve Forecasting using Machine Learning and Econometrics: A Comparative Analysis
May 11, 2026Aman Singh, Tokunbo Ogunfunmi, Sanjiv DasQuantitative Finance Time Series Forecasting
Department of Electrical and Computer Engineering, Santa Clara University, USA · School of Business, Santa Clara University, USA
Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption
May 10, 2026Marja Wahl, Daniel R. Bayer, Sven Rausch +1Energy-Efficient ML Time Series Forecasting
RAUSCH Technology GmbH, Würzburg, Germany · Modeling and Simulation, University of Würzburg, Würzburg, Germany
ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics
May 10, 2026Henok Tenaw Moges, Charalampos Skokos, Deshendran MoodleyForecasting Benchmarks Synthetic Benchmark Generation
Centre for Artificial Intelligence Research (CAIR) University of Cape Town, Cape Town, South Africa · Nonlinear Dynamics and Chaos Group, Department of Mathematics and Applied Mathematics University of Cape Town, Cape Town, South Africa
TailedTS: Benchmark Dataset for Heavy-Tailed Time Series Prediction and Periodicity Quantification
May 9, 2026Xinyu Chen, HanQin Cai, Lijun Ding +1Time Series Forecasting Heavy-Tailed Noise
University of Central Florida Orlando, FL 32816 · Massachusetts Institute of Technology Cambridge, MA 02139
LEAF: A Living Benchmark for Event-Augmented Forecasting
May 9, 2026Mingtian Tan, Mihir Parmar, Palash Goyal +5Financial Forecasting Time Series Forecasting
Google · University of Virginia
FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution
May 8, 2026Jianrong Ding, Jianyuan Zhong, Zhengyan Shi +1AI for Science LLM Evaluation
Department of Computer Science and Engineering The Chinese University of Hong Kong
SpatialEpiBench: Benchmarking Spatial Information and Epidemic Priors in Forecasting
May 7, 2026Ruiqi Lyu, Alistair Turcan, Bryan WilderEpidemic Forecasting Forecasting Benchmarks
1Carnegie Mellon University
AirQualityBench: A Realistic Evaluation Benchmark for Global Air Quality Forecasting
May 7, 2026Xing Xu, Xu Wang, Yudong Zhang +3Benchmark Design Time Series Forecasting
University of Science and Technology of China (USTC), Hefei, Anhui, China · 2Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China · 3The Hong Kong Polytechnic University, Hong Kong, China
OracleProto: A Reproducible Framework for Benchmarking LLM Forecasting via Knowledge Cutoffs and Temporal Masking
May 5, 2026Yiding Ma, Chengyun Ruan, Kaibo Huang +3LLM Evaluation Forecasting Benchmarks
Beijing University of Posts and Telecommunications
FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models
May 5, 2026Seunghan Lee, Jun Seo, Jaehoon Lee +7Time Series Reasoning CoT Reasoning
LG AI Research · Seoul, South Korea
CombinationTS: A Modular Framework for Understanding Time-Series Forecasting Models
May 2, 2026Xiaorui Wang, Fanda Fan, Chenxi Wang +9Time Series Forecasting Forecasting Benchmarks
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Beijing Normal University - Hong Kong Baptist University United International College, Zhuhai, China +1
Extreme Weather Bench: A framework and benchmark for evaluation of high-impact weather
May 1, 2026Amy McGovern, Taylor Mandelbaum, Daniel Rothenberg +6Forecasting Benchmarks Extreme Weather Forecasting
Brightband, San Francisco, CA, USA. · School of Meteorology and School of Computer Science, University of Oklahoma, Norman, 73072, OK, USA. · Bureau of Meteorology, Australia. +5

May 26, 2026J/K move · Enter open · S save

Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting

Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang+3

Shanghai University of Finance and Economics Shanghai, China · Ant Group Shanghai, China · Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics Shanghai, China

Multivariate Time Series Forecasting Forecasting Benchmarks Automated Machine Learning

Open paper AskPDF arXiv

While previous research in multivariate time series forecasting has focused on developing complex holistic models, this work advocates for a shift toward a granular, component-level understanding of their impacts. We propose TSCOMP, the first large-scale benchmark that systematically deconstructs deep forecasting methods into their core, fine-grained components--spanning series preprocessing, encoding strategies, network architectures including specific and large time-series models, and optimization methods. Using constrained orthogonal experimental design and extensive evaluations, we conduct multi-view analyses that reveal component effectiveness across different backbones, data characteristics, and their interactions. Beyond providing insights, this benchmark establishes a fine-grained performance corpus comprising over 20,000 model-dataset evaluations, which supports the learning of automated component selection, enabling zero-shot model construction on new datasets. Our experiments demonstrate that the corpus-driven approach, despite its simplicity, consistently outperforms state-of-the-art methods, validating the soundness of our evaluation design and confirming that systematic component selection surpasses manually designed complex architectures. All code and the performance corpus are publicly available at https://github.com/SUFE-AILAB/TSCOMP.