Forecasting Benchmarks

Ask about this topic

Overview

Forecasting benchmarks test how well models predict future values or events in settings such as energy, weather, health, finance, and prediction markets. They compare classical statistical methods, machine-learning models, time-series foundation models, and language-model agents using fixed test sets, rolling or live evaluations, and zero-shot or fine-tuned setups. Stronger benchmarks use leakage controls, simple baselines, probabilistic scores, and tests across changing regimes, missing data, extreme events, or downstream decisions. A central challenge is that model rankings can change with the dataset, evaluation protocol, metric, or operating conditions, so low average error alone does not establish reliable or useful forecasts.

Latest papers

CardsList

224 papers37 papers in the last four weeks, up 32% on the four weeks before. 0.2% of all new papers.

  1. Beyond Local Accuracy: Benchmarking the Rollout Reliability of PDE Foundation Models

    Oct 22, 2026Jaewan Park, Seid Koric, Giri Krishnan +2PDE Foundation ModelsPDE Surrogate Modeling

    National Center for Supercomputing Applications (NCSA), University of Illinois Urbana–Champaign · Georgia Institute of Technology

  2. WhatIf-Bench: Benchmarking Intervention-Aware Time Series Forecasting

    Oct 19, 2026Arjun Ashok, Mihir Parmar, Palash Goyal +6Time Series ForecastingTime Series Forecasting with Covariates

    Google · Mila - Quebec AI Institute · Université de Montréal +1

  3. Beyond Static Leaderboards: Capability-Focused Evaluation with Structural Diversity for Time Series Foundation Models

    Oct 19, 2026Hongyin Zhang, Mingyi Xiang, Rongzhao Chen +7Forecasting BenchmarksTime-Series Foundation Models

    School of Software, BNRist, Tsinghua University · Timecho Ltd, Beijing, China · Harbin Institute of Technology

  4. World Models, Trees and Foundation Models for Retail Forecasting: Evidence from Four Public Data Sets

    Oct 18, 2026Karthik Iyer, Madan DabbeeruTime Series ForecastingTime Series Forecasting with Covariates

  5. PathFM: Fast Generative Forecasting of Multivariate Trajectories

    Oct 15, 2026Jiachen Yao, Xiyuan Zhang, Annan Yu +5Time Series ForecastingMultivariate Time Series Forecasting

    California Institute of Technology · Amazon

  6. Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones

    Oct 15, 2026Nathan Chalumeau, Xavier Renard, Remi Meynadier +3Tropical Cyclone Intensity ForecastingWeather Forecasting

    INRIA, Paris, France · AXA, Paris, France · Sorbonne Université, CNRS, LIP6, F-75005 Paris, France +1

  7. When Does "Post-Call" Begin? An Availability Contract for Earnings-Call NLP Benchmarks

    Oct 15, 2026Xiaoyu Ma, Kelvin J. L. Koa, Yuncong Liu +1Look-Ahead Bias in Language ModelsForecasting Benchmarks

    National University of Singapore Asian Institute of Digital Finance Singapore

  8. JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making

    Oct 14, 2026Zhongwei Xie, Xingsu Li, Jingqiao Yu +8LLM Agent EvaluationForecasting Benchmarks

    The Hong Kong University of Science and Technology · Tsy Capital Limited · Independent researcher +1

  9. Beyond RMSE and Granger Gain: Multi-Horizon Forecast-Error Entropy for Nonlinear Predictive Coupling

    Oct 13, 2026Andrei VelichkoTime-Series Causal DiscoveryMultivariate Time Series Forecasting

    Institute of Physics and Technology, Petrozavodsk State University, Petrozavodsk, Russia

  10. GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation

    Oct 13, 2026Xu Liu, Hanshu Yan, Taha Aksu +6Time Series ForecastingForecasting Benchmarks

    Moirai Team, Salesforce AI Research

  11. IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models

    Oct 12, 2026Vebjørn Haugland, Pavel Gulin, David Foss +5Time Series ForecastingTime-Series Foundation Models

    Cognite

  12. CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds

    Oct 12, 2026Ang Li, Haofei Yu, Zhengwei Tao +7LLM World ModelsWorld Model Evaluation

    The Chinese University of Hong Kong · University of Illinois at Urbana-Champaign · UniPat AI +3

  13. XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting

    Oct 12, 2026Jaehoon Lee, Jun Seo, Seunghan Lee +10Financial ForecastingStock Price Forecasting

    LG AI Research, Seoul, South Korea · Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea

  14. World2TS: Real-World Event Context for Time-Series Forecasting

    Oct 10, 2026Zhiding Liu, Jiqian Yang, Mingyue Cheng +2Time Series ForecastingForecasting Benchmarks

    State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, China · Shenzhen International Graduate School, Tsinghua University, China

  15. CESNET-TimeSeries26: A Multi-Year Network Traffic Time Series Dataset for Annual Seasonality, Concept Drift, and Long-Term Anomaly Detection

    Oct 10, 2026Josef Koumar, Karel Hynek, Kamil Jerabek +1Forecasting BenchmarksLong-Term Time Series Forecasting

    Czech Technical University in Prague, Thákurova 9, 160 00 Prague 6, Czech Republic · CESNET a.l.e., Generála Píky 430/26, 160 00 Prague 6, Czech Republic

  16. SeoulMMOD: A Large-Scale Multimodal Origin-Destination Flow Benchmark

    Oct 9, 2026Taeyoung Yu, Seonbin Jo, Jiwon Kim +1Forecasting BenchmarksTraffic Forecasting

    The University of Queensland · POSTECH · Chung-Ang University

  17. Machine Learning and Time-Series Foundation Models for Day-Ahead Electricity Forecasting in a University Building

    Oct 8, 2026Amey Vasulkar, Samuel Kernan Freire, Arie-Willem de Leeuw +1Electricity Load ForecastingForecasting Benchmarks

  18. Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective

    Oct 8, 2026Dmitry Osin, Egor Surkov, Petr Mokrov +5Conditional Generative ModelingForecasting Benchmarks

    Applied AI Institute · ITMO University · Vega Institute Foundation +2

  19. RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting

    Oct 8, 2026Shengsheng Lin, Jing Hu, Zhengyang Hu +8Exogenous Time Series ForecastingTime Series Forecasting

    South China University of Technology, China · Didi Chuxing, China · The University of Hong Kong, China

  20. Deep Learning vs. Statistical Models for Multi-Horizon Price Forecasting of Second-Hand Electronics: A Systematic Benchmark

    Oct 7, 2026Mateusz Buczyński, Michał Woźniak, Konrad Kaczyński +2Time Series ForecastingForecasting Benchmarks

    Faculty of Economic Sciences, University of Warsaw, Dluga 44/50, Warsaw, 00-241, PL · WeSub, Branickiego 15, Warsaw, 02-972, PL · Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, Warsaw, 00-662, PL

  21. Are We Really Benchmarking Forecasting Models? The Impact of Preprocessing on Time Series Performance

    Oct 6, 2026Guilherme Afonso Galindo Padilha, Paulo Salgado Gomes de Mattos Neto, Rafael Menelau Oliveira e CruzTime Series ForecastingForecasting Benchmarks

    LIVIA, Department of Software and IT Engineering École de technologie supérieure (ÉTS) Montreal, QC, Canada · Center for Informatics (CIn) Universidade Federal de Pernambuco (UFPE) Recife, PE, Brazil

  22. Machine Learning for German Redispatch Forecasting under Data Delays and Temporal Distribution Shift

    Oct 6, 2026Faraz Shamim, Faris ShamimDistribution Shift RobustnessTime Series Forecasting

    KIST Medical College and Teaching Hospital, Nepal · OTH Regensburg

  23. HouseholdBench: Evaluating Large Language Models as Predictors of Household Economic Behavior

    Oct 6, 2026Jin Huang, Diego Ferreras Garrucho, Yutong Xie +4LLM EvaluationLLM Fine-Tuning

    University of Michigan, Ann Arbor · London School of Economics · MobLab Inc +1

  24. Towards Explainable Benchmarking for Data-driven Post-Wildfire Debris Flow Prediction

    Oct 5, 2026Zhisheng Qi, Li Zhu, Utkarsh Sahu +3Perturbation-Based Feature AttributionForecasting Benchmarks

    University of Georgia Athens, Georgia, USA · University of Oregon Eugene, Oregon, USA

  25. Benchmarking Time Series Foundation Models for Load Forecasting Under Covariate Uncertainty

    Oct 5, 2026Tomas Kaljevic, Ivan Arzola, Yu ZhangZero-Shot Time Series ForecastingTime Series Forecasting

    Department of Electrical and Computer Engineering University of California, Santa Cruz

  26. ClimateBench v2.0: Probabilistic Climate Model Benchmarking

    Oct 3, 2026Duncan Watson-Parris, Willa Tobin, Aytaç Paçal +32Climate ModelingBenchmark Design

    Scripps Institution of Oceanography, University of California San Diego · Halicioglu Data Science Institute, University of California San Diego · University of Tübingen, connAIx Research School for Applied AI, Heilbronn, Germany +23

  27. Time Series Forecasting Benchmarks Need Scenario-Grounded Stress Testing

    Oct 2, 2026Yuyang Zhao, Lian Xu, Hao XueRobust Time Series ForecastingTime Series Forecasting

    The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The University of Western Australia, Perth, Australia

  28. Foundations without Fundamentals: Zero-Shot Blind Spots in Time Series FMs

    Oct 1, 2026Nafiseh Ghoroghchian, Haipeng Zhang, Shuyi Han +2Multivariate Time Series ForecastingZero-Shot Time Series Forecasting

    Layer6 AI

  29. Do Your Own Research: Learning to Forecast by Learning to Search

    Oct 1, 2026Yusuf Afifi, Artur Kiulian, Anton Polishko +3Agentic RLForecasting Benchmarks

    Future Principle

  30. On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models

    Oct 1, 2026Haochen Zhang, Jiaheng Guo, Zhen Xu +4World ModelsTime Series Forecasting

    University of North Carolina at Chapel Hill · Foci Labs · Stevens Institute of Technology