Forecasting Benchmarks

Ask about this topic

Overview

Forecasting benchmarks test how well models predict future values or events in settings such as energy, weather, health, finance, and prediction markets. They compare classical statistical methods, machine-learning models, time-series foundation models, and language-model agents using fixed test sets, rolling or live evaluations, and zero-shot or fine-tuned setups. Stronger benchmarks use leakage controls, simple baselines, probabilistic scores, and tests across changing regimes, missing data, extreme events, or downstream decisions. A central challenge is that model rankings can change with the dataset, evaluation protocol, metric, or operating conditions, so low average error alone does not establish reliable or useful forecasts.

Latest papers

CardsList

224 papers37 papers in the last four weeks, up 32% on the four weeks before. 0.2% of all new papers.

  1. Evaluating Strategic Reasoning in Forecasting Agents

    Apr 28, 2026Tom Liptay, Dan Schwarz, Rafael Poyiadzi +2LLM Agent EvaluationForecasting Benchmarks

    FutureSearch

  2. Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting

    Apr 27, 2026Max Kleinebrahm, Jonathan Berrisch, Philipp Eiser +11Benchmark DesignTime Series Forecasting

    Karlsruhe Institute of Technology, Institute for Industrial Production, Karlsruhe, Germany · University of Duisburg-Essen, House of Energy Markets and Finance, Essen, Germany · Karlsruhe Institute of Technology, Institute for Automation and Applied Informatics, Eggenstein-Leopoldshafen, Germany +5

  3. Machine Learning and Deep Learning Models for Short Term Electricity Price Forecasting in Australia's National Electricity Market

    Apr 26, 2026Wei Lu, Jay Wang, Dingli Duan +3Gradient-Boosted Decision TreesElectricity Price Forecasting

    School of Engineering, Computer & Mathematical Sciences, Auckland University of Technology (AUT), Auckland 1010, New Zealand · Department of Thermal Energy and Power Engineering, Yantai University, Yantai, China · Department of Building Environment and Equipment, School of Civil Engineering, Hefei University of Technology, Hefei, China +1

  4. FETS Benchmark: Foundation Models Outperform Dataset-specific Machine Learning in Energy Time Series Forecasting

    Apr 24, 2026Marco Obermeier, Marco Pruckner, Florian Haselbeck +1Time Series ForecastingForecasting Benchmarks

    Julius-Maximilians-Universität Würzburg, Modeling and Simulation Lab, Am Hubland, Würzburg, 97074, Germany · Weihenstephan-Triesdorf University of Applied Sciences, Smart Farming, Am Staudengarten 1, Freising, 85354, Germany · Weihenstephan-Triesdorf University of Applied Sciences, Digital Energy Transition, Hans-Carl-von-Carlowitz-Platz 3, Freising, 85354, Germany

  5. Deep FinResearch Bench: Evaluating AI's Ability to Conduct Professional Financial Investment Research

    Apr 22, 2026Mirazul Haque, Antony Papadimitriou, Samuel Mensah +6Quantitative FinanceDeep Research Agents

    AI Research, JPMorganChase

  6. Agentic Forecasting with Structured Linguistic Beliefs

    Apr 20, 2026Kevin MurphyForecasting BenchmarksProbabilistic Forecasting

    Department of Computer Science University of British Columbia Vancouver, BC, Canada

  7. IDOBE: Infectious Disease Outbreak forecasting Benchmark Ecosystem

    Apr 20, 2026Aniruddha Adiga, Jingyuan Chou, Anshul Chiranth +7Epidemic ForecastingForecasting Benchmarks

    Biocomplexity Institute, University of Virginia, Charlottesville, Virginia, USA · Department of Public and Ecosystem Health Cornell University College of Veterinary Medicine, Ithaca, New York, USA · Johns Hopkins University, Baltimore, Maryland, USA +2

  8. CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Prediction

    Apr 17, 2026Jianyou Wang, Youze Zheng, Longtian Bao +11HealthcareForecasting Benchmarks

    Laboratory for Emerging Intelligence, University of California, San Diego · Elsevier · Department of Dermatology, University of California, San Diego

  9. QuantSightBench: Evaluating LLM Quantitative Forecasting with Prediction Intervals

    Apr 17, 2026Jeremy Qin, Maksym AndriushchenkoLLM EvaluationForecasting Benchmarks

    ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center

  10. Do VLMs Truly "Read" Candlesticks? A Multi-Scale Benchmark for Visual Stock Price Forecasting

    Apr 14, 2026Kaiqi Hu, Linda Xiao, Shiyue Xu +2Stock Price ForecastingFinancial Forecasting

    University of Leeds, Leeds, UK · Sun Yat-sen University, Zhuhai, China · Sun Yat-sen University, GuangZhou, China +1

  11. TempusBench: An Evaluation Framework for Time-Series Forecasting

    Apr 13, 2026Denizalp Goktas, Gerardo Riaño-Briceño, Omkar Tekawade +14Time Series ForecastingForecasting Benchmarks

    Simulacrum New York City, NY, USA

  12. SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future

    Apr 7, 2026Timothy B. Higgins, Antonios Mamalakis, Chirag AgarwalVLM EvaluationForecasting Benchmarks

    Department of Environmental Sciences, University of Virginia, Charlottesville, 22903, Virginia, United States. · School of Data Science, University of Virginia, Charlottesville, 22903, Virginia, United States.

  13. TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems

    Apr 7, 2026Md Atik Ahamed, Mihir Parmar, Palash Goyal +7Forecasting BenchmarksTime Series Reasoning

    Google · University of Kentucky

  14. TimeSeek: Temporal Reliability of Agentic Forecasters

    Apr 5, 2026Hamza Mostafa, Om Shastri, Dennis LeeLLM Agent ReliabilityForecasting Benchmarks

    The Wharton School, University of Pennsylvania

  15. PolyBench: Benchmarking LLM Forecasting and Trading Capabilities on Live Prediction Market Data

    Apr 3, 2026Pu Cheng, Juncheng Liu, Yunshen LongForecasting BenchmarksLLM Evaluation

    College of Electrical Engineering, Sichuan University, Chengdu, Sichuan, China · School of Economics, Sichuan University, Chengdu, Sichuan, China

  16. YC Bench: a Live Benchmark for Forecasting Startup Outperformance in Y Combinator Batches

    Apr 1, 2026Mostapha BenhendaForecasting BenchmarksBenchmark Design

  17. Prediction Arena: Benchmarking AI Models on Real-World Prediction Markets

    Mar 28, 2026Jaden Zhang, Gardenia Liu, Oliver Johansson +3AI Agent EvaluationForecasting Benchmarks

    Arcada Labs · Harvard University

  18. A Benchmark of Classical and Deep Learning Models for Agricultural Commodity Price Forecasting on A Novel Bangladeshi Market Price Dataset

    Mar 27, 2026Tashreef Muhammad, Tahsin Ahmed, Meherun Farzana +7Time Series ForecastingForecasting Benchmarks

    Department of Computer Science and Engineering, Southeast University, Dhaka, Bangladesh · Department of Computer Science and Engineering, University of Dhaka, Bangladesh

  19. QuitoBench: A High-Quality Open Time Series Forecasting Benchmark

    Mar 27, 2026Siqiao Xue, Zhaoyang Zhu, Wei Zhang +7Time Series ForecastingTime Series Modeling

    Ant Group

  20. Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting

    Mar 23, 2026Qilin WangProbabilistic Time Series ForecastingForecasting Benchmarks

    Independent Researcher

  21. Benchmarking Scientific Machine Learning Models for Air Quality Data

    Mar 22, 2026Khawja Imran Masud, Venkata Sai Rahul Unnam, Sahara AliAir Quality PredictionForecasting Benchmarks

    University of North Texas, 76205 Texas, USA

  22. Rolling-Origin Validation Reverses Model Rankings in Multi-Step PM10 Forecasting: XGBoost, SARIMA, and Persistence

    Mar 19, 2026Federico Garcia Crespi, Eduardo Yubero Funes, Marina Alfosea SimonTime Series ForecastingAir Quality Prediction

    Departamento de Ingeniería de Computadores, Universidad Miguel Hernández, Elche, Spain · Departamento de Física Aplicada, Universidad Miguel Hernández, Elche, Spain

  23. Beyond Accuracy: Evaluating Forecasting Models by Multi-Echelon Inventory Cost

    Mar 17, 2026Swata Marik, Swayamjit Saha, Garga ChatterjeeInventory ControlTime Series Forecasting

    Department of Home Science, University of Calcutta, 20 E, Judges Court Road, Alipore, Kolkata, 700027, West Bengal, India. · Department of Computer Science and Engineering, Mississippi State University, 665 George Perry Street, Starkville, 39759, MS, USA. · Psychology Research Unit, Indian Statistical Institute Kolkata, 203, B.T. Road, Kolkata, 700108, West Bengal, India.

  24. msData: A Millisecond-Resolution Network Dataset for Advancing Time Series Foundation Models

    Mar 17, 2026Subina Khanal, Seshu Tirupathi, Merim Dzaferagic +2Wireless Local Area NetworksTime-Series Foundation Models

    Department of Computer Science Aalborg University · IBM Research, Ireland · Trinity College Dublin

  25. Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains

    Mar 16, 2026Raeid Saqur, Christoph Bergmeir, Blanka Horvath +3Time Series ForecastingTime Series Modeling

    Dept. of Computer Science, University of Toronto · Dept. of Mathematics, University of Oxford · Vector Institute +5

  26. Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting

    Mar 9, 2026Azul Garza, Renée Rosillo, Rodrigo Mendoza-Smith +5Forecasting BenchmarksTime Series Forecasting

    TimeCopilot · Independent Researcher · ELLIS Institute Tübingen +3

  27. Are We Winning the Wrong Game? Revisiting Evaluation Practices for Long-Term Time Series Forecasting

    Mar 9, 2026Thanapol Phungtua-eng, Yoshitaka YamamotoTime Series ForecastingLong-Term Time Series Forecasting

    Shizuoka University, Shizuoka, Japan

  28. ViroGym: Realistic Large-Scale Benchmarks for Evaluating Viral Proteins

    Mar 6, 2026Yichen Zhou, Jonathan Golob, Amir Karimi +2Protein Language ModelsProtein Fitness Prediction

    GlaxoSmithKline · Technical University of Munich · University of Washington +1

  29. How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

    Mar 2, 2026Adir Saly-Kaufmann, Kieran Wood, Jan-Peter Calliess +1Financial Time SeriesTime Series Forecasting

    Machine Learning Research Group, Department of Engineering Science, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford

  30. Time Series Foundation Models as Strong Baselines in Transportation Forecasting: A Large-Scale Benchmark Analysis

    Feb 27, 2026Javier Yanes-Pulido, Filipe RodriguesTime-Series Foundation ModelsZero-Shot Time Series Forecasting

    Technical University of Denmark, Kgs. Lyngby, Denmark