cs.AINov 11, 2026

Science Prediction Universe

Authors: Zengji Tu, Xiang Liu, Jiarui Sun, Ruoyu Wu, Zhihui Qi, Chengda Wu, Yutao Zhou, Bo Yang, 

Abstract

Humanity has long sought to explain natural phenomena. Reliable prediction of experimental outcomes is a central test of our understanding of the processes that produce them. If artificial general intelligence (AGI) is to surpass human scientific capabilities, it must also predict experimental outcomes more reliably than humans. Such capabilities could accelerate scientific discovery across disciplines, help address humanity's most pressing challenges, and improve human well-being. We introduce Science Prediction Universe (SPU), a benchmark of hidden experimental outcomes with 100 public verified tasks from chemistry, materials science, biology and medicine, and 16 private tasks from unpublished wet-lab experiments. Given an experimental setup and candidate conditions, an agent works with any resources it needs to rank the candidates by a measured outcome. We evaluate thirteen agent configurations spanning twelve frontier models across two harnesses (Codex and Claude Code). Current agents remain far from saturating wet-lab outcome prediction, with substantial performance disparities across models: on the public benchmark, configurations separate into two distinct tiers, with task-adjusted Spearman correlations ranging from 0.227 for the lowest-performing configuration to 0.478 for the best. Analysis of 1,155 scored public trajectories (227,103 tool calls) suggests that the bottleneck lies in scientific judgment rather than tool execution: agents rely on shared monotone priors, agreeing far more with one another than with experimental reality (0.52 vs. 0.33), and frequently compute trivial physical or chemical proxies that do not influence the final outcome.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?

    Apr 12, 2026Udari Madhushani Sehwag, Elaine Lau, Haniyeh Ehsani Oskouie +14AI-Assisted Scientific ResearchScientific Reasoning in Language Models

  2. PreScience: A Dataset and Benchmark for Scientific Forecasting

    Feb 24, 2026Anirudh Ajith, Amanpreet Singh, Jay DeYoung +7AI for ScienceForecasting Benchmarks

  3. Science sandboxes measure the scientific capability of AI agents

    Aug 31, 2026Arya S. Rao, Rodrigo I. Castro, Sager J. Gosai +8Protein Fitness PredictionAI Agent Evaluation