cs.AIOct 31, 2026

Decision-Centric Research: A Calibrated Decision Model and Benchmark for Scientific Agents

Authors: Shaoning Zeng, Wen Huang, Xinzhong Zhu, Yi Duan, Jianhang Zhou, Jianping Gou

Abstract

Scientific agents make consequential choices about evidence, next actions and research strategy, but choices embedded in generated text are difficult to evaluate independently. Decision-Centric Research (DCR) is introduced to make individual choices the common unit of supervision, prediction and evaluation through typed predictions over an explicit Research State. DCR defines 24 decision types, ten of which are instantiated in the Research Decision Benchmark (RD-Bench) of 2,000 rule-generated cases. The Decision-Centric Research Model (DCRM) shares a candidate-token readout across types to produce answers and option probabilities in one forward pass. Under a held-out research-domain split, a 0.6B DCRM (state-aware) achieves 89.4% Top-1 accuracy, compared with 27.8% for DCRM (question-only) trained with the same recipe and 36.4% for DeepSeek-V4.1-Flash. All 30 type-seed comparisons against DCRM (question-only) are significant after Holm correction. After held-out calibration, the resulting prediction sets contain a mean of 1.515 candidates at 97.1% empirical label coverage, reducing mean set size by 57.4% relative to DCRM (question-only) at similar coverage. These results establish a compact state-aware decision layer with accurate typed prediction and an inspectable uncertainty interface.

Figures & tables

Explore similar work

CardsList
  1. DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports

    Jan 13, 2026Ruizhe Li, Mingxuan Du, Benfeng Xu +3Deep Research AgentsLLM Agent Evaluation

  2. ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment

    May 30, 2026Qiuyu Tian, Haojie Yin, Yingce Xia +2LLM Agent EvaluationAI Agent Benchmarks