cs.AINov 17, 2026

SciWorkBench: A Benchmark for Real Scientific Workflows

Authors: Yuchen Zhang, Taolin Han, Jinghang Wang, Jinxin Wang, Yun Wu, Wai Yuet Chiu, Xiaogang Li, Yuting Fu, 

Organizations: Alibaba Group · Tsinghua University · University of Chinese Academy of Sciences · University of Alberta · 

Abstract

Language models are increasingly being used as agents capable of planning and executing projects, as well as refining their actions through feedback. This shift raises a question: can current agents complete projects like real scientists? To examine this question, we introduce SciWorkBench, a benchmark for evaluating agents' abilities on real science tasks across biology, chemistry, materials science, and physics. SciWorkBench is built from peer-reviewed literature or expert-contributed research scenarios. We also propose a unified scientific analysis framework validated by domain experts, which can categorize most tasks in SciWorkBench. Each task covers one or more stages of this framework. We evaluate up-to-date models within the Claude Code and Codex agent frameworks under a unified execution and grading protocol. The best-performing agent achieves a mean score of 57.38, indicating that current agents still struggle with realistic scientific workflows. Our failure analysis identifies three recurring weaknesses at the workflow level: difficulty translating scientific knowledge into task-specific procedures, weak integration of evidence across different modalities, and limited capacity for scientific self-correction. These findings highlight the importance of workflow-based evaluation for measuring progress toward reliable AI systems for scientific research.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SCI-AGENT-EXAM: Benchmarking Agents on Scientific Research Tasks

    Oct 23, 2026Tianyun Zhong, Wei Wang, Jialin Li +17Scientific Agent BenchmarksLLM Agent Evaluation

  2. Benchmarking AI Agents for Addressing Scientific Challenges Across Scales

    Jun 10, 2026Tianyu Liu, Allen Xin Wang, Antonia Panescu +30AI for ScienceScientific Agent Benchmarks

  3. SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration

    Jul 23, 2026Yinhao Tang, Youqing Fang, Yanan Sun +6AI Agent BenchmarksInformation Retrieval