cs.LGNov 10, 2026

BioXArena-Pro: a research-grade benchmark for autonomous biomedical machine learning agents

Authors: Loka Li, Duzhen Zhang, Xingbo Du, Leonard Song, Kun Zhang, Le Song

Organizations: Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates. · The Westminster Schools, Atlanta, Georgia, United States. · Carnegie Mellon University, Pittsburgh, Pennsylvania, United States.

Abstract

Coding agents are increasingly proposed as autonomous builders of biomedical models, but existing benchmarks are small, near saturation or open to leaked information. We present BioXArena-Pro, 64 research-level modelling tasks from eight biomedical domains, each rebuilt from a published study; most hold out whole biological units such as species or cohorts. Across 22 agent configurations, the best scores 55.1 of 100, and the scaffold matters almost as much as the model. We re-ran a published method for every task: where the published recipe applies unchanged, the best agent and the methods tie; on rebuilt tasks the methods trail by 8.2 points, suggesting that agents gain by adapting to new data. Pretrained models (including biological foundation models) matter where the strongest agent uses them: forbidding them decreases its score on those tasks by 9 points. Every task was audited in several AI-assisted rounds and a final human review.

Figures & tables

Explore similar work

CardsList
  1. BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents

    May 7, 2026Jinge Wu, Hongjian Zhou, Mingde Zeng +8AI Agents for Scientific DiscoveryDeep Research Agents

  2. Auto-MD-bench: Evaluating Autoresearch Agents for Real-World Medical Tasks

    Oct 18, 2026Shenqi Zhou, Qiran Zou, Wenhao Zhao +2Medical AI AgentsAutonomous Research Agents

  3. BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies

    Oct 6, 2026David Li, Shaamil Karim, Christian GensbiglerScientific Agent BenchmarksAI Agent Benchmarks