An Artifact-based Agent Framework for Adaptive and Reproducible Medical Image Processing

Lianrui Zuo; Yihao Liu; Gaurav Rudravaram; Karthik Ramadass; Aravind R. Krishnan; Michael D. Phillips; Yelena G. Bodien; Mayur B. Patel

cs.AIMar 31, 2026

An Artifact-based Agent Framework for Adaptive and Reproducible Medical Image Processing

Authors: Lianrui Zuo, Yihao Liu, Gaurav Rudravaram, Karthik Ramadass, Aravind R. Krishnan, Michael D. Phillips, Yelena G. Bodien, Mayur B. Patel,

Organizations: Dept. of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA · Dept. of Computer Science, Vanderbilt University, Nashville, TN, USA · Dept. of Surgery, Vanderbilt University Medical Center, Nashville, TN, USA · Dept. of Neurology, Vanderbilt University Medical Center, Nashville, TN, USA · Dept. of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA · Dept. of Thoracic Surgery, Vanderbilt University Medical Center, Nashville, TN, USA · Veteran Affairs, Tennessee Valley Healthcare System, Nashville, TN, USA · Dept. of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA · Peabody College, Vanderbilt University, Nashville, TN, USA · Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA · Dept. of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA · Dept. of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA · Vanderbilt University Institute of Imaging Science, Nashville, TN, USA

Abstract

Medical imaging research is increasingly shifting from controlled benchmark evaluation toward real-world clinical deployment. In such settings, applying analytical methods extends beyond model design to require dataset-aware workflow configuration and provenance tracking. Two requirements therefore become central: adaptability, the ability to configure workflows according to dataset-specific conditions and evolving analytical goals; and reproducibility, the guarantee that all transformations and decisions are explicitly recorded and re-executable. Here, we present an artifact-based agent framework that introduces a semantic layer to augment medical image processing. The framework formalizes intermediate and final outputs through an artifact contract, enabling structured interrogation of workflow state and goal-conditioned assembly of configurations from a modular rule library. Execution is delegated to a workflow executor to preserve deterministic computational graph construction and provenance tracking, while the agent operates locally to comply with most privacy constraints. We evaluate the framework on real-world clinical CT and MRI cohorts, demonstrating adaptive configuration synthesis, deterministic reproducibility across repeated executions, and artifact-grounded semantic querying. These results show that adaptive workflow configuration can be achieved without compromising reproducibility in heterogeneous clinical environments.

Figures & tables

Figure 1: The typical analytical workflow and operational reality in imaging research. Imaging data originate in hospital and research PACS, where heterogeneous file structures, mixed modalities, and nested archives are common. Extensive curation and processing are often required before most analytical algorithms can be deployed.

Figure 2: Overview of the proposed agentic system. User requests are routed to either (1) Knowledge Agent (generic questions), (2) a Semantic Query Agent operating on artifacts $\mathcal{A}$ , or (3) a workflow pathway for analytical goals. The Workflow Planning and Assembly Agents construct a configuration $C$ from rule library $\mathcal{S}$ and artifacts $\mathcal{A}$ , which is executed by a workflow manager to transform dataset $\mathcal{D}$ into artifacts $\mathcal{A}$ .

Dataset Characterization						Assembly		Reprod.	Adaptability
Dataset	Modality	Cohort	Heteroge.	#subj	#ses	Analytical goal	#Rules	DAG	IRM(%) $\uparrow$	PL $\downarrow$	FO(%) $\uparrow$
NLST [ kramer2011lung ]	CT	Research	Mid.	$200$	$592$	convert+curate	11	$\checkmark$	$100$	1	$98.8$
						lung lobe seg	14	$\checkmark$	$85.7$	4	$97.9$
						lung nodule seg	14	$\checkmark$	$100$	3	$97.4$
LungCaTrial	CT	Clinical	High	636	$2,950$	convert+curate	12	$\checkmark$	$100$	2	$99.0$
						lung nodule seg	15	$\checkmark$	$100$	4	$90.7$

Table 1: Dataset characteristics and quantitative evaluation of reproducibility and adaptability. For each dataset and analytical goal, we report the number of rules in the domain-expert ground-truth configuration, reproducibility via DAG equivalence across repeated executions, and adaptability metrics † .

Dataset	Model	Contract?	Status	Filter	Provenance
NLST	DeepSeek	✗	95%	20%	80%
	Qwen	$\checkmark$	100%	90%	100%
	DeepSeek	$\checkmark$	100%	95%	100%
LungCaTrial	DeepSeek	✗	90%	15%	80%
	Qwen	$\checkmark$	100%	85%	100%
	DeepSeek	$\checkmark$	100%	90%	100%

Table 2: Semantic query evaluation ( deepseek-r1:14b and qwen2.5:14b ) and ablation experiment with/without artifact contract. Accuracy denotes exact-match percentage over 20 queries per category.

Agentic Workflows

Agentic workflows coordinate AI agents, tools, and humans across multi-step tasks, emphasizing reliable execution, efficiency, and oversight.

See all 218 papers

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
Jun 1, 2026Junqi Liu, Selena Song, Yuhan Wang +12AI Agent Evaluation AI Agent Benchmarks
University of California, Santa Cruz · 2NVIDIA
MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflows
Mar 25, 2026Weixiang Shen, Chengzhi Shen, Yanzhu Hu +12Medical VLMs Medical Image Benchmarks
Technical University of Munich (TUM) · TUM University Hospital · LMU Munich +6
Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery
Mar 6, 2026Lin Fan, Pengyu Dai, Zhipeng Deng +4Self-Evolving Agents LLM Agent Skill Learning
Southwest Jiaotong University, China · The University of Tokyo, Japan · RIKEN, Japan +2

Jun 1, 2026J/K move · Enter open · S save

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

Junqi Liu, Selena Song, Yuhan Wang, Jiawei Mao+11

University of California, Santa Cruz · 2NVIDIA

AI Agent Evaluation AI Agent Benchmarks Scientific Workflow Automation Autonomous Research Agents Medical AI Agents

Open paper AskPDF arXiv

Autonomous agents are increasingly expected to support end-to-end medical-AI research workflows, moving beyond isolated prediction tasks or short-form clinical question answering. However, existing medical agent benchmarks primarily evaluate final outputs, providing limited visibility into agent behavior within the research process. To address this gap, we present AutoMedBench, a workflow-aware benchmark for autonomous medical-AI research across diverse medical imaging and multimodal inference tasks, organizing agent execution into a unified five-stage workflow (S1-S5): Plan, Setup, Validate, Inference, and Submit. It comprises long-horizon tasks with each run averaging 33 agent turns, spanning five research tracks: segmentation, image enhancement, visual question answering (VQA), report generation, and lesion detection. Each task is evaluated under two difficulty tiers, Lite and Standard, which use the same data and metrics but differ in the amount of task-brief scaffolding, and each run is scored using both final task performance and S1-S5 stage scores, enabling stage-level analysis from the initial task brief to the final submitted artifact. Across thousands of recorded runs, stage-level scoring reveals that Validate is the weakest workflow stage on average, whereas Setup is the strongest, suggesting that current agents are better at making pipelines executable than at verifying their reliability. Post-run error analysis further shows that verification and submission failures dominate tagged errors, accounting for 37.7% and 38.1% of fired codes respectively, whereas task-understanding errors are rare at 0.9%, and runs with one fired error code have a 48% lower overall score than runs with no error code on average.

Abstract

Agentic Workflows

Explore similar work

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflows

Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery