Can Coding Agents Build Robust Baselines? A Skill-Based Approach for Automating the Medical Imaging Model-Development Pipeline

Eugenia Moris; José Ignacio Orlando

cs.CVAug 24, 2026

Can Coding Agents Build Robust Baselines? A Skill-Based Approach for Automating the Medical Imaging Model-Development Pipeline

Authors: Eugenia Moris, José Ignacio Orlando

Organizations: Arionkoder LLC, Boston, MA 02114, United States · Yatiris, PLADEMA, UNICEN-CONICET, Tandil, Buenos Aires, Argentina

Abstract

Developing competitive deep learning baselines for medical imaging remains a highly iterative process requiring literature review, implementation, experimentation, and expert refinement. Existing automation approaches typically optimize isolated components, such as architecture search or hyperparameter tuning, rather than the complete baseline development process. We present an agentic AI Scientist workflow that combines literature-guided reasoning, automated code generation, and hypothesis-driven experimentation to generate competitive baseline models for medical imaging challenges. The framework is evaluated on four public benchmarks spanning segmentation, classification, and detection. Across all tasks, the Experimentation Pipeline consistently improves validation performance, achieving competitive leaderboard results, including 6th place on both PUMA tracks (15 teams) and 31st place on MILK10k (125 teams). On MIDOG25, the resulting model also demonstrates strong domain generalization across scanners, tumor types, and species. Using the same workflow across all challenges without task-specific redesign, we demonstrate that skill-based, literature-guided agentic workflows can substantially reduce the engineering effort required to develop competitive medical imaging baselines.

Figures & tables

Figure 1: Overview of the proposed skill-based agentic framework. Raw inputs are converted into a structured task description and a standardized dataset. Three phases then synthesize task-specific knowledge, construct an executable modeling pipeline, and iteratively select, execute, and analyze experiments while maintaining a persistent experiment plan and tree. The task description persists across all phases and the researcher invokes each skill in order, reviewing and running the experiments the framework proposes.

Challenge	Task	Train/Val/Test	Metric	Evaluation
PUMA	Segmentation + Detection	174 / 31 / Hidden	Dice / Macro-F1	Leaderboard
MILK10k	Classification	4454 / 786 / Hidden	Macro-F1	Leaderboard
MIDOG25	Detection	8626 / 1747 / 1564	Object F1	Held-out test

Table 1: Summary of the benchmark challenges used to evaluate the workflow.

Challenge	Metric	Validation			Test				Time to model (days) †
Challenge	Metric	LR selection	Data augmentation	Experiment exploration	Final baseline (ours)	Similar method	First-place model	Rank	Time to model (days) †
PUMA Tissue segmentation	Macro Dice	0.672	0.675	0.732	0.691	0.554 $(+0.137)$	0.783 $(-0.092)$	6/15	2
PUMA Nuclei detection	Macro-F1	0.459	0.467	0.717	0.624	0.586 $(+0.038)$	0.658 $(-0.034)$	6/15	4
MILK10k ISIC Challenge	Macro-F1	0.474	0.493	0.528	0.490	0.515 * $(-0.025)$	0.609 * $(-0.119)$	31/125 *	5
MIDOG25 Atypical mitosis classification	F1	0.757	0.791	0.855	— $\ddagger$	0.882	0.907	—	7

Table 2: Performance across challenges with open leaderboard evaluations. Validation scores are reported after learning-rate (LR) selection, data augmentation, and experiment exploration. Test scores include results of our final baseline vs. the most similar method submitted and the leading model.

Figure 2: Experimentation workflow on MIDOG25. Left: experiment tree showing validation $F_{1}$ progression, with nodes colored by decision category and gray nodes indicating rejected experiments. Right: chronological sequence of executed experiments. The final accepted configuration achieved the highest validation score ( $F_{1}=0.855$ ).

Explore similar work

Towards Fully Automated Medical Imaging Code Generation via Validation-based Context Engineering
Aug 29, 2026Zixiao Zhao, Jing Sun, Zhe Hou +5LLM Code Generation
University of Auckland Auckland, New Zealand · Griffith University Brisbane, Australia · Suzhou Industrial Park Monash Research Institute of Science and Technology Suzhou, China +2
AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
Jun 1, 2026Junqi Liu, Selena Song, Yuhan Wang +12AI Agent Evaluation AI Agent Benchmarks
University of California, Santa Cruz · 2NVIDIA
Towards Autonomous and Auditable Medical Imaging Model Development
Jul 12, 2026Shengyuan Liu, Jia-Xuan Jiang, Boyun Zheng +8Medical Imaging Large Language Model-Guided Optimization
The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences · Independent Researcher +2

Aug 29, 2026J/K move · Enter open · S save

Towards Fully Automated Medical Imaging Code Generation via Validation-based Context Engineering

Zixiao Zhao, Jing Sun, Zhe Hou, Cheng-Hao Cai+4

University of Auckland Auckland, New Zealand · Griffith University Brisbane, Australia · Suzhou Industrial Park Monash Research Institute of Science and Technology Suzhou, China+2

LLM Code Generation

Open paper AskPDF arXiv

Large language models (LLMs) have demonstrated considerable promise in program generation for small-scale and conventional application development; however, they remain limited when applied to complex, domain-specific tasks such as medical image processing. General-purpose models lack explicit domain knowledge and robust validation mechanisms to ensure correctness, often requiring substantial human intervention to produce reliable processing pipelines. To address these limitations, we propose AutoMedImg, a multi-agent framework for fully automated medical image processing code generation. AutoMedImg orchestrates specialised agents across two phases: a Planning Phase that performs dataset analysis and architecture design with semantic and formal verification, and a Coding Phase that generates modules in parallel with static checking, execution testing, and assembly validation. This multi-stage validation mitigates error propagation throughout generation, while comprehensive auto-context engineering combining domain-specific knowledge bases, shared memory, and validation feedback automates context construction without manual prompting. A cross-project adaptive pipeline synthesis mechanism further accumulates validated pipelines and retrieves proven components for new tasks based on project similarity, enhancing generation efficiency through cross-project learning. Extensive evaluation across six diverse and well-established medical imaging datasets with five backbone LLMs demonstrates that AutoMedImg achieves zero human intervention, with Dice scores of up to 0.90 for segmentation tasks and 99% accuracy for classification.

Abstract

Explore similar work

Towards Fully Automated Medical Imaging Code Generation via Validation-based Context Engineering

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

Towards Autonomous and Auditable Medical Imaging Model Development