Organizations: School of Computer Science, University of Auckland, Auckland, New Zealand · School of Information and Communication Technology, Griffith University, Brisbane, Australia · Metasics, China. Xiamen, China · School of Cyberspace Science and Technology, Beijing Institute of Technology, Beijing, China · DSA Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Department of Computer Science, National University of Singapore, Singapore
Recent advances in large language models have enabled agentic systems that automate machine learning pipeline construction from minimal user input. However, existing agentic AutoML methods either depend on rigid templates that limit adaptability, target narrow task domains, or rely on free-form code generation that lacks reliability guarantees, leaving intermediate errors to propagate silently and degrade the final model. We present EvoML, an agentic framework for end-to-end ML pipeline synthesis built on expert-grounded skills that evolve across runs, executed through a validation-gated harness. The framework decomposes pipeline construction into four phases coordinated by three agents: an Orchestrator that enforces validation gates, advancing each phase only when its outputs pass structural, execution, and semantic checks, which precludes error propagation and keeps every artifact auditable; an Executor that runs expert-grounded skills, grounded in the nnU-Net workflow and pairing reusable templates with online adaptation; and a Curator that performs post-run evolution, consolidating the scripts and guidance recovered during a run into the skill library so it improves across datasets. We evaluate EvoML on eight datasets spanning six sophisticated, real-world ML task types, against agentic AutoML systems and command-line coding tools. EvoML achieves full automation and the best final model quality on all eight, spanning segmentation (up to 0.94 Dice), classification (0.99 accuracy), reconstruction (0.95 SSIM), and denoising (0.79 normalized PSNR), while transferring to broader ML tasks.
Figures & tables
Fig. 1 : Overview of the proposed validation-gated agentic ML pipeline synthesis framework. The Orchestrator progresses through four phases and invokes the corresponding Executor skill group in sequence: Task Understanding (skills 1–2), Planning (3–4), Training (5–7), and Post-train Processing (8–9). After each phase, generated artifacts and execution evidence are returned for gated validation; passed phases advance, while failed phases trigger recovery. The Curator operates after the run to evolve the skill library from accumulated execution evidence.
Fig. 2 : An example validation-gated recovery cycle in the Planning phase.
Fig. 3 : The Curator’s post-run skill evolution loop after a 3D reconstruction task with HDF5 dataset format.
Fig. 4 : Comparison of EvoML against agentic AutoML systems and command-line coding tools across the benchmarking datasets: EvoML attains full automation and the best FMQ on every dataset, with markedly lower variance than the baselines.
Method
Input Pairing
Model
Loss / Output
FMQ
Codex
Synthesised from target
U-Net3D
L1+SSIM, 1-ch
0.8579
Claude Code
Synthesised from target
U-Net3D
L1+SSIM, 1-ch
0.8521
MLZero
Backprojection → temptv
U-Net2D
L1, 1-ch
0.6953
AutoML-Agent
Zero-filled → temptv
U-Net2D
SSIM, 4-ch sigmoid
0.6583
M3Builder
Zero-filled → temptv
U-Net3DTiny
L1+SSIM, 4-ch sigmoid
0.1979
EvoML
Undersampled → full k-space RSS
VarNet3D
L1+SSIM, 1-ch
0.9498
TABLE I : FastMRI reconstruction pipeline design choices across methods, with resulting mean FMQ (SSIM).
Fig. 5 : Effect of validation gating: without gates, EAC and FMQ both decline, sharply on the harder reconstruction, denoising, and semi-supervised tasks.
Fig. 6 : Effect of expert grounding: without the nnU-Net grounding, both automation and quality fall and variance roughly doubles.
Fig. 7 : Effect of post-run evolution: evolution reduces synthesis time and token consumption across the dataset sequence.
Method
CIFAR-10
VOC2012
AG News
M3Builder
0.9216 ± 0.0049
0.6097 ± 0.0123
0.9113 ± 0.0044
AutoML-Agent
0.9419 ± 0.0027
0.7328 ± 0.0084
0.9103 ± 0.0056
MLZero
0.9466 ± 0.0030
0.7117 ± 0.0068
0.9441 ± 0.0033
Codex
0.9424 ± 0.0042
0.7036 ± 0.0088
0.9207 ± 0.0061
Claude Code
0.9391 ± 0.0032
0.7086 ± 0.0065
0.9457 ± 0.0039
EvoML
0.9674 ± 0.0026
0.7401 ± 0.0058
0.9548 ± 0.0030
TABLE II : Generalisation to three general ML tasks: FMQ (mean ± std).
Method
Manual-Artifact- Free Input
Expert Grounding
Stage-Gated Validation
Cross-run Evolution
M 3 Builder
✗ (DataCard required)
✓ (Fixed templates)
✗ (Shallow output check)
✗
AutoML-Agent
✗ (Skeleton required)
✗
✗ (Validation on pseudo execution)
✗
MLZero
✓
✗ (Document retrieval)
✗ (Execution only)
✗
Codex
✓
✗
✗ (Execution only)
✗
Claude Code
✓
✗
✗ (Execution only)
✗
EvoML
✓
✓
✓
✓
TABLE III : Architectural comparison of EvoML against agentic AutoML systems and command-line coding tools.
Dataset
Human Leaderboard
EvoML
FLARE22
0.910
0.920
ACDC
0.936
0.919
BTCV
0.921
0.837
FLARE21
0.909
0.935
Promise12
0.919
0.920
OrganMNIST3D
0.996
0.992
TABLE IV : Cost-efficiency comparison against expert leaderboard solutions.
Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands · Netherlands Organization for Applied Scientific Research (TNO), The Netherlands