Organizations: School of Computer Science, University of Auckland, Auckland, New Zealand · School of Information and Communication Technology, Griffith University, Brisbane, Australia · Metasics, China. Xiamen, China · School of Cyberspace Science and Technology, Beijing Institute of Technology, Beijing, China · DSA Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Department of Computer Science, National University of Singapore, Singapore
Recent advances in large language models have enabled agentic systems that automate machine learning pipeline construction from minimal user input. However, existing agentic AutoML methods either depend on rigid templates that limit adaptability, target narrow task domains, or rely on free-form code generation that lacks reliability guarantees, leaving intermediate errors to propagate silently and degrade the final model. We present EvoML, an agentic framework for end-to-end ML pipeline synthesis built on expert-grounded skills that evolve across runs, executed through a validation-gated harness. The framework decomposes pipeline construction into four phases coordinated by three agents: an Orchestrator that enforces validation gates, advancing each phase only when its outputs pass structural, execution, and semantic checks, which precludes error propagation and keeps every artifact auditable; an Executor that runs expert-grounded skills, grounded in the nnU-Net workflow and pairing reusable templates with online adaptation; and a Curator that performs post-run evolution, consolidating the scripts and guidance recovered during a run into the skill library so it improves across datasets. We evaluate EvoML on eight datasets spanning six sophisticated, real-world ML task types, against agentic AutoML systems and command-line coding tools. EvoML achieves full automation and the best final model quality on all eight, spanning segmentation (up to 0.94 Dice), classification (0.99 accuracy), reconstruction (0.95 SSIM), and denoising (0.79 normalized PSNR), while transferring to broader ML tasks.
Figures & tables
Fig. 1 : Overview of the proposed validation-gated agentic ML pipeline synthesis framework. The Orchestrator progresses through four phases and invokes the corresponding Executor skill group in sequence: Task Understanding (skills 1–2), Planning (3–4), Training (5–7), and Post-train Processing (8–9). After each phase, generated artifacts and execution evidence are returned for gated validation; passed phases advance, while failed phases trigger recovery. The Curator operates after the run to evolve the skill library from accumulated execution evidence.
Fig. 2 : An example validation-gated recovery cycle in the Planning phase.
Fig. 3 : The Curator’s post-run skill evolution loop after a 3D reconstruction task with HDF5 dataset format.
Fig. 4 : Comparison of EvoML against agentic AutoML systems and command-line coding tools across the benchmarking datasets: EvoML attains full automation and the best FMQ on every dataset, with markedly lower variance than the baselines.
Method
Input Pairing
Model
Loss / Output
FMQ
Codex
Synthesised from target
U-Net3D
L1+SSIM, 1-ch
0.8579
Claude Code
Synthesised from target
U-Net3D
L1+SSIM, 1-ch
0.8521
MLZero
Backprojection → temptv
U-Net2D
L1, 1-ch
0.6953
AutoML-Agent
Zero-filled → temptv
U-Net2D
SSIM, 4-ch sigmoid
0.6583
M3Builder
Zero-filled → temptv
U-Net3DTiny
L1+SSIM, 4-ch sigmoid
0.1979
EvoML
Undersampled → full k-space RSS
VarNet3D
L1+SSIM, 1-ch
0.9498
TABLE I : FastMRI reconstruction pipeline design choices across methods, with resulting mean FMQ (SSIM).
Fig. 5 : Effect of validation gating: without gates, EAC and FMQ both decline, sharply on the harder reconstruction, denoising, and semi-supervised tasks.
Fig. 6 : Effect of expert grounding: without the nnU-Net grounding, both automation and quality fall and variance roughly doubles.
Fig. 7 : Effect of post-run evolution: evolution reduces synthesis time and token consumption across the dataset sequence.
Method
CIFAR-10
VOC2012
AG News
M3Builder
0.9216 ± 0.0049
0.6097 ± 0.0123
0.9113 ± 0.0044
AutoML-Agent
0.9419 ± 0.0027
0.7328 ± 0.0084
0.9103 ± 0.0056
MLZero
0.9466 ± 0.0030
0.7117 ± 0.0068
0.9441 ± 0.0033
Codex
0.9424 ± 0.0042
0.7036 ± 0.0088
0.9207 ± 0.0061
Claude Code
0.9391 ± 0.0032
0.7086 ± 0.0065
0.9457 ± 0.0039
EvoML
0.9674 ± 0.0026
0.7401 ± 0.0058
0.9548 ± 0.0030
TABLE II : Generalisation to three general ML tasks: FMQ (mean ± std).
Method
Manual-Artifact- Free Input
Expert Grounding
Stage-Gated Validation
Cross-run Evolution
M 3 Builder
✗ (DataCard required)
✓ (Fixed templates)
✗ (Shallow output check)
✗
AutoML-Agent
✗ (Skeleton required)
✗
✗ (Validation on pseudo execution)
✗
MLZero
✓
✗ (Document retrieval)
✗ (Execution only)
✗
Codex
✓
✗
✗ (Execution only)
✗
Claude Code
✓
✗
✗ (Execution only)
✗
EvoML
✓
✓
✓
✓
TABLE III : Architectural comparison of EvoML against agentic AutoML systems and command-line coding tools.
Dataset
Human Leaderboard
EvoML
FLARE22
0.910
0.920
ACDC
0.936
0.919
BTCV
0.921
0.837
FLARE21
0.909
0.935
Promise12
0.919
0.920
OrganMNIST3D
0.996
0.992
TABLE IV : Cost-efficiency comparison against expert leaderboard solutions.
The purpose of our paper is to develop a unified multi-agent architecture that automates end-to-end machine learning (ML) pipeline generation from datasets and natural-language (NL) goals, improving efficiency, robustness and explainability. A five-agent system is proposed to handle profiling, intent parsing, microservice recommendation, Directed Acyclic Graph (DAG) construction and execution. It integrates code-grounded Retrieval-Augmented Generation (RAG) for microservice understanding, an explainable hybrid recommender combining multiple criteria, a self-healing mechanism using Large Language Model (LLM)-based error interpretation and adaptive learning from execution history. The approach is evaluated on 150 ML tasks across diverse scenarios. The system achieves an 84.7% end-to-end pipeline success rate, outperforming baseline methods. It demonstrates improved robustness through self-healing and reduces workflow development time compared to manual construction. The study introduces a novel integration of code-grounded RAG, explainable recommendation, self-healing execution and adaptive learning within a single architecture, showing that tightly coupled intelligent components can outperform isolated solutions.
Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space. We present LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator. To our knowledge, LACE is the first to formulate general tabular pipeline AutoML this way, evaluated on standardized OpenML tasks under a leakage-controlled protocol that withholds dataset identity from the generator. Because every candidate is ordinary Python, the returned pipeline and the search that produced it can be inspected and edited directly, rather than only through a framework's model objects. On 68 OpenML classification tasks, LACE with GPT-5.4-mini significantly outperforms auto-sklearn, H2O, and a fixed XGBoost baseline, with no detectable difference against AutoGluon, the strongest search-based system evaluated, while covering the full benchmark. Newer tabular foundation models are more accurate on the subset of tasks they support, but apply a fixed pretrained predictor rather than returning an editable task-specific program. LACE's contribution is therefore not raw accuracy but a search space defined by code: complete coverage, pipelines practitioners can reuse directly, and a component set extended by editing the prompt rather than the framework.
Sofoklis Kitharidis, Cor J. Veenman, Jan N. van Rijn +2
Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands · Netherlands Organization for Applied Scientific Research (TNO), The Netherlands
Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes. We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE through outcome-guided supervision. Windowed Interaction RL (WIRL) treats the multi-turn decisions between successive experiment results as an execution window and assigns it a shared outcome-based reward. Asynchronous collection and policy correction support policy updates before the trajectory ends. On MLE-Bench Lite, MLExpert improves the average medal rate of the base model from 37.9% to 72.7%, achieving performance comparable to larger frontier models such as Kimi-K3 and GLM-5.3. Further experiments show that MLExpert improves performance on NatureBench Lite, agent scaffold optimization, and algorithm optimization, suggesting that the learned optimization capability generalizes to other tasks.
Shangheng Du, Jinxin Shi, Shuaiyu Zhang +7
East China Normal University · Shanghai Artificial Intelligence Laboratory