A Multi-Agent LLM System for Automated Tuning of Specialized Neural Network Models

Ivan Moloshnikov; Dmitry Kunitsyn; Alexander Sboev; Maksim Skorokhodov; Roman Rybka; Yuanchao Liu

cs.AIOct 9, 2026

A Multi-Agent LLM System for Automated Tuning of Specialized Neural Network Models

Authors: Ivan Moloshnikov, Dmitry Kunitsyn, Alexander Sboev, Maksim Skorokhodov, Roman Rybka, Yuanchao Liu

Organizations: National Research Centre “Kurchatov Institute”, 1 Akademika Kurchatova Square, Moscow, 123182, Moscow, Russia · National Research Nuclear University MEPhI, 31 Kashirskoye Shosse, Moscow, 115409, Moscow, Russia · Moscow Institute of Physics and Technology, 9 Institutskiy Lane, Dolgoprudny, 141701, Moscow, Russia · Harbin Institute of Technology, No. 92 West Dazhi Street, Nangang District, Harbin, 150001, Heilongjiang, China

Abstract

Many machine learning approaches are two-level: an expert selects a class of methods, within this class the choice of key components is critical and usually manual. This paper addresses the automatic tuning of these components and proposes a framework transferable across models of different nature. The multi-agent large language model (LLM) framework has three roles---analyst, coder, and recorder~---with an experiment log and a sandboxed execution environment. Unlike systems that optimize the entire ML script or search architectures in a discrete space, the framework operates over an open but semantically narrow space: the agent generates code only for the specified components, leaving the overall network structure unchanged. The approach is validated in two directions: tuning a classification spiking neural network with correlation learning, and solving high-order differential equations with Physics-Informed Neural Networks (PINN). Across five tasks---spiking classification on MNIST and Digits, and boundary-value problems for the generalized nonlinear Schrödinger equation (GNLSE) and Kuramoto--Sivashinsky (KS) equations---the agent outperforms the baselines: classification accuracy is 13 percentage points (MNIST) and 6 percentage points (Digits) higher; the error against the analytical single-soliton solution is reduced approximately 2491-fold; the invariant-preservation error drops approximately 325-fold in the selected GNLSE run with a Gaussian perturbation and by more than 100-fold on average over repeated initializations; and the mass drift drops 14-fold for the KS problem with a Gaussian perturbation. The framework thus tunes key components on tasks of different nature without changing its internal structure.

Figures & tables

Figure 1: Architecture of the multi-agent framework. The Human Researcher configures the task, agent-modifiable block(s), target metric, and software contract once. At each iteration, the LLM Analyst reads the experiment log and produces a technical specification, which the LLM Coder converts into contract-constrained code. The generated module is first functionally tested and then evaluated in a Docker sandbox; runtime errors are returned to the Coder, while a successful evaluation is passed to the LLM Recorder. The Recorder appends a structured JSON record to the experiment log, which serves as the external memory for the next iteration. The detailed-prompt mode additionally provides the Analyst with a method-class taxonomy and a dynamically updated coverage map. Only task-specific prompts, the software contract, the editable model block, and the target metric change between tasks; the agent roles and loop structure remain unchanged.

Figure 2: SNN pipeline. The input encoding, the SNN trained with STDP, and the output activity vector $\mathbf{r}$ (left and middle) are fixed. Only the decoder module $\mathrm{Decoder}(\mathbf{r})$ (right) is generated and optimized by the framework.

Figure 3: Reference solutions of the two GNLSE boundary-value problems, shown as a surface (top) and a space–time heat map (bottom). Left: the analytical single soliton. Right: the segmented PINN solution of the soliton with a Gaussian perturbation from Zavertyaev et al. (2024) (Figure 5). The Kuramoto–Sivashinsky problem is not shown because it is solved without a reference solution.

Figure 4: PINN architecture. The coordinates $(x,t)$ pass through the transformation layer $F_{I}$ , the fully connected network, and the transformation layer $F_{O}$ , producing predictions $\hat{u},\hat{v}$ . Automatic differentiation computes the derivatives for the loss function. The loss function ( $\mathcal{L}$ ) consists of the residual in the differential equation ( $\mathcal{L}_{pde}$ ) and the errors in the initial ( $\mathcal{L}_{ic}$ ) and boundary conditions ( $\mathcal{L}_{bc}$ ). The optimizer updates the network weights.

Task (metric)	Baseline (single run)	Best agent result (best in series)	Improvement over baseline (pp / factor)
SNN, MNIST (Acc)	80 %	93 %	$+13$ pp
SNN, scikit-learn digits (Acc)	91 %	97 %	$+6$ pp
PINN, GNLSE, single soliton ( $\mathrm{Rel}_{h}$ )	$5.51\cdot 10^{-2}$	$2.21\cdot 10^{-5}$	$2491\times$
PINN, GNLSE, soliton + Gaussian ( $\mathrm{Rel}_{i}$ )	$2.15\cdot 10^{-1}$	$6.61\cdot 10^{-4}$	$325\times$
PINN, KS, traveling wave + Gaussian ( $\Delta_{m}^{(u)}$ )	$9.31\cdot 10^{-2}$	$6.56\cdot 10^{-3}$	$14\times$

Table 1: Baseline and best agent results across five tasks.

Model	Prompt	Mean maximum accuracy $\pm$ std, %	Best accuracy, %	Runs with accuracy $\geq 90\,\%$
Claude Opus 4.6	simple	$91.13\pm 0.44$	91.65	10 out of 10
Claude Opus 4.6	detailed	$92.15\pm 0.62$	92.83	10 out of 10
Claude Opus 4.6	detailed without coverage map	$91.02\pm 0.25$	91.45	10 out of 10
Perplexity Sonar	simple	$79.88\pm 0.00$	79.88	0 out of 10
Perplexity Sonar	detailed	$91.76\pm 0.56$	92.70	10 out of 10
Perplexity Sonar	detailed without coverage map	$82.34\pm 0.70$	83.18	0 out of 10

Table 2: Comparison of analyst models and prompt variants on MNIST. For each configuration, the table reports the number of complete agent runs, the mean and standard deviation of the maximum accuracy achieved within an individual run, the highest accuracy across all runs, and the number of runs in which the 90 % threshold was reached. Ten matched runs were performed for every configuration.

Method	Mode	$\mathrm{Acc}_{min}$ , %	$\mathrm{Acc}_{max}$ , %	$\mathrm{k}_{min}$	$\mathrm{k}_{max}$
TPOT	classic	89.60	90.15	201	243
TPOT	sequential	89.38	89.93	67	210
Random Choice	LLM sampler	89.48	89.93	-	-
Qwen3-30B- A3B-Instruct-2507	detailed	82.28	89.16	15	209
Perplexity Sonar	detailed	90.93	92.70	8	147
Claude Opus 4.6	detailed	91.22	92.83	2	43

Table 3: Comparison of the agent system, TPOT evolutionary search ( classic and sequential modes), and the Random Choice baseline for the SNN decoder on MNIST with a budget of 250 evaluated models.

Metric	Claude det.	Sonar det.	Qwen det.	Random Choice
Performance
$\mathrm{Acc}_{\min}$ , %	$91.22$	$90.93$	$82.28$	$89.48$
$\mathrm{Acc}_{\max}$ , %	$92.83$	$92.70$	$89.16$	$89.93$
$k_{\min}$	$2$	$8$	$15$	—
$k_{\max}$	$43$	$147$	$209$	—
Resources per run, tokens in millions (M)

Table 4: Performance, resources, and cost of the agent framework and the Random Choice baseline on MNIST (detailed prompt, budget of 250 evaluated decoders per run). The performance block is taken from Table 3 .

Model	Baseline $\mathrm{Rel}_{h}$	Best $\mathrm{Rel}_{h}$	Change relative to baseline
sonar-pro	$4.69\cdot 10^{-2}$	$1.97\cdot 10^{-4}$	$238\times$ better
anthropic-claude-opus-4.6	$5.51\cdot 10^{-2}$	$2.21\cdot 10^{-5}$	$2491\times$ better
qwen-qwen3-30b-a3b-instruct-2507	$4.85\cdot 10^{-2}$	$8.86\cdot 10^{-3}$	$5.5\times$ better

Table 5: PINN results for the single soliton (

\mathrm{Rel}_{h}

metric; problem with an analytical solution). Each value is the mean over three internal seeds. Because baseline variation is non-negligible, the change for each analyst is calculated relative to the identity baseline from the same agent run.

Model	Result	$\mathrm{Rel}_{i}$	$\mathrm{Res}_{\mathrm{bvp}}$	Change in $\mathrm{Rel}_{i}$ relative to baseline
Human-designed PINN from prior work Zavertyaev et al. (2024)	Reported result	$9.87\cdot 10^{-4}$	$9.11\cdot 10^{-5}$	$130\times$ better
Identity baseline (best across runs)		$1.28\cdot 10^{-1}$	$1.04\cdot 10^{-3}$	—
sonar-pro	Best $\mathrm{Rel}_{i}$	$6.35\cdot 10^{-3}$	$2.37\cdot 10^{-2}$	$20\times$ better
anthropic-claude-opus-4.6	Best $\mathrm{Rel}_{i}$	$6.61\cdot 10^{-4}$	$3.95\cdot 10^{-5}$	$194\times$ better
anthropic-claude-opus-4.6	Best $\mathrm{Res}_{\mathrm{bvp}}$	$1.40\cdot 10^{-3}$	$2.08\cdot 10^{-5}$	$91\times$ better
qwen-qwen3-30b-a3b-instruct-2507	Best $\mathrm{Rel}_{i}$	$9.43\cdot 10^{-2}$	$1.38\cdot 10^{-3}$	$1.36\times$ better

Table 6: Single-run PINN results for the soliton with a Gaussian perturbation (problem without an analytical solution). The shared identity-baseline row contains the best value of each metric across the analyst runs; these values may therefore come from different runs. Improvements in

\mathrm{Rel}_{i}

are computed relative to this shared baseline.

Configuration	$\mathrm{Res}_{\mathrm{bvp}}$	$\Delta_{m}^{(u)}$	Hypothesis
Baseline ( $F_{I},F_{O}$ identity)	$1.41\cdot 10^{-3}$	$9.31\cdot 10^{-2}$	—
Best $\mathrm{Res}_{\mathrm{bvp}}$ ( $9\times$ better)	$1.51\cdot 10^{-4}$	$3.47\cdot 10^{-2}$	MultiplicativeIC_DualAdditive_DualZoom
Best $\Delta_{m}^{(u)}$ ( $14\times$ better)	$1.70\cdot 10^{-4}$	$6.56\cdot 10^{-3}$	BoundedMultAdd_SechZoom_DualPhase

Table 7: Best single-run PINN results for the Kuramoto–Sivashinsky traveling wave with a Gaussian perturbation (Claude Opus 4.6, one seed per hypothesis).

\mathrm{Res}_{\mathrm{bvp}}

is the metric the analyst optimized;

\Delta_{m}^{(u)}

is an independent physical diagnostic. The two columns report the best value of each metric over the run; they are attained by different hypotheses, which are named in the last column.

Figure 5: Predicted solution of the Kuramoto–Sivashinsky problem at $t=0,6,10$ for the baseline (top) and for the hypothesis with the lowest mass drift, BoundedMultAdd_SechZoom_DualPhase (bottom). No numerical reference is available for this problem, so only the PINN prediction is plotted; the initial condition is shown in the left panels for comparison. The horizontal axis is the coordinate $x$ , the vertical axis is $u$ . The baseline pulse loses roughly half of its amplitude by $t=10$ and develops spurious low-amplitude oscillations across the domain, whereas the agent hypothesis keeps a localized traveling pulse whose amplitude stays close to the initial value of $1/2$ .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Single soliton	Soliton + Gaussian
Number of hidden layers	2	4
Neurons per layer	32	64
Activation (hidden layers)	$\sin$	$\sin$
Activation (output layer)	linear	linear
Optimizer	Adam	Adam
Learning rate	$10^{-3}$	$10^{-3}$

Appendix

Table 8: Hyperparameters of the baseline PINN for the two GNLSE configurations. The Kuramoto–Sivashinsky configuration is given separately in Table 9 .

Parameter	Value
Number of hidden layers	4
Neurons per layer	32
Network output	$u$ (one real scalar)
Activation (hidden layers)	$\sin$
Activation (output layer)	linear
Optimizer	Adam, $\mathrm{clipnorm}=1$

Appendix

Table 9: Hyperparameters of the baseline PINN for the Kuramoto–Sivashinsky traveling-wave problem with a Gaussian perturbation.

Parameter	Value
$V_{rest}$ (resting potential)	$-70$ mV
$V_{reset}$ (post-spike potential)	$-70$ mV
$V_{th}$ (threshold potential)	$-56$ mV
$\tau_{m}$ (membrane time constant)	$10^{3}$ ms
$\tau_{s}$ (teacher-spike shift)	$0.1$ ms
$N_{ref}$ (spacing between spikes in $S_{0}$ )	3

Appendix

Table 10: Hyperparameters of the baseline SNN and its training, common to all configurations.

Configuration	Mean $\pm$ std	Median	Min–max	CV	Reduction factor in $\mathrm{Rel}_{i}$ relative to baseline mean
Baseline ( $n=10$ )	$(1.65\pm 0.18)\cdot 10^{-1}$	$1.66\cdot 10^{-1}$	$1.29\cdot 10^{-1}$ – $1.94\cdot 10^{-1}$	$11.1\,\%$	$1\times$
Residual-selected configuration ( $n=10$ )	$(1.42\pm 0.59)\cdot 10^{-3}$	$1.50\cdot 10^{-3}$	$5.40\cdot 10^{-4}$ – $2.41\cdot 10^{-3}$	$41.8\,\%$	$116.1\times$
Invariant-selected configuration ( $n=10$ )	$(1.60\pm 0.81)\cdot 10^{-3}$	$1.59\cdot 10^{-3}$	$7.57\cdot 10^{-4}$ – $3.31\cdot 10^{-3}$	$50.4\,\%$	$103.3\times$

Appendix

Table 11: Repeated runs for the GNLSE soliton problem with a Gaussian perturbation. The table reports the mean and sample standard deviation, median, range, and coefficient of variation (CV) of

\mathrm{Rel}_{i}

Method class	Methods	Best acc, %	Status
Ensembles	154	92.70	explored
LDA / QDA	32	87.90	explored
Softmax / calibration	13	82.32	explored
$k$ -nearest neighbors	11	88.56	explored
Naive Bayes	11	83.73	explored
Mahalanobis	9	86.05	limited data

Appendix

Table 12: Coverage map of decoder method classes for one 250-iteration run (Perplexity Sonar, detailed prompt, MNIST).

Pair no.	Simple, %	Detailed, %	$\Delta$ , pp
1	79.88	82.28	2.40
2	79.88	83.35	3.47
3	79.88	88.57	8.69
4	79.88	86.48	6.60
5	79.88	89.16	9.28
6	79.88	84.10	4.22

Appendix

Table 13: Maximum accuracy in ten matched pairs of complete agent-system runs with the simple and detailed prompts for Qwen3-30B-A3B-Instruct-2507 on MNIST. Within each pair, the same trained SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy for the detailed prompt minus the accuracy for the simple prompt.

Pair no.	With map, %	Without map, %	$\Delta$ , pp
1	82.28	79.88	2.40
2	83.35	80.42	2.93
3	88.57	79.88	8.69
4	86.48	81.15	5.33
5	89.16	79.88	9.28
6	84.10	80.73	3.37

Appendix

Table 14: Comparison of maximum accuracy in ten matched Qwen3-30B-A3B-Instruct-2507 runs on MNIST for the detailed prompt with and without the coverage map. Within each pair, the same SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy with the coverage map minus the accuracy without the coverage map.

Pair no.	Simple, %	Detailed, %	$\Delta$ , pp
1	79.88	92.70	12.82
2	79.88	92.57	12.69
3	79.88	91.95	12.07
4	79.88	91.82	11.94
5	79.88	91.77	11.89
6	79.88	91.74	11.86

Appendix

Table 15: Maximum accuracy in ten matched pairs of complete agent-system runs with the simple and detailed prompts for Perplexity Sonar on MNIST. Within each pair, the same trained SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy for the detailed prompt minus the accuracy for the simple prompt.

Pair no.	With map, %	Without map, %	$\Delta$ , pp
1	92.70	83.18	9.52
2	92.57	82.92	9.65
3	91.95	82.77	9.18
4	91.82	82.66	9.16
5	91.77	82.65	9.12
6	91.74	82.47	9.27

Appendix

Table 16: Comparison of maximum accuracy in ten matched Perplexity Sonar runs on MNIST for the detailed prompt with and without the coverage map. Within each pair, the same SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy with the coverage map minus the accuracy without the coverage map.

Pair no.	Simple, %	Detailed, %	$\Delta$ , pp
1	90.20	92.83	2.63
2	91.15	91.88	0.73
3	91.65	91.58	-0.07
4	91.60	92.49	0.89
5	91.33	91.22	-0.11
6	90.71	92.35	1.64

Appendix

Table 17: Maximum accuracy in ten matched pairs of complete agent-system runs with the simple and detailed prompts for Claude Opus 4.6 on MNIST. Within each pair, the same trained SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy for the detailed prompt minus the accuracy for the simple prompt.

Pair no.	With map, %	Without map, %	$\Delta$ , pp
1	91.22	90.66	0.56
2	91.24	90.73	0.51
3	91.58	90.89	0.69
4	91.88	90.91	0.97
5	92.35	90.95	1.40
6	92.49	91.01	1.48

Appendix

Table 18: Comparison of maximum accuracy in ten matched Claude Opus 4.6 runs on MNIST for the detailed prompt with and without the coverage map. Within each pair, the same SNN instance and identical output spike data were used. The difference

\Delta

is calculated as the accuracy with the coverage map minus the accuracy without the coverage map.

Model	Prompt	$n$	Mean maximum accuracy $\pm$ std, %	Best accuracy, %	Runs with accuracy $\geq 93\,\%$
Claude Opus 4.6	simple	10	$95.07\pm 0.52$	95.56	10 out of 10
Claude Opus 4.6	detailed	10	$96.12\pm 0.66$	96.67	10 out of 10
Perplexity Sonar	simple	10	$93.44\pm 0.57$	93.89	9 out of 10
Perplexity Sonar	detailed	10	$93.31\pm 0.93$	94.17	8 out of 10
Qwen3-30B-A3B-Instruct-2507	simple	10	$91.63\pm 0.74$	92.78	0 out of 10
Qwen3-30B-A3B-Instruct-2507	detailed	10	$93.66\pm 0.79$	94.17	7 out of 10

Appendix

Table 19: Comparison of analyst models and prompt variants on the digits

8\times 8

dataset. For each configuration, the table reports the number of complete agent runs, the mean and standard deviation of the maximum accuracy achieved within an individual run, the highest accuracy across all runs, and the number of runs in which the 93 % threshold was reached. Ten matched runs of 250 iterations were performed for every configuration; the baseline (argmax decoder) gives an accuracy of 90.83 %.

Panel of Figure 6 — Figure 6: Search dynamics for the decoding module on the MNIST dataset. The blue line shows the accuracy of the solution obtained at each iteration, the red dashed line shows the best result achieved, and the gray horizontal line shows the baseline accuracy of 79.88 %. The columns correspond to analyst models, and the upper and lower rows correspond to the simple and detailed prompts, respectively.

Panel of Figure 7 — Figure 7: Search dynamics for the decoding module on the digits dataset. The blue line shows the accuracy of the solution obtained at each iteration, the red dashed line shows the best result achieved, and the gray horizontal line shows the baseline accuracy of 90.83 %. The columns correspond to analyst models, and the upper and lower rows correspond to the simple and detailed prompts, respectively.

Method class	Catalog operator	Implementation
Prototypes and centroids	NearestCentroid	Classification by distance to the class centroid
$k$ -nearest neighbors	KNeighborsClassifier	The number of neighbors, weighting rule, and metric are selected from the grid
Correlation, cosine, and local-density rules	KNeighborsClassifier	Implemented using different local-proximity metrics
Covariance methods, LDA, and QDA	LinearDiscriminant Analysis , QuadraticDiscriminant Analysis	Linear and quadratic discriminant rules
Naive Bayes classifier	GaussianNB	Gaussian class-conditional feature model
Threshold and tree-based rules	DecisionTreeClassifier , ExtraTreeClassifier	Threshold partitions of the spike-rate space

Appendix

Table 20: Mapping of decoder method classes to TPOT catalog operators.

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 332 papers

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
Apr 15, 2026Zerun Ma, Guoqiang Wang, Xinchen Xie +7LLM Fine-Tuning LLM Agent Training
Shanghai AI Laboratory · Fudan University
Evolving Excellence: Automated Optimization of LLM-based Agents
Dec 9, 2025Paul Brookes, Vardan Voskanyan, Rafail Giavrimis +18Large Language Model-Guided Optimization LLM Agent Workflow Optimization
TurinTech AI, London, UK · University of Surrey, Guildford, United Kingdom · University of Leeds, Leeds, UK +4
Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning
Sep 6, 2026Haoran Zhao, Wei Du, Dingwen Yang +7Fine-Tuning Language Models LLM Fine-Tuning
Fudan University · Ant Group · Shanghai Jiaotong University

Apr 15, 2026J/K move · Enter open · S save

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration

Zerun Ma, Guoqiang Wang, Xinchen Xie, Yicheng Chen+6

Shanghai AI Laboratory · Fudan University

LLM Fine-Tuning LLM Agent Training Multi-Agent LLM Systems

Open paper AskPDF arXiv

While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.

Abstract

Neural Network Optimization

Explore similar work

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration

Evolving Excellence: Automated Optimization of LLM-based Agents

Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning