cs.CLOct 4, 2026

Building LLM Agent Systems the Deep Learning Way: From Modular Design to Architecture Search

Authors: Tao Feng, Pengrui Han, Zhongjie Dai, Jiaxuan You

Organizations: University of Illinois Urbana-Champaign

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Models (LLMs) have revolutionized AI research and enabled exciting agent systems. To build a complex LLM agent system, most existing research relies on insights from other domains or heuristics to manually build the agent system. However, this approach often requires heavy hand-engineering and fails to fully optimize for the downstream task of interest. Inspired by the tremendous success of deep learning, we propose to construct LLM agent systems in a modular manner, similar to building a deep neural network. Our key insight is to make analogies between LLM building blocks, such as retrievals, memories, and prompting strategies, and the successful deep learning modules, such as MLPs, attention, and recurrent modules. We further design forward inference and feedback mechanisms for LLMs, where prompts in LLMs are considered as the weights in deep models, and the prompt optimization from feedback is analogous to the back-propagation algorithm. We additionally leverage a search algorithm to search for the best configuration of LLM agent systems, similar to the neural architecture search (NAS) in deep learning research. Comprehensive experimental results demonstrate that the proposed deep learning recipe for LLM agent systems is highly effective, in particular: (1) Organizing LLM modules into deep-learning-style architectures yields noticeable performance gain; (2) Automatic prompt optimization, equivalent to backpropagation, is efficient in incorporating feedback from the task of interest and achieves at least 5% performance improvement; (3) NAS equivalent algorithm works well for further optimizing the LLM agent system architecture with 11% performance gain compared with randomly designed architectures. Overall, our research demonstrates the exciting opportunity of transferring the success of deep learning to building LLM agent systems.

Figures & tables

Figure 1: Basic Building Blocks of LLM agent systems . (a) A vanilla LLM can be translated into a single layer MLP, which receives a query qq and outputs a response rr ; (b) Memory processor can be regarded as RNN since they both utilize historical data for future planning; (c) Retrieval is analogous to the attention network of DL because they both extract the importance information of the corpus context; (d) Prompting strategies, such as Tree/Graph of thoughts, are translated to GNNs since they decompose query qq into interconnected thoughts, which are connected and related to each other.
Figure 2: Forward and Back Propagation Algorithms for Large Language Model . In this process, meta prompt is regarded as trainable parameters in DL. For the forward inference, the input query goes through the composition of LLM building blocks to obtain the response, which is guided through the meta prompt. For the feedback propagation, the feedback from the self-reflection updates the parameters (meta prompt) of LLM.
Figure 3: Build a Cross-attention Transformer with LLM Building Blocks . The multi-query retrieval is equivalent to the cross-attention network, and the LLM is translated to MLP in DL. Additionally, the process of appending the raw query to the generated multiple queries is equivalent to the residual.
Figure 4: An example of the meta prompt used for RNN-MLP on the GSM8K dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Table 5
Figure 5: Optimization on transformer to maximize F1.
Figure 7
 074378
F1 Score0.5410.6850.6940.712
Table 3: As the number of LLM architecture searches increases, the LLM agent system is getting better.
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

LLM agent systemsDeep Learning Architecture
Vanilla LLMSingle layer MLP
Memory processorRecurrent neural network
RetrievalAttention Network
Prompting strategies (ToT & GoT)Graph neural network
PromptsTraining parameters
Max tokens of each LLMSize of MLP
Appendix
Table 4: Deep Learning Recipe for LLM Agent Systems.
Figure 8: An example of the meta prompt used for RNN-MLP on the TSP dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 9: An example of the meta prompt used for Tree-MLP/GNN-MLP on the TSP dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 10: An example of the meta prompt used for RNN-MLP on GSM8K dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 11: An example of the meta prompt used for Tree-MLP/GNN-MLP on GSM8K dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 12: An example of the meta prompt used for RNN-Att-MLP on StrategeyQA dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 13: An example of the meta prompt used for Tree-Att-MLP/GNN-Att-MLP on StrategeyQA dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 14: An example of the meta prompt used for RNN-Att-MLP on Related-multi dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
Figure 15: An example of the meta prompt used for Tree-Att-MLP/GNN-Att-MLP on Related-multi dataset. The blue text contains input questions and previous responses/evaluations. The purple text describes the output format instructions.
 Range
Output Size m1m_{1}[64, 128, 192, 256, 320]
Output Size m2m_{2}[64, 128, 192, 256, 320]
Output Size m3m_{3}[64, 128, 192, 256, 320]
Head Num hh[1, 2, 3, 4, 5]
Retrieve Num nn[1, 2, 3, 4, 5]
Appendix
Table 5: Optimization range of each hyperparameter.
Figure 16: The optimization process on transformer . The x-axis represents the number of trials. For each trial, we will get a set of pr,F1pr,F1 ; the y-axis represents the F1F1 value.
DatasetTask TypeCases
Without External Knowledge
TSPOptimization1500
GSM8KMath Problem1,319
With External Knowledge
StrategyQACommonsense Reasoning2,290
Related-multiSummary1000
Appendix
Table 6: Overview of Datasets.
More in this topic

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 681 papers

Explore similar work

CardsList
Apr 15, 2026

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration

While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.

Zerun Ma, Guoqiang Wang, Xinchen Xie +7

Shanghai AI Laboratory · Fudan University

LLM Fine-TuningLLM Agent TrainingMulti-Agent LLM Systems
AskPDF
Dec 9, 2025

Evolving Excellence: Automated Optimization of LLM-based Agents

Agentic AI systems built on large language models (LLMs) offer significant potential for automating complex workflows, from software development to customer support. However, LLM agents often underperform due to suboptimal configurations; poorly tuned prompts, tool descriptions, and parameters that typically require weeks of manual refinement. Existing optimization methods either are too complex for general use or treat components in isolation, missing critical interdependencies. We present ARTEMIS, a no-code evolutionary optimization platform that jointly optimizes agent configurations through semantically-aware genetic operators. Given only a benchmark script and natural language goals, ARTEMIS automatically discovers configurable components, extracts performance signals from execution logs, and evolves configurations without requiring architectural modifications. We evaluate ARTEMIS on four representative agent systems: the ALE Agent for competitive programming on AtCoder Heuristic Contest, achieving a \textbf{13.6%13.6\% improvement} in acceptance rate; the Mini-SWE Agent for code optimization on SWE-Perf, with a statistically significant 10.1% performance gain; and the CrewAI Agent for cost and mathematical reasoning on Math Odyssey, achieving a statistically significant \textbf{36.9%36.9\% reduction} in the number of tokens required for evaluation. We also evaluate the MathTales-Teacher Agent powered by a smaller open-source model (Qwen2.5-7B) on GSM8K primary-level mathematics problems, achieving a 22% accuracy improvement and demonstrating that ARTEMIS can optimize agents based on both commercial and local models.

Paul Brookes, Vardan Voskanyan, Rafail Giavrimis +18

TurinTech AI, London, UK · University of Surrey, Guildford, United Kingdom · University of Leeds, Leeds, UK +4

Large Language Model-Guided OptimizationLLM Agent Workflow OptimizationLLM AgentsGenetic Algorithms
AskPDF
Apr 3, 2026

Self-Optimizing Multi-Agent Systems for Deep Research

Given a user's complex information need, a multi-agent Deep Research system iteratively plans, retrieves, and synthesizes evidence across hundreds of documents to produce a high-quality answer. In one possible architecture, an orchestrator agent coordinates the process, while parallel worker agents execute tasks. Current Deep Research systems, however, often rely on hand-engineered prompts and static architectures, making improvement brittle, expensive, and time-consuming. We therefore explore various multi-agent optimization methods to show that enabling agents to self-play and explore different prompt combinations can produce high-quality Deep Research systems that match or outperform expert-crafted prompts.

Arthur Câmara, Vincent Slot, Jakub Zavrel

Zeta Alpha, Amsterdam, The Netherlands

Deep Research AgentsMulti-Agent LLM SystemsMulti-Agent System OptimizationPrompt Optimization
AskPDF