cs.AIMar 23, 2026
SaveFrom Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
Organizations: Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180 USA · IBM Research, Yorktown Heights, NY 10598 USA · Department of Mechanical, Aerospace, and Nuclear Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180 USA
Abstract
Large language model (LLM)-based systems are becoming increasingly popular for solving tasks by constructing executable workflows that interleave LLM calls, information retrieval, tool use, code execution, memory updates, and verification. This survey reviews recent methods for designing and optimizing such workflows, which we treat as agentic computation graphs (ACGs). We organize the literature based on when workflow structure is determined, where structure refers to which components or agents are present, how they depend on each other, and how information flows between them. This lens distinguishes static methods, which fix a reusable workflow scaffold before deployment, from dynamic methods, which select, generate, or revise the workflow for a particular run before or during execution. We further organize prior work along three dimensions: when structure is determined, what part of the workflow is optimized, and which evaluation signals guide optimization (e.g., task metrics, verifier signals, preferences, or trace-derived feedback). We also distinguish reusable workflow templates, run-specific realized graphs, and execution traces, separating reusable design choices from the structures actually deployed in a given run and from realized runtime behavior. Finally, we outline a structure-aware evaluation perspective that complements downstream task metrics with graph-level properties, execution cost, robustness, and structural variation across inputs. Our goal is to provide a clear vocabulary, a unified framework for positioning new methods, a more comparable view of existing body of literature, and a more reproducible evaluation standard for future work in workflow optimizations for LLM agents.
Figures & tables
Figure 1: An overview of workflow optimization for LLM Agents.
| Survey family | Central lens | How workflow appears | Relation to this survey |
|---|---|---|---|
| Workflow / infrastructure ( Yu et al., 2025 ; Li et al., 2024a ) | workflow systems, orchestration abstractions, and MAS system organization | workflow as system scaffold, lifecycle, or engineering abstraction | adjacent but not centered on structure search, per-run graph construction, or workflow-specific evaluation |
| Planning ( Huang et al., 2024 ) | decomposition, plan selection, reflection, memory, and external planning modules | workflow as one internal phase or capability of an agent | covers a crucial ingredient of agents, but not workflow optimization as a broader graph-design problem |
| Tool learning ( Xu et al., 2025b ) | tool retrieval, tool planning, tool invocation, and tool-use evaluation | workflow as a tool-use pipeline and environment interface | centers tool competence rather than topology, routing, or graph generation/editing |
| MAS collaboration / applications ( Chen et al., 2024b ; Tran et al., 2025 ; Zhang et al., 2025f ) | collaboration mechanisms, communication protocols, and application domains | workflow as a communication pattern or application-specific coordination scheme | studies collaborative behavior and applications, not reusable-template or realized-graph design as the main object |
| Broad agent optimization ( Du et al., 2026 ) | parameter-driven and parameter-free optimization of agents overall | workflow as one optimization lever among prompts, training, retrieval, memory, and other factors | does not isolate workflow structure as the primary object of comparison |
| Agent architecture / evaluation ( V et al., 2026 ) | architectures, taxonomies, and evaluation of agentic AI systems | workflow as one architectural manifestation within a larger agent taxonomy | provides a broad architecture-and-evaluation view, but not a workflow-optimization-specific synthesis |
Table 1: Positioning this survey in the broader survey landscape. Most prior surveys center on adjacent topics such as planning, tool learning, collaboration, architecture, or broad agent optimization, whereas this survey centers workflow structure itself.
| Method | Setting (GDT/GPM) | Level | Representation | Feedback / evidence | Update mechanism | Cost handling |
|---|---|---|---|---|---|---|
| AFlow ( Zhang et al., 2025e ) | offline / none | graph | typed operator graph | metric: task score | search: MCTS | soft objective ($) |
| ADAS ( Hu et al., 2025a ) | offline / none | joint | runnable code | metric: task score | search: archive meta-search | none |
| A 2 Flow ( Zhao et al., 2025 ) | offline / none | graph | abstract operator graph | supervision: demos + execution | hybrid: operator learning + search | none |
| Multi-Agent Design ( Zhou et al., 2025 ) | offline / none | joint | topology + prompts | metric: task score | hybrid: staged alternation | none |
| Learning Multi-Agent Communication from Graph Modeling Perspective ( Hu et al., 2024 ) | offline / none | joint | differentiable communication graph | proxy: training objective | continuous relaxation | none |
| Optima ( Chen et al., 2025 ) | offline / none | node | fixed scaffold + trajectories | metric: quality–efficiency reward | hybrid: generate-rank-select-train | soft objective (efficiency) |
Table 2: Representative core static workflow-optimization methods, summarized with a unified comparison card. When a paper uses multiple evidence types or update mechanisms, columns report the dominant workflow-relevant entry in each field.
| Method | Setting (GDT/GPM) | Level | Representation | Feedback / evidence | Update mechanism | Cost handling |
|---|---|---|---|---|---|---|
| Pre-execution generation or selection | ||||||
| Difficulty-Aware Agentic Orchestration ( Su et al., 2025a ) | pre-exec / select | joint | modular operator workflow | proxy: difficulty estimate | controller: router + allocator | soft objective (cost) |
| Assemble Your Crew ( Li et al., 2025b ) | pre-exec / generate | graph | query-conditioned DAG | supervision: task labels | generator: autoregressive DAG | none |
| G-Designer ( Zhang et al., 2025d ) | pre-exec / generate | graph | generated communication graph | metric: quality score | generator: VGAE | soft objective (cost) |
| Dynamic Generation of Multi-LLM Agents Communication Topologies ( Jiang et al., 2025 ) | pre-exec / generate | graph | diffusion-generated topology | proxy: reward proxy | generator: graph diffusion | soft objective (cost) |
| Multi-Agent Architecture Search via Agentic Supernet ( Zhang et al., 2025b ) | pre-exec / generate | graph | sampled supernet architecture | metric: quality score | generator: architecture distribution | soft objective (cost) |
Table 3: Representative core dynamic workflow-optimization methods, summarized with the same comparison card as Table 2 . When a paper uses multiple evidence types or update mechanisms, columns report the dominant workflow-relevant entry in each field.
| Role in workflow research | Representative resources | Main artifact or evidence | Structure-aware? | Best used for |
|---|---|---|---|---|
| Workflow-generation benchmarks | WorFBench ( Qiao et al., 2025 ) , FlowBench ( Xiao et al., 2024 ) , constrained graphical domains such as ComfyUI-R1 ( Xu et al., 2025c ) | reference workflows, schema-valid graphs, workflow plans | / | evaluating structural fidelity, validity, and workflow-level robustness |
| Training corpora and executable validators | WorkflowBench ( Fan et al., 2024 ) , HumanEval ( Chen et al., 2021 ) , MBPP ( Austin et al., 2021 ) , GSM8K ( Cobbe et al., 2021 ) , MATH ( Hendrycks et al., 2021 ) , DROP ( Dua et al., 2019 ) , HotpotQA ( Yang et al., 2018 ) | workflow code pairs, hidden tests, labeled reasoning tasks | / | supervised workflow generation and controlled validation of local or global design choices |
| Interactive tool-use benchmarks | GAIA ( Mialon et al., 2024 ) , -Bench ( Yao et al., 2025 ) , -Bench ( Barres et al., 2025 ) , APIBank ( Li et al., 2023b ) , T-Eval ( Chen et al., 2024c ) , ToolBench ( Xu et al., 2023 ) , AppWorld ( Trivedi et al., 2024 ) , MCP-Universe ( Luo et al., 2025 ) , MCP-Bench ( Wang et al., 2025f ) , MCP-RADAR ( Gao et al., 2025b ) , MCPEval ( Liu et al., 2025b ) , LiveMCPBench ( Mo et al., 2025 ) , MCPWorld ( Yan et al., 2025 ) | long-horizon execution traces under realistic tools | end-to-end orchestration, schema compliance, recovery, and tool-selection quality | |
| Software and operations environments | SWE-bench ( Jimenez et al., 2024 ) , Terminal-Bench ( Merrill et al., 2026 ) , SOP-Bench ( Nandi et al., 2025 ) , AssetOpsBench ( Patel et al., 2025 ) | repository edits, terminal traces, procedural execution logs | planning, verification, and recovery under hard constraints |
Table 4: Representative evaluation assets, grouped by the role they play in workflow optimization research.
| Dimension | What should be reported | Why it matters |
|---|---|---|
| Workflow representation | code, DSL, graph IR, schema constraints, executable interpreter, available operators and tools | determines what can be searched, validated, or edited |
| Structural setting | static or dynamic, GDT, GPM, admissible edits, routing policy, stopping rules | clarifies what kind of structural variation the method actually allows |
| Model and tool configuration | base models, decoding settings, tool registry, verifier placement, memory policy | separates workflow effects from backbone or tool effects |
| Offline optimization cost | search budget, number of evaluated candidates, training compute, verifier cost, human annotation if any | distinguishes one-time optimization expense from deployment expense |
| Online inference cost | tokens, LLM calls, tool calls, latency, wall-clock time, dollars, cost-per-success | makes quality–cost trade-offs scientifically comparable |
| Trace statistics | number of rounds, retries, edits, failures, fallbacks, termination causes | exposes hidden control-flow behavior and failure modes |
Table 5: A proposed minimum reporting protocol for workflow-optimization papers.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Paper | What is optimized | Feedback signal | Method | Why it matters for workflows |
|---|---|---|---|---|
| DSPy ( Khattab et al., 2023 ) | module prompts and demonstrations | user-defined task metric | compiler / teleprompting | direct local-node optimization in a fixed pipeline |
| Large Language Models as Optimizers ( Yang et al., 2023 ) | prompts or instructions | external scalar score | LLM-as-optimizer search | black-box optimizer for workflow-local instructions |
| EvoPrompt ( Guo et al., 2023 ) | prompt population | development metrics | evolutionary search | node-level instruction search without graph change |
| CAPO ( Zehle et al., 2025 ) | instructions and few-shot content | quality + token-aware cost | cost-aware evolution | makes node tuning sensitive to deployment cost |
| GEPA ( Agrawal et al., 2025 ) | prompts inside a larger system | task outcomes + reflection | reflective evolution | example of trace-derived text updating workflow-local prompts |
Table 6: Node-level prompt optimization methods commonly used inside larger workflows.
| Paper | Setting | What changes at inference time | Signal / method | Why it is adjacent to workflow optimization |
|---|---|---|---|---|
| Cut the Crap ( Zhang et al., 2025c ) | offline compression | communication edges and message passing | importance proxies + task quality | post-hoc communication sparsification of a discovered scaffold |
| Adaptive Graph Pruning ( Li et al., 2025a ) | runtime select | agents and communication edges | joint node/edge pruning | learns sparse task-adaptive subgraphs |
| AgentDropout ( Wang et al., 2025g ) | runtime select | redundant agents and links | utility versus token cost | dynamically reduces collaboration overhead |
| DAGP ( Wang and Tong, 2025 ) | runtime select | difficulty-conditioned edges | difficulty estimation + task reward | allocates denser communication only when needed |
| DyLAN ( Liu et al., 2023 ) | runtime select | agent team and collaboration pattern | preliminary trials + agent importance | team selection before or during collaboration |
| MasRouter ( Yue et al., 2025 ) | runtime route | collaboration mode, role allocation, model routing | RL under quality–cost objective | routes across multi-agent designs rather than emitting one graph artifact |
Table 7: adjacent methods for selection, pruning, sparsification, and routing.
| Paper | Workflow abstraction | Main contribution | Why it matters in this survey |
|---|---|---|---|
| CAMEL ( Li et al., 2023a ) | role-conditioned multi-agent dialogue | early scalable role-playing template for autonomous cooperation | widely used baseline interaction scaffold |
| AutoGen ( Wu et al., 2023 ) | conversation programming over agents and tools | generic infrastructure for multi-agent conversations and tools | reusable scaffold later optimized by search or runtime control |
| EvoAgentX ( Wang et al., 2025d ) | layered framework for workflow generation, execution, and evolution | open-source platform integrating generation and evolutionary refinement | makes the workflow life cycle explicit in one system |
| Scaling Large-Language-Model-based Multi-Agent Collaboration ( Qian et al., 2025 ) | DAG-based collaboration network (MacNet) | studies how topology and scale affect performance | motivates topology as a meaningful design variable |
| OpenHands Software Agent SDK ( Wang et al., 2025b ) | software-agent execution loop with terminal and repository tools | open platform for SWE-style agents | important scaffold for workflow experiments in software settings |
| OneFlow ( Xu et al., 2026 ) | single-agent simulator over a role-conditioned graph | shows that many homogeneous multi-agent workflows can be simulated by one LLM | strong baseline against overclaiming multi-agent gains |
Table 8: background frameworks, scaffolds, and strong baselines that frequently anchor workflow experiments.
| Field | Meaning in this survey |
|---|---|
| Setting | static or dynamic, together with GDT (offline / pre-execution / in-execution) and GPM (none / select / generate / edit) |
| Optimized level | node / graph / joint |
| Representation | code / text / DSL / explicit graph / typed operator graph / constrained IR / orchestration state |
| Feedback / evidence | dominant workflow-relevant evidence used to accept, rank, or revise structures; encode as a coarse tag plus a short instantiation, e.g., metric, verifier, supervision, preference, trace, or proxy |
| Update mechanism | dominant structural update mechanism; encode as a coarse tag plus a short instantiation, e.g., search, generator, controller, supervised learning, RL, preference optimization, repair/edit, continuous relaxation, or hybrid |
| Cost handling | whether cost enters the method and how: none / eval-only / soft objective / hard constraint, optionally annotated with a resource type such as tokens, tool calls, latency, dollars, or budget |
Table 9: Field definitions and coding rules for the compact classification card used in the main-text tables.
More in this topic
LLM Agent Workflow Optimization
LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.
See all 256 papers