From Agent Loops to Structured Graphs:A Scheduler-Theoretic Framework for LLM Agent Execution

Hu Wei

cs.AIApr 13, 2026

From Agent Loops to Structured Graphs:A Scheduler-Theoretic Framework for LLM Agent Execution

Authors: Hu Wei

Abstract

The dominant paradigm for building LLM based agents is the Agent Loop, an iterative cycle where a single language model decides what to do next by reading an ever growing context window. This paradigm has three structural weaknesses: implicit dependencies between steps, unbounded recovery loops, and mutable execution history that complicates debugging. We characterize the Agent Loop as a single ready unit scheduler: at any moment, at most one executable unit is active, and the choice of which unit to activate comes from opaque LLM inference rather than an inspectable policy. This perspective places Agent Loops and graph based execution engines on a single semantic continuum. We propose SGH, Structured Graph Harness, which lifts control flow from implicit context into an explicit static DAG. SGH makes three commitments: execution plans are immutable within a plan version, planning execution and recovery are separated into three layers, and recovery follows a strict escalation protocol. These choices trade some expressiveness for controllability, verifiability, and implementability. Our contributions are fourfold: a scheduler unified framework that applies classical scheduling theory to LLM agent execution and identifies challenges introduced by non deterministic LLM nodes; a trade off analysis of controllability, expressiveness, and implementability across 70 surveyed systems; a formal specification including a node state machine with termination and soundness guarantees; and an attributable experimental framework with a seven group design for future validation. This is a position paper and design proposal. We provide a theoretical framework, design analysis, and experimental protocol, not a production implementation or empirical results.

Figures & tables

Symbol	Definition
$\mathcal{E}=(\mathcal{S},\mathcal{U},\mathcal{P},\mathcal{O},\Delta)$	Execution system tuple (Def. 3.1 )
$\mathcal{U}(s)\subseteq\mathcal{A}$	Ready set at state $s$
$\|\mathcal{U}(s)\|$	Ready-set cardinality
$\Pi=(\mathit{id},\mathit{version},V,E,\sigma,\kappa)$	Execution plan (Def. 5.1 )
$\sigma:V\to\text{NodeConfig}$	Node configuration mapping
$\kappa$	Plan-level output contract

Table 1: Summary of notation used throughout this paper.

Panel of Figure 1 — Figure 1: Motivating example: a bug-fix task as a DAG. Blue nodes form the first parallel wave; green nodes form the second. The any_of join selects one patch; the other is skipped. Note: This example illustrates SGH’s potential benefits under ideal conditions: (1) the planner correctly identifies all parallel structure, (2) dependencies are fully known at planning time, and (3) the LLM executes each node correctly. Real-world tasks may not exhibit such clean parallelism, and the planner’s ability to recognize independent operations is a prerequisite for these benefits. See Section 3.7 for discussion of what happens when the planner fails.

Dimension	LangGraph	Graph Harness
Plan mutability	Mutable at runtime (add/remove nodes and edges)	Immutable per plan version
Recovery	Retry + conditional routing	Three-level escalation protocol
Routing policy	LLM-driven conditional edges	Deterministic topology-based
Failure attribution	State-channel snapshots	Plan-version-audited traces
Expressiveness target	Maximum flexibility	Maximum controllability
Production maturity	Widely deployed; community-validated	Design only; no empirical validation

Table 2: Graph Harness vs. LangGraph.

Classical concept	LLM agent challenge	Graph Harness mechanism
Deterministic node output	Same input may yield different outputs; hallucination	Contract validation + validation gap analysis
Infrastructure failures	Reasoning errors, hallucinations, misinterpretation	Three-level recovery distinguishing error types
Scheduling = optimization	Scheduling involves side-effect safety	Side-effect classification (Principle 4)
Context = task parameters	Context includes reasoning history that may corrupt subsequent steps	Execution/diagnostic context separation
Retry = idempotent	Retry may produce different results; non-idempotent side effects	Bounded retry + side-effect-aware scheduling

Table 3: Classical DAG scheduling vs. LLM agent scheduling: why the analogy is not trivial.

Dimension	Airflow / Luigi / Prefect	Graph Harness
Node behavior	Deterministic (Python code)	Non-deterministic (LLM inference)
Node output	Predictable, type-checked	Unpredictable, contract-validated
Failure causes	Primarily infrastructural	Primarily reasoning errors, hallucinations
Recovery	Manual / simple retry	Three-level escalation protocol
Context management	Task-scoped variables	Execution/diagnostic context separation
Plan evolution	Redeploy new DAG version	Plan-version immutability within execution

Table 4: Graph Harness vs. classical workflow engines.

System	Multi-	Det.	Bounded	Immut.	Recovery	Contract	Target
	Ready	Policy	Recovery	Plan	Protocol	Valid.	Domain
Agent Loop ( Yao et al., 2023 )	No	No	No	No	Ad-hoc LLM	None	General
AutoGPT	No	No	No	No	Ad-hoc LLM	None	General
CrewAI	No	No	No	No	Ad-hoc LLM	None	General
Plan-and-Act ( Erdogan and others, 2025 )	No	No	No	Partial	Re-plan	Optional	Web nav
TDP ( Li et al., 2026 )	No	No	No	Mutable	Node-local	Optional	General

Table 5: Systematic comparison of Graph Harness with related LLM agent and workflow systems.

Approach	Execution commitment	Recovery	Scheduler type	LLM-aware?
Agent Loop ( Yao et al., 2023 )	None	Unbounded LLM decision	Single-ready-unit	Implicit
Plan-and-Act ( Erdogan and others, 2025 )	Plan as prompt	Re-plan	Single-ready-unit	Partial
TDP ( Li et al., 2026 )	Mutable DAG sub-tasks	Node-local replan	Single-ready-unit	Partial
LangGraph ( Team, 2024 )	Mutable runtime graph	Retry + conditional routing	Multi-ready-unit	Partial
GPTSwarm ( Zhuge et al., 2024 )	Optimizable graph	Edge reconnection	Multi-ready-unit	No
DynTaskMAS ( Yu et al., 2025 )	Dynamic DAG	Dynamic reschedule	Multi-ready-unit	No

Table 6: Positioning Graph Harness relative to existing approaches.

Panel of Figure 2 — Figure 2: The scheduler continuum (extended). The new “Parallel Loop” category represents Agent Loops with parallel tool calls (e.g., OpenAI’s tool_calls field). These achieve $|\mathcal{U}|\geq 1$ but with a non-deterministic policy, distinguishing them from Graph Harness ’s explicit, topology-based scheduling.

Category	Expressiveness	Controllability	Implementability
Agent Loop (60%)	Low	Low	High
Event-driven (15%)	Low	Medium	High
State-machine (10%)	Medium	High	Medium
Graph/flow orchestration (5%)	High	Low	Low
Hybrid (10%)	Medium	Medium	Medium

Table 7: Observed trade-offs across 70 surveyed agent systems.Categories are mutually exclusive based on the primary execution pattern observed. Percentages are approximate; sensitivity analysis within

\pm

10% does not change the ordinal ranking (Agent Loop is most common, Graph/flow is least common). See Section A.5 for methodology.

Principle	Sacrificed	Gained
Controllability first	Competitive parallelism, recursive expansion, parent-chain rollback	Predictable execution, verifiable traces
Stable commitment	Dynamic plan modification	Auditable plan history, reliable attribution
Bounded recovery	Ad-hoc LLM-driven recovery	Deterministic escalation, failure-loop prevention
Side-effect classification	Unrestricted parallel dispatch	Safety guarantees, risk-bounded scheduling

Table 8: Design principles: what is sacrificed and what is gained.

Topology	Expressiveness	Predictability	Verifiability
Static DAG (chosen)	Medium	High	High
Recursive expansion graph	High	Low	Low
Parent-chain rollback graph	High	Low	Low
Dynamic topology graph	High	Low	Low

Table 9: Graph topology alternatives and their trade-offs.

Panel of Figure 3 — Figure 3: Three-layer separation: planning, execution, and recovery.

Panel of Figure 4 — Figure 4: Node state machine. Terminal states (gray) are stable: once entered, they are never left. Transitions: running $\to$ failed_retryable occurs on transient errors (network timeout, rate limit) that may succeed on retry; running $\to$ failed occurs on structural errors (missing dependencies, invalid plan) that cannot be fixed by retrying the same node.

Level	Trigger	Scope
local_retry	Transient errors (network, timeout)	Current node only; plan structure unchanged
local_patch	Contract violation, auth error	Current node’s configuration; plan structure unchanged
request_replan	Missing dependency, invalid plan structure	Entire plan version; new $(V^{\prime},E^{\prime})$ generated

Table 10: Recovery levels, triggers, and scope.

Capability	Graph Harness	Full dynamic graph
Sequential execution	✓	✓
Parallel execution	✓	✓
Alternative paths	✓	✓
Competitive parallelism	—	✓
Recursive sub-graph expansion	—	✓
Dynamic topology change	—	✓

Table 11: Expressiveness boundary of Graph Harness .

Group	Scheduler type	Structure	Recovery
G0: SOTA Loop	Single-ready-unit	Planner prompt + reflection	Inline replan
G1: Naive Loop	Single-ready-unit	None	Context continuation
G2: Planner Loop	Single-ready-unit	None	Context + replan
G3: Structured Loop	Single-ready-unit	Scaffold	Scaffold recovery
G4: GH-Core	Multi-ready-unit	Static DAG	Retry only
G5: GH+Patch	Multi-ready-unit	Static DAG	Retry + local patch

Table 12: Seven-group experimental design. Each row adds exactly one feature to the previous row.

Tier	Characteristics	Expected result
Simple (1–3 steps)	Linear chain; no branching dependencies; all_of joins only	SGH may show fixed overhead (no parallelism to exploit)
Medium (4–8 steps)	2–3 branches; both all_of and any_of joins; recovery opportunities	Primary evaluation tier (parallelism and recovery benefits emerge)
Complex (9+ steps)	Nested branches; deep dependency chains; multiple recovery paths	SGH advantage should widen (structure provides maximal value)

Table 13: Task tier design and expected behavior.

Excluded capability	Reason	Task impact
first_of competitive parallelism	Non-deterministic ready set; requires compensation protocol	Cannot express “try all, take first”
Recursive sub-graph expansion	Unbounded $\|V\|$ ; unpredictable ready set	Cannot express dynamic decomposition
Parent-chain rollback	Plan invariant violation	Cannot express deep hierarchical recovery
Dynamic topology modification	Plan invariant violation	Cannot adapt structure mid-execution

Table 14: Excluded capabilities and their rationale.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

From	To	Trigger	Condition
pending	ready	Dependencies satisfied	$\forall p\in\mathit{dep}(v):\sigma(p)\in\Sigma_{\mathit{term}}^{+}$
ready	running	Scheduler dispatch	Selected by $\mathcal{P}$
ready	blocked	Dependency lost	A predecessor left terminal-success state
ready	skipped	Sibling completed	any_of join; another candidate executed
running	executed	Action success	Output contract $\kappa_{v}$ satisfied
running	failed_retryable	Transient error	Network, timeout; retry budget $>0$

Appendix

Table 15: Complete state transition table.

Project	Category	Key innovation	Rating
ironclaw	Agent Loop	LoopDelegate trait, three-phase tool execution	5.0
openclaw	Agent Loop	Three-tier degradation, Auth Profile rotation	4.5
clawdroid	Event-driven	OODA loop, MessageBus architecture	4.85
openhands	Event-driven	EventStream controller, sandboxed execution	4.2
vtcode	State-machine	Four-layer Phase FSM, compile-time exhaustiveness	5.0
codex	State-machine	SQ/EQ dual-queue protocol	5.0

Appendix

Table 16: Nine representative projects from the 70-project survey. Full project list available in supplementary material.

LLM Agent Orchestration

Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.

See all 446 papers

Jul 15, 2026

MyAG: A Graph-Based Framework for Designing and Analyzing Composable LLM Agent Systems

We present MyAG, a graph-based framework for designing and analyzing composable LLM agent systems. Our framework separates agent system construction into three graph abstractions: a component graph for agents, environments, and modules; a workflow graph for execution control; and a search graph for runtime execution. This separation allows users to flexibly reuse the same components with different strategies. We further support hierarchical composition through recursive system nodes and provide monitoring and visualization tools for inspecting agent execution. Experiments on representative agent applications show that our framework supports flexible agent system design and helps analyze performance-efficiency tradeoffs. Our framework is publicly available and fully open-source.

Zhisong Zhang

Department of Computer Science, City University of Hong Kong

LLM Agent Orchestration LLM Agents LLM Agent Harnesses

Ask PDF

Jun 14, 2026

LLM-as-Code: Agentic Programming for Agent Harness

Every major LLM agent framework gives the LLM the role of orchestrator; the model decides what to do next, when to call tools, and when to stop. We argue that token explosion, control-flow hallucination, and unreliable completion are not implementation bugs but architectural consequences of assigning the deterministic work of looping, branching, and sequencing to a probabilistic system. A better prompt or a stronger model cannot guarantee the reliability of the LLM agent. We therefore propose Agentic Programming, in which the program governs all control flow, and the LLM is itself part of it, an adaptive component we call LLM-as-Code and invoke only where a task calls for reasoning or generation. Within each call the model keeps full flexibility, but it cannot alter the program's execution path. With control in the program, the LLM's context is built from the execution history's call tree and forms a directed acyclic graph (DAG). Each call's context length is then determined by its call depth rather than by accumulation over steps. A case study of computer-use agents shows that the design is practical, not just a theoretical stance, substantially improving the stability of long visual operation sequences.

Junjia Qi, Zichuan Fu, Jingtong Gao +4

City University of Hong Kong · Tencent Jarvis Lab

AI Agent Reliability LLM Agent Orchestration Long-Horizon LLM Agents LLM Agent Reliability

Ask PDF

Jun 22, 2026

GRADE: Graph Representation of LLM Agent Dependency and Execution

Traces record what LLM agents did, leaving implicit what each step relied on. GRADE holds both in one graph; six published graph views are its projections. Execution edges come free; each costly dependency edge is graded as observed in the trace, declared by added logging, or inferred under an assumption, a schema we test one grade at a time. We recover the dependency layer from six tool-use, coding, and web corpora and measure its failure-prediction gain over run size. First, over a linear run-size baseline, it improves prediction on three corpora, but the gain depends on the learner and task sampling. On held-out corpora the layer stays above chance where run size does not, under one learner, with the same task-sampling caveat. Second, structure can pass for signal it does not carry: a layer inferred under full history restates run size. Within a corpus, a preregistered test does not separate the recovered layer from a degree-matched counterfeit that rewires each step's dependencies but keeps their number. The recovered layer's largest transfer lead over it reverses when the held-out corpus's task family is withheld. Before crediting structure, grade each edge, keep a run-size baseline, and score degree-matched attachment controls.

Yue Zhao

University of Southern California

LLM Agent Evaluation LLM Agent Reliability

Ask PDF

Abstract

LLM Agent Orchestration

Explore similar work

MyAG: A Graph-Based Framework for Designing and Analyzing Composable LLM Agent Systems

LLM-as-Code: Agentic Programming for Agent Harness

GRADE: Graph Representation of LLM Agent Dependency and Execution