Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures

Benjamin Rombaut

cs.SEApr 3, 2026

Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures

Authors: Benjamin Rombaut

Abstract

LLM-based coding agents can localize bugs, generate patches, and run tests with diminishing human oversight, yet the scaffolding code that surrounds the language model (the control loop, tool definitions, state management, and context strategy) remains poorly understood. Existing surveys classify agents by abstract capabilities (tool use, planning, reflection) that cannot distinguish between architecturally distinct systems, and trajectory studies observe what agents do without examining the scaffold code that determines why. This paper presents a source-code-level architectural taxonomy derived from analysis of 13 open-source coding agent scaffolds at pinned commit hashes. Each agent is characterized across 12 dimensions organized into three layers: control architecture, tool and environment interface, and resource management. The analysis reveals that scaffold architectures resist discrete classification: control strategies range from fixed pipelines to Monte Carlo Tree Search, tool counts range from 0 to 37, and context compaction spans seven distinct strategies. Five loop primitives (ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search) function as composable building blocks that agents layer in different combinations; 11 of 13 agents compose multiple primitives rather than relying on a single control structure. Dimensions converge where external constraints dominate (tool capability categories, edit formats, execution isolation) and diverge where open design questions remain (context compaction, state management, multi-model routing). All taxonomic claims are grounded in file paths and line numbers, providing a reusable reference for researchers studying agent behavior and practitioners designing new scaffolds.

Figures & tables

Agent	Category	Language	Stars	Origin
OpenCode ( Anomaly, 2025 )	CLI	TypeScript	135k	SST
Gemini CLI ( Google, 2025 )	CLI	TypeScript	100k	Google
Codex CLI ( OpenAI, 2025 )	CLI	Rust / TS	72k	OpenAI
OpenHands ( Wang et al., 2025 )	SWE-bench	Python	70k	All Hands AI
Cline ( Rizwan, 2024 )	CLI	TypeScript	60k	Independent
Aider ( Gauthier, 2024 )	CLI	Python	43k	Independent

Table 1: Agents analyzed, ordered by GitHub stars as a proxy for community adoption ( Borges and Valente, 2018 ) . “Category” distinguishes interactive CLI tools from agents designed for automated SWE-bench evaluation. All agents were analyzed at pinned commit hashes (see Appendix B ).

Figure 1: Taxonomy overview: 12 dimensions organized into three architectural layers. Each dimension is a continuous spectrum with observed endpoints from the corpus of 13 agents. Agents occupy positions along these spectra rather than falling into discrete categories (Section 5.1 ).

Position	Agents	Mechanism
Fixed pipeline	Agentless	10-stage pipeline of independent scripts connected by JSONL files on disk. No feedback loop between stages.
User-driven loop	Aider	Outer loop is user-initiated (each edit cycle requires user input). Inner generate-test-repair loop is autonomous for up to max_reflections iterations.
Sequential ReAct loop	SWE-agent , OpenHands , Codex CLI , Gemini CLI , mini-swe-agent , Cline , OpenCode	Standard thought–tool–observation cycle. LLM selects the next action; loop terminates on completion signal or budget exhaustion.
Phased loop	AutoCodeRover , Prometheus	Distinct stages with different tool access. AutoCodeRover : search-then-patch phase separation. Prometheus : LangGraph ( LangChain, 2024 ) state machine with explicit edges.
Depth-first tree search	DARS-Agent	ReAct steps form nodes in a search tree. At branch points, the environment is reset and replayed from root; an LLM critic selects among sampled alternatives.
Full MCTS	Moatless Tools	Select–Expand–Simulate–Backpropagate with reward values ( $-100$ to $+100$ ) and visit counts. Pluggable selector interface; a discriminator selects the best finished trajectory.

Table 2: Control loop strategies. Agents ordered from least to most flexible exploration strategy.

Driver	Agents
User-driven	Aider : the LLM has 0 callable tools. The user selects files ( /add ), runs searches, and provides context. The LLM produces edits in a text format parsed by the scaffold ( base_coder.py:2296--2304 ).
Scaffold-driven	Agentless , AutoCodeRover : the scaffold controls sequencing and calls the LLM at fixed points. In Agentless , each LLM call is single-turn with no conversation state. In AutoCodeRover , the scaffold manages phase transitions across up to four stages: optional reproducer generation, optional SBFL fault localization, search, and patch generation.
LLM-driven	SWE-agent , OpenHands , Codex CLI , Gemini CLI , Cline , mini-swe-agent , Moatless Tools , DARS-Agent , OpenCode : the LLM selects tools and controls exploration. Prometheus is a hybrid: within each graph node, the LLM drives tool selection via ReAct, but transitions between nodes are scaffold-controlled (conditional edges on state fields such as state["reproduced_bug"] ).

Table 3: Loop driver strategies. Who decides what happens next?

Implementation	Agents
Imperative while loop	SWE-agent , OpenHands , Codex CLI , Gemini CLI , mini-swe-agent , Aider , OpenCode
Fixed pipeline (no loop)	Agentless : sequential scripts; each LLM call is single-turn with no agent loop. The Anthropic path contains a bounded for loop (up to 10 iterations, model.py:148--284 ), but the primary architecture has no loop.
Recursion	Cline : recursivelyMakeClineRequests ( task/index.ts:2268 ). The call stack grows linearly with conversation length.
Graph-as-control-flow	Prometheus : LangGraph compiled state machine with explicit edges ( issue_graph.py:22--134 ). Cycles in the graph create loops; recursion limits (30 for IssueBugSubgraph , 150 for BugReproductionSubgraph ; dynamic formulas scale to hundreds of steps for multi-candidate runs) serve as the termination guarantee.
Exception-based signaling	mini-swe-agent : InterruptAgentFlow hierarchy ( Submitted , LimitsExceeded , FormatError ) carries messages as payloads. The run() method catches these, injects messages into history, and either continues or breaks ( default.py:88--96 ).

Table 4: Control flow implementation variants. This table classifies agents by code-level mechanism, which is orthogonal to the semantic loop strategy in Table 2 : an agent classified as “user-driven” by loop strategy (e.g., Aider ) may still use an imperative while loop at the implementation level.

Agent	Tools	Read	Search	Edit	Execute	Validate	Notes
Aider	0			✓ ∗			∗ Text-parsed edits, 13 formats
Agentless	0–1			✓ ∗			∗ 1 simulated tool in Anthropic path
AutoCodeRover	8	✓	✓				All search/read; no edit tools
OpenHands	9+	✓		✓	✓		+MCP; search via bash
Moatless Tools	15–37	✓	✓	✓	✓	✓	37 classes; ${\sim}$ 15 typical per session
SWE-agent	3–35	✓	✓	✓	✓		3 default; 35 across 15 bundles

Table 5: Tool set size and capability coverage. Tool count reflects LLM-callable tools; user-facing commands (e.g., Aider ’s

{\sim}

38 slash commands) are excluded.

Format	Agents
String replacement (function calling)	OpenHands and SWE-agent ( str_replace_editor ), Codex CLI ( apply_patch ), OpenCode ( edit )
Write tool (function calling)	Gemini CLI , Cline
Text-parsed edit blocks	Aider (13 formats), DARS-Agent
Simulated tool use	Agentless (Anthropic path)
Pydantic-schema actions	Moatless Tools , Prometheus

Table 6: Edit/patch format variants.

Strategy	Agents
Static (all at init)	Aider , Agentless , AutoCodeRover , mini-swe-agent , DARS-Agent , Moatless Tools
Per-phase scoping (static per node)	Prometheus , AutoCodeRover
Config-conditional	SWE-agent , OpenHands
Per-turn dynamic rebuild	Codex CLI ( built_tools() called every sampling request, codex.rs:6156--6164 )
Dynamic (MCP + subprocess)	Gemini CLI (tool discovery subprocess, tool-registry.ts:312--439 ), Cline (MCP servers connected/disconnected at runtime), OpenCode (static core tools + dynamic custom tools from config dirs, plugin system, and MCP integration)

Table 7: Tool discovery strategies.

Strategy	Agents	Mechanism
Keyword/regex search	SWE-agent , OpenHands , Codex CLI , Gemini CLI , Cline , mini-swe-agent , DARS-Agent , OpenCode	LLM invokes grep , find , ripgrep as tools.
Repo map (static analysis)	Aider	PageRank-weighted tree-sitter tag index. Builds a NetworkX dependency graph; identifiers mentioned in conversation get a 10x boost; chat files get a 50x boost ( repomap.py:365--574 ).
AST-aware search	AutoCodeRover , Moatless Tools , DARS-Agent , Prometheus	search_class , search_method : structure-aware queries over pre-built AST indices. AutoCodeRover ’s AST tools are Python-only; Prometheus ’s tree-sitter-based tools cover 20 languages.
Knowledge graph traversal	Prometheus	Neo4j graph built from tree-sitter ASTs covering 20 languages. 11 tools (10 graph traversal plus read_file ) query FileNode , ASTNode , TextNode entities ( graph_traversal.py:93--586 ).
Embedding-based semantic search	Moatless Tools	FAISS ( Johnson et al., 2021 ) vector store via LlamaIndex ( LlamaIndex, 2022 ) ( code_index.py:57 ). The only agent with embedding-based retrieval as an LLM-callable tool.
Hierarchical localization	Agentless	File $\to$ class/function $\to$ line-level narrowing across pipeline stages. Each level sees only what the previous level identified ( FL.py:313--681 ).

Table 8: Context retrieval strategies. Agents may use multiple strategies.

Isolation level	Agents
None (local shell)	Gemini CLI , Aider , OpenCode
Stateless subshells	mini-swe-agent ( subprocess.run() , local.py:28--39 )
Platform sandboxing	Codex CLI (Bubblewrap+Landlock on Linux, Seatbelt on macOS)
Docker container	SWE-agent , OpenHands , DARS-Agent , AutoCodeRover , Prometheus
Shadow git checkpoints	Cline (isolated git repo tracks state per tool execution, CheckpointTracker.ts )
Shadow mode (in-memory)	Moatless Tools ( shadow_mode flag, agent.py:57 )

Table 9: Execution isolation strategies.

Strategy	Agents
Destructive	Aider : summarization overwrites done_messages ( base_coder.py:1024--1034 )
Flat list, preserved	SWE-agent , Codex CLI , Gemini CLI : raw history kept; filtered views created for LLM. mini-swe-agent : raw history kept with no filtering (messages sent to the LLM as-is)
Typed event log	OpenCode : SQLite-backed message/part hierarchy with 12 part types (Drizzle ORM). Append-only messages with mutable part states ( message-v2.ts )
Graph-scoped	Prometheus : separate message lists per graph node ( edit_messages , analyzer_messages , context_provider_messages ), reset at retry boundaries
Tree-structured (MCTS)	Moatless Tools : nodes in a tree with per-node file context snapshots, visit counts, and reward values
Tree-structured (greedy)	DARS-Agent : nodes in a tree with per-node expansion candidates and critic responses ( dars_agent.py:294--297 ). No MCTS statistics; branching uses greedy LLM critic selection, and state is recovered via Docker reset and action replay rather than file context snapshots.

Table 10: State management strategies.

Strategy	Agents	Mechanism
None (crash on overflow)	mini-swe-agent	Unbounded growth; agent crashes on ContextWindowExceededError .
Rule-based truncation	SWE-agent , DARS-Agent	SWE-agent : 7 composable processors; keep first + last $N$ observations, elide rest. Polling parameter for prompt-cache preservation. DARS-Agent : adds Last5Observations in its fork.
Structural isolation	Prometheus , AutoCodeRover	Prometheus : per-node message scoping + resets at retry boundaries. AutoCodeRover : round limits (15) + result truncation.
Token-based selective inclusion	Moatless Tools	Greedy recent-first selection within token budget; per-observation summary fallback.
LLM summarization (scaffold-triggered)	Aider , OpenHands , Gemini CLI , Codex CLI , OpenCode	Automatic at token threshold. Aider : recursive hierarchical summarization. Codex CLI : pre-turn and mid-turn modes. OpenCode : two-phase (prune old outputs, then LLM summarization).
LLM summarization + verification	Gemini CLI	Summarize, then “Probe” verification turn to check for information loss.

Table 11: Context compaction strategies.

Strategy	Agents	Mechanism
Single model	mini-swe-agent , Agentless (per path)	One model throughout.
Router abstraction (default single)	OpenHands	RouterLLM base with MultimodalRouter : routes by image presence and token limits ( llm/router/rule_based/impl.py:16--81 ). Default: single model via noop_router .
Role-based	Aider , OpenCode	Main/weak/editor for different subtasks. Aider : weak model for summarization and commit messages ( models.py:607--608 ); editor model for architect mode ( architect_coder.py:22--25 ). OpenCode : per-agent model overrides for build, plan, explore, and compaction agents ( agent/agent.ts:78--233 ).
Plan/Act mode-based	Cline	Independent model per mode ( api/index.ts:76--149 ). LLM switches modes via tool calls.
Per-attempt cycling	SWE-agent , AutoCodeRover	Different models for retry attempts. SWE-agent : round-robin through agent_configs ( agents.py:303--319 ). AutoCodeRover : round-robin through model_names ( inference.py:98--114 ).
Safety-focused (Guardian)	Codex CLI	Separate model ( gpt-5.4 ) evaluates tool call risk ( guardian.rs ).

Table 12: Multi-model routing strategies.

Strategy	Agents	Mechanism
None	SWE-agent , OpenHands , AutoCodeRover , mini-swe-agent , DARS-Agent	No cross-session persistence. Each run starts fresh. Trajectory files are output artifacts, not consumed by future runs.
Pipeline resumability	Agentless	No cross-task learning, but JSONL outputs enable pipeline resumability ( --skip_existing ), embedding indices persist via --persist_dir , and pre-computed repo structures are cached via PROJECT_FILE_LOC . These are consumed by future runs of the same pipeline, not cross-task knowledge.
Config file loading	Aider ( .aider.conf.yml )	Static, user-written configuration. Tags cache ( repomap.py:43, 217--265 ) persists AST analysis across sessions as a performance optimization.
LLM-writable rules/memory	Cline ( .clinerules/ ), Gemini CLI ( GEMINI.md )	The LLM actively writes persistent instructions. Cline : new_rule tool ( tools.ts:31 ). Gemini CLI : save_memory tool appends under “Gemini Added Memories” in GEMINI.md ( memoryTool.ts ).
Full session persistence	OpenCode	SQLite-backed session history with resume capability. All messages, tool outputs, token usage, and costs persist ( session/session.sql.ts:14--76 ).
Background extraction pipeline	Codex CLI	Two-phase: Phase 1 extracts memories from recent rollouts (parallel, gpt-5.1-codex-mini ); Phase 2 consolidates via sub-agent ( gpt-5.3-codex ). Usage-ranked, stale memories pruned ( memories/README.md ).

Table 13: Persistent memory strategies.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Agent	Disposition	Exclusion rationale
Gemini CLI	Included	—
OpenHands	Included	—
Aider	Included	—
Cline	Included	—
SWE-agent	Included	—
Codex CLI	Included	—

Appendix

Table 14: Full candidate pool. Agents are grouped by disposition: the 13 included agents appear first (ordered as in Table 1 ), followed by the 9 excluded agents grouped by exclusion criterion.

Agent	Commit hash	Repository URL
Gemini CLI	dd8d4c98b3	https://github.com/google-gemini/gemini-cli
OpenHands	922e3a2431	https://github.com/OpenHands/OpenHands
Aider	861a1e4d15	https://github.com/Aider-AI/aider
Cline	71e312e92a	https://github.com/cline/cline
SWE-agent	e72a7e4660	https://github.com/SWE-agent/SWE-agent
Codex CLI	9dba7337f2	https://github.com/openai/codex

Appendix

Table 15: Pinned commit hashes for all 13 analyzed agents. Commits are listed in the same order as Table 1 .

Explore similar work

Jul 4, 2026

Don't Blame the Large Language Model: How Scaffolding Evolution Shapes Coding Agent Quality

Coding agents, autonomous systems that use large language models (LLMs) to resolve software engineering tasks, rely on agentic scaffolding: a middleware layer in between a developer and a large language model that orchestrates system prompts, tool execution, context management, and iterative reasoning loops. While these scaffoldings evolve at extreme velocities, no study has examined how this evolution affects agent quality (i.e., effectiveness and efficiency) over time. Practitioners regularly report quality regressions after scaffolding updates, yet consistently attribute them to the underlying model rather than the scaffolding itself. In this paper, we address this gap by conducting the first controlled longitudinal study that isolates the scaffolding's contribution. Unlike prior work that fixes the scaffolding and varies the model, we fix the model and vary only the scaffolding, evaluating 35 sequential releases to measure their impact on agent effectiveness and efficiency. We first empirically study the development and release evolution of five major open-source scaffoldings (i.e., Codex, Qwen Code, Gemini, OpenCode, and OpenHands), revealing extreme release velocities exceeding two releases per day and thousands of issues within months. We then perform a controlled deep dive into 35 sequential releases of the Qwen Code CLI, evaluating each against 50 stratified SWE-bench Verified tasks while holding the underlying LLM constant. We trace the resulting quality fluctuations to specific development patterns and architectural components, and illustrate our findings with concrete qualitative evidence linking individual pull requests to measured quality shifts.

Oussama Ben Sghaier, Hao Li, Bram Adams +1

Queen’s University, Canada

Software Engineering Agents LLM Agent Orchestration LLM Agent Evaluation

Ask PDF

Feb 28, 2026

Theory of Code Space: Do Code Agents Understand Software Architecture?

AI code agents excel at isolated tasks yet struggle with multi-file software engineering requiring architectural understanding. We introduce Theory of Code Space (ToCS), a benchmark that evaluates whether agents can construct, maintain, and update coherent architectural beliefs during codebase exploration. Agents explore procedurally generated codebases under partial observability -- opening files under a budget -- and periodically externalize their belief state as structured JSON, producing a time-series of architectural understanding. Three findings emerge from experiments with four baselines and six frontier LLMs. First, the Active-Passive Gap is model-dependent: one model builds better maps through active exploration than from seeing all files at once, while another shows the opposite -- revealing that active exploration is itself a non-trivial capability absent from some models. Second, retaining structured belief maps in context acts as self-scaffolding for some models but not others, showing that the mechanism is model-dependent. Third, belief state maintenance varies dramatically: a smaller model maintains perfectly stable beliefs across probes while its larger sibling suffers catastrophic belief collapse -- forgetting previously-discovered components between probes. We release ToCS as open-source software. Code: https://github.com/che-shr-cat/tocs

Grigory Sapunov

Intento

Software Engineering AI Agent Benchmarks

Ask PDF

Oct 7, 2026

Code Understanding is a Bottleneck for Coding Agents

Repository benchmarks (e.g., SWE-bench) for coding agents often assume that lines of code edited can predict task difficulty, but such datasets' poor control over code and task types makes it hard to know which abilities truly drive agent errors. We present CABRA: a Coding Ability Blueprint for Rigorous Agent evaluation. CABRA builds tasks from scratch as call graph transformations and scales difficulty via a task size parameter on four axes: function traversal, search, runtime resolution, and instruction following. We run eight LLMs and six coding agents on 6,840 CABRA tasks to show: 1) LLM accuracy falls as task size~grows, but agents stay near-perfect by offloading work to tools (e.g., grep); 2) Larger CABRA tasks elicit more tool calls for reading and analysis, while a separate study on SWE-bench Verified shows these tool call counts predict agents' accuracy better than lines of code edited, suggesting task difficulty for agents can lie in understanding code to edit, not just in making edits; 3) Extending CABRA to an intense understanding task where models analyze divergent logic across two classes backs this finding, as agent accuracy finally falls. More broadly, we argue for synthetic evaluations like CABRA to unmask LLM weaknesses trivialized by tools (e.g., needle-in-a-haystack) and abilities beyond just editing (e.g., understanding) that coding agents still find difficult, pairing SWE-bench-style tasks with controlled diagnosis.

Nishant Balepur, Kiran Tomlinson, Tobias Schnabel

Microsoft Research · University of Maryland

Coding Agents LLM Agent Evaluation AI Agent Benchmarks Tool-Using Agents

Ask PDF