Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

Yuyuan Feng; Zhishang Xiang; Chaobin Yang; Qichao Ma; Zerui Chen; Yujing Zhang; Ke Huang; Chuanjie Wu

cs.IRAug 21, 2026

Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

Authors: Yuyuan Feng, Zhishang Xiang, Chaobin Yang, Qichao Ma, Zerui Chen, Yujing Zhang, Ke Huang, Chuanjie Wu, +27 more

Abstract

LLMs have evolved from language generators to autonomous agents capable of complex, long-horizon tasks. This evolution has produced paradigms including Prompt Engineering to elicit model capabilities, Context Engineering to manage information access, Harness Engineering to organize external tools and resources, and Loop Engineering to support continual reflection and self-improvement. Yet as tasks grow more complex, individual intelligence faces a fundamental limit: many tasks require heterogeneous expertise, interdependent subtasks, parallel execution, independent verification, and persistent state, exceeding any single agent's organizational capacity. Augmenting one agent's capabilities or context cannot resolve this architectural mismatch; intelligence must instead be distributed across specialized agents and organized at the system level. We call this System Intelligence: an agent system's ability to organize and coordinate multiple intelligent components into a coherent, adaptive whole pursuing a shared objective. Achieving it requires more than adding agents; it demands explicit structures to organize work, coordinate heterogeneous agents, and maintain evolving execution states. We introduce Graph Engineering, an emerging paradigm for next-generation agent systems. Unlike prior paradigms that mainly optimize individual interactions or agent-level behavior, Graph Engineering constructs explicit, dynamic, evolving graph structures representing tasks, agents, and system states. These abstractions provide a unified foundation for organizing complex objectives, orchestrating heterogeneous agents, modeling system dynamics, and enabling scalable agent evolution. We systematically review the principles, methodologies, and applications of Graph Engineering for LLM agents. Related papers, open-source data, and projects are collected at https://github.com/DEEP-JLU/Awesome-Graph-Engineering.

Figures & tables

Figure 1: Overview From Model Intelligence to System Intelligence. Prompt and Context Engineering elicit and condition the access of foundation models to realize model intelligence. Harness and Loop Engineering extend and orchestrate agentic capabilities to enable Individual Intelligence. Graph Engineering builds on the foundation of individual agents through Task Organization, Agent Coordination, and Runtime State Management, empowering System Intelligence.

Figure 3: From model intelligence to individual intelligence. Foundation-model capability is progressively transformed through task conditioning, persistent execution support, and feedback-controlled interaction into a capable, persistent, and goal-directed agent. The limitations of individual intelligence motivate the subsequent transition toward system intelligence.

Figure 4: An Illustrative Conceptualization of System Intelligence and Its Related Technologies. The exam analogy depicts the progression from Model, Prompt, and Context Engineering to tool-enabled Harness Engineering, feedback-driven Loop Engineering, team-oriented Graph Engineering, and ontology-based collaboration. The lower panel summarizes representative technologies associated with these layers.

Figure 5: Overview of Graph Engineering. Task Organization structures the objective into explicit subtasks and executable workflows; Agent Coordination matches capabilities to work, defines team topology, and routes communication among agents; Runtime State Management records execution states, detects and localizes anomalies, and supports recovery and structural updates. Together, these coupled graph views organize work, coordinate agents, and use runtime evidence to evolve the system toward reliable task completion.

Figure 6: Overview of Task Organization. Goal Decomposition translates a high-level objective into explicit subtasks, exposes their dependencies for scheduling, and refines the remaining task graph using intermediate execution results. Workflow Optimization compiles semantic subgoals into executable workflows, searches and optimizes alternative control flows, and adapts execution in response to runtime feedback. These mechanisms specify what work must be accomplished, how it should be operationalized, and how the work structure evolves during execution.

Figure 7: Overview of Agent Coordination. The Agent Capability Graph maps agents to their capabilities and accessible resources; the Agent Team Graph organizes agents into task-dependent collaboration structures; and the Communication Graph specifies and adapts information flow among agents. These graphs determine who should perform the work and how agents collaborate during execution.

Figure 8: Overview of Runtime State Management. Runtime State Management structures how to manage state by recording consistent and traceable runtime views, localizing failures from execution evidence, and recovering from validated states. These capabilities turn distributed execution histories into a reliable substrate for monitoring, diagnosis, recovery, and continual system evolution.

Name	Type	Primary Unit	Focus	Evaluation	Link
Model Intelligence
MMLU/MMLU-Pro [ 120 , 355 ]	B/D	QA instances	Knowledge	Broad knowledge and problem solving; MMLU-Pro increases reasoning difficulty and prompt robustness.
GPQA [ 293 ]	B/D	Expert science QA	Reasoning	Graduate-level scientific knowledge and difficult multi-step reasoning.
NPPC [ 396 ]	B/D	NP instances	Reasoning	Scalable, automatically verifiable reasoning over NP-complete problems.
OlymMATH [ 323 ]	B/D	Math problems	Reasoning	Olympiad-level reasoning with objective and formal verification.
IFEval [ 468 ]	B/D	Instruction pairs	Following	Verifiable instruction following under objectively checkable constraints.

Table 1: Representative benchmarks, datasets, and executable environments across Model, Individual, and System Intelligence. Type: B = benchmark or evaluation protocol; D = released dataset, annotations, or traces; E = executable or interactive environment. Focus denotes the principal capability or structural property evaluated.

Project/System	Focus	Abstraction	Engineering Paradigm	License	Link
Model Intelligence
Transformers [ 366 ]	Model interface	Unified model definitions, configurations, tokenizers, and generation APIs	Model loading, training, generation, multimodal models, and integration with downstream training and inference stacks	Python; Apache-2.0
Megatron Core [ 317 , 255 ]	Pretraining	Distributed transformer training building blocks	Tensor, pipeline, data, expert, and context parallelism; mixed precision and scalable distributed training	Python; Apache-2.0
LLaMA-Factory [ 462 ]	Post-training	Unified fine-tuning and post-training recipes	Continued pretraining, SFT, preference optimization, reward modeling, PPO, LoRA, and quantized fine-tuning	Python; Apache-2.0
verl [ 314 ]	RL post-train	Distributed RL post-training dataflow	PPO, GRPO and related algorithms; integration with FSDP/Megatron for training and vLLM/SGLang for rollout generation	Python; Apache-2.0
slime [ 477 ]	RL scaling	Training–rollout–data-buffer loop	Megatron training, SGLang rollout, custom rewards, verifiers, tool interaction, sandboxes, and asynchronous agentic data generation	Python; Apache-2.0

Table 2: Representative open-source projects and engineering systems across Model, Individual, and System Intelligence. Libraries are grouped by their primary engineering target rather than exclusive functionality. Focus summarizes the main engineering concerns exposed by each system.

System	Application Domain	Level	Focus	Structural role	Evidence or artifact	Link
Software Engineering and IT Operations
MetaGPT [ 122 ]	End-to-end software production	S	Work/Team	SOP-derived stages assign requirements, architecture, implementation, and review to specialized roles with structured intermediate artifacts	Collaborative software generation with executable projects and role-specific artifacts
SWE-agent [ 403 ]	Repository issue resolution	I	Work/State	Agent-computer interface constrains repository navigation, editing, commands, and test feedback within an iterative execution trajectory	Patch resolution on real repository issues with observable action and test trajectories
OpenHands [ 348 ]	General software development	I/S	Team/State	Event-stream runtime connects code, shell, browser, observations, and delegation while preserving execution history	Reproducible platform and evaluation across software-engineering tasks
Codex [ 258 ]	Production software engineering	I/S	Work/Team/State	Multiple coding agents execute parallel tasks in isolated worktrees with project threads, skills, review, and background automation	Deployed coding system supporting parallel long-running engineering work
Claude Code [ 11 ]	Repository-scale coding	I/S	Work/Team/State	Agent loop combines repository tools, checkpoints, hooks, background tasks, subagents, and parallel agent teams	Long-running coding workflows and demonstrated parallel agent-team software development

Table 3: Representative applications of System Intelligence. Level denotes the primary intelligence level: I = Individual Intelligence, S = System Intelligence, and I/S = systems spanning both. Focus summarizes the principal Graph Engineering concerns: Work Organization, Agent Team, Runtime State, and System Evolution.

Survey / study	Harness	Loop	Planning	Workflow	MAS	State	Self-Evolution	Ontology
LLM Agents’26 [ 170 ]	$\circ$	–	✓	–	✓	–	–	–
Graphs Meet Agents’25 [ 25 ]	$\circ$	–	✓	$\circ$	✓	–	–	–
Graph-Aug. Agents’25 [ 216 ]	$\circ$	–	✓	$\circ$	✓	–	$\circ$	–
Agent Intelligence + Graphs’26 [ 306 ]	$\circ$	$\circ$	$\circ$	$\circ$	✓	–	–	–
QA-to-Task Completion’26 [ 108 ]	✓	✓	$\circ$	✓	$\circ$	✓	✓	–
Agent Harness’26 [ 236 ]	✓	✓	✓	$\circ$	✓	✓	$\circ$	–

Table 4: Topic coverage of representative surveys related to LLM agents and agent systems.

Explore similar work

Sep 4, 2026cs.AI

Inference-Time Graph Engineering for Multi-Agent LLM Workflows

Recent multi-agent LLM systems increasingly rely on graph-structured communication to coordinate specialized agents. We revisit multi-agent orchestration from a graph-engineering perspective: rather than optimizing a static topology, we synthesize a task-conditioned temporal workflow graph that jointly specifies agent connectivity and edge-level communication semantics. We introduce ReActNet, a training-free framework that compiles a query and a set of role-specialized agents into a sequence of directed communication graphs. Each graph snapshot corresponds to one reasoning stage, and each edge carries a natural-language instruction specifying the message that a source agent should provide to a target agent. The compiled temporal graph is then executed through structured message passing: agents update their reasoning states by integrating their previous states with messages from controller-assigned neighbors, and a final aggregator synthesizes the resulting states into the answer. This design separates graph compilation from graph execution, making multi-agent coordination explicit, inspectable, and task-conditioned without requiring reinforcement learning or gradient-based topology optimization. Across knowledge reasoning, mathematical problem solving, code generation, and GAIA-style assistant tasks, ReActNet consistently improves over fixed-topology and learned-topology baselines while maintaining competitive inference cost. These results suggest that effective multi-agent orchestration depends not only on which agents communicate, but also on engineering executable workflow graphs that encode when, why, and how information should flow during reasoning.

Katherine Tieu, Dongqi Fu, Yinglong Xia +3

Meta · University of Illinois Urbana-Champaign · Work done at Meta

Multi-Agent LLM Systems Multi-Agent Orchestration Multi-Agent Communication Graph Generation

Ask PDF

Jul 26, 2026cs.AI

Focus Is All You Need: Adaptive Goal-aware Attention Orchestration for Multi-Agent Graph Systems

Large language models (LLMs) enable autonomous agents for reasoning, planning, and tool use. Recent systems increasingly organize these agents as graphs of specialized, interconnected nodes. Although graph-based orchestration supports flexible decomposition and coordination, it creates a key challenge: \textbf{attention allocation}. As workflows grow, existing approaches often execute graph components uniformly, wasting resources on irrelevant or low-impact tasks. We introduce \textbf{Attention Orchestration}, a paradigm that extends Transformer-style attention from token representations to workflow-level agent coordination. Our framework, \textbf{Adaptive Goal-aware Attention Orchestration (AGAO)}, dynamically estimates agent importance based on user objectives, graph dependencies, and computational constraints. AGAO combines three components: (1) goal-aware attention, measuring semantic relevance between user goals and agent capabilities; (2) topology-aware attention, modeling structural dependencies in agent graphs; and (3) resource-aware attention, allocating budgets and execution priorities across heterogeneous agents. Together, these mechanisms transform static agent graphs into adaptive systems that focus computation on goal-critical reasoning paths. Experiments across diverse multi-agent workloads show that AGAO improves task effectiveness while reducing unnecessary computation, latency, and token consumption compared with existing graph-based execution strategies. Our work establishes \textbf{Attention Engineering} as a direction for scalable, intelligent multi-agent systems. Code: https://github.com/MingzhouFan97/AGAO.

Mingzhou Fan, Siyuan Xu, Mingxuan Yuan

Multi-Agent System Optimization LLM Agent Orchestration LLM Agent Workflow Optimization Multi-Agent Systems Multi-Agent LLMs

Ask PDF

May 21, 2026cs.LG

GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving

Large Language Model (LLM)-based agents demonstrate strong reasoning and execution capabilities on complex tasks when guided by structured instructions, commonly referred to as workflows. However, existing workflow-assisted agent serving systems typically rely on predefined templates and shallow matching mechanisms, which limit their ability to capture deep semantic relationships and generalize to previously unseen tasks. To address these limitations, we propose a new workflow management paradigm that represents workflows using a unified graph, termed wGraph, where each node corresponds to an atomic operation. wGraph serves as a shared substrate from which task-specific workflows are dynamically instantiated. Building on wGraph primitives, we introduce GraphFlow, a system that efficiently integrates workflows into agent serving through two key designs. First, adaptive workflow generation dynamically constructs workflows from wGraph based on task semantics and constraint requirements. Second, workflow state management exploits wGraph structure to efficiently manage Key-Value (KV) caches, reducing redundant computation during agent serving. Extensive experiments across five benchmark datasets show that GraphFlow consistently outperforms state-of-the-art methods, yielding an average performance improvement of approximately 4.95 percentage points, while achieving an approximately 4

\times

reduction in memory footprint.

Ao Li, Shangpeng Yang, Fahao Chen +3

School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China · School of Artificial Intelligence, Shandong University, Jinan, China · 3Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China.

LLM Agent Orchestration Agentic Workflows LLM Agent Workflow Optimization LLM Agent Serving

Ask PDF