Procedural Memory for Language Agents: A Neurosymbolic Perspective on Explicit Goal Structure

Jonathan Leung; Zhiqi Shen; Deheng Ye

cs.AIOct 19, 2026

Procedural Memory for Language Agents: A Neurosymbolic Perspective on Explicit Goal Structure

Authors: Jonathan Leung, Zhiqi Shen, Deheng Ye

Organizations: Tencent-NTU Joint Research Laboratory (CENTURY), Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract

Large language models remain unreliable on long-horizon tasks requiring procedural reasoning. We argue that a key cause is the absence of explicit procedural memory: accessible, modifiable knowledge of how tasks are accomplished, distinct from semantic and episodic memory. Goal-Oriented Graphs offer such a representation, encoding goals and prerequisite dependencies as inspectable symbolic structure. We examine their construction, the challenges they pose, and principles for integrating them with neural reasoning.

Figures & tables

Domain	Example Goal	Prerequisite Structure	Knowledge Source
Game	Craft diamond axe	Tool tiers, material chains	Wiki pages, recipe files
Software	Deploy web service	Dependency ordering, build artifacts	Documentation, runbooks
Medicine	Diagnose condition X	Prior test results, clinical criteria	Clinical guidelines
Manufacturing	Assemble device Y	Component availability, tool requirements	Technical manuals

Table 1: Goal-Oriented Graph structure across representative domains. The fundamental structure is consistent: goals decompose into subgoals with explicit prerequisites. The primary variation is in the source of procedural knowledge available for graph construction. The game domain (Minecraft) is the only one that has been empirically evaluated [ 30 ] ; the other rows are conceptual illustrations of the representation pattern.

Dimension	Implicit	Experience-based	Symbolic
	(fine-tuning)	(templates)	(GoG, PDDL)
Inspectability	$\times$	$\sim$	$\checkmark$
Compositional generalization	$\sim$	$\sim$	$\checkmark$
Construction cost	High	Medium	Variable
Adaptability	Low	High	Low–Medium
Cross-domain transfer	$\sim$	$\sim$	$\sim$

Table 2: Expected tradeoffs among procedural memory representations for LLM agents. A “

\sim

” indicates partial or moderate support. These ratings are conceptual expectations based on representational properties rather than a quantitative benchmark comparison.

Procedural Memory for LLM Agents

See all 12 papers

Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Sep 8, 2026Yuxing Lu, Yicheng Chen, Shanchan Wu +1Graph-Based Agent Memory Long-Horizon LLM Agents
Google · Georgia Institute of Technology · Peking University
Procedural Knowledge at Scale Improves Reasoning
Apr 1, 2026Di Wu, Devendra Singh Sachan, Wen-tau Yih +1Retrieval-Augmented Reasoning LLM Reasoning
University of California, Los Angeles · Meta FAIR
PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning
Jul 22, 2026Alexis Fox, Junlin Wang, Paul Rosu +1Memory-Augmented Language Model Agents Persistent Memory for Language Models
Duke University

Sep 8, 2026J/K move · Enter open · S save

Procedural Graphs: Self-Evolving Execution Structures for LLM Agents

Yuxing Lu, Yicheng Chen, Shanchan Wu, Sercan Ö. Arık

Google · Georgia Institute of Technology · Peking University

Graph-Based Agent Memory Long-Horizon LLM Agents LLM Agent Planning Self-Evolving Agents

Open paper AskPDF arXiv

Large language models are increasingly deployed as agents that plan over long horizons and act through external tools. Most agents select actions through unconstrained generation over an accumulating history, leaving implicit the procedural knowledge of what to do, in what order, and under which conditions. As trajectories lengthen, agents can lose track of their objectives, invoke tools out of order, and repeat unproductive actions. We introduce the Procedural Graph: just as a knowledge graph organizes factual knowledge into (entity, relation, entity) triplets for what-is questions, a Procedural Graph organizes procedural knowledge into (procedure, relation, procedure) triplets for what-to-do questions. At each decision step, the framework localizes the agent's active node, and a guidance model translates the surrounding subgraph into step-level situational guidance that biases the solver's next action without dictating it. The graph is self-evolving: an LLM refiner contrasts failed trajectories with successful ones and edits the graph's topology and attributes, committing edits that preserve or improve held-out validation performance while retaining rejected ones to discourage repetition. Starting from a minimal skeleton, the loop builds graphs that match or surpass hand-designed ones. It can also repair a flawed expert prior. Across multiple datasets, task types, and LLMs, the Procedural Graph delivers consistent gains over memory-based baselines, and self-evolution further improves performance without manual engineering.

Abstract

Procedural Memory for LLM Agents

Explore similar work

Procedural Graphs: Self-Evolving Execution Structures for LLM Agents

Procedural Knowledge at Scale Improves Reasoning

PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning