Large language model (LLM) agents are increasingly built less by changing model weights than by reorganizing the runtime around them. Capabilities that earlier systems expected the model to recover internally are now externalized into memory stores, reusable skills, interaction protocols, and the surrounding harness that makes these modules reliable in practice. This paper reviews that shift through the lens of externalization. Drawing on the idea of cognitive artifacts, we argue that agent infrastructure matters not merely because it adds auxiliary components, but because it transforms hard cognitive burdens into forms that the model can solve more reliably. Under this view, memory externalizes state across time, skills externalize procedural expertise, protocols externalize interaction structure, and harness engineering serves as the unification layer that coordinates them into governed execution. We trace a historical progression from weights to context to harness, analyze memory, skills, and protocols as three distinct but coupled forms of externalization, and examine how they interact inside a larger agent system. We further discuss the trade-off between parametric and externalized capability, identify emerging directions such as self-evolving harnesses and shared agent infrastructure, and discuss open challenges in evaluation, governance, and the long-term co-evolution of models and external infrastructure. The result is a systems-level framework for explaining why practical agent progress increasingly depends not only on stronger models, but on better external cognitive infrastructure.
Figures & tables
Figure 1: Externalization as the organizing principle of LLM agent design. Upper panel: The arc of human cognitive externalization from thought through language, writing, printing, to digital computation. Middle panel: The corresponding externalization arc for LLM agents, from weights through three externalization dimensions—Memory (externalized state), Skills (externalized expertise), and Protocols (externalized interaction)—to the Harness that unifies them. Lower panel: A literature landscape mapping representative works onto three capability layers—Weights, Context, and Harness—illustrating how research threads have progressively migrated outward. The parallel between the two arcs encodes a recursive claim: LLM agents achieve reliable agency by externalizing cognitive burdens along the same representational dimensions that have driven human cognitive history.
Figure 2: Community theme evolution across three capability layers. The stacked layers—Weights, Context, and Harness—show how the center of gravity in the LLM agent community has shifted outward over time, from parametric knowledge and prompting toward harness-level infrastructure such as tool ecosystems, protocols, skills, and multi-agent orchestration.
Figure 3: Externalization architecture of a harnessed LLM agent. The Harness sits at the center; three externalization dimensions— Memory (working context, semantic knowledge, episodic experience, personalized memory), Skills (operational procedures, decision heuristics, normative constraints), and Protocols (agent–user, agent–agent, agent–tools)—orbit around it. Operational elements such as sandboxing, observability, compression, evaluation, approval loops, and sub-agent orchestration mediate the interaction between the harness core and the externalized modules.
Figure 4: Memory as externalized state. Raw context from the ephemeral context window and environment feedback is converted into four persistent memory dimensions—working context, episodic experience, semantic knowledge, and personalized memory. These dimensions are organized through progressively more managed architectures: monolithic context, retrieval stores, hierarchical orchestration (with extraction, consolidation, forgetting, and OS-style hot/cold swapping), and adaptive memory systems (with dynamic modules and feedback-based strategy optimization via MOE, RL, etc.). On the harness side, execution traces from skills and protocols flow into externalized memory, which in turn supplies task-relevant content back to the agent core through direct recall and curated snapshots.
Figure 5: Skills as externalized expertise. The figure traces the full lifecycle of a skill through three phases—invocation, selection, and procedure. Skill Acquisition shows four pathways by which procedural know-how enters the system: authored by experts, distilled from episodic memory and trajectories, discovered through environment exploration and self-induction, or composed from existing units. Skill Artifact packages that know-how into operational procedures, decision heuristics, and normative constraints, accompanied by a manifest declaring capabilities, preconditions, and scope. Activation Pipeline handles registry-based discovery via semantic abstraction, progressive disclosure from abstract summaries to full guides, and composition that binds skills to tools, APIs, files, agents, and protocols. Runtime shows how the active context and the LLM execute the selected skill, while boundary conditions—staleness, portability limits, context-dependent degradation, and unsafe composition—constrain reliability.
Figure 6: Protocols as externalized interaction. Upper panel: The evolutionary trajectory of agent interaction—from isolated model calls with limited model-to-model communication, through hardcoded API connections, to standardized protocols that provide unified interaction, task allocation, tool integration, and secure access, and ultimately toward a decentralized and networked agentic web. Lower panel: The harness implements externalized interaction management through three functional surfaces: Interact (interfacing with external APIs, tools, and environments), Perceive (perception of environment, context, memory, and feedback), and Collaborate (collaboration with other LLMs, agents, and humans).
Figure 7: The harness as cognitive environment. The Foundation Model (Agent Core) sits at the center; six harness dimensions form a coordinated ring around it. Three externalization modules— Memory (state persistence, failure recording, cross-session context), Skills (reusable routines, staged loading, failure-driven revision), and Protocols (deterministic interfaces, structured invocation, schema contracts)—supply the externalized cognitive content. Three operational surfaces— Permission (sandboxing, filesystem isolation, network restrictions), Control (recursion bounds, cost ceilings, timeout), and Observability (structured logging, execution traces, aggregate metrics)—govern how that content is accessed, constrained, and monitored at runtime. Arrows indicate the continuous flow among dimensions within the harness loop.
Figure 8: Couplings among memory, skills, and protocols. The six arrows summarize how the three externalization modules reinforce one another inside a harness. Memory supplies evidence for skill formation and protocol routing; skills turn stored experience into reusable procedures and invoke protocolized actions; protocols constrain execution and write normalized outcomes back into memory.