Recurrent Harness for Agentic Continual Learning

Yuxin Xiao; Zhiyun Lu; Jianyu Wang; Yangtian Zhang; Yang Song; Qifan Wang; Bailin Wang

cs.LGOct 8, 2026

Recurrent Harness for Agentic Continual Learning

Authors: Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang

Organizations: Massachusetts Institute of Technology · Meta MRS Research · Yale University

Abstract

Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.

Figures & tables

Figure 1: From stateful agent harnesses to ReHarness. (a) Traditional stateful harness propagates information through forward write and read, allowing an overgeneralized reflection to persist and mislead future instances. (b) ReHarness adds backward revision to refine retrieved reflections using downstream feedback, preventing the same error from propagating to future instances.

Method	CL-Bench			MemoryBench			MemoryArena			Average
Method	Claude	GPT	Gemini	Claude	GPT	Gemini	Claude	GPT	Gemini	Average
Mem0	$0.103$	$0.220$	$0.092$	$0.049$	$0.016$	$0.053$	$0.093$	$0.322$	$0.235$	$0.131$
RBank	$0.073$	$0.065$	$0.040$	$0.023$	$0.004$	$0.045$	$0.014$	$0.017$	$0.023$	$0.034$
DoT	$0.125$	$0.081$	$-0.008$	$0.053$	$0.027$	$0.010$	$0.129$	$0.179$	$0.042$	$0.071$
ACE	$0.042$	$0.088$	$0.036$	$0.085$	$-0.018$	$0.028$	$0.045$	$0.004$	$0.073$	$0.043$
GEPA-Online	$0.076$	$0.239$	$0.132$	$0.138$	$0.064$	$-0.134$	$0.126$	$0.129$	$0.058$	$0.092$

Table 1: Mean normalized gain across task streams for each benchmark–backbone combination. Highest gains are in bold. ReHarness outperforms all baselines, suggesting that backward revision of accumulated experience effectively adapts agent harnesses to continual distribution shifts.

Figure 2: Task-level normalized gain for each method, averaged across the three backbone LLMs. Performance varies across tasks, while ReHarness remains effective on nearly all of them.

LLM	Grounded Rate $\mathbf{(\uparrow)}$		Leakage Rate $\mathbf{(\downarrow)}$		Unsupported Rate $\mathbf{(\downarrow)}$
LLM	Average	$\mathbf{\Delta}$ / Revision	Average	$\mathbf{\Delta}$ / Revision	Average	$\mathbf{\Delta}$ / Revision
Claude	$0.953$	$-0.003$	$0.021$	$0.006$	$0.026$	$0.002$
GPT	$0.987$	$-0.002$	$0.008$	$0.003$	$0.005$	$0.001$
Gemini	$0.963$	$0.007$	$0.013$	$0.004$	$0.024$	$-0.002$

Table 2: Average grounded, leakage, and unsupported claim rates of revised reflections on CL-Bench for each backbone, together with the average change per revision. The grounded rate remains above

95\%

for all three LLMs and decreases only marginally after revision, suggesting that ReHarness uses the downstream trajectory primarily as feedback rather than as a source of leaked information.

Revision	Revision	Revised Harness	CL-Bench
Setting	Based On	Applied To	Cohort	Codebase	Database	Sales	Spectrum	Poker	Average
No Revision	–	–	$-0.053$	$0.359$	$0.279$	$0.258$	$0.033$	$-0.012$	$0.144$
ReHarness	$\tau_{t}$	$\tau_{t+1}$	$-0.019$	$0.378$	$0.361$	$0.408$	$0.162$	$0.009$	$0.217$
Counterfactual	$\tau_{t}$	$\tau_{t}$	$\mathbf{0.046}$	$\mathbf{0.509}$	$\mathbf{0.402}$	$\mathbf{0.698}$	$\mathbf{0.314}$	$\mathbf{0.041}$	$\mathbf{0.335}$

Table 3: Normalized gain under counterfactual revision on CL-Bench with Claude. Counterfactual replay reapplies the revised harness to the same instance and substantially improves over ReHarness and the no-revision baseline. This shows that the backward mechanism enables meaningful revision.

Figure 3: Qualitative examples of ReHarness revision during continual learning. (a) In blind spectrum monitoring, ReHarness corrects an early overgeneralized reflection with evidence-grounded guidance. (b) In database exploration, ReHarness adapts an outdated reflection to a schema shift.

Method	Claude		GPT		Gemini		Average
Method	Stability	Plasticity	Stability	Plasticity	Stability	Plasticity	Stability	Plasticity
Mem0	$0.002$	$0.101$	$0.014$	$0.206$	$-0.026$	$0.118$	$-0.003$	$0.142$
RBank	$\mathbf{0.004}$	$0.069$	$0.001$	$0.065$	$-0.022$	$0.062$	$-0.006$	$0.065$
DoT	$-0.003$	$0.127$	$-0.002$	$0.083$	$0.000$	$-0.007$	$-0.002$	$0.068$
ACE	$0.000$	$0.042$	$0.011$	$0.077$	$-0.028$	$0.065$	$-0.006$	$0.061$
GEPA-Online	$0.000$	$0.076$	$0.006$	$0.223$	$-0.017$	$0.149$	$-0.004$	$0.153$

Table 4: Stability–plasticity decomposition on CL-Bench. ReHarness achieves the highest average gains in both, showing robust transfer across distribution shifts and effective adaptation afterward.

Figure 4: (a) Harness state dynamics by ReHarness on CL-Bench codebase adaptation with Claude. Revisions diminish as reflections stabilize before the distribution shift, then rise afterward to update outdated reflections. (b) Top- $k$ scaling on CL-Bench with Claude. ReHarness improves sharply up to $k=5$ and stabilizes for moderate $k$ , while larger values yield diminishing returns and higher cost.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

LLM: Claude Opus 4.8
Method	Cohort	Codebase	Database	Sales	Spectrum	Poker	Average
Mem0	$-0.045\pm 0.014$	$0.408\pm 0.022$	$-0.016\pm 0.014$	$0.284\pm 0.011$	$0.000\pm 0.018$	$-0.013\pm 0.002$	$0.103\pm 0.006$
RBank	$-0.065\pm 0.002$	$0.238\pm 0.007$	$-0.119\pm 0.006$	$0.376\pm 0.011$	$0.000\pm 0.001$	$0.005\pm 0.003$	$0.073\pm 0.002$
DoT	$-0.074\pm 0.002$	$\mathbf{0.499\pm 0.009}$	$0.056\pm 0.006$	$0.252\pm 0.026$	$0.033\pm 0.004$	$-0.017\pm 0.001$	$0.125\pm 0.007$
ACE	$\mathbf{-0.015\pm 0.004}$	$0.231\pm 0.004$	$-0.005\pm 0.004$	$0.051\pm 0.006$	$0.000\pm 0.001$	$-0.010\pm 0.003$	$0.042\pm 0.001$
GEPA-Online	$-0.028\pm 0.006$	$0.238\pm 0.023$	$0.040\pm 0.014$	$0.171\pm 0.004$	$-0.023\pm 0.004$	$\mathbf{0.056\pm 0.002}$	$0.076\pm 0.008$

Appendix

Table 5: Normalized gain for each LLM on each CL-Bench task, with the highest value in bold.

LLM: Claude Opus 4.8
Method	HB-Academic	HB-Creative	WB-Academic	WB-Creative	WB-Politics	Average
Mem0	$0.073\pm 0.004$	$\mathbf{0.182\pm 0.009}$	$0.000\pm 0.004$	$0.053\pm 0.003$	$-0.065\pm 0.009$	$0.049\pm 0.002$
RBank	$-0.055\pm 0.004$	$0.111\pm 0.004$	$0.026\pm 0.002$	$0.033\pm 0.003$	$0.000\pm 0.003$	$0.023\pm 0.002$
DoT	$0.077\pm 0.002$	$0.179\pm 0.004$	$-0.013\pm 0.002$	$0.047\pm 0.003$	$-0.026\pm 0.003$	$0.053\pm 0.003$
ACE	$0.115\pm 0.004$	$0.134\pm 0.001$	$0.064\pm 0.007$	$0.113\pm 0.005$	$0.000\pm 0.004$	$0.085\pm 0.004$
GEPA-Online	$0.166\pm 0.005$	$0.132\pm 0.002$	$0.090\pm 0.029$	$\mathbf{0.173\pm 0.003}$	$\mathbf{0.130\pm 0.006}$	$0.138\pm 0.013$

Appendix

Table 6: Normalized gain for each LLM on each MemoryBench task, with the highest value in bold.

LLM: Claude Opus 4.8
Method	Travel	Shopping	Search	Math	Physics	Average
Mem0	$\mathbf{0.451\pm 0.022}$	$0.029\pm 0.028$	$-0.057\pm 0.003$	$0.016\pm 0.002$	$0.024\pm 0.014$	$0.093\pm 0.005$
RBank	$0.037\pm 0.003$	$0.043\pm 0.002$	$0.019\pm 0.002$	$-0.020\pm 0.003$	$-0.010\pm 0.004$	$0.014\pm 0.001$
DoT	$0.085\pm 0.004$	$0.121\pm 0.010$	$0.124\pm 0.002$	$0.013\pm 0.004$	$0.300\pm 0.008$	$0.129\pm 0.008$
ACE	$-0.002\pm 0.001$	$0.079\pm 0.001$	$0.048\pm 0.004$	$0.012\pm 0.002$	$0.089\pm 0.007$	$0.045\pm 0.003$
GEPA-Online	$0.009\pm 0.004$	$0.219\pm 0.014$	$0.105\pm 0.007$	$0.077\pm 0.002$	$0.218\pm 0.007$	$0.126\pm 0.005$

Appendix

Table 7: Normalized gain for each LLM on each MemoryArena task, with the highest value in bold.

Figure 5: Prompts used by the decision (top), memory (middle), and revision (bottom) agents in ReHarness.

Figure 6: Full qualitative revision example for blind spectrum monitoring from Figure 3 (a).

Figure 7: Full qualitative revision example for database exploration from Figure 3 (b).

Continual Learning for LLM Agents

Continual learning for LLM agents studies how agents adapt from experience while retaining skills and knowledge across tasks.

See all 199 papers

Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Aug 19, 2026Borui Kang, Jinrui Gu, Junhan Lv +3Continual Learning for LLM Agents Continual Learning
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia
Harness-Model Collaboration for Long-Horizon Continual Learning
Oct 26, 2026Zhen Bi, Zhizhi Peng, Yan Wang +6Continual Learning for LLM Agents Continual Learning
Huzhou Normal University · Alibaba Group · Zhejiang University +1
Harness Learning Enables Generalizable Test-Time Adaptation
Sep 28, 2026Alvin Zhang, Xuecheng Liu, Zixuan Wang +6Test-Time Adaptation Meta-Learning

Aug 19, 2026J/K move · Enter open · S save

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li+2

State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia

Continual Learning for LLM Agents Continual Learning Agent Harness Optimization

Open paper AskPDF arXiv

Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.

Abstract

Continual Learning for LLM Agents

Explore similar work

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

Harness-Model Collaboration for Long-Horizon Continual Learning

Harness Learning Enables Generalizable Test-Time Adaptation