cs.AIOct 26, 2026

Harness-Model Collaboration for Long-Horizon Continual Learning

Authors: Zhen Bi, Zhizhi Peng, Yan Wang, Xueshu Chen, Haofei Zhang, Yaobo Zhang, Zeyu Yang, Qing Shen, 

Organizations: Huzhou Normal University · Alibaba Group · Zhejiang University · Ningxia University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Long-horizon agents continuously encounter reusable procedural knowledge, yet existing systems typically rely either on model adaptation or external memory, trading persistent learning for rapid reuse. This limitation is especially important for Web agents that repeatedly execute related multi-step workflows. We propose HaMo, a Harness-Model collaborative framework that combines fast non-parametric experience accumulation with slow parametric adaptation. HaMo distills trajectories into strategy-level experience cards for immediate retrieval, while an experience-conditioned continual learning objective trains the model to interpret, ground, and execute retrieved strategies across changing contexts. A retrieval-coverage-based selection mechanism further identifies compact and complementary trajectories for parameter updates. Experiments show that HaMo outperforms harness-only and model-only adaptation under within-domain and cross-domain shifts, improving both retention and adaptation. These results highlight the importance of jointly preserving procedural experience and internalizing the competence to transfer and execute it across tasks.

Figures & tables

Figure 1. Motivation for our method in continual learning for long-horizon agents. Harness- and model-only methods either enable fast experience reuse but tend to plateau, or suffer from instability and forgetting. HaMo combines both mechanisms to achieve steady improvement within domains and better retention across domain shifts.
Figure 2. Overview of the HaMo framework. HaMo continuously distills expert trajectories into a persistent harness memory, selects representative anchor trajectories based on retrieval coverage, and combines them with historical replay to update a LoRA adapter through experience-conditioned behavioral cloning. An optional feedback step releases memory entries associated with trained high-quality trajectories after adaptation. This collaboration enables fast experience reuse at the harness level and gradual knowledge internalization at the model level.
Figure 3. Within-domain data-incremental continual learning performance (stage-wise and average performance) on ScienceWorld. The training set is divided into four equally sized, sample-disjoint subsets, each covering all task types. Each stage introduces one subset and evaluates the entire test set. Average reports the mean stage score for adapting methods and the fixed reference score for Base. Higher is better. Four panels show stage-wise performance for Qwen3-8B and Qwen3-14B on the left, and average performance for the same models on the right.
Figure 4. Cross-domain continual learning with Qwen3-8B in the order ScienceWorld (S), ALFWorld (A), and WebShop (W). Panels (a)–(c) show performance after each training stage. Bar groups compare methods, and colors indicate evaluation domains. Higher scores are better.
Figure 5. Component ablation on Qwen3-8B with the training order ScienceWorld (S), ALFWorld (A), and WebShop (W). Each group reports performance after training on the indicated dataset.
Figure 6. Training-order ablation on Qwen3-8B. S, A, and W denote ScienceWorld, ALFWorld, and WebShop, respectively. Final-stage total (S+A+W) sums the three dataset scores measured after the final training stage for each training order.
Figure 7. Performance on the first-trained domain throughout continual learning under three training orders. The curves track ScienceWorld in S → A → W, ALFWorld in A → W → S, and WebShop in W → S → A.
StageHaMoBase
User GoalChocolate-coated wafers; caramel flavor; 4.41 ounce; price below $60.Chocolate-coated wafers; caramel flavor; 4.41 ounce; price below $60.
SearchSearches for “chocolate coated wafers 4.41 ounce”.Searches for “caramel chocolate coated wafers 4.41 ounce”.
CandidateOpens Loacker Tortina Chocolate Coated Wafer ($41.99). Notices “Triple Dark” in the title and checks the available flavor options .Opens Loacker Quadratini Hazelnut Wafer Cookies ($14.99), incorrectly treating the hazelnut product as a caramel match.
VerificationAttempts to select the requested options: click[caramel] followed by click[4.41 ounce] ; neither changes the visible observation.The product page offers no caramel option. Makes no flavor or size selection before buying.
PurchaseChecks 41.9941.99 < 60 and executes click[buy now] after attempting both option selections.Executes click[buy now] immediately after opening the mismatched product.
Interactions5 actions; completes the purchase.3 actions; completes the purchase.
Table 1. WebShop case study. Trajectories are summarized from the recorded interactions. Purple text backgrounds mark HaMo’s key constraint-grounding behaviors. Scores are reported on the same 0–100 scale as the main experiments.
Figure 8. Performance of HaMo across model scales (1.7B, 4B, and 8B) with the training order ScienceWorld (S), ALFWorld (A), and WebShop (W). The largest LLM backbone achieves the strongest overall results, supporting HaMo’s applicability across model capacities.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

SettingValue
GPU hardwareNVIDIA A800, 80 GB
Training epochs per stage3
Learning rate2×10−42\times 10^{-4}
LoRA rank rr16
LoRA scaling parameter α\alpha32
Backbone parametersFrozen
Appendix
Table 2. HaMo implementation and evaluation settings.
More in this topic

Continual Learning for LLM Agents

Continual learning for LLM agents studies how agents adapt from experience while retaining skills and knowledge across tasks.

See all 199 papers

Explore similar work

CardsList

  1. Harness Continual Learning: Continual Adaptation Beyond Model Parameters

    Aug 19, 2026Borui Kang, Jinrui Gu, Junhan Lv +3Continual Learning for LLM AgentsContinual Learning

    State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia

  2. Recurrent Harness for Agentic Continual Learning

    Oct 8, 2026Yuxin Xiao, Zhiyun Lu, Jianyu Wang +4Continual Learning for LLM AgentsAgent Harness Optimization

    Massachusetts Institute of Technology · Meta MRS Research · Yale University

  3. Position: Modular Memory is the Key to Continual Learning Agents

    Mar 2, 2026Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21Continual Learning for LLM AgentsMemory-Augmented Agents

    Toyota Motor Europe · University of Bremen · University of Florence +17