CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection

Linbo Liu; Guande Wu; Han Ding; Yawei Wang; Qiang Zhou; Yuzhe Lu; Zhichao Xu; Huan Song

cs.AIApr 8, 2026

CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection

Authors: Linbo Liu, Guande Wu, Han Ding, Yawei Wang, Qiang Zhou, Yuzhe Lu, Zhichao Xu, Huan Song, +2 more

Organizations: AWS AI

Abstract

Large language model agents rely on effective model context to obtain task-relevant information for decision-making. Many existing context engineering approaches primarily rely on the context generated from the past experience and retrieval mechanisms that reuse these context. However, retrieved context from past tasks must be adapted by the execution agent to fit new situations, placing additional reasoning burden on the underlying LLM. To address this limitation, we propose a generative context augmentation framework using Contrastive Learning of Experience via Agentic Reflection (CLEAR). CLEAR first employs a reflection agent to perform contrastive analysis over past execution trajectories and summarize useful context for each observed task. These summaries are then used as supervised fine-tuning data to train a context augmentation model (CAM). Then we further optimize CAM using reinforcement learning, where the reward signal is obtained by running the task execution agent. By learning to generate task-specific knowledge rather than retrieve knowledge from the past, CAM produces context that is better tailored to the current task. We conduct comprehensive evaluations on the AppWorld and WebShop benchmarks. Experimental results show that CLEAR consistently outperforms strong baselines. It improves task completion rate from 72.62% to 81.15% on AppWorld test set and averaged reward from 0.68 to 0.74 on a subset of WebShop, compared with baseline agent. Our code is publicly available at https://github.com/awslabs/CLEAR.

Figures & tables

Figure 1: CLEAR training framework design. First, we execute each task $q_{i}\sim\mathcal{D}_{\text{train}}$ for $m$ times and collect groups of replay $\Gamma_{i}$ for $q_{i}$ . We employ reflection agents $\pi^{R}$ to generate instance-level instruction $c_{i}$ for each $q_{i}$ , collected into $\mathcal{D}_{\text{SFT}}$ . Then, we initialize CAM from an open-source LLM and perform SFT on $\mathcal{D}_{\text{SFT}}$ . Finally, we further perform RL on the trained CAM, which leverages the reward signal from $\pi^{E}$ for policy update of CAM.

Figure 2: During inference, a new task $q_{\text{new}}$ is sampled from $\mathcal{D}_{\text{test}}$ and is passed into $\pi^{C}_{\theta}$ to generate $c_{\text{new}}\sim\pi^{C}_{\theta}(q_{\text{new}})$ . The auxiliary context $c_{\text{new}}$ is appended to $q_{\text{new}}$ and the execution agent $\pi^{E}$ starts with $q_{\text{new}}\oplus c_{\text{new}}$ .

Method	TGC $\uparrow$		SGC $\uparrow$
Method	Avg	Pass@3	Avg	Pass@3
Baseline	72.62(2.59)	86.90	52.38(2.73)	66.07
RAG	72.02(2.15)	86.31	54.67(3.72)	71.43
ACE Zhang et al. (2025)	74.40(3.57)	85.71	58.93(6.19)	73.21
GEPA Agrawal et al. (2025)	75.99(0.74)	89.88	57.14(3.86)	78.57
MIPRO Opsahl-Ong et al. (2024)	74.40(1.46)	88.10	55.36(1.46)	80.36

Table 1: AppWorld experiments results. Task Goal Completion (TGC) and Scenario Goal Completion (SGC) on the Test-N split are reported. Results are averaged over three runs (standard deviation in parentheses) except the Pass@3 metric.

Method	Avg. Reward	Std.
Baseline	0.6799	0.0119
RAG	0.7252	0.0076
CLEAR (ours)	0.7406	0.0044

Table 2: WebShop-40k experiments results. Averaged reward on the test dataset is reported. Results are averaged over three runs with standard deviation also reported.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Experiment ID	Method	TGC $\uparrow$		SGC $\uparrow$
Experiment ID	Method	Avg	Pass@3	Avg	Pass@3
1	Qwen/Qwen3-32B	67.40(2.95)	83.33	48.81(4.49)	66.07
2	SFT without CL	68.65 (2.68)	88.69	42.86 (4.72)	67.86
3	SFT with CL	74.21(1.72)	90.48	50.60(2.73)	71.43
4	CLEAR	81.15 (2.48)	91.67	66.67 (4.49)	82.14

Appendix

Table 3: Ablation study on AppWorld Test-N split. For the CAM, we use the following variant: (1) a Qwen/Qwen3-32B model without SFT and RL as in experiment 1. (2) a Qwen3-32B model after SFT on

\mathcal{D}_{\text{SFT\_no\_CL}}

(no RL) as in experiment 2. (3) a Qwen3-32B model after SFT with CL (no RL) as in experiment 3. (4) full CLEAR framework as in experiment 4. Results are averaged over three runs (standard deviation in parentheses) except the Pass@3 metric.

Method	Run time (s)	# of turns	CAM throughput (tokens/s)	CAM latency (s)
Baseline	63.4	17.3	NA	NA
CLEAR (ours)	77.2	18.5	69.5	1.21

Appendix

Table 4: CAM Latency study. Task run time, number of turns of the execution agent

\pi^{E}

, throughput of CAM, and latency for invoking CAM. All numbers are averaged over AppWorld Test-N split. The model for

\pi^{E}

is Claude-Sonnet-4 .

Model for $\pi^{E}$	Method	TGC $\uparrow$		SGC $\uparrow$
Model for $\pi^{E}$	Method	Avg	Pass@3	Avg	Pass@3
DeepSeek-V3.1	Baseline	41.27(3.05)	64.29	19.05(2.06)	26.79
	RAG	33.33(1.57)	53.57	10.71(4.72)	21.43
	ACE	32.54(1.50)	53.57	13.69(1.03)	28.57
	CLEAR (ours)	42.95 (2.85)	66.07	24.40 (4.49)	33.93

Appendix

Table 5: AppWorld experiments results. Task Goal Completion (TGC) and Scenario Goal Completion (SGC) on the Test-N split are reported. Results are averaged over three runs (standard deviation in parentheses) except the Pass@3 metric.

Explore similar work

Learning Agent-Compatible Context Management for Long-Horizon Tasks
May 29, 2026Lu Yi, Runlin Lei, Liuyi Yao +6Long-Horizon Agent Tasks LLM Context Management
1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group +2
From History to State: Constant-Context Skill Learning for LLM Agents
May 6, 2026Haoyang Xie, Xinyuan Wang, Yancheng Wang +2LLM Agent Skill Learning LLM Agent Training
School of Computing and Augmented Intelligence, Arizona State University
Context Language Models
Sep 29, 2026Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin +10Memory-Augmented Language Models LLM Context Management
University of Washington · Meta Superintelligence Labs · MIT +1

May 29, 2026J/K move · Enter open · S save

Learning Agent-Compatible Context Management for Long-Horizon Tasks

Lu Yi, Runlin Lei, Liuyi Yao, Yuexiang Xie+5

1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group+2

Long-Horizon Agent Tasks LLM Context Management RL for LLM Agents

Open paper AskPDF arXiv

LLM agents increasingly face long-horizon tasks such as web search and deep research in real-world applications, where accumulated context can cause long-context degradation and reasoning failures. Prior work mitigates this through context management with agent-side context control or fixed strategies such as summarization, which require training the agent itself for adaptation - making it impractical for closed-source agents and ignoring that different agents may require different strategies. We introduce Adaptive Context Management (AdaCoM), which trains an external LLM to manage the context of a frozen agent through flexible modification actions and end-to-end reinforcement learning. Across diverse agents on web search and deep research benchmarks, AdaCoM substantially improves performance by preserving task constraints and progress while pruning stale content. The learned strategies reveal a Fidelity-Reliability Trade-off: agents with higher vanilla ReAct performance benefit from higher-fidelity context preservation, whereas lower-performing agents require more aggressive compression to stay within a reliable reasoning regime. Transfer experiments show that AdaCoM generalizes most effectively across agents with similar capability (measured by vanilla ReAct performance), suggesting a practical path toward reusable context managers for agent systems.

Abstract

Explore similar work

Learning Agent-Compatible Context Management for Long-Horizon Tasks

From History to State: Constant-Context Skill Learning for LLM Agents

Context Language Models