ParamMem: Augmenting Language Agents with Parametric Reflective Memory

Tianjun Yao; Yongqiang Chen; Yujia Zheng; Pan Li; Zhiqiang Shen; Kun Zhang

cs.LGFeb 26, 2026

ParamMem: Augmenting Language Agents with Parametric Reflective Memory

Authors: Tianjun Yao, Yongqiang Chen, Yujia Zheng, Pan Li, Zhiqiang Shen, Kun Zhang

Organizations: Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University · Georgia Institute of Technology

Abstract

Self-reflection enables language agents to iteratively refine solutions, yet often produces repetitive outputs that limit reasoning performance. Recent studies have attempted to address this limitation through various approaches, among which increasing reflective diversity has shown promise. Our empirical analysis reveals a strong positive correlation between reflective diversity and task success, further motivating the need for diverse reflection signals. We introduce ParamMem, a parametric memory module that encodes cross-sample reflection patterns into model parameters, enabling diverse reflection generation through temperature-controlled sampling. Building on this module, we propose ParamAgent, a reflection-based agent framework that integrates parametric memory with episodic and cross-sample memory. Extensive experiments on code generation, mathematical reasoning, and multi-hop question answering demonstrate consistent improvements over state-of-the-art baselines. Further analysis reveals that ParamMem is sample-efficient, enables weak-to-strong transfer across model scales, and supports self-improvement without reliance on stronger external model, highlighting the potential of ParamMem as an effective component for enhancing language agents.

Figures & tables

Figure 1: Correlation between reflective diversity (measured by average pairwise cosine distance) and task performance across five datasets using LLaMA-3.1-8B under Reflexion, DoT, and DoT-bank.

Figure 2: Comparison of memory mechanisms across different frameworks. Pale Mint denotes episodic memory only, as in Reflexion and DoT. Lavender Gray indicates episodic memory combined with cross-sample memory, as in DoT-bank. Soft Sand represents the full integration of episodic, cross-sample, and parametric memory, as in P aram A gent-plus . When using only episodic and parametric memory, the framework reduces to P aram A gent .

Panel of Figure 3 — Figure 3: Illustration of the output produced by P aram M em .

Domain	Dataset	Method	Llama-3.1-8B		Mistral-7B-v0.2		Qwen2-1.5B
			Score	#Prompt Tokens	Score	#Prompt Tokens	Score	#Prompt Tokens
Code	HumanEval	Base	59.15	$1.00$	32.93	$1.00$	41.46	$1.00$
		Reflexion	76.22 $\uparrow$ 17.07	$9.29$	51.22 $\uparrow$ 18.29	$28.54$	49.39 $\uparrow$ 7.93	$18.30$
		Retroformer	67.68 $\uparrow$ 8.53	$11.28$	42.94 $\uparrow$ 10.01	$38.37$	46.34 $\uparrow$ 4.88	$12.77$
		DoT	73.17 $\uparrow$ 14.02	$17.45$	46.95 $\uparrow$ 14.02	$43.06$	56.56 $\uparrow$ 15.10	$15.26$
		DoT-bank	79.56 $\uparrow$ 20.41	$24.71$	54.26 $\uparrow$ 21.33	$61.62$	60.10 $\uparrow$ 18.64	$31.28$

Table 1: Performance on HumanEval/MBPP, MATH, HotpotQA, and 2WikiMultiHopQA. Bold denotes the best result, and underline marks the second best.

\uparrow

and

\downarrow

indicate the absolute improvement or decrease relative to the Base method. For clarity, the prompt token usage of the Base method is normalized to 1. Score is Pass@1 for HumanEval/MBPP and Accuracy for MATH/QA.

Panel of Figure 4 — Figure 4: Reflection diversity induced by P aram M em . (a) Higher pairwise distance indicates more diverse outputs. (b) Higher optimal $K$ and silhouette scores confirm greater semantic variation. (c) Silhouette score as a function of cluster number $K$ .

Method	HumanEval	HotpotQA
Base	59.15	57.67
Reflexion	76.22 $\uparrow$ 17.07	71.33 $\uparrow$ 13.66
DoT	73.17 $\uparrow$ 14.02	66.67 $\uparrow$ 9.00
DoT-bank	79.56 $\uparrow$ 20.41	72.00 $\uparrow$ 14.33
P aram A gent	78.05 $\uparrow$ 18.90	76.33 $\uparrow$ 18.66
P aram A gent-plus	86.59 $\uparrow$ 27.44	83.33 $\uparrow$ 25.66

Table 2: Self-improvement results using Llama-3.1-8B as both the agent and data generator Bold denotes the best, underline the second best. denotes Llama-3.1-8B-Instruct as the data generator.

Figure 5: The performance of P aram A gent and P aram A gent-plus in HumanEval dataset, with 3 iterative process.

Method	LiveCodeBench	HumanEval	HotpotQA
Simple	52.00	90.24	71.33
Reflexion	62.00 $\uparrow$ 10.00	96.34 $\uparrow$ 6.10	82.33 $\uparrow$ 11.00
DoT	60.67 $\uparrow$ 8.67	96.34 $\uparrow$ 6.10	79.67 $\uparrow$ 8.34
DoT-bank	62.00 $\uparrow$ 10.00	96.95 $\uparrow$ 6.71	83.33 $\uparrow$ 12.00
P aram A gent	61.33 $\uparrow$ 9.33	97.56 $\uparrow$ 7.32	79.67 $\uparrow$ 8.34
P aram A gent-plus	63.33 $\uparrow$ 11.33	98.17 $\uparrow$ 7.93	85.00 $\uparrow$ 13.67

Table 3: Weak-to-strong transfer on LiveCodeBench, HumanEval, and HotpotQA with Qwen3-Next-80B-A3B-Instruct as the base LLM in the agent. and denote P aram M em instantiated by Llama-3.1-8B-Instruct and Qwen3-Next-30B-A3B-Instruct.

Method	HumanEval	MBPP
DoT	73.17	61.21
DoT-bank	79.56	64.82
P aram A gent ( 8000 samples)	82.93	67.00
P aram A gent (500 samples)	81.71	64.99
P aram A gent-plus (500 samples)	86.59	65.49

Table 4: Sample efficiency of P aram M em . Models are trained on 500 diverse samples via

K

-means clustering. Bold denotes the best result, underline the second best.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Task Type	Dataset Name	Size	Metric
Programming	HumanEval	164 problems, $\sim$ 3 visible test cases/problem	Pass@1
Programming	MBPP	397 sampled problems	Pass@1
Math	MATH	278 sampled problems across 7 subjects	0-1 Acc
Multi-hop QA	HotpotQA	300 sampled problems (100 per difficulty)	0-1 Acc
Multi-hop QA	2WikiMultiHopQA	300 sampled problems (75 per type)	0-1 Acc

Appendix

Table 5: Datasets used for Programming, Math, and Multi-hop QA tasks.

Dataset	Method	Llama-3.1-70B-Instruct		Qwen2.5-72B-Instruct
		Pass@1	#Prompt Tokens	Pass@1	#Prompt Tokens
HumanEval	Base	80.49	$1.00$	82.92	$1.00$
	Model-based Reflection	87.80 $\uparrow$ 7.31	$6.39$	89.64 $\uparrow$ 6.72	$3.48$
	Reflexion	90.24 $\uparrow$ 9.75	$4.31$	88.41 $\uparrow$ 5.49	$3.48$
	DoT	90.85 $\uparrow$ 10.36	$7.51$	87.80 $\uparrow$ 4.88	$6.05$
	DoT-bank	92.68 $\uparrow$ 12.19	$9.14$	90.24 $\uparrow$ 7.32	$8.17$

Appendix

Table 6: Performance on HumanEval. Bold denotes the best result, and underline marks the second best.

\uparrow

and

\downarrow

indicate absolute change relative to the Base method. For clarity, the prompt token usage of the Base method is normalized to 1.

Dataset	Method	Llama-3.1-70B-Instruct		Qwen2.5-72B-Instruct
		Acc	#Prompt Tokens	Acc	#Prompt Tokens
HotpotQA	Base	70.00	$1.00$	73.33	$1.00$
	Model-based CoT	73.67 $\uparrow$ 3.67	$1.43$	74.10 $\uparrow$ 1.05	$1.44$
	Reflexion	82.33 $\uparrow$ 12.33	$3.02$	82.67 $\uparrow$ 9.34	$2.81$
	DoT	73.67 $\uparrow$ 3.67	$3.43$	80.67 $\uparrow$ 7.34	$4.30$
	DoT-bank	80.00 $\uparrow$ 10.00	$5.24$	82.33 $\uparrow$ 9.00	$7.87$

Appendix

Table 7: Performance on HotpotQA dataset. Bold denotes the best result, and underline marks the second best.

\uparrow

and

\downarrow

indicate the absolute improvement or decrease relative to the Base method. For clarity, the prompt token usage of the Base method is normalized to 1.

Method	HumanEval				HotpotQA
	#Prompt	#Completion	Total Cost	Pass@1	#Prompt	#Completion	Total Cost	Acc
	Tokens	Tokens	($)	(%)	Tokens	Tokens	($)	(%)
Base	37,463	13,506	0.00917	59.15	164,013	1,801	0.02985	57.67
Model-based Reflection	342,805	82,280	0.07652	78.05	236,548	1,212	0.04280	61.67
Reflexion	348,068	73,538	0.07589	76.22	703,192	68,612	0.13892	71.33
DoT	653,981	169,986	0.14831	72.56	1,164,812	106,806	0.22889	66.67

Appendix

Table 8: Token usage and cost on HumanEval and HotpotQA datasets with Llama3.1-8B as backbone LLM. Best and second-best metrics are in bold and underline respectively.

Panel of Figure 8 — Figure 8: Selected reflections on the mbpp_769_Diff problem. Red = critical errors that directly invalidate the implementation; Orange = conceptual pitfalls or misleading reasoning patterns; Green = key insights leading to a working solution; Gray = background noise (irrelevant or stylistic).

Panel of Figure 10 — Figure 10: 1-shot example for reflective dataset construction in math reasoning.

Panel of Figure 11 — Figure 11: 1-shot example used in P aram A gent for semantic decomposition dataset construction in multi-hop QA.

Panel of Figure 12 — Figure 12: Instruction prompt used by P aram A gent to generate next-round solutions for programming tasks.

Panel of Figure 13 — Figure 13: Instruction prompt used by P aram A gent to generate next-round solutions for math reasoning.

Panel of Figure 14 — Figure 14: The prompt of P aram A gent to generate next-round answers for multi-hop QA tasks.

Self-Reflection in Language Model Agents

See all 36 papers

CoMem: Learning to Anticipate Failures from Retrieved Experience
Oct 19, 2026Yun Shen, Ming Shen, Tianjun YaoMemory-Augmented Language Model Agents Self-Reflection in Language Model Agents
The Ohio State University · Shenzhen Loop Area Institute · Shanghai Jiaotong University
Training Language Agents to Learn from Experience
May 19, 2026Yuval Shalev, Zifeng Ding, Mateja JamnikContinual Learning for LLM Agents Self-Improving Agents
University of Cambridge
Agent-Reflex: Teaching Language Agents to Act Reflectively
Oct 13, 2026Guanting Dong, Xinyu Yang, Jiajie Jin +9LLM Agent Training Self-Reflection in Language Model Agents
Gaoling School of Artificial Intelligence, Renmin University of China · National University of Singapore

Oct 19, 2026J/K move · Enter open · S save

CoMem: Learning to Anticipate Failures from Retrieved Experience

Yun Shen, Ming Shen, Tianjun Yao

The Ohio State University · Shenzhen Loop Area Institute · Shanghai Jiaotong University

Memory-Augmented Language Model Agents Self-Reflection in Language Model Agents LLM Agent Reliability Retrieval-Augmented Reasoning

Open paper AskPDF arXiv

Reflection lets LLM agents improve from their own attempts, but the memory it builds is local to a single task, so an agent reflecting on its own tends to revisit the same explanation for failure. Cross-task reflective memory addresses this in two forms: retrieved memory keeps past experience explicit and looks up related cases, which is specific but bounded by the bank, and parametric memory internalizes experience into parameters, which generalizes but is grounded in nothing beyond the task description. When both are used, existing agents combine them additively, feeding their outputs to the solver side by side, so retrieved experience never changes what the learned memory produces. We introduce CoMem (Compounding Memory), which makes retrieved experience an input to the learned memory itself, at training and at inference. The agent builds a bank of its own successful and failed reflections, learns from retrieved cases to write failure hypotheses, candidate explanations of how an attempt is likely to go wrong, and at test time reasons from newly retrieved cases to guide each attempt. Varying the retrieved cases gives the memory different perspectives, yielding reflective diversity that is not merely sampled but also grounded in retrieved experience. Across two backbones and five benchmarks in math, code, and multi-hop QA, CoMem achieves strong overall performance. Additional evaluations show that it generalizes to out-of-distribution tasks and enables smaller models to provide effective guidance to stronger solvers.

Abstract

Self-Reflection in Language Model Agents

Explore similar work

CoMem: Learning to Anticipate Failures from Retrieved Experience

Training Language Agents to Learn from Experience

Agent-Reflex: Teaching Language Agents to Act Reflectively