What Makes Agent Memory Useful for Reliable Unanswerable Question Handling?

Chuanyuan Tan; Junjie Yu; Yuxin Wang; Yining Zheng; Xipeng Qiu; Wenliang Chen

cs.CLAug 28, 2026

What Makes Agent Memory Useful for Reliable Unanswerable Question Handling?

Authors: Chuanyuan Tan, Junjie Yu, Yuxin Wang, Yining Zheng, Xipeng Qiu, Wenliang Chen

Organizations: School of Computer Science and Technology, Soochow University · Suzhou Key Lab of Multi-modal Data Fusion and Intelligent Healthcare, Suzhou City University · Shanghai Key Lab of Intelligent Information Processing · College of Computer Science and Artificial Intelligence, Fudan University ·

Abstract

Reliable handling of unanswerable questions (UAQs) is critical for trustworthy LLM-based agents. Although memory is widely used in agent systems, its role in reliable UAQ handling remains unclear. We present a systematic study of agent memory for UAQ handling under a unified agentic RAG framework, evaluating four representative memory methods across three UAQ-related datasets and two base models. We find that memory can improve UAQ performance in some settings, but such gains are selective rather than universal and remain fragile under dataset shift. Interestingly, cross-model memory reuse is often more feasible than cross-dataset transfer, suggesting that shifts in answerability patterns pose a greater challenge to memory reuse than changes in the base model itself. We further find that UAQ gains are more strongly preserved through decision guidance than through trajectory shaping, and that memory effectiveness depends strongly on representation. In particular, procedural and rule-based memories often provide the most reliable support for UAQ handling, while memory composition is most effective when procedural guidance is combined with complementary behavioral signals. Overall, our findings suggest that reliable UAQ memory depends less on storing larger amounts of experience and more on preserving transferable behavioral guidance.

Figures & tables

Figure 1: Overview of agent with external memory. Retrieved memory is injected as initial guidance before interaction begins. The resulting interaction trajectory is then distilled into new memory and stored for future reuse.

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
No-memory	19.50	51.00	41.55	65.33	64.33	64.63
Human Hint	60.67	44.67	49.47	87.00	51.17	61.92
Expel	33.00	51.50	45.95	73.67	60.67	64.57
MemEvolve	38.83	51.83	47.93	71.83	62.50	65.30

Table 1: Effectiveness of memory methods in in-distribution setting. All results are evaluated after memory update on the training set.

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
Human Hint	60.67	44.67	49.47	87.00	51.17	61.92
Expel	87.50	38.00	52.85	94.83	46.17	60.77
MemEvolve	79.67	41.00	52.60	90.67	44.33	58.23
AWM	81.17	41.83	53.63	94.83	45.33	60.18

Table 2: Effectiveness of memory mechanisms combined with Human Hint under in-distribution setting.

Memory Source	KUQ-Train						UAQFact-Train
Test Set	UAQFact			RefuNQ			KUQ			RefuNQ
Method	AR	Acc	JS ( $\Delta$ )	AR	Acc	JS ( $\Delta$ )	AR	Acc	JS ( $\Delta$ )	AR	Acc	JS ( $\Delta$ )
DeepSeek-V3.2
No-memory	65.33	64.33	64.63	39.33	58.17	52.52	19.50	51.00	41.55	39.33	58.17	52.52
Expel	72.33	59.50	63.35 ( -1.28 )	42.83	57.83	53.33 ( +0.81 )	28.33	51.17	44.32 ( +2.77 )	42.50	59.67	54.52 ( +2.00 )
MemEvolve	68.50	62.50	64.30 ( -0.33 )	36.17	59.17	52.27 ( -0.25 )	30.50	51.00	44.85 ( +3.30 )	42.33	57.67	53.07 ( +0.55 )

Table 3: Cross-dataset transfer results.

\Delta

denotes the performance gap compared to the No-memory baseline.

Method	Cross JS	$\Delta_{C-N}$	$\Delta_{C-B}$
DeepSeek-V3.2
Expel	43.95	+2.40	-2.00
MemEvolve	42.85	+1.30	-5.08
AWM	46.53	+4.98	+1.25
AgentKB	39.73	-1.82	-1.99
Qwen3-235B

Table 4: Cross-model transfer results on KUQ using JS. Cross JS denotes using memory updated by the other base model.

\Delta_{C-N}

and

\Delta_{C-B}

compare Cross JS against No-memory and same-model memory, respectively, and are computed based on mean JS scores. Detailed results are listed in Appendix B.2.2 .

Figure 2: JS score comparison for different functional channels of memory: decision guidance (G), trajectory shaping (T), and their combination (G&T). The horizontal line denotes the No-memory setting performance (No-mem). Detailed results are listed in Appendix B.3 .

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
No-memory	19.50	51.00	41.55	65.33	64.33	64.63
Compact trajectory	33.00	51.83	46.18	74.67	58.67	63.47
Principle	31.17	52.00	45.75	68.83	59.33	62.18
Insight	27.17	50.83	43.73	69.50	59.50	62.50

Table 5: Main results on different memory types.

Without procedural memory
Combination	DeepSeek-V3.2	Qwen3-235B
No-memory	41.55	41.98
Compact trajectory + principle	48.05	41.58
Principle + insight	47.42	43.17
With workflow
Compact trajectory + workflow	56.07	41.45

Table 6: JS performance of pairwise memory content compositions on KUQ dataset. Combinations involving procedural memory generally achieve the strongest performance. Full results are provided in Appendix B.5 .

DeepSeek-V3.2	Both	Success	Failure
No-memory	41.55	–	–
Compact trajectory	46.18	47.80	43.80
Principle	45.90	43.52	46.90
Insight	43.57	42.85	43.17
Qwen3-235B	Both	Success	Failure
No-memory	41.98	–	–

Table 7: JS performance of memory constructed from successful, failed, and mixed experiences on the KUQ dataset. Full results are provided in Appendix B.6 .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Memory Type	Example
Compact Trajectory	Topic: Health insurance regulations (U.S.-centric)
	Key Finding: Under the Affordable Care Act (ACA), private health insurers must accept all eligible applicants on a guaranteed-issue basis, regardless of health status, pre-existing conditions, age, or other factors.
	Search Strategy: (1) Initial broad search (“private insurance companies required to accept”) pointed to the “Health insurance” page; (2) lookup for “guaranteed issue” and “pre-existing conditions” revealed ACA context; (3) direct search for “Guaranteed issue” provided the definitive answer.
	Reusable Notes: Focus on terms such as “guaranteed issue” and “pre-existing conditions” for insurance regulation questions. The “Guaranteed issue” page is a concise source for ACA-related acceptance rules.
Insight	• Start with a direct search for the most specific and standard term related to the question. • When search returns multiple candidate pages, select the most directly relevant one without overcomplicating exploration. • Use section headings or table of contents to efficiently locate the target information within a page. • Search regulatory questions using canonical terms such as “Affordable Care Act” rather than paraphrased descriptions.
Principle	When direct search results for a speculative or fictional concept are inconclusive, shift toward verifying its scientific basis and consulting authoritative overviews to assess whether the concept corresponds to a recognized phenomenon.

Appendix

Table 8: Example memory representations extracted from interaction trajectories. Different memory content types preserve different levels of abstraction, ranging from descriptive trajectory summaries to reusable procedural workflows.

Dataset	Human AR	LLM AR	Cohen’s Kappa
KUQ	0.21	0.23	0.8835
UAQFact	0.60	0.59	0.8963
RefuNQ	0.38	0.38	0.9151

Appendix

Table 9: Human annotation validation for LLM-as-judge. We report human/LLM AR and Cohen’s Kappa on 100 randomly sampled UAQs per dataset (no-memory setting). High Cohen’s Kappa indicates strong agreement between human annotations and LLM-as-judge.

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
No-memory	19.50 $\pm$ 1.32	51.00 $\pm$ 0.87	41.55 $\pm$ 0.46	65.33 $\pm$ 1.15	64.33 $\pm$ 1.15	64.63 $\pm$ 0.70
Human Hint	60.67 $\pm$ 2.75	44.67 $\pm$ 2.08	49.47 $\pm$ 0.65	87.00 $\pm$ 1.50	51.17 $\pm$ 0.76	61.92 $\pm$ 0.13
Expel	33.00 $\pm$ 3.04	51.50 $\pm$ 0.50	45.95 $\pm$ 1.00	73.67 $\pm$ 3.25	60.67 $\pm$ 0.29	64.57 $\pm$ 1.16
MemEvolve	38.83 $\pm$ 7.79	51.83 $\pm$ 3.40	47.93 $\pm$ 1.78	71.83 $\pm$ 0.76	62.50 $\pm$ 1.32	65.30 $\pm$ 0.70

Appendix

Table 10: Effectiveness of memory methods in in-distribution setting. All results are evaluated after memory update on the training set.

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
Human Hint	60.67 $\pm$ 2.75	44.67 $\pm$ 2.08	49.47 $\pm$ 0.65	87.00 $\pm$ 1.50	51.17 $\pm$ 0.76	61.92 $\pm$ 0.13
Expel	87.50 $\pm$ 2.78	38.00 $\pm$ 0.87	52.85 $\pm$ 1.11	94.83 $\pm$ 2.93	46.17 $\pm$ 1.61	60.77 $\pm$ 1.73
MemEvolve	79.67 $\pm$ 4.16	41.00 $\pm$ 3.12	52.60 $\pm$ 0.94	90.67 $\pm$ 1.26	44.33 $\pm$ 0.58	58.23 $\pm$ 0.52
AWM	81.17 $\pm$ 3.01	41.83 $\pm$ 1.89	53.63 $\pm$ 0.53	94.83 $\pm$ 1.44	45.33 $\pm$ 3.06	60.18 $\pm$ 1.74

Appendix

Table 11: Effectiveness of memory methods combined with Human Hint under in-distribution setting.

	KUQ			UAQFact
Method (GPT-5.5)	AR	Acc	JS	AR	Acc	JS
No-memory	22.50	56.00	45.95	50.00	70.00	64.00
Expel	30.00	58.00	49.60	51.50	71.50	65.50
MemEvolve	24.00	60.50	49.55	49.50	73.00	65.95
AWM	26.00	60.00	49.80	53.00	73.00	67.00

Appendix

Table 12: Closed-source in-distribution evaluation with GPT-5.5. Results are from a single run; unlike the main experiments, no standard deviations are reported.

DeepSeek-V3.2
Memory Source	KUQ-Train						UAQFact-Train
Test Set	UAQFact			RefuNQ			KUQ			RefuNQ
Method	AR	Acc	JS	AR	Acc	JS	AR	Acc	JS	AR	Acc	JS
No-memory	65.33 ±1.15	64.33 ±1.15	64.63 ±0.70	39.33 ±1.26	58.17 ±1.04	52.52 ±1.10	19.50 ±1.32	51.00 ±0.87	41.55 ±0.46	39.33 ±1.26	58.17 ±1.04	52.52 ±1.10
Expel	72.33 ±1.26	59.50 ±1.32	63.35 ±1.03	42.83 ±3.06	57.83 ±0.58	53.33 ±1.07	28.33 ±1.53	51.17 ±2.93	44.32 ±2.46	42.50 ±1.00	59.67 ±1.61	54.52 ±0.91
MemEvolve	68.50 ±2.00	62.50 ±2.18	64.30 ±1.10	36.17 ±1.61	59.17 ±1.53	52.27 ±1.19	30.50 ±7.37	51.00 ±1.80	44.85 ±1.03	42.33 ±2.25	57.67 ±1.89	53.07 ±0.93

Appendix

Table 13: Cross-dataset transfer results (mean

{}_{\pm\text{std}}

Method	DeepSeek			Qwen3-235B
	AR	Acc	JS	AR	Acc	JS
Expel	27.50 ±2.29	51.00 ±1.32	43.95 ±0.93	29.83 ±1.15	49.17 ±0.58	43.37 ±0.38
MemEvolve	22.67 ±0.76	51.50 ±2.29	42.85 ±1.43	35.00 ±7.86	47.83 ±1.04	43.98 ±2.01
AWM	35.33 ±5.73	51.33 ±1.53	46.53 ±2.81	36.83 ±6.93	48.67 ±1.15	45.12 ±2.86
AgentKB	18.50 ±1.73	48.83 ±2.31	39.73 ±1.93	15.17 ±0.76	48.67 ±2.25	38.62 ±1.81

Appendix

Table 14: Cross-model transfer results on KUQ (mean

{}_{\pm\text{std}}

	DPSK / KUQ			DPSK / UAQFact			Qwen / KUQ			Qwen / UAQFact
Method	G	T	G&T	G	T	G&T	G	T	G&T	G	T	G&T
Expel	46.60	42.27	45.95	63.78	62.48	64.57	41.76	39.43	44.38	60.73	62.38	63.42
MemEvolve	47.40	44.12	47.93	62.30	63.55	65.30	41.08	38.31	38.27	60.22	61.57	61.18
AWM	45.92	42.75	45.28	65.82	63.27	66.37	43.36	39.58	43.72	61.57	59.96	61.95
AgentKB	43.15	41.13	41.72	62.48	63.07	62.65	39.59	39.88	38.48	62.09	61.63	60.55

Appendix

Table 15: JS comparison for decision guidance (G), interaction trajectories (T), and their combination (G&T).

Method	KUQ			UAQFact
	AR	Acc	JS	AR	Acc	JS
DeepSeek-V3.2
Expel
G.	37.50 ±4.36	50.50 ±2.18	46.60 ±1.43	71.83 ±2.75	60.33 ±0.76	63.78 ±1.26
T.	25.00 ±2.60	49.67 ±0.76	42.27 ±1.03	62.83 ±3.33	62.33 ±1.04	62.48 ±0.85
G.&T.	33.00 ±3.04	51.50 ±0.50	45.95 ±1.00	73.67 ±3.25	60.67 ±0.29	64.57 ±1.16

Appendix

Table 16: Comparison of full results for guidance-only (G), trajectory-only (T), and their combination (G&T) (mean

{}_{\pm\text{std}}

DeepSeek-V3.2
	KUQ			UAQFact
Method	AR	Acc	JS	AR	Acc	JS
No-memory	19.50 $\pm$ 1.32	51.00 $\pm$ 0.87	41.55 $\pm$ 0.46	65.33 $\pm$ 1.15	64.33 $\pm$ 1.15	64.63 $\pm$ 0.70
Compact trajectory	33.00 $\pm$ 9.12	51.83 $\pm$ 1.53	46.18 $\pm$ 1.98	74.67 $\pm$ 3.18	58.67 $\pm$ 1.53	63.47 $\pm$ 0.71
Principle	31.17 $\pm$ 9.61	52.00 $\pm$ 2.50	45.75 $\pm$ 1.18	68.83 $\pm$ 3.25	59.33 $\pm$ 1.04	62.18 $\pm$ 1.10
Insight	27.17 $\pm$ 1.53	50.83 $\pm$ 0.76	43.73 $\pm$ 0.75	69.50 $\pm$ 0.50	59.50 $\pm$ 1.32	62.50 $\pm$ 0.79

Appendix

Table 17: Main results on different memory types (mean

{}_{\pm\text{std}}

Without procedural memory
Combination	DeepSeek-V3.2			Qwen3-235B
	AR	Acc	JS	AR	Acc	JS
No-memory	19.50 ±1.32	51.00 ±0.87	41.55 ±0.46	20.17 ±1.89	51.33 ±0.29	41.98 ±0.77
Compact trajectory + principle	36.50 ±3.12	53.00 ±1.32	48.05 ±0.40	24.67 ±8.22	48.83 ±1.53	41.58 ±2.68
principle + insight	31.67 ±7.00	49.50 ±1.61	44.15 ±2.59	25.67 ±2.65	50.67 ±2.00	43.17 ±2.07
With workflow

Appendix

Table 18: Full results of memory content compositions on KUQ-Train (mean

{}_{\pm\text{std}}

	DeepSeek-V3.2			Qwen3-235B
Settings	AR	Acc	JS	AR	Acc	JS
No-memory	19.50 ±1.32	51.00 ±0.87	41.55 ±0.46	20.17 ±1.89	51.33 ±0.29	41.98 ±0.77
Compact trajectory	33.00 ±9.12	51.83 ±1.53	46.18 ±1.98	23.83 ±4.16	48.17 ±2.47	40.87 ±1.37
Success only	42.67 ±2.00	50.00 ±2.18	47.80 ±1.10	48.00 ±6.02	48.50 ±1.50	48.35 ±2.72
Failure only	24.67 ±2.08	52.00 ±1.80	43.80 ±0.79	16.67 ±1.04	49.33 ±1.44	39.53 ±1.14
Principle (Both)	37.50 ±9.61	49.50 ±2.50	45.90 ±1.18	31.67 ±4.01	48.00 ±1.89	43.10 ±0.59

Appendix

Table 19: Ablation of memory learned from successful, failed, and mixed experiences on the KUQ dataset (mean

{}_{\pm\text{std}}

	DeepSeek-V3.2			Qwen3-235B
Settings	AR	Acc	JS	AR	Acc	JS
No-memory	19.50 ±1.32	51.00 ±0.87	41.55 ±0.46	20.17 ±1.89	51.33 ±0.29	41.98 ±0.77
Compact trajectory	33.67 ±8.52	51.67 ±1.26	46.27 ±2.58	19.50 ±2.65	51.50 ±0.50	41.90 ±0.93
Success only	52.33 ±9.33	51.67 ±0.76	51.87 ±2.84	31.83 ±6.98	47.50 ±0.87	42.80 ±2.49
Failure only	22.83 ±3.40	52.50 ±0.87	43.60 ±1.02	19.83 ±2.84	48.50 ±1.80	39.90 ±2.11
Principle (Both)	31.50 ±8.72	51.33 ±2.57	45.38 ±4.20	23.50 ±5.27	49.17 ±1.04	41.47 ±2.19

Appendix

Table 20: Ablation analysis of memory learned from successful and failed cases on the KUQ dataset with equal sample quantity (mean

{}_{\pm\text{std}}

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 376 papers

Feb 13, 2026

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning

Long-context LLMs and Retrieval-Augmented Generation defer state tracking and evidence consolidation to query time, which is brittle when facts evolve and answers depend on latent states. We introduce Unified Memory Agent (UMA) for a one-to-many setting: query-agnostic external memory is constructed once from a stream and reused across multiple future QA sessions. A single policy maintains a structured Memory Bank through CRUD operations and answers using both the Memory Bank and raw context. Task-Stratified GRPO uses the mean reward of QA trajectories branching from each sampled memory state to supervise memory maintenance, while normalizing memory and per-question QA groups separately. We also introduce Ledger-QA, a diagnostic benchmark for long-horizon state tracking over accumulated updates. At the 16k budget, UMA-Generalist achieves the highest average score among compared methods across the test-time-learning and accurate-retrieval benchmarks and transfers to Ledger-QA without task-specific training; UMA-Specialist further improves long-horizon tracking after task adaptation. These results support learned proactive memory management for long-context reasoning.

Kehao Zhang, Shangtong Gui, Sheng Yang +2

1Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · University of Chinese Academy of Sciences, Beijing, China · 4Li Auto Inc. +1

Persistent Memory for Language Models Long-Context Inference Agent Memory Management RL for LLM Agents Memory-Augmented Language Model Agents

Ask PDF

Feb 25, 2026

Towards Autonomous Memory Agents

Recent memory agents improve LLMs by extracting experiences and conversation history into an external storage. This enables low-overhead context assembly and online memory update without expensive LLM training. However, existing solutions remain passive and reactive; memory growth is bounded by information that happens to be available, while memory agents seldom seek external inputs in uncertainties. We propose autonomous memory agents that actively acquire, validate, and curate knowledge at a minimum cost. U-Mem materializes this idea via (i) a cost-aware knowledge-extraction cascade that escalates from cheap self/teacher signals to tool-verified research and, only when needed, expert feedback, and (ii) semantic-aware Thompson sampling to balance exploration and exploitation over memories and mitigate cold-start bias. On both verifiable and non-verifiable benchmarks, U-Mem consistently beats prior memory baselines and can surpass RL-based optimization, improving HotpotQA (Qwen2.5-7B) by 14.6 points and AIME25 (Gemini-2.5-flash) by 7.33 points.

Xinle Wu, Rui Zhang, Mustafa Anis Hussain +1

National University of Singapore, Singapore

Memory-Augmented Language Model Agents Memory Management for LLM Agents Agent Memory Management

Ask PDF

Oct 15, 2026

From Relevant Memories to Sufficient Evidence: Rethinking Memory Utilization in LLM Agents

Long-term memory enables LLM agents to reuse past experiences, yet retrieving relevant memories does not necessarily provide sufficient evidence for answering a query. Essential information may remain fragmented across memory records, obscured by abstraction, or disconnected from the relations required for reasoning. We identify this relevance-sufficiency gap and propose QEMem, a framework that reformulates memory utilization as the construction of query-specific evidence sets, shifting the focus from individual retrieval relevance to collective evidential support. QEMem integrates three complementary mechanisms: query-guided discovery retrieves evidence across multiple memory granularities; source-grounded reification organizes it into structured ViewCards with explicit evidential roles and source references; and sufficiency-guided refinement selectively recovers missing information. Operating on existing memory stores without modifying memory formation, QEMem achieves 91.43% LLM-judge accuracy on LoCoMo with Claude Opus 4.5, outperforming the strongest evaluated baseline, MemR

^3

, by 7.6 percentage points and NEMORI by 10.7 points under a shared memory store. Further analyses demonstrate improved evidence coverage and strong performance on questions requiring cross-session evidence integration.

Shixuan Li, Wei Yang, Peiyu Zhang +7

University of Southern California · Georgia Institute of Technology

LLM Agent Memory LLM Agents Retrieval-Augmented Reasoning

Ask PDF

Abstract

LLM Agent Memory

Explore similar work

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning

Towards Autonomous Memory Agents

From Relevant Memories to Sufficient Evidence: Rethinking Memory Utilization in LLM Agents