Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

Jiaqi Ding; Guorong Wu

cs.CLSep 24, 2026

Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

Authors: Jiaqi Ding, Guorong Wu

Organizations: Department of Computer Science UNC-Chapel Hill · Department of Psychiatry UNC-Chapel Hill

Abstract

Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often collapse memory behavior into final-answer accuracy. We introduce MemProbe, a cognitive-science-inspired framework for diagnosing stability-plasticity tradeoffs in agent memory. The framework is motivated by a core insight from cognitive memory research: memory is reconstructive and shaped by interference, source reliability, reinforcement, and reactivation. MemProbe turns this insight into four reusable experimental paradigms (interference, misinformation, consolidation strength, and reconsolidation window) that manipulate when a memory should be updated, preserved, or treated as uncertain. It further decomposes correctness into behavioral profiles that reveal how systems update, preserve, attribute, and temporally organize information. We instantiate these paradigms in a 56-episode diagnostic suite and evaluate six incremental memory systems under a unified protocol. Results show that systems with similar aggregate scores exhibit distinct behavioral profiles. MemProbe provides such a diagnostic lens, turning aggregate performance into interpretable profiles of memory maintenance over time. Code is available at https://github.com/jq-ding/MemProbe.

Figures & tables

Figure 1: Illustrative depiction of how the same accuracy can mask distinct memory-update behaviors, motivating behavioral evaluation beyond final-answer correctness.

Figure 2: Overview of the MemProbe evaluation framework. Cognitive-science-inspired paradigms generate controlled trial episodes that are processed by a memory system and queried through multi-dimensional probes. Observable responses are converted into behavioral stability–plasticity profiles.

Figure 3: Input protocols used in MemProbe . Full-context reading, retrieval-based reading, heuristic baselines, batch memory, and incremental memory correspond to different computational interpretations of memory and are therefore reported separately. Incremental memory is the primary setting for evaluating stability–plasticity behavior.

Metric	Oracle RAG (diag.)	Full-context	Mem0	Graphiti	LangMem	Cognee	A-MEM	MemoryOS	Naive RAG	Time-aware RAG
Overall [95% CI]	93.3 [89.7, 96.4]	88.8 [84.4, 92.9]	41.5 [35.7, 46.9]	62.5 [55.8, 68.8]	65.2 [57.6, 72.3]	81.7 [76.3, 86.6]	88.4 [83.9, 92.4]	60.3 [53.6, 66.5]	68.3 [60.3, 76.3]	68.8 [61.2, 75.9]

Table 1: Overall behavioral performance across references, incremental memory systems and retrieval baselines.

Figure 4: Probe-level performance across systems. Many systems perform well on current-value, previous-value and change-detection probes, but degrade on source, conflict-value and temporal probes.

System	Plast.	Stab.	SP-Bal	PreserveErr.	UpdateErr.
Mem0	28.6	54.5	37.5	45.5	71.4
Graphiti	58.9	66.1	62.3	33.9	41.1
LangMem	63.4	67.0	65.1	33.0	36.6
Cognee	75.0	88.4	81.1	11.6	25.0
A-MEM	91.1	85.7	88.3	14.3	8.9
MemoryOS	54.5	66.1	59.7	33.9	45.5

Table 2: Stability–plasticity metrics for incremental memory systems.

Figure 5: Stability–plasticity plane. Systems closer to the lower-left corner have better SP-balance.

System	Interference	Misinformation	Consolidation	Reconsolidation	Overall
Oracle RAG (diag.)	89.3	92.9	98.2	92.9	93.3
Full-context	82.1	91.1	91.1	91.1	88.8
Mem0	41.1	46.4	33.9	44.6	41.5
Graphiti	53.6	60.7	75.0	60.7	62.5
LangMem	60.7	66.1	69.6	64.3	65.2
Cognee	80.4	80.4	85.7	80.4	81.7

Table 3: Paradigm-wise accuracy across the four paradigms. Interference is difficult even for strong references, while consolidation sharply separates structured memory systems from flatter extraction or retrieval systems.

System	Gemini	Qwen3
A-MEM	88.4	72.8
Cognee	81.7	68.3
LangMem	65.2	60.3
Graphiti	62.5	61.6
MemoryOS	60.3	51.8
Mem0	41.5	41.5

Table 4: Reader ablation. The same stored memories are answered by two readers from different model families. Absolute scores drop under the weaker reader, but the ordering is largely preserved.

System	Hist.	Source	Temp.
Oracle RAG (diag.)	100.0	75.8	77.0
Full-context	100.0	66.1	77.0
Mem0	71.0	17.7	36.0
Graphiti (Zep)	96.0	41.9	45.0
LangMem	68.0	46.8	41.0
Cognee	96.0	64.5	73.0

Table 5: Historical, source and temporal fidelity across systems

System	Read-out	Storage-side
A-MEM	6	2
Cognee	9	1
LangMem	9	10
Graphiti	8	11
MemoryOS	9	13
Mem0	7	20

Table 6: Manual classification of source-probe errors. Systems are ordered by how much of the original context they preserve for the reader. Read-out errors are roughly constant, whereas storage-side losses grow as systems compress memory more aggressively.

Metric	Naive RAG	Time-aware RAG	Oracle RAG	Full-context
${\Delta_{\mathrm{FC}}}$	20.5	20.0	-4.5	–
${\mathrm{RAG}_{\mathrm{norm}}}$	0.769	0.775	1.051	–
Non-split	79.0	78.4	94.3	90.9
Split	29.2	33.3	89.6	81.2
Split CV	8.3	8.3	91.7	66.7
Split CD	8.3	8.3	100.0	91.7

Table 7: Diagnostic RAG ablation for split-perturbation episodes.

\Delta_{\mathrm{FC}}

measures the loss relative to full-context reading, and

\mathrm{RAG}_{\mathrm{norm}}

measures the fraction of full-context performance recovered by retrieved evidence. Split CV denotes current-value accuracy, and Split CD denotes change-detection accuracy.

Figure 6: Split penalty/gain by systems. Negative values indicate that performance drops when perturbation evidence is distributed across sessions.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Suite property	Value
Number of episodes	56
Number of sessions	1,246
Number of probes	224
Paradigms	4
Domains	Personal, Work, Agentic
Split-perturbation episodes	12

Appendix

Table 8: Summary statistics of the 56-episode MemProbe demonstration suite.

Domain	Example fact types
Personal / lifestyle	Current gym, dietary preference, coffee preference, commute plan, routine preference
Work / productivity	Submission deadline, project owner, team standup time, deployment cadence, review schedule
Agentic / tool-state	Reported port number, active branch, selected database, retry limit, configuration value, tool-reported state

Appendix

Table 9: Domains and representative fact types used in the demonstration suite.

	Pearson		Spearman
Axis	mean	min	mean	min
4 paradigms	0.93	0.85	0.88	0.77
6 probe types	0.78	0.43	0.75	0.43

Appendix

Table 10: Off-diagonal correlation summary across the two decomposition axes, computed over the six incremental memory systems.

Probe type	Judge	Full-context	Naive RAG
Current value	98.2%	96.4%	71.4%
Previous value	100.0%	100.0%	92.9%
Change detection	100.0%	94.6%	65.5%
Temporal	86.4%	85.7%	81.8%
Source	97.1%	97.1%	97.1%
Conflict value	100.0%	96.4%	100.0%

Appendix

Table 11: Probe-level validation accuracy for the 56-episode suite. Judge and Full-context scores are high for most probe types, indicating that structured gold labels are largely recoverable when all evidence is visible. Naive RAG shows the largest degradation on current-value and change-detection probes, suggesting that retrieval-based systems struggle most when they must identify the accepted current state and determine whether a real update occurred.

Baseline	Accuracy
LLM judge	97.8%
Full-context reader	95.5%
Naive RAG	81.2%
Latest-value heuristic	18.8%
FC–RAG gap	14.3 pp

Appendix

Table 12: Validation results for the 56-episode demonstration suite. The judge and Full-context reader indicate that the episodes are largely readable and internally consistent, while the RAG gap and low heuristic score indicate that the suite is not solved by simple retrieval or recency shortcuts.

Subset	Judge	Full-context	Naive RAG
Split episodes	95.8%	93.6%	56.2%
Non-split episodes	98.3%	96.0%	88.0%

Appendix

Table 13: Split versus non-split validation results. Split perturbations preserve Full-context readability but sharply reduce RAG performance, indicating that they diagnose distributed evidence integration rather than simply making the text ambiguous.

Baseline	GPT-5.4	Gemini-3-Flash	Qwen3-32B
Judge	96.9	94.2	90.2
Full-context	92.4	88.8	78.1
Naive RAG	68.8	68.3	59.4
FC–RAG gap	23.6	20.5	18.7

Appendix

Table 14: Cross-model validation summary under structured JSON scoring. All three readers show the same qualitative trend: Full-context performance is substantially higher than naive RAG, indicating that the suite exposes retrieval and evidence-integration difficulty beyond Full-context readability.

Probe type	GPT-5.4	Gemini-3-Flash	Qwen3-32B
Current value	96.4	92.9	94.6
Previous value	100.0	100.0	96.4
Change detection	96.4	98.2	94.6
Source	76.5	73.5	41.2
Conflict value	92.9	78.6	42.9
Temporal	86.4	77.3	72.7

Appendix

Table 15: Full-context accuracy by probe type across reader models. State-oriented probes are robust across models, whereas source and conflict-value probes reveal larger differences, especially for Qwen3-32B. This suggests that provenance and conflict-status reasoning are more demanding than extracting current or previous values.

System	Interf.	Misinf.	Consol.	Recons.	Overall [95% CI]
A-MEM	97.5	89.5	97.3	97.5	95.5 [91.7, 98.7]
Cognee	86.5	92.1	92.5	94.6	91.4 [86.3, 96.1]
MemoryOS	84.6	76.9	68.4	80.0	77.6 [70.4, 84.3]
LangMem	82.5	70.3	86.8	63.2	75.8 [69.4, 82.6]
Graphiti	75.0	71.1	66.7	83.8	74.0 [66.4, 81.8]
Mem0	45.0	34.2	43.2	48.6	42.8 [36.7, 48.4]

Appendix

Table 16: Paradigm-wise and overall accuracy on the second suite. Confidence intervals are episode-level bootstrap intervals over the 40 episodes.

Agent Memory

Agent memory lets AI systems retain, update, and use facts, experiences, and skills across sessions and long-horizon tasks.

See all 191 papers

Controlled Memory Interference in Continual LLM Agents
Aug 7, 2026Ao Ding, Hongzong LI, Shiqin Tang +4Continual Learning for LLM Agents Agent Memory Poisoning
China University of Geosciences (Beijing) · Northwestern Polytechnical University · The Hong Kong University of Science and Technology +4
MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery
Jun 23, 2026Enze Ma, Yufan Zhou, Wei-Chieh Huang +7Agent Memory AI Agent Benchmarks
University of Illinois Chicago · KU Leuven · UC San Diego
Remember Before You're Asked: MemDream for Self-Probing Memory Evolution
Sep 28, 2026Mingfei Lu, Mengjia Wu, Runsong Jia +2LLM Agent Memory Graph-Based Agent Memory
Australian Artificial Intelligence Institute (AAII) University of Technology Sydney

Aug 7, 2026J/K move · Enter open · S save

Controlled Memory Interference in Continual LLM Agents

Ao Ding, Hongzong LI, Shiqin Tang, Li Zhang+3

China University of Geosciences (Beijing) · Northwestern Polytechnical University · The Hong Kong University of Science and Technology+4

Continual Learning for LLM Agents Agent Memory Poisoning

Open paper AskPDF arXiv

Long-term memory enables AI agents to maintain continuity across sessions, personalize behavior, and evolve through accumulated experience. Yet memory evolution is not simply a process of storing more information: new experiences may reinforce, revise, or interfere with existing memory states. Existing systems mainly emphasize memory construction and relevance-based retrieval, but several memories may remain simultaneously relevant while differing in state, temporal validity, or authority. We introduce Controlled Memory Interference (CMI), a controlled diagnostic and data-generation framework for studying how agent memory evolves under different memory relationships. Across controlled memory evolution, benign accumulation has limited effects, whereas relationship-specific interference sharply suppresses update plasticity with little stability gain, either by blocking target-memory exposure or by disrupting its downstream use. Lexical and Dense retrieval exhibit distinct interference pathways, while poisoning is more sensitive to update-authority cues than to recency alone. Beyond diagnosis, CMI provides targeted examples for interference-aware memory learning, improving the distinction between valid updates and interference-inducing memories while preserving performance on original memory tasks. These findings show that memory evolution is shaped not only by memory scale, but also by interactions among accumulated experiences. More broadly, memory interference emerges as an important factor for reliable continual agent memory systems.

Abstract

Agent Memory

Explore similar work

Controlled Memory Interference in Continual LLM Agents

MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery

Remember Before You're Asked: MemDream for Self-Probing Memory Evolution