Organizations: Manning College of Information and Computer Sciences, University of Massachusetts Amherst · VA Bedford Healthcare · Division of Computational Health Sciences, Department of Surgery, University of Minnesota · Department of Health Outcomes & Biomedical Informatics, University of Florida College of Medicine · Miner School of Computer & Information Sciences, University of Massachusetts Lowell
Large language models (LLMs) have demonstrated considerable potential in biomedical applications, while agentic memory systems have emerged as a promising approach to preserving and utilizing information across tasks and interactions. However, a fundamental question remains: how effectively and reliably do these systems retain, retrieve, update, and utilize biomedical information? In this study, we systematically investigate agentic memory in the biomedical domain, focusing on task effectiveness, retrieval quality, information fidelity, and long-term robustness. We evaluate five memory approaches across four LLMs in three representative settings: biomedical knowledge reasoning, clinical prediction, and long-term conversational memory. Our analyses reveal how memory construction, evidence relevance, and conversational history shape downstream biomedical performance, highlighting the distinct challenges of retrieving relevant information, preserving its fidelity, and effectively utilizing it for reasoning. Building on these findings, we introduce Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that jointly enhances evidence accessibility and task-aware utilization to improve biomedical reasoning and long-term conversational memory. HMRU achieves an average relative improvement of 8.6% across biomedical question-answering and conversational memory benchmarks, with additional gains in selected clinical prediction tasks. It further reduces conversational answer errors by 24.9% and closes 48.0% of the performance gap to full-context inference. Our findings provide a systematic characterization of biomedical agentic memory and practical insights into developing more effective and reliable memory-augmented LLM systems.
Figures & tables
Method
RiTeK-PharmKG
RiTeK-ADInt
Mort.-MIMIC-III
Readm.-MIMIC-III
TCGA-Reports
MIMIC-IV OOD
P∗
R∗
F1∗
P†
R†
F1†
P∗
R∗
F1∗
P†
R†
F1†
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
Qwen3-8B
A-MEM
26.97
34.88
28.53
29.25
37.76
31.00
37.59
48.86
40.17
41.94
55.81
45.24
9.61
56.76
59.25
53.61
83.08
97.88
4.15
84.21
MemZero
16.56
26.28
18.92
18.70
29.00
21.19
23.81
38.76
27.59
27.42
45.41
31.98
8.98
53.07
58.23
53.22
78.61
96.61
3.13
76.67
FullContext
14.81
20.10
15.83
18.06
24.80
19.51
21.19
28.98
22.87
27.08
38.55
29.84
9.13
54.93
54.37
47.87
20.87
65.51
4.33
76.31
NaiveRAG
28.34
35.08
29.57
30.84
37.75
32.16
41.10
48.20
42.74
45.94
54.56
48.15
10.22
59.60
57.79
53.75
81.29
97.79
5.76
81.64
Table 1: Comparative evaluation of agentic memory approaches on biomedical benchmarks. Bold and underlined values indicate the best and second-best distinct results within each language model, respectively. For RiTeK datasets, ∗ denotes exact match and † denotes ROUGE-1.
Figure 1: Combined Sensitivity (test). a , Effect of training data volume. b , Effect of retrieval depth (Top-k) on biomedical memory performance. c , Effect of class distribution on biomedical memory performance.
Figure 2: Memory retrieval quality and utilization across biomedical tasks. a , Evidence-to-outcome flows for NaiveRAG on the pooled RiTeK test questions. b , Distribution of the strongest retrieved evidence category across five memory methods. c , Answer accuracy stratified by whether retrieved memory contains an answer matching the ground truth. d , Proportions of predictions matching a retrieved source answer, separated by correct and incorrect predictions. e , Answer accuracy by BGE-M3 similarity between the test question and the strongest answer-matching source question; bins contain at least ten questions. f , Retrieval enrichment for the ground-truth class relative to its training-set prior in four classification tasks; error bars show 95 per cent confidence intervals.
Figure 3: Memory evolution in A-MEM. a , Proportion of memory nodes with unsupported context during sequential memory construction across six biomedical tasks. b , Association between context rewrite frequency and unsupported context rate, with 95 per cent Wilson confidence intervals. c , Proportion of memory notes containing source-unsupported context in their initial and final states after memory evolution. d , Evolution trajectories of the ten most frequently rewritten nodes in MIMIC-IV OOD. Gray and red indicate supported and unsupported context, respectively. Unsupported content is determined by an LLM evaluator against the original training record.
Method
Memory Category
Session Type
Dyn. Inc.
Dyn. Upd.
Proac. Align.
Temp. Reas.
Single Sess.
Multi Sess.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
Qwen3-8B
A-MEM
74.60
76.60
68.70
65.80
71.50
68.20
58.30
62.50
76.50
74.60
68.50
73.60
MemZero
58.00
39.70
55.60
30.60
65.90
34.30
58.30
50.00
59.70
37.20
57.30
39.90
FullContext
78.70
82.30
85.90
89.10
70.70
72.80
100.00
100.00
77.70
81.30
80.30
83.10
Table 2: Memory performance on the RealMem medical subset. Performance of five memory methods across four language-model backbones, four memory categories, and two session types. QA and Rec. denote QA Score and Memory Recall, respectively. Scores are percentages. Bold and underlined values indicate the best and second-best distinct scores within each backbone and metric. Dyn. Inc., dynamic incremental; Dyn. Upd., dynamic updating; Proac. Align., proactive alignment; Temp. Reas., temporal reasoning.
Figure 4: Conversational memory robustness. a , Effects of conversation history length on QA performance and memory recall under Short, Medium, and Long conditions. b , Relationship between evidence distance and memory performance, measured by the number of tokens separating the last relevant evidence from the question. c , QA performance conditioned on whether all required evidence was retrieved, comparing A-MEM and LightMem. Filled and open circles indicate complete and incomplete evidence retrieval, respectively.
Figure 5: Performance and robustness of long-term conversational memory systems. a , Paired comparison of perfect-answer outcomes between four memory methods and FullContext (exact McNemar test; *** P<0.001 ). b , Perfect-answer rates conditional on retrieving none, some, or all gold-standard memories. c , Perfect-answer rates across five conversation history-length groups, with adjusted odds ratios for history length. Error bars indicate 95% confidence intervals.
Method
RiTeK-PharmKG
RiTeK-ADint
Mort.-MIMIC-III
Readm.-MIMIC-III
TCGA-Reports
MIMIC-IV OOD
P∗
R∗
F1∗
P†
R†
F1†
P∗
R∗
F1∗
P†
R†
F1†
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
Base method
39.95
39.23
39.27
42.18
41.40
41.51
53.32
53.08
52.94
58.84
58.62
58.53
9.81
59.88
61.91
56.69
99.06
99.93
4.94
82.16
HMRU-R
43.47
43.04
42.95
45.53
45.05
45.01
55.08
54.53
54.60
60.28
59.86
59.90
9.66
60.43
61.62
57.68
98.84
99.94
4.28
84.08
HMRU-I
44.31
43.29
43.48
46.01
45.02
45.23
58.30
57.33
57.64
63.34
62.57
62.81
9.67
61.16
62.17
57.89
98.84
99.94
4.17
78.43
HMRU-C
44.17
43.26
43.35
45.92
45.05
45.16
56.54
55.67
55.91
62.00
61.29
61.50
10.03
61.87
61.72
57.92
98.84
99.94
3.91
76.96
Table 3: Comparison of HMRU variants and baseline memory performance across six biomedical tasks. All scores are percentages. Bold and underlined values indicate the best and second-best distinct scores for each metric, respectively.
Method
Memory Category
Session Type
Dyn. Inc.
Dyn. Upd.
Proac. Align.
Temp. Reas.
Single Sess.
Multi Sess.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
QA
Rec.
Base Method
91.90
90.34
91.92
86.01
95.93
90.16
91.67
62.50
93.83
92.25
90.13
85.18
HMRU-R
93.25
92.63
94.95
91.31
95.12
91.99
91.67
75.00
93.83
93.88
93.33
89.51
HMRU-I
93.12
92.73
91.92
86.26
95.12
96.86
91.67
75.00
93.50
93.62
92.80
90.39
HMRU-T
90.96
92.87
79.80
83.33
88.62
91.38
66.67
75.00
90.33
93.38
87.47
88.49
Table 4: Comparison of HMRU variants and baseline memory performance on RealMem using HuatuoGPT-3-27B. QA and Rec. denote QA Score and Memory Recall, respectively. All scores are percentages. Bold and underlined values indicate the best and second-best distinct scores for each metric, respectively.
Figure 6: Mechanistic analysis of Hybrid Memory Retrieval and Utilization on biomedical question answering. a , Distribution of retrieved evidence categories before and after BM25 augmentation. b , Decomposition of answer performance and failure modes across successive retrieval and utilization enhancements. c , Distribution of evidence utilization errors before and after task-aware adaptation. Results are reported for RiTeK-PharmKG and RiTeK-ADInt using HuatuoGPT-3-27B.
Figure 7: Performance improvements of HMRU-R on long-term conversational memory using HuatuoGPT-3-27B. a , Overall perfect-answer rates compared with the baseline and FullContext. b , Performance across five conversation history-length groups. c , Question-level improvements and regressions stratified by changes in gold-memory recall.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Memory data
Biomedical knowledge reasoning
Clinical prediction and pathology classification
RiTeK-PharmKG
RiTeK-ADint
Mortality
Readmission
TCGA-Reports
MIMIC-IV OOD
P∗
R∗
F1∗
P†
R†
F1†
P∗
R∗
F1∗
P†
R†
F1†
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
HuatuoGPT-3-27B
A-MEM
25%
28.14
27.78
27.66
31.38
31.00
30.89
38.28
37.66
37.83
46.67
46.25
46.34
9.11
57.47
58.45
53.00
98.76
99.80
4.91
73.08
50%
33.76
33.48
33.23
36.52
36.28
36.05
43.36
43.15
43.08
50.23
50.13
50.00
9.67
59.66
61.25
56.24
99.01
99.85
3.87
82.96
100%
39.95
39.23
39.27
42.18
41.40
41.51
53.32
53.08
52.94
58.84
58.62
58.53
9.81
59.88
61.91
56.69
99.06
99.93
4.94
82.16
Appendix
Table 5: Effect of memory construction data volume across six biomedical tasks. Nested memory-construction subsets contain 250, 500, and 1,000 training samples, corresponding to 25%, 50%, and 100% of the full subset. The test sets and other experimental settings are fixed. RiTeK tasks report precision ( P ), recall ( R ), and F1 ( F1 ) using exact match ( ∗ ) and ROUGE-1 ( † ). Mortality and readmission use MIMIC-III. TCGA-Reports results are macro-averaged. Clinical prediction and pathology classification report AUPRC and AUROC. All results are percentages. Within each model and memory data volume, bold and underlined values indicate the best and second-best scores among memory approaches, respectively.
Method
Top- k
Biomedical knowledge reasoning
Clinical prediction and pathology classification
RiTeK-PharmKG
RiTeK-ADint
Mortality
Readmission
TCGA-Reports
MIMIC-IV OOD
P∗
R∗
F1∗
P†
R†
F1†
P∗
R∗
F1∗
P†
R†
F1†
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
AUROC
Qwen3-30B-A3B-Instruct-2507
A-MEM
1
26.03
31.02
27.33
28.72
34.46
30.28
34.98
41.94
36.71
40.28
50.02
42.94
8.99
56.72
56.17
52.02
96.02
99.56
6.42
78.18
3
29.14
33.46
30.32
31.29
36.14
32.68
41.13
46.23
42.57
46.92
54.37
49.09
8.43
54.76
55.54
50.89
97.03
99.63
4.10
73.39
5
30.45
35.57
31.81
32.63
38.03
34.12
43.25
49.20
44.95
49.01
57.27
51.50
8.86
54.00
56.98
52.28
97.97
99.77
3.08
74.98
Appendix
Table 6: Effect of retrieval depth on biomedical memory performance. We evaluate four memory methods (A-MEM, MemZero, LightMem, and NaiveRAG) with two language-model backbones across six biomedical tasks. For each benchmark, the same 1,000 samples are used to construct the memory or retrieval corpus, and the number of retrieved memory items varies over k∈{1,3,5,10} . The test sets and other experimental settings are held fixed. RiTeK tasks report precision ( P ), recall ( R ), and F1 ( F1 ) using exact match ( ∗ ) and ROUGE-1 ( † ). Mortality and readmission are evaluated on MIMIC-III; TCGA-Reports results are macro-averaged. Clinical prediction and pathology classification report AUPRC and AUROC. All results are percentages. Within each model and retrieval depth, bold and underlined values indicate the best and second-best scores among the four memory approaches, respectively. The Top-10 results reproduce the corresponding full-data baseline results.
Method
Pos:Neg
Mortality–MIMIC-III
Readmission–MIMIC-III
AUPRC
AUROC
AUPRC
AUROC
Qwen3-30B-A3B-Instruct-2507
A-MEM
10:90
8.57
54.49
55.06
49.29
25:75
9.67
54.66
57.92
52.23
50:50
7.11
49.38
56.82
51.30
Natural
9.37
51.17
55.01
49.76
Appendix
Table 7: Effect of class distribution on biomedical memory performance. Five memory methods are evaluated using two language-model backbones on MIMIC-III mortality and readmission prediction. For each task, 1,000 samples are selected for memory construction or context preparation under positive-to-negative ratios of 10:90, 25:75, and 50:50, or under the natural distribution without explicit class balancing. Test sets retain their original class distributions, and other experimental settings are held fixed. Performance is reported as AUPRC and AUROC in percentages. Within each backbone and class-ratio setting, the best and second-best results across methods are shown in bold and underlined, respectively. Natural rows reproduce the corresponding baseline results.
Figure 8: Task-aware memory utilization instructions in HMRU-I. The adapter applies task-specific evidence utilization rules while preserving the original memory representation.
Figure 9: Compact memory representation in HMRU-C. The adapter retains the task-aware utilization instructions of HMRU-I while removing auxiliary memory metadata, including keywords, tags, and timestamps.
Figure 10: RealMem HMRU-I: relevance-ordered memory utilization. Retrieved memories retain their original A-MEM representation, with task-aware instructions for personalized, temporally consistent responses.
Figure 11: RealMem HMRU-T: chronological memory organization and state tracking. The same retrieved memories are reorganized by session date and original turn order, with a compact state checklist for dialogue continuity, user preferences and temporal updates.
Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory architecture comprising six cognitive mechanisms: (1) sleep-phase consolidation, (2) interference-based forgetting, (3) engram maturation, (4) reconsolidation upon retrieval, (5) entity knowledge graphs, and (6) hybrid multi-cue retrieval. Each mechanism addresses a specific failure mode of naive memory accumulation. We introduce a synthetic calibration methodology that derives all pipeline thresholds without benchmark data exposure, eliminating a common source of evaluation leakage. We evaluate on two benchmarks. First, a VSCode issue-tracking dataset (13K issues, 120K events) where deduplication-based consolidation achieves 97.2% retention precision with 58% store reduction (+21.8 pp over baseline). Second, the LongMemEval personal-chat benchmark where we conduct the first streaming M-tier evaluation (475 sessions, ~540K unique turns). At a 200K-token context budget, our pipeline matches raw retrieval accuracy (70.1% vs. 71.2%, overlapping 95% CI) while exposing a tunable accuracy/store-size operating curve. At S-tier scale (50 sessions), dedup-based consolidation yields a +13.3 pp improvement in preference recall.
Doga Kerestecioglu, Alexei Robsky, Clemens Vasters +2
Long-term memory has become increasingly important for LLM agents that operate across extended interactions and evolving task contexts. Recent memory systems have made past experiences more persistent, compact, and retrievable, but retrieval alone does not ensure that a memory provides valid evidence for the current query. When experiences are compressed into reusable fragments, memories from different situations may appear equally relevant if they involve recurring entities or user states. We refer to this failure as context collapse: memories lose the surrounding context needed to judge whether they provide valid evidence for the current query. To address this problem, we propose Contextual Reinstatement for Agentic Memory (RaMem), a framework that turns retrieved memory fragments into contextually verifiable evidence. RaMem operates through four coordinated stages: (i) evidence anchoring grounds each memory in its original episodic conditions, especially event time, mention time, session span, and participants; (ii) recall condition induction derives the evidence conditions implied by the query; (iii) validity-aware retrieval uses these conditions to prioritize context-compatible memories while retaining content-relevant candidates as fallback evidence; and (iv) context-preserved synthesis keeps the selected memories' structured context available to the generator. Experiments on long-term memory benchmarks show that RaMem consistently improves performance over strong memory baselines, with average F1 gains of more than 10% across several backbones.
Long-term memory is essential for LLM-based agents operating over extended interactions. Existing memory systems primarily update memory when new information arrives, treating retrieval as the endpoint of memory access rather than a driver of memory evolution. Consequently, retrieval feedback is rarely exploited to reorganize memory for future access continuously. Moreover, most existing approaches rely on predefined memory structures together with fixed retrieval pipelines, limiting the agent's ability to organize and evolve its own memory autonomously. Inspired by memory reconsolidation in cognitive neuroscience, we propose REALM, a reconsolidation-evolution agentic long-term memory framework. It models long-term memory as a continual lifecycle by autonomously organizing memories into a heterogeneous cognitive graph, retrieving evidence via adaptively composed graph-search atoms, and continually reconsolidating memories based on retrieval feedback. REALM achieves an average accuracy of 75.97% on LoCoMo and 65.11% on LongMemEval, outperforming the strongest baselines by 7.17 and 1.31 points respectively. Ablation studies confirm that memory reconsolidation consistently boosts performance, with further analyses revealing that it progressively reorganizes related memory units into more coherent local structures for collective evidence recall and utilization during reasoning. These results suggest that retrieval-driven memory reconsolidation provides an effective mechanism for continually evolving long-term memory in LLM agents.
Yuanyi Song, Yukai Wang, Xinbei Ma +7
Shanghai Jiao Tong University · OPPO · National University of Singapore