D-MEM: Dopamine-Gated Agentic Memory via Reward Prediction Error Routing

Yuru Song; Qi Xin

q-bio.NCMar 15, 2026

D-MEM: Dopamine-Gated Agentic Memory via Reward Prediction Error Routing

Authors: Yuru Song, Qi Xin

Organizations: UC San Diego · Carnegie Mellon University

Abstract

Autonomous LLM agents require structured long-term memory, yet current "append-and-evolve" systems like A-MEM face O(N^2) write-latency and excessive token costs. We introduce D-MEM (Dopamine-Gated Agentic Memory), a biologically inspired architecture that decouples short-term interaction from cognitive restructuring via a Fast/Slow routing system based on Reward Prediction Error (RPE). A lightweight Critic Router evaluates stimuli for Surprise and Utility. Routine, low-RPE inputs are bypassed or cached in an O(1) fast-access buffer. Conversely, high-RPE inputs, such as factual contradictions or preference shifts, trigger a "dopamine" signal, activating the O(N) memory evolution pipeline to reshape the agent's knowledge graph. To evaluate performance under realistic conditions, we introduce the LoCoMo-Noise benchmark, which injects controlled conversational noise into long-term sessions. Evaluations demonstrate that D-MEM reduces token consumption by over 80%, eliminates O(N^2) bottlenecks, and outperforms baselines in multi-hop reasoning and adversarial resilience. By selectively gating cognitive restructuring, D-MEM provides a scalable, cost-efficient foundation for lifelong agentic memory.

Figures & tables

Figure 1: LoCoMo-Noise benchmark construction pipeline. An LLM-based noise generator synthesizes three categories of noise—Filler (40%), Status (30%), and Tangent (30%)—and interleaves them with the original session at a target noise ratio of 75%. The core factual turns (“needles”) are preserved at their original positions, while synthetic noise is injected at random intervals to simulate real-world conversational dynamics.

Method	Overall F1	Single	Temp	Open	Multi	Adv	Total Tokens
A-MEM (Synchronous)	0.336	0.208	0.408	0.111	0.365	0.388	1,648K
D-MEM (Ours)	0.369	0.246	0.440	0.085	0.412	0.412	319K (–80%)

Table 1: Evaluation under extreme noise conditions (

\rho=0.75

) using GPT-4o-mini. D-MEM establishes a new state-of-the-art across complex reasoning tasks while eradicating 80% of the computational token cost caused by synchronous memory evolution.

Figure 2: RPE component decomposition over 700 turns. The top panel shows the RPE signal (black) overlaid with its Surprise (orange) and Utility (purple) components, along with the routing thresholds $\theta_{low}=0.3$ and $\theta_{high}=0.7$ . The bottom panel visualizes the resulting routing tier assignment per turn: SKIP (gray), CONSTRUCT_ONLY (blue), and FULL_EVOLUTION (red). The dominance of blue and the sparsity of red confirm that expensive memory evolution is reserved for genuinely paradigm-shifting inputs.

Figure 3: Routing analysis on the LoCoMo-Noise benchmark. (a) Scatter plot of all turns in the Surprise–Utility space, colored by the routing tier assigned by the Critic Router. All SKIP decisions are tightly concentrated in the “Early cutoff” region (

\text{Utility}<0.3

, yellow dashed line), confirming that the routing is governed by Utility rather than Surprise. (b) Routing distribution stratified by input type (Noise vs. Real). Counter-intuitively, real turns are skipped at a higher rate (53.9%) than noise turns (43.2%), revealing a calibration asymmetry discussed in the text.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Overall	Single Hop	Temporal	Open Domain	Multi Hop	Adversarial
LoCoMo (Full Context)	—	40.4 / 29.1	18.4 / 14.8	12.0 / 11.2	25.0 / 19.8	69.2 / 68.8
ReadAgent	—	9.7 / 7.7	12.6 / 8.9	5.3 / 5.1	9.2 / 6.5	9.8 / 9.0
MemoryBank	—	6.6 / 5.2	9.7 / 7.0	5.6 / 5.9	5.0 / 4.8	7.4 / 6.5
MemGPT	—	41.0 / 34.3	25.5 / 19.4	9.2 / 7.4	26.7 / 17.7	43.3 / 42.7
A-MEM	35.94	44.7 / 37.1	45.9 / 36.7	12.1 / 12.0	27.0 / 20.1	50.0 / 49.5
D-MEM (ours)	37.4 / 31.2	21.6 / 16.4	43.4 / 12.1	10.4 / 33.7	42.7 / 31.9	43.1 / 42.5

Appendix

Table 2: Full baseline comparison on the clean LoCoMo Benchmark. Metrics are F1 / BLEU-1 (%). Bold indicates the best result among memory-augmented methods per column. D-MEM leads on Overall F1 and Multi-hop by a substantial margin; the Single-hop gap relative to A-MEM is discussed in Section 6.1 and Section 7 .

Figure 4: Attention heatmap: cosine similarity between query turns and memory slots. Each row corresponds to a dialogue turn (y-axis) and each column to a memory slot index (x-axis). The characteristic staircase-diagonal structure confirms that the active memory frontier advances monotonically as the session progresses. Vertical high-similarity bands (deep red) indicate memory slots that remain persistently salient across hundreds of subsequent turns—the physical substrate of Multi-hop reasoning. The left-side tier strip encodes the routing decision for each turn: gray ( SKIP ), blue ( CONSTRUCT_ONLY ), red ( FULL_EVOLUTION ).

Figure 5: D-MEM Memory Manifold (UMAP). Each point represents a memory entry, colored by Turn Number (purple $\to$ yellow) and shaped by routing tier: CONSTRUCT (star), EVOLVE (circle), SKIP (cross). Blue diamonds mark LTM (final) nodes retained in the persistent knowledge graph; gray dots mark STM (final) buffer entries. LTM nodes occupy dense, well-separated topical clusters across the manifold, while STM entries form a compact, isolated region—indicating that the hierarchical routing produces a structurally stable latent space rather than representation collapse.

Method	Overall	Single	Temp	Open	Multi	Adv	Skip Rate	Total Tokens
D-MEM-v1 (Linear)	0.334	0.235	0.434	0.110	0.354	0.354	9.6%	489K
D-MEM-v2 (Strict)	0.291	0.172	0.338	0.081	0.327	0.341	68.5%	497K
D-MEM-v3 (Safety Net)	0.324	0.235	0.455	0.101	0.345	0.312	41.1%	531K
D-MEM-v4 (Final)	0.369	0.246	0.440	0.085	0.412	0.412	41.1%	319K

Appendix

Table 3: Ablation study of D-MEM variants under extreme noise conditions (

\rho=0.75

Agent Memory Management

Agent memory management studies how AI agents store, update, retrieve, compress, and forget information across long-running tasks.

See all 446 papers

D-Mem: A Dual-Process Memory System for LLM Agents
Mar 19, 2026Zhixing You, Jiachen Yuan, Jason CaiAgent Memory Management Memory-Augmented Language Model Agents
Einstein Institute of Mathematics The Hebrew University of Jerusalem, Israel · Independent Researcher · AWS AI
MemFit: Efficient Long-Term Agentic Memory
Oct 1, 2026Mitchell Piehl, Muchao YeLLM Agent Memory Persistent Memory for Language Models
Department of Computer Science The University of Iowa
LeanMem: Simple and Efficient Long-Term Memory for LLM Agents
Aug 4, 2026Yuxin Liao, Le Wu, Min Hou +3LLM Agent Memory Persistent Memory for Language Models
Hefei University of Technology

Mar 19, 2026J/K move · Enter open · S save

D-Mem: A Dual-Process Memory System for LLM Agents

Zhixing You, Jiachen Yuan, Jason Cai

Einstein Institute of Mathematics The Hebrew University of Jerusalem, Israel · Independent Researcher · AWS AI

Agent Memory Management Memory-Augmented Language Model Agents Long-Term Conversational Memory

Open paper AskPDF arXiv

Driven by the development of persistent, self-adapting autonomous agents, equipping these systems with high-fidelity memory access for long-horizon reasoning has emerged as a critical requirement. However, prevalent retrieval-based memory frameworks often follow an incremental processing paradigm that continuously extracts and updates conversational memories into vector databases, relying on semantic retrieval when queried. While this approach is fast, it inherently relies on lossy abstraction, frequently missing contextually critical information and struggling to resolve queries that rely on fine-grained contextual understanding. To address this, we introduce D-Mem, a dual-process memory system. It retains lightweight vector retrieval for routine queries while establishing an exhaustive Full Deliberation module as a high-fidelity fallback. To achieve cognitive economy without sacrificing accuracy, D-Mem employs a Multi-dimensional Quality Gating policy to dynamically bridge these two processes. Experiments on the LoCoMo and RealTalk benchmarks using GPT-4o-mini and Qwen3-235B-Instruct demonstrate the efficacy of our approach. Notably, our Multi-dimensional Quality Gating policy achieves an F1 score of 53.5 on LoCoMo with GPT-4o-mini. This outperforms our static retrieval baseline, Mem0

^\ast

(51.2), and recovers 96.7% of the Full Deliberation's performance (55.3), while incurring significantly lower computational costs.

Abstract

Agent Memory Management

Explore similar work

D-Mem: A Dual-Process Memory System for LLM Agents

MemFit: Efficient Long-Term Agentic Memory

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents