MemFuse: Multi-Source Memory Fusion from Fragmented Observations

Chao Li; Yuanfa Li; Wenhao Wu; Xule Liu; Zhi Wang; Kun Shao

cs.CLAug 19, 2026

MemFuse: Multi-Source Memory Fusion from Fragmented Observations

Authors: Chao Li, Yuanfa Li, Wenhao Wu, Xule Liu, Zhi Wang, Kun Shao

Organizations: Xiaomi Inc. · Nanjing University

Abstract

Long-term memory is essential for agents that operate across extended interactions, yet existing memory systems and benchmarks predominantly focus on single-source textual histories. In realistic settings, however, relevant information is often fragmented across applications and devices, as well as across users and time, requiring agents to integrate dispersed observations into coherent episodic memories while preserving their source provenance. To address these gaps, we introduce MemFuseBench, a benchmark for multi-source memory fusion. MemFuseBench is built with a Scene-to-Sensor pipeline that synthesizes controllable scenarios into source-tagged observations, evidence-grounded questions, and adversarial distractors. It enables systematic evaluation of temporal reasoning, cross-source evidence fusion, and robustness to noise. We further propose MemFuse, a structured memory system that preserves source-level evidence in event-layer atomic memory and organizes related atomic events into cluster-layer fused memory within a causal fusion graph. During retrieval, MemFuse retrieves and organizes related evidence fragments while maintaining traceability to original source events. Experiments on MemFuseBench show that MemFuse achieves the best overall performance among the evaluated memory systems under all three LLM settings and consistently improves performance on questions requiring cross-source evidence fusion.

Figures & tables

Figure 1: A MemFuseBench instance requiring fusion across a household conversation, work calendar, and phone call, while rejecting a plausible distractor on the wrong day.

Figure 2: Overall MemFuseBench construction framework. The left panel summarizes the construction pipeline , whose six stages are organized into three phases: event construction, QA generation, and noise injection. The right panel shows reviewer-corrector validation , where candidate samples are checked, revised, and iterated until the final accepted sample is produced.

Category	#Q	Avg. Events	Description
Cross-source causality (Causal)	63	8.5	Infer causal relations across source observations.
Cross-source fusion (Fusion)	71	15.2	Combine partial observations across sources.
Cross-user aggregation (User Agg.)	52	11.4	Aggregate evidence across users.
Cross-user query (User Query)	61	7.1	Answer queries about another user’s events.
Conflict arbitration (Conflict)	70	2.8	Resolve conflicting source reports.
Perspective difference (Perspective)	40	13.3	Account for different user or source perspectives.

Table 1: Per-category statistics of the current MemFuseBench release. Short labels in parentheses are used in result tables. #Q is the number of questions; Avg. Events is the average number of distinct evidence events required per question.

Figure 3: Overall MemFuse framework. Left: online construction from source-tagged events through agentic fusion. Middle: a dual-layer causal fusion graph connecting atomic events and fused memories. Right: fusion-aware retrieval that returns traceable event-layer evidence for grounded answering.

		Answer Checklist Score							Token Usage (M)
System	Setting	Overall	Causal	Fusion	User Agg.	User Query	Conflict	Perspective	Ingest	Inference
Qwen3-30B-A3B
Long context	all	0.4424	0.4521	0.3737	0.3575	0.3874	0.6655	0.3531	–	40.83
Naive RAG	$k=20$	0.3178	0.2920	0.1690	0.2516	0.2939	0.5817	0.2832	–	0.90
A-MEM	$k=20$	0.3154	0.2956	0.1738	0.2397	0.3107	0.5903	0.2221	23.18	1.19
EverMemOS	$k=20$	0.3336	0.3336	0.2404	0.2717	0.3369	0.4899	0.3013	76.52	13.26

Table 2: MemFuseBench answer checklist scores and token usage. Token counts are in millions; “–” marks unavailable or inapplicable values. Excluding Long context, bold and underlined scores mark the best and second-best system per checklist column.

Panel of Figure 4 — Figure 4: Ablation results on MemFuseBench. Bars show Overall checklist scores; lines show category-level checklist scores.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Implementation
Atomic-event store	SQLite.
Dense index	L2-normalized, 1,024-dimensional BGE-M3 embeddings with FAISS.
Sparse index	BM25L with Jieba search-mode tokenization over atomic-event content and fused summaries.
Causal fusion graph	NetworkX.

Appendix

Table 3: Storage and indexing backends used by MemFuse.

Parameter group	Setting	Meaning
Retrieval budget	Seed top- $k=30$ , final top- $k=20$ .	Retrieval keeps 30 candidates before graph expansion, and the final reader context keeps 20 events.
Ranking constants	RRF constant $=60$ , $\beta=0.7$ .	Reciprocal-rank fusion is smoothed with 60; hop decay uses factor 0.7.
Graph expansion	Causal hops $=2$ , semantic hops $=1$ , semantic threshold $=0.8$ .	Causal traversal expands up to 2 hops, semantic traversal up to 1 hop, and semantic edges require similarity at least 0.8.
Fusion Agent	Accumulation zone (sliding window) $=10$ , max members per fused node $=10$ .	The session keeps a 10-turn sliding window over previously processed events, and each fused node keeps at most 10 member events.
Query planner	Confidence threshold $=0.6$ , rewrites $\leq 3$ .	The planner produces up to 3 rewritten queries and applies retrieval constraints only when confidence is at least 0.6.
Answer-time agentic retrieval	$2$ – $5$ rounds, per-round top- $k=20$ .	The retrieval agent may take 2–5 search rounds, and each round returns up to 20 candidates.

Appendix

Table 4: Key fixed parameters used by MemFuse in the reported experiments.

Agent Memory

Agent memory lets AI systems retain, update, and use facts, experiences, and skills across sessions and long-horizon tasks.

See all 192 papers

MemoryLACE: Memory Lifecycle-Aware Consolidation and Evidence Retrieval
Sep 2, 2026Meriem Yacoubi, Pia Schmidt, Nenad Petrovic +3Persistent Memory for Language Models Long-Term Conversational Memory
Chair of Robotics, Artificial Intelligence and Real-Time Systems, Technical University of Munich · inovex GmbH · Cerebras Systems Inc.
SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory
May 15, 2026Huacan Chai, Yukai Wang, Yingxuan Yang +7Multimodal Memory Multimodal Reasoning
Shanghai Jiao Tong University, China · OPPO, China
GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory
Aug 27, 2026Geng Li, Yuhao Wang, Dong Li +2Multimodal Memory Agent Memory Management
Wangxuan Institute of Computer Technology, Peking University · MemoraX AI

Sep 2, 2026J/K move · Enter open · S save

MemoryLACE: Memory Lifecycle-Aware Consolidation and Evidence Retrieval

Meriem Yacoubi, Pia Schmidt, Nenad Petrovic, Ahmed Frikha+2

Chair of Robotics, Artificial Intelligence and Real-Time Systems, Technical University of Munich · inovex GmbH · Cerebras Systems Inc.

Persistent Memory for Language Models Long-Term Conversational Memory Memory Consolidation for LLM Agents Agent Memory Management Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Long-term LLM agents must preserve information across interactions while distinguishing repeated evidence, historical states, updates, and unresolved contradictions. Existing textual memory systems retrieve semantically relevant memories efficiently but often leave these relationships implicit, whereas richer structured approaches model them through global graphs, hierarchical abstractions, or reflection at greater complexity. We introduce MemoryLACE (MemLACE), a lightweight memory framework that explicitly models the lifecycle of textual evidence through sparse merge, supersession, and contradiction relations while preserving atomic natural-language memories and their provenance. Rather than retrieving memories independently, MemLACE reconstructs relation-aware evidence units that expose current, historical, supporting, and conflicting evidence for downstream reasoning. Across BEAM and StructMemEval, using open-weight and proprietary LLM backbones, MemLACE achieves the highest overall performance in same-backbone comparisons while reducing end-to-end runtime on BEAM by 66.6% relative to Hindsight, the strongest reported reflective-memory baseline. Ablation studies identify lifecycle expansion and temporal awareness as the principal contributors to these gains. Together, the results demonstrate that explicitly modeling the local lifecycle of textual evidence is sufficient to substantially improve long-term memory reasoning without requiring comprehensive knowledge graphs or global reflection.

Abstract

Agent Memory

Explore similar work

MemoryLACE: Memory Lifecycle-Aware Consolidation and Evidence Retrieval

SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory

GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory