MemoWM: How World Models Change What Agents Need to Remember

Bingfan Zeng; Zhisheng Chen; Chenbo Sang; Zhengwei Xie; Jinpeng Wang; Xiangchen Guan; Rui Qian; Zheng Lu

cs.LGOct 7, 2026

MemoWM: How World Models Change What Agents Need to Remember

Authors: Bingfan Zeng, Zhisheng Chen, Chenbo Sang, Zhengwei Xie, Jinpeng Wang, Xiangchen Guan, Rui Qian, Zheng Lu, +1 more

Organizations: South China University of Technology · University of the Chinese Academy of Sciences · Beijing University of Posts and Telecommunications · University of Science and Technology of China · Tsinghua University · Peking University · Fudan University · Shanghai Jiao Tong University

Abstract

Long-term agents face growing storage demands as they accumulate experience. World models capture reusable regularities that can reduce the information stored for each experience. We formulate the problem of memory allocation conditioned on a world model and introduce MemoWM, a framework that uses shared predictions to compress retained information and reconstruct omitted content. Its task-aware allocation rule balances the expected impact of reconstruction errors against storage cost, retaining information with downstream value beyond the predictive prior. Across five long-term agent-memory benchmarks, MemoWM achieves 42.42% average answer accuracy, exceeding the strongest baseline by 2.62 percentage points, while reducing average experience-specific storage by 53.9% relative to MIRIX, the most storage-efficient baseline. Further analysis shows that stronger world models reduce per-experience storage at comparable task quality. Accounting for model parameters reveals a trade-off between shared model capacity and recurring storage costs, with the capacity that minimizes total storage increasing as more interactions are retained. Our code is available at https://github.com/Feld-maxiu/MemoWM.

Figures & tables

Figure 1: Average accuracy versus experience-specific storage.

Figure 2: Overview of the MemoWM framework. Coordinates are retained when their estimated task impact exceeds the weighted storage cost; others are filled by a causal world model. (A) Task-relevant retention, (B) shared-history reconstruction, and (C) offline fitting and calibration. Reconstructed states are retrieved through the Reader Bridge to answer future queries.

Method	WorldMemArena	AMA-Bench	LME-V2	WebQuest	MemEye	Avg.
RAG	52.97 $\pm$ 0.12	44.87 $\pm$ 0.70	40.25 $\pm$ 0.69	22.35 $\pm$ 0.50	35.42 $\pm$ 0.93	39.17
UniversalRAG	40.38 $\pm$ 0.19	40.35 $\pm$ 0.88	34.56 $\pm$ 1.18	21.68 $\pm$ 0.75	37.95 $\pm$ 1.24	34.98
MEM1	45.66 $\pm$ 0.31	14.96 $\pm$ 1.64	29.41 $\pm$ 1.83	20.25 $\pm$ 1.01	32.05 $\pm$ 1.93	28.47
MemAgent	43.03 $\pm$ 0.37	29.18 $\pm$ 1.60	29.33 $\pm$ 1.91	16.90 $\pm$ 0.78	30.88 $\pm$ 2.34	29.86
A-Mem	55.37 $\pm$ 0.28	34.85 $\pm$ 1.31	41.28 $\pm$ 1.48	26.70 $\pm$ 0.85	36.33 $\pm$ 1.28	38.91
MemGPT	58.43 $\pm$ 0.20	35.44 $\pm$ 0.81	44.97 $\pm$ 1.29	24.55 $\pm$ 0.87	35.61 $\pm$ 1.16	39.80

Table 1: Main results across five long-horizon memory benchmarks. (a) The upper table reports answer accuracy (%); (b) the middle table reports the experience-specific storage rate (kbit/observation); (c) the break-even point is the number of retained observations at which MemoWM’s total storage is no greater than the listed comparator’s. LME-V2 denotes LongMemEval-V2, Omni-SM denotes Omni-SimpleMem, and Avg. is the unweighted mean across benchmarks.

	WMA		AMA-Bench		LME-V2
Variant	AC	Rate	AC	Rate	AC	Rate
No quantizer	60.33	262.5	43.75	263.0	43.04	272.2
Quantizer only	60.27	6.2	43.27	7.1	42.98	14.3
WM, all-send	60.27	5.6	43.27	3.5	42.98	11.2
MemoWM	60.18	4.3	43.58	2.3	42.91	9.4
M2A	50.35	1814.8	42.61	6.1	40.18	80.6

Table 2: Ablation and M2A integration. AC: accuracy (%); Rate: experience-specific storage (kbit/observation).

Panel of Figure 3 — Figure 3: (a) Upper left: all-coordinate coding rate and (b) upper right: gated total rate with amortized FP16 weights, both versus model size at three training-data fractions. (c) Lower left: all-coordinate rate under four predictive contexts; blue curves show increases over full context (upper axis). (d) Lower right: storage-optimal gated model size versus retained transitions.

Figure 4: Left: where decoder-side reconstruction quality comes from, with the omitted coordinates and storage cost held fixed so that only the fill source changes. Right: answer distortion and storage over four rollout-step ranges for teacher-forced oracle and closed-loop reconstruction.

Gate	$\|\Delta\mathrm{NLL}\|$	Rate	Rate $\uparrow$
All-send	0.000	3737.5	$+58.8\%$
MemoWM	1.536	2354.2	–
Static- $e$	1.536	2754.3	$+17.0\%$
Uniform- $V$	1.536	2954.8	$+25.5\%$
Random	1.536	3134.2	$+33.1\%$
Uncertainty	1.536	3342.1	$+42.0\%$

Table 3: Task-aware gating at matched distortion. Rate: bits/state; Rate

\uparrow

: increase over MemoWM.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Subset	Coords.	Low- $V_{j}$	Unseen query	Random
Bottom 5%	51	0.212	0.323	37.4
Bottom 10%	102	0.221	0.375	38.8
Bottom 15%	154	0.231	0.423	38.3
Bottom 20%	205	0.232	0.565	39.8

Appendix

Table 4: Held-out checks of the two risk components. (a) Answer-NLL damage conditioned on prediction errors for the original queries and independently constructed unseen-query variants (

10^{-3}

bits/answer). (b) Frozen entropy lookup versus observed prediction error (%).

Explore similar work

MemWM: Memory-Augmented Text-Based World Model
Aug 7, 2026Yujun Wang, Tao Zhang, Jinhe Bi +9LLM World Models World Model Learning
LMU Munich · Munich Center for Machine Learning (MCML) · Huawei Heisenberg Research Center +4
When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense
Jun 20, 2026Pranav SinghAgent Memory Bayesian Inference
Indian Institute of Technology Ropar
MemRefine: LLM-Guided Compression for Long-Term Agent Memory
Jun 11, 2026Minjae Kim, Jinheon Baek, Soyeong Jeong +1Persistent Memory for Language Models Long-Term Conversational Memory
Korea University · KAIST

Aug 7, 2026, cs.AIJ/K move · Enter open · S save

MemWM: Memory-Augmented Text-Based World Model

Yujun Wang, Tao Zhang, Jinhe Bi, Aniri+8

LMU Munich · Munich Center for Machine Learning (MCML) · Huawei Heisenberg Research Center+4

LLM World Models World Model Learning World Model-Based Planning Model-Based Planning World Model Evaluation Memory-Augmented Agents

Open paper AskPDF arXiv

World models are increasingly used to support planning in agents by predicting how environment states evolve in response to agent actions. Yet fluent next-state predictions can still omit task-critical facts, corrupt product attributes, or apply incorrect transition rules. To address such systematic prediction errors, we introduce MemWM, a memory-augmented text-based world model. MemWM uses world memory, a curated memory bank of transition rules, state caches, and hard-to-predict facts, to condition next-state imagination. We evaluate factual state preservation with Structured State Fidelity (SSF), which scores predicted states through benchmark-specific facts and fields. Compared with SFT, memory-augmented training improves SSF by up to 206.3%. In the full planning setting, we keep the policy model frozen and provide policy-side world skill: retrieved task-level skills and step-wise corrective guidance for action selection. Across ALFWorld, WebShop, and ScienceWorld, memory-augmented agents improve downstream success over an SFT-trained world-model agent, with up to a 65.4% relative gain. Sensitivity analyses further show that retrieved memory improves task success and efficiency under different memory and action-budget settings.

Abstract

Explore similar work

MemWM: Memory-Augmented Text-Based World Model

When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense

MemRefine: LLM-Guided Compression for Long-Term Agent Memory