MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings

Himchan Hwang; Hyeokju Jeong; Gene Chung; Seungyeon Kim; Sangwoong Yoon; Frank Chongwoo Park

cs.LGMay 17, 2026

MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings

Authors: Himchan Hwang, Hyeokju Jeong, Gene Chung, Seungyeon Kim, Sangwoong Yoon, Frank Chongwoo Park

Organizations: Seoul National University · Saige · Ulsan National Institute of Science and Technology (UNIST)

Abstract

We propose MATE, a simple yet effective memory architecture for solving Contextual Markov Decision Processes (CMDPs), a family of MDPs parameterized by an unobserved context. In CMDPs, an optimal agent can adapt online by maintaining the posterior belief over contexts. MATE replaces this intractable posterior with a sum-aggregated memory, leveraging the posterior's permutation invariance to retain provably sufficient expressiveness. Compared to prior memory architectures, MATE avoids the growing per-step rollout cost of Transformers and the gradient issues commonly associated with Recurrent Neural Networks (RNNs). Extensive evaluations across diverse benchmarks demonstrate that MATE provides clear computational advantages while achieving performance comparable to standard sequence-model baselines.

Explore similar work

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Recurrent Model Efficient Long-Context Inference
Multi-Head Recurrent Memory Agents
Jul 1, 2026Jiatong Li, Samuel Yeh, Sharon LiRecurrent Model Efficient Long-Context Inference
Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
May 7, 2026Haodong Liang, Lifeng LaiIn-Context Learning Transformer Architectures

Aug 12, 2026 · cs.LGJ/K move · Enter open · S save

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua+5

Shanghai AI Laboratory · Tsinghua University · Fudan University

Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length. This flexibility, however, incurs a quadratic computation complexity during training and a key--value cache that grows linearly during autoregressive inference. Recurrent alternatives offer efficient decoding by compressing the entire history into a fixed-size state, but often underperform on recall-intensive tasks since earlier associations usually get overwritten by subsequent updates, and only the most recent contextual information is retained. In this paper, we introduce Memory-Anchor Routing across Context History (MARCH), a network architecture that effectively scales state-space models beyond a fixed-size dimension, while maintaining computational efficiency over long-sequences. MARCH periodically caches cumulative recurrent-state checkpoints as state anchors and associates each anchor with a compact, content-conditioned anchor key. This lets MARCH maintain a memory bank, which can grow as context length increases, providing a controllable trade-off between historical resolution and memory cost. At each token, MARCH produces an anchor query to attend all causally available state anchors, and the output is calculated as an attention-style aggregation over all historical anchors along the current state. We show that after standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning, LongBench, and in-context retrieval. These results demonstrate that content-routed state caching substantially strengthens recurrent long-range memory while preserving its native computation path.

Recurrent Model Efficient Long-Context Inference Residual Anchoring Token-Level Uncertainty

Open paper Ask about it PDF

Abstract

Explore similar work

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

Multi-Head Recurrent Memory Agents

Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement