Memory-Augmented Language Models

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 142% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 196

All topics

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement
Sep 15, 2026Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang +1Continual Learning for LLMs Memory-Augmented Language Models
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Sep 14, 2026Albert Ge, Chandan Singh, Yufan Zhuang +3Memory-Augmented Language Models Diffusion Language Models
University of Wisconsin–Madison · Microsoft Research · UC San Diego
MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup
Sep 14, 2026Muchen Li, Leonid Sigal, Renjie LiaoMemory-Augmented Language Models Persistent Memory for Language Models
University of British Columbia · 2Vector Institute for AI · 3Canada CIFAR AI Chair +1
ConvMem: Convolutional Memory for Long-Context Reasoning
Sep 9, 2026Hongming Zhang, Zhaozhen Gu, Fengshuo Bai +6Memory-Augmented Language Models Long-Context Modeling
Institute of Automation, Chinese Academy of Sciences
From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora
Sep 9, 2026Christoph Wigbels, Ali Abusaleh, Markus T. Jansen +2Memory-Augmented Language Models Memorization in Language Models
Bergische Universität Wuppertal · Goethe-Universität Frankfurt
Less Is Personal: Learning Minimal Sufficient User Profiles for Personalized Language Models
Sep 8, 2026Minghang Liu, Qiang Qiu, Yuanzhuo Wang +2LLM Personalization Memory-Augmented Language Models
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences, Beijing, China
Separating Stream Stability from Long-Term Recall in Language Models
Sep 7, 2026Peipei Cao, Xin Zhang, Jie Tang +3Memory-Augmented Language Models Long-Context Language Model Evaluation
School of Computer and Electronic Information, Guangxi University · School of Information Science and Engineering, Chongqing Jiaotong University · School of Computer Science and Technology, Guangdong University of Technology
ProtLingo: Efficient Protein Language Modeling via Conditional Memory and Expert Routing
Sep 7, 2026Mingrui Li, Sixian Shen, Minzhang Li +4Mixture-of-Experts Language Models Protein Fitness Prediction
ShanghaiTech University
RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory
Sep 3, 2026Yuxiang Wang, Kunyu Feng, Yingda Shen +3Memory-Augmented Language Models Recurrent Transformers
The Chinese University of Hong Kong, Shenzhen · The Chinese University of Hong Kong · Tianjin University +2
Replacing Training with Memory: Listwise Selection for Text-to-SQL
Sep 1, 2026Yeonseok Jeong, Soyoung Yoon, Seongjun Lee +1Memory-Augmented Language Models Learning to Rank
IPAI, Seoul National University · KAIST
Safin-1: Safety from Within through Memory-Native State Evolution
Aug 31, 2026Ming Zhang, Kaisen Yang, Shu Yu +15Memory-Augmented Language Models LLM Safety Alignment
Shanghai AI Laboratory
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
Aug 31, 2026Asa ShepardLLM Inference Efficiency Memory-Augmented Language Models
Department of Computer Science, Williams College
Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache
Aug 31, 2026Tong Yuan, Chengxi Liao, Zeyi WenMemory-Augmented Language Models KV Caching
Data Science and Analytics Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou)
When Errors Become Memories: Causal Pathway Tracing in Multi-Turn Memory-Augmented LLMs
Aug 31, 2026Shuyao Xiao, Shengling Wang, Xuan Chen +8Memory-Augmented Language Models Causal Interventions in Language Models
College of Artificial Intelligence, Beijing Normal University, Beijing, China · 2Beike Language and Intelligence, Beijing, China · Center for Data Science, New York University, New York, NY, USA
Memory Is Not Always Needed: Characterizing Conditional Memory in Scientific Reasoning
Aug 25, 2026Zhen Bi, Xueshu Chen, Yan Wang +6Memory-Augmented Language Models Scientific Reasoning
Huzhou Normal University · Alibaba Group · Bota Biosciences +1
Does Episodic Memory Help Close the Lexical Frequency Gap in Sensitivity to Syntactic Contrasts? A Test Using Retrieval-Augmented Language Models
Aug 24, 2026Jing Liu, Najoung KimMemory-Augmented Language Models Language Modeling
CoML Team, LSCP ENS, Université PSL, EHESS, CNRS · Department of Linguistics Boston University
Frozen Memory Is Not Enough: Rethinking External Memory as Extraction
Aug 17, 2026Mingyuan Li, Guangsheng Yu, Xu Wang +1Memory-Augmented Language Models Knowledge Augmentation for Language Models
ELLIS Institute Finland · University of Turku · University of Technology Sydney
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Long-Context Retrieval In-Context Retrieval
Shanghai AI Laboratory · Tsinghua University · Fudan University
LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining
Aug 12, 2026Qiuwu Chen, Zimo Liu, Yuchen Li +8Language Model Pretraining Memory-Augmented Language Models
AIGCode · South China University of Technology · Pazhou Laboratory
EvoMem: Memory-Augmented Evolution for Code Optimization
Aug 11, 2026Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8Memory-Augmented Language Models Evolutionary Optimization
AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia
AquiLLM: An Architecture for Supporting Tacit Knowledge Capture in Research Groups
Aug 9, 2026Jack Stark, Srinath Saikrishnan, Vikram Seenivasan +3Memory-Augmented Language Models Knowledge Augmentation for Language Models
University of California, Los Angeles, Los Angeles, California, USA · Southern Oregon University, Ashland, Oregon, USA
PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
Aug 4, 2026Dawei Liu, Haixu Song, Shuang Cheng +9Memory-Augmented Language Models Long-Context QA
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University +3
Maglev: Sliding Recurrent Memory
Aug 3, 2026Bo Liu, Qiang LiuMemory-Augmented Language Models Next-Token Prediction
The University of Texas at Austin
Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection
Aug 3, 2026Anusha Madan Gopal, Aras Pirbadian, Kristofor D. Carlson +2Retrieval-Augmented Generation On-Device Language Model Inference
BrainChip Inc. 23041 Avenida de la Carlota, Laguna Hills, CA
LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference
Aug 3, 2026Zhichen Liu, Ruihan Sun, Hengjie Yang +4Memory-Augmented Language Models LLM Inference
Southern University of Science and Technology · Xidian University
Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling
Aug 3, 2026Qinwen Wang, Jieping Luo, Aoxiang Qin +5Long-Context Retrieval Memory-Augmented Neural Networks
The Chinese University of Hong Kong, Shenzhen · Hainan Bielefeld University of Applied Sciences · BrainGalaxy
DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
Aug 3, 2026Yixiao Qian, Song Chen, Pengkai Wang +3Memory-Augmented Language Models KV Caching
College of Control Science and Engineering, Zhejiang University · Department of Mathematics, National University of Singapore · Hong Kong Polytechnic University +1
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Aug 3, 2026Shicheng Xu, Liang Pang, Liyi Chen +7Retrieval-Augmented Generation Continual Learning for LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Xiaohongshu Inc.
Tokenizer-Agnostic Engram Module
Jul 31, 2026Jia Peng Lim, Hai Leong ChieuMemory-Augmented Language Models Language Modeling
Singapore Management University · DSO National Laboratories
TransMem: Transforming Hidden States into Memory for Large Language Models
Jul 31, 2026Haodong Lei, Junming Liu, Yirong Chen +4Memory-Augmented Language Models LLM Agent Memory
Southeast University, Nanjing, Jiangsu, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China

Sep 15, 2026, cs.CLJ/K move · Enter open · S save

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang, Hsin-Hsi Chen

Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan

Continual Learning for LLMs Memory-Augmented Language Models Language Model Self-Improvement

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remains a key challenge. Existing memory-augmented approaches retrieve individual past examples as direct references, but do not explicitly synthesize actionable strategies from them, causing the same types of errors to recur. We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement. We evaluate DRPG across six benchmarks spanning text-to-SQL, question answering, medical diagnosis, and Python programming, using seven LLMs from both proprietary and open-weight families. DRPG outperforms strong baselines across most datasets and models. Further analysis demonstrates that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators. We also find that the benefit of policy-level guidance depends on task characteristics, offering practical insights into when and under what conditions this mechanism is most effective.

Momentum

Related topics

Latest papers 196