Working Memory in Language Models

Follow this topic Ask about this topic

Latest papers

14 papers1 paper in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Language Models Share a Readout Bottleneck in Working Memory
Oct 15, 2026Dongyu GongWorking Memory in Language Models Attention Mechanisms
Yale University
In-Context Binding Capacity in Language Models
Sep 24, 2026Manas Venkata Sai Ravulapalli, Samrath Singh ChadhaWorking Memory in Language Models Language Model Scaling Laws
Efficient Computation Inc.
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
Jul 27, 2026Maruthi Vemula, Neeraj Praneeth GajulaKV Caching KV-Cache Eviction
University of North Carolina at Chapel Hill · New York University
Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents
Jul 14, 2026Ning LiuMulti-Hop QA Agent Memory
Independent Researcher
Memory in the Loop: In-Process Retrieval as ExtendedWorking Memory for Language Agents
Jul 6, 2026Yusuf Khan, Carlo LipizziAgent Memory Management Memory-Augmented Language Model Agents
Stevens
Unlocking the Working Memory of Large Language Models for Latent Reasoning
May 28, 2026Lukas Aichberger, Sepp HochreiterImplicit Reasoning in Language Models Efficient Language Model Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria · NXAI GmbH, Linz, Austria
A Dual-Task Paradigm to Investigate Sentence Comprehension Strategies in Language Models
Apr 29, 2026Rei Emura, Saku SugawaraCognitive Modeling Working Memory in Language Models
Tohoku University · National Institute of Informatics · The University of Tokyo
Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity
Apr 22, 2026Pranava Madhyastha, Dagmar AdamcovaTransformer Attention Language Modeling
△City, University of London · ∞The Alan Turing Institute · ⊕Grounded Machines
Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions
Apr 8, 2026Parth Patil, Dhruv Kumar, Yash Sinha +1Working Memory in Language Models LLM Mathematical Reasoning
Department of Computer Science & Information Systems BITS Pilani · School of Computer Engineering, KIIT Bhubaneshwar
In-context superposition: human-like working memory interference in large language models
Apr 1, 2026Hua-Dong Xiong, Li Ji-An, Jiaqi Huang +3Working Memory in Language Models LLM Interpretability
School of Psychological and Brain Sciences, Georgia Tech · Department of Psychology, New York University · Department of Cognitive Science, Indiana University Bloomington +3
WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Tracking
Mar 28, 2026Dengzhe Hou, Lingyu Jiang, Deng Li +3Working Memory in Language Models LLM Evaluation
Tohoku University · Lappeenranta-Lahti University of Technology LUT · Texas A&M University +1
Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind
Mar 27, 2026Christopher AckermanWorking Memory in Language Models Theory of Mind
Are they human? Detecting large language models by probing human memory constraints
Mar 10, 2026Simon Schug, Brenden M. LakeWorking Memory in Language Models
Department of Computer Science, Princeton University · Department of Psychology, Princeton University
MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Models
Mar 2, 2026Kejing Xia, Mingzhe Li, Lixuan Wei +5Diffusion Language Models Persistent Memory for Language Models
Georgia Institute of Technology · University of Massachusetts Amherst · Harvard University

Oct 15, 2026J/K move · Enter open · S save

Language Models Share a Readout Bottleneck in Working Memory

Dongyu Gong

Yale University

Working Memory in Language Models Attention Mechanisms LLM Interpretability

Open paper AskPDF arXiv

Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position. We ask whether this shared behavioral signature reflects a shared mechanism. The literature offers two candidate accounts. On a precision account, softmax attention cannot concentrate its weight on the item N positions back, so adjacent items are retrieved instead. On an interference account, attention selects the correct item, but the content of competing items corrupts the decision. We dissociate these accounts in 15 pretrained LLMs from three families (70 million to 14 billion parameters) and in minimal transformers trained from scratch. In all three families, the attention heads that carry the task attend to the target on most missed matches. Linear probes decode near-perfectly whether a matching letter occurred at the target or at a lure position, even on trials where the model false-alarms to the lure, although its own decisions barely distinguish the two. Sharpening attention yields negligible gains, whereas suppressing the letter identity of non-target items improves sensitivity (d') in nearly all model-by-load conditions. In larger models the task is spread over more heads, and removing interference helps more. In minimal transformers, precision limits a single attention layer, but a second layer removes this constraint. The working-memory limit of LLMs therefore lies in reading items out under interference rather than in locating them, consistent with interference accounts of human working memory.