LLMs Remember First, Forget Last: Dual-Process Interference in Large Language Models

Sourav Chattaraj; Kanak Raj

cs.IRFeb 27, 2026

LLMs Remember First, Forget Last: Dual-Process Interference in Large Language Models

Authors: Sourav Chattaraj, Kanak Raj

Organizations: Thomson Reuters, Bengaluru, India

Abstract

Large language models can process millions of tokens, yet how they handle conflicting information within context remains poorly understood. From patient health logs tracking evolving vital signs to legal documents with superseding clauses, real-world applications routinely require models to retrieve specific values from streams of semantically similar, competing updates. We adapt classical interference paradigms from cognitive psychology to compare retroactive interference (RI; recalling initial values after updates) and proactive interference (PI; recalling recent values despite competing prior encodings) across 39 LLMs spanning 1B to 2.5T parameters. Every model exhibits the same pattern: PI causes substantially greater degradation than RI (d = 1.73), the opposite of the typical human finding where new information more readily disrupts old. Four lines of evidence indicate that RI and PI engage distinct mechanisms: model size predicts RI resistance but not PI; the two scores correlate only weakly; error analysis reveals qualitatively different failure profiles, with RI failures reflecting passive retrieval failure and PI failures reflecting active primacy intrusion; and hallucination rates remain below 1% in both conditions. An adaptive bootstrap design further shows that RI decays smoothly while PI collapses bimodally at high interference, consistent with gradual dilution versus winner-take-all attention dynamics. These findings characterize a systematic primacy bias in transformer attention, with implications for reliable deployment in settings where in-context information evolves over time.

Figures & tables

Figure 1: Interference Asymmetry Across 39 LLMs. RI accuracy (left) stays near ceiling through $N$ =50 before declining to 25% at $N$ =300; PI accuracy (right) drops from 94% to 3%, near-total failure. Every model shows PI $>$ RI, the inverse of the dominant human pattern MacLeod (2024) . Shaded bands indicate 95% bootstrap CIs.

Figure 2: Shared-Stimulus Design. Both conditions receive the same sequence of 46 initial pairs plus $N{-}1$ updates per category; RI probes the first value (green), PI the most recent (red). Identical stimuli ensure that performance differences reflect interference direction, not search difficulty.

Top 5 (Most Resistant)				Bottom 5 (Least Resistant)
Model	RIES	@3	@300	Model	RIES	@3	@300
o1	186.4	100	100	Llama-3.2-3b 1 1 1 Llama-3.2-3b produced unparseable output at $N$ =3 across all sessions (0% extracted accuracy); higher $N$ values yielded valid responses.	113.6	0	76
o1-preview	186.4	100	99	Llama-3.2-1b	126.1	74	2
GPT-5	186.4	100	38	Nova-micro	126.1	95	2
o3	185.8	100	100	GPT-4.1-nano	128.2	83	7
o3-mini	183.3	100	68	Llama-3.1-8b	128.3	59	14

Table 1: Range of RI Resistance. The five highest- and five lowest-scoring models by RIES, with accuracy at minimal (

N

=3) and maximal (

N

=300) interference.

Figure 3: Scale Helps RI but Not PI. (A) Parameter count accounts for 49% of variance in RIES ( $p<0.0001$ ). (B) Context window length is uninformative ( $R^{2}=0.003$ , n.s.). The same regression applied to PIES yields $R^{2}=0.06$ (n.s.), confirming that model size does not predict PI resistance.

Panel of Figure 4 — Figure 4: Decay Curves by Model Size: RI (left) vs PI (right). Both plots use the same 39 models with complete paired data. RI shows clear size stratification (L tier $>$ 50% at $N$ =300, XS $<$ 25%). PI shows no such ordering: the M tier outperforms the larger L tier, confirming that size predicts RI ( $R^{2}=0.49$ ) but not PI ( $R^{2}=0.06$ ).

Figure 5: Dissociation Between RI and PI Resistance. (A) RIES vs. PIES across 39 models: the correlation is weak ( $r=0.21$ , 95% CI $[-0.11,0.49]$ , $p=0.20$ ) and every model falls above the identity line (RIES $>$ PIES throughout). (B) Bland-Altman analysis shows a systematic offset of +69.5 points (95% limits of agreement: $-9.4$ to $+148.5$ ), confirming that the two scores capture different aspects of model performance.

Error Type	RI (%)	PI (%)
Retrieval failure	50.8	42.5
Same-key intrusion	46.0	56.1
Cross-key intrusion	0.6	0.7
Hallucination	0.9	0.2
Partial match	1.8	0.7

Table 2: How RI and PI Errors Differ.

	RI Resistance	PI Resistance	Key Insight
Reasoning models	High (+18%)	Lowest	Consolidation $\neq$ retrieval
Large dense	Strong	Strong	Balanced interference profile
Parameter count	Predictive	Not predictive	Capacity helps RI only
Context length	No effect	No effect	Buffer size irrelevant

Table 3: Summary of Key Empirical Patterns. Derived from a single paradigm; application-specific validation is needed before generalization.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	B	A	RIES	PIES	Model	B	A	RIES	PIES
GPT-5	2500	M	186.4	99.2	GPT-3.5-turbo	175	M	156.1	120.0
GPT-5-mini	2500	M	162.5	109.1	Llama-4-scout	109	M	153.9	40.2
GPT-5-nano	2500	M	160.6	21.0	Gem-2.0-flash	100	M	162.5	101.6
o3 R	2000	M	185.8	76.0	Gem-2.0-fl-lite	100	M	143.9	98.6
o3-mini R	2000	M	183.3	75.6	Llama-3.2-90b	90	D	134.6	87.1
o1 R	1760	M	186.4	22.0	Nova-pro	70	D	157.5	63.7

Appendix

Table 4: All 39 Models with RIES and PIES Scores. Sorted by parameter count (B = billions). A = architecture (D = Dense, M = MoE). R = reasoning model. Cl = Claude, Gem = Gemini, DS = DeepSeek.

Figure 6: Positional Origins of Intrusion Errors. At $N$ =3, RI intrusions originate from late positions (0.72); PI from early positions (0.12). Both converge toward 0.5 as $N$ grows.

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 789 papers

In-context superposition: human-like working memory interference in large language models
Apr 1, 2026Hua-Dong Xiong, Li Ji-An, Jiaqi Huang +3Working Memory in Language Models LLM Interpretability
School of Psychological and Brain Sciences, Georgia Tech · Department of Psychology, New York University · Department of Cognitive Science, Indiana University Bloomington +3
Language Models Share a Readout Bottleneck in Working Memory
Oct 15, 2026Dongyu GongWorking Memory in Language Models Attention Mechanisms
Yale University
Learning to Forget: Sleep-Inspired Memory Consolidation for Resolving Proactive Interference in Large Language Models
Mar 15, 2026Ying XieKV-Cache Management Knowledge Conflicts in Language Models
Kennesaw State University

Apr 1, 2026J/K move · Enter open · S save

In-context superposition: human-like working memory interference in large language models

Hua-Dong Xiong, Li Ji-An, Jiaqi Huang, Robert C. Wilson+2

School of Psychological and Brain Sciences, Georgia Tech · Department of Psychology, New York University · Department of Cognitive Science, Indiana University Bloomington+3

Working Memory in Language Models LLM Interpretability Representation Geometry in Language Models

Open paper AskPDF arXiv

Intelligent systems must maintain and manipulate task-relevant information online to adapt to dynamic environments. This capacity, known as working memory, is fundamental to human reasoning. Yet, human working memory is strikingly limited, maintaining only three to four items in a brain with billions of neurons. Surprisingly, large language models (LLMs), despite different substrates and direct access to prior context through attention, exhibit similar working memory limitations. Why should such different systems face analogous constraints? We propose that working memory limitations reflect a general trade-off of shared representations: representational compression and reuse support efficient learning and generalization, but also cause simultaneously active representations to interfere. We show a two-layer transformer trained on a working memory task can solve it perfectly, but diverse trained LLMs exhibit human-like limitations: performance declines with memory load, while retrieval is biased by recency and stimulus statistics. Mirroring humans, working memory performance in LLMs is also associated with broader model capability. Mechanistically, we show that LLMs encode multiple memories in entangled representations --- a condition we call in-context superposition --- and progressively suppress competing content while aligning the target with the readout. Moreover, a causal intervention that suppresses interfering information improves performance. Together, these findings suggest that working memory capacity reflects the ability to select task-relevant information under interference, a computational challenge shared by biological and artificial systems.

Abstract

LLM Reliability

Explore similar work

In-context superposition: human-like working memory interference in large language models

Language Models Share a Readout Bottleneck in Working Memory

Learning to Forget: Sleep-Inspired Memory Consolidation for Resolving Proactive Interference in Large Language Models