Language Models Keep Abandoned Analyses: Residue and Adaptation after Garden-Path Sentences

Xiang Ding

cs.CLOct 27, 2026

Language Models Keep Abandoned Analyses: Residue and Adaptation after Garden-Path Sentences

Authors: Xiang Ding

Organizations: Nagoya University

Abstract

Reading "While the man hunted the deer ran into the woods" requires giving up the analysis in which "the deer" is the object of "hunted". People who have just read such a sentence still produce more transitive structures, so the abandoned analysis primes later language. We ask whether language models keep using an analysis they have abandoned, and what in the model carries it. In 24 open models from 124M to 70B parameters, a garden-path sentence leaves two traces in the context. A residue of the abandoned analysis raises later object attachment, by about three quarters of the effect of a sentence in which the noun phrase really is an object, and it appears in sampled text, on held-out and human-normed items and for a second type of garden path. An adaptation toward the resolved structure scales with the sentence's disambiguation error and acts mainly on comprehension-like measures. Activation patching separates the two. Later text reads the residue from the role information in the noun phrase's states, which were computed under the object analysis before the disambiguation, and replacing these states removes almost all of the residue in the 32B and 70B models. The adaptation is read from the states that processed the disambiguation. Overridden statements show the same persistence (a corrected value keeps pulling later predictions in 8 of 9 models), and in most models the pull is carried by the positions after the statement. Finally, the human paradigm transfers only with controls for punctuation, because the models also write garden-path sentences themselves, which reverses the paradigm's raw statistic.

Figures & tables

Panel of Figure 1 — Figure 1: Two traces of a garden-path sentence in a causal language model. Later text reads a residue of the abandoned object analysis from the states of the deer , which were computed before the disambiguation (§ 6 ). The main-clause states carry an adaptation toward the resolved structure. The two traces push later processing in opposite directions, and their balance differs between production-like and comprehension-like measures (§ 5 ).

Prime	Example
gp	While the man hunted the deer ran into the woods.
cue	While the man hunted, the deer ran into the woods.
tb	While the man hunted the deer it ran into the woods.
nb / nc	While the man slept(,) the deer ran into the woods.
nom	While the man hunted they ran into the woods.
pass	While the man was hunted the deer ran into the woods.

Table 1: Prime conditions and target quantities, illustrated with one prime and one target family. nom and pass test punctuation and local coherence (§ 4 ).

Figure 2: The residue of the abandoned object analysis. Left:

A_{L}

(Eq. 2 ) per model on the discovery, held-out and Christianson et al. (2001) item sets. Right: the same contrast on sampled completions, as a difference in the proportion of transitive continuations. Error bars are 95% intervals from a two-way bootstrap over prime and target families. Models are ordered by size.

Figure 3: Patching the garden-path prime. Left: residue on

L

retained when the target reads counterfactual subject-analysis states at the noun phrase (blue) or at the main clause (orange). Right: noun-phrase states replaced only from a given fraction of the network’s depth onward. Values for both measures, with intervals, are in Appendix E .

Figure 4: Sampled production after garden-path primes. The raw contrast with comma primes used in the human study (orange) is negative in most Llama and Qwen models. Without the completions that are themselves comma-less garden-path sentences it is positive in every model (gray), and so is the controlled residue

A_{gen}

(blue).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Item-level predictors of the residue on the discovery set. Each point is the

t

value of one predictor in a regression of the family-level contrast

A

on the model’s object expectation after the ambiguous verb (blue) and on the garden-path strength of the prime, the surprisal of its disambiguating word without versus with the comma (orange). Dotted lines mark

|t|=1.96

. Object expectation raises

A

on both measures. Garden-path strength lowers it on the comprehension-like measure

S

in 21 of 22 models.

Experiment	Pre-specified prediction	Outcome
Held-out families	(i) object expectation predicts the residue on $S$ (ii) error predicts it with the sign found in discovery (iii) sign of the net residue	(i) 7/17 (not confirmed) (ii) 11/17, same sign in 16/17 (sign test $p=.0001$ ) (iii) 5/9
Within-family error	more surprising disambiguation lowers the residue in the models with a discovery error effect	confirmed in 3/3, and null in the 2 models predicted null
NP/S	the two forces generalize to NP/S garden paths	not confirmed (1/5 and 0/4)
MV/RR	error and commitment relations in main-verb/reduced-relative garden paths	(i) error 6/9 (ii) commitment 0/8
Case ( nom )	(i) object expectation predicts the residue without punctuation (ii) error relation negative (iii) comma part of the predictor vanishes	(i) 22/22 (ii) 20/22 (iii) not confirmed (reinterpreted as part of object expectation)
Measures	(i) residue on $L$ positive (ii) error relation stronger on $S$ than on $L$	(i) 16/16 (ii) 14/16

Appendix

Table 4: Pre-specified predictions and outcomes. Counts are models in which the prediction held. Outcomes that did not confirm the prediction are in italics.

Model	$A_{L}$	$\rho$ , NP	$\rho$ , MC	$\Delta A_{S}$ , NP	$\Delta A_{S}$ , MC
Pythia-1.4B	0.46 (0.33, 0.59)	0.65 (0.56, 0.73)	0.85 (0.76, 0.92)	$-$ 0.26 ( $-$ 0.30, $-$ 0.22)	0.22 (0.11, 0.34)
Pythia-2.8B	0.49 (0.34, 0.65)	0.50 (0.41, 0.59)	0.74 (0.64, 0.83)	$-$ 0.35 ( $-$ 0.42, $-$ 0.29)	0.40 (0.25, 0.55)
Llama-3.2-3B	0.82 (0.62, 1.03)	0.39 (0.30, 0.48)	0.83 (0.72, 0.95)	$-$ 0.51 ( $-$ 0.60, $-$ 0.42)	1.12 (0.92, 1.33)
Mistral-7B	0.86 (0.64, 1.09)	0.40 (0.28, 0.49)	0.89 (0.74, 1.10)	$-$ 0.55 ( $-$ 0.66, $-$ 0.46)	0.93 (0.74, 1.11)
Qwen2.5-7B	0.90 (0.66, 1.16)	0.14 ( $-$ 0.05, 0.27)	1.34 (1.12, 1.71)	$-$ 0.84 ( $-$ 0.99, $-$ 0.71)	1.13 (0.93, 1.33)
Llama-3.1-8B-Inst.	1.09 (0.87, 1.36)	0.28 (0.17, 0.38)	1.03 (0.91, 1.17)	$-$ 0.83 ( $-$ 0.96, $-$ 0.71)	1.33 (1.11, 1.55)

Appendix

Table 5: Patching results with 95% two-way bootstrap intervals.

\rho

: retained fraction of the residue on

L

\Delta A_{S}

: change of the residue on

S

in nats. NP and MC: the target reads counterfactual states at the noun phrase or at the main clause.

Syntactic Generalization in Language Models

See all 18 papers

LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms
Feb 4, 2026Dario Paape, Tal Linzen, Shravan VasishthLatent Variable Models Cognitive Modeling
Department of Linguistics, University of Potsdam · Department Linguistik, Universität Potsdam, Karl-Liebknecht-Straße 24–25, D-14476, Germany. · Center for Data Science/Department of Linguistics, New York University
Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis
May 14, 2026William Timkey, Brian Dillon, Tal LinzenSurprisal Theory Language Modeling
Department of Linguistics, New York University, 10 Washington Pl, New York, NY 10003, USA · Department of Linguistics, University of Massachusetts Amherst, 650 N Pleasant St, Amherst, MA 01002, USA · Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA
Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction
Sep 30, 2026Grayson Wycliffe Storer, Julia Witte ZimmermanText Embeddings Language Modeling
Department of Mathematics and Statistics. · Department of Computer Science, University of Vermont, Burlington, 05405, Vermont, United States. · Computational Story Lab. +1

Feb 4, 2026J/K move · Enter open · S save

LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms

Dario Paape, Tal Linzen, Shravan Vasishth

Department of Linguistics, University of Potsdam · Department Linguistik, Universität Potsdam, Karl-Liebknecht-Straße 24–25, D-14476, Germany. · Center for Data Science/Department of Linguistics, New York University

Latent Variable Models Cognitive Modeling Computational Psycholinguistics

Open paper AskPDF arXiv

Temporarily ambiguous garden-path sentences ("While the team trained the striker wondered... ") are known to cause processing difficulty, which can manifest itself in a variety of reading behaviors (in-situ slowdowns, rereading), as well as in miscomprehension or outright rejection of the sentence as ungrammatical. Which types of reading behavior are observed critically depends on the experimental method used to collect the data, which makes comparing results between reading paradigms difficult. To address this problem, we present a latent-process multinomial processing tree (MPT) model of human reading and comprehension/judgment behavior in garden-path sentences that we fit to combined data from four different reading paradigms (eye tracking, uni- and bidirectional self-paced reading, Maze). The model distinguishes between the probability of adopting an incorrect initial analysis, the cost of encountering an incompatible continuation, and the cost of syntactic reanalysis. By taking into account trials with inattentive reading, more realistic estimates of the cost parameters are obtained. Cross-validation reveals that the MPT model has a better predictive fit to human reading patterns and end-of-trial task data than a model based solely on LLM-derived surprisal values. We also test several models that assume an influence of surprisal within the MPT architecture, and find that adding surprisal as an additional predictor or reading time and/or garden-path cost further improves predictive fit.

Abstract

Syntactic Generalization in Language Models

Explore similar work

LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms

Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis

Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction