Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden-Paths and Digging-In Effects

Amani Maina-Kilaas; Roger Levy

cs.CLMar 12, 2026

Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden-Paths and Digging-In Effects

Authors: Amani Maina-Kilaas, Roger Levy

Organizations: Department of Brain and Cognitive Sciences, MIT

Abstract

Under surprisal theory, linguistic representations affect processing difficulty only through the bottleneck of surprisal. Our best estimates of surprisal come from large language models, which have no explicit representation of structural ambiguity. While LLM surprisal robustly predicts reading times across languages, it systematically underpredicts difficulty when structural expectations are violated -- suggesting that representations of ambiguity are causally implicated in sentence processing. Particle filter models offer an alternative where structural hypotheses are explicitly represented as a finite set of particles. We prove several algorithmic consequences of particle filter models, including the amplification of garden-path effects. Most critically, we demonstrate that resampling, a common practice with these models, inherently produces real-time digging-in effects -- where disambiguation difficulty increases with ambiguous region length. Digging-in magnitude scales inversely with particle count: fully parallel models predict no such effect.

Figures & tables

Figure 1: Expected surprisal during resampling. Assume only two structures,

T_{1}

and

T_{2}

, which give word

w

the in-context probability

Q(w\mid T_{1},C)=0.004

and

Q(w\mid T_{2},C)=0.5

, such that

w

strongly disambiguates to

T_{2}

. Using 25 particles, we simulate the change in surprisal with a context that prefers

T_{1}

\pi_{\text{AMB}}(T_{1}\mid C)=0.8

, and a context that prefers

T_{2}

\pi_{\text{UNAMB}}(T_{1}\mid C)=0.2

( not unambiguous in typical sense ). Expected surprisal is lower-bounded by

S\left(\pi\right)

, reflecting full parallelism, and upper-bounded by

\E_{\pi^{(N)}_{s_{\infty}}}\left[S\left(\pi^{(N)}_{s_{\infty}}\right)\right]

, at which point only a single structure is entertained by the parser. Shaded regions indicate ±1 stdev of the sample (50 thousand trials); 95% error bars would not exceed the line width.

Panel of Figure 2 — Figure 2: Expected surprisal (top) and garden-path effects (bottom) for a hypothetical digging-in experiment with short and long ambiguous regions, varying disambiguation strength (through $Q_{1}=Q(w\mid T_{1},C)$ ) and particles $N$ . Long versions typically add 2-3 words and so we use 2 resampling steps. Fixed parameters: $Q(w\mid T_{2},C)=0.5$ , $\pi_{\text{AMB}}(T_{1}\mid C)=0.8$ , $\pi_{\text{UNAMB}}(T_{1}\mid C)=0.2$ ( where preferred is correct ). Using 50 thousand trials; 95% error bars would not be visible.

Figure 3: Expected surprisal in early resampling steps, varying disambiguation strength (through

Q_{1}=Q(w\mid T_{1},C)

) and the number of particles

N

. Fixed parameters:

Q(w\mid T_{2},C)=0.5

\pi_{\text{AMB}}(T_{1}\mid C)=0.8

\pi_{\text{UNAMB}}(T_{1}\mid C)=0.2

(again, this is not unambiguous in the traditional sense : if there is truly only one compatible structure, then the expected surprisal is fixed). The second-order approximation is computed at each empirical sample of

\pi^{(N)}_{s_{i}}

and cumulatively summed from the empirical starting point

\E_{\pi^{(N)}_{s_{0}}}\left[S\left(\pi^{(N)}_{s_{0}}\right)\right]

. The linear-diffusion approximation applies a constant slope estimated from the empirical sample of

\pi^{(N)}_{s_{0}}

. Shaded regions indicate ±1 stdev of the sample (1 million trials); 95% error bars would not exceed the line width.

Figure 4: True per-step surprisal increase by approximated value, using the data from Figure 3 . Top: visualized on linear scale. Bottom: visualized on log-log scale.

Computational Psycholinguistics

Computational psycholinguistics models how people learn, produce, and understand language using formal theories, algorithms, and behavioral data.

See all 86 papers

Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths
Mar 24, 2026Amani Maina-Kilaas, Roger LevySurprisal Theory Computational Psycholinguistics
Department of Brain and Cognitive Sciences, MIT
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal
Apr 20, 2026Ryo Yoshida, Shinnosuke Isono, Taiga Someya +2Computational Psycholinguistics Surprisal Theory
♠The University of Tokyo · ♡NINJAL · ♢NII LLMC +1
Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis
May 14, 2026William Timkey, Brian Dillon, Tal LinzenSurprisal Theory Language Modeling
Department of Linguistics, New York University, 10 Washington Pl, New York, NY 10003, USA · Department of Linguistics, University of Massachusetts Amherst, 650 N Pleasant St, Amherst, MA 01002, USA · Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA

Mar 24, 2026J/K move · Enter open · S save

Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths

Amani Maina-Kilaas, Roger Levy

Department of Brain and Cognitive Sciences, MIT

Surprisal Theory Computational Psycholinguistics

Open paper AskPDF arXiv

Digging-in effects, where disambiguation difficulty increases with longer ambiguous regions, have been cited as evidence for self-organized sentence processing, in which structural commitments strengthen over time. In contrast, surprisal theory predicts no such effect unless lengthening genuinely shifts statistical expectations, and neural language models appear to show the opposite pattern. Whether digging-in is a robust real-time phenomenon in human sentence processing -- or an artifact of wrap-up processes or methodological confounds -- remains unclear. We report two experiments on English NP/Z garden-path sentences using Maze and self-paced reading, comparing human behavior with predictions from an ensemble of large language models. We find no evidence for real-time digging-in effects. Critically, items with sentence-final versus nonfinal disambiguation show qualitatively different patterns: positive digging-in trends appear only sentence-finally, where wrap-up effects confound interpretation. Nonfinal items -- the cleaner test of real-time processing -- show reverse trends consistent with neural model predictions.

Abstract

Computational Psycholinguistics

Explore similar work

Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths

An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal

Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis