Surprisal Theory

Surprisal theory posits that the difficulty of processing linguistic units, like words, is directly related to their predictability within a given context, quantified as surprisal (negative log-probability). Current research focuses on refining surprisal's predictive power by investigating its relationship with other contextual factors, such as pointwise mutual information and frequency, and exploring its cross-linguistic validity using diverse language models. While surprisal theory successfully predicts processing time in many cases, recent studies highlight limitations, particularly in accurately capturing the magnitude of effects like garden-path sentences, suggesting that other factors beyond predictability influence language processing. This ongoing refinement of surprisal theory contributes to a deeper understanding of human language comprehension and its computational modeling.