Failures and Successes to Learn a Core Conceptual Distinction from the Statistics of Language

Zhimin Hu; Jeroen van Paridon; Gary Lupyan

cs.CLJul 5, 2026

Failures and Successes to Learn a Core Conceptual Distinction from the Statistics of Language

Authors: Zhimin Hu, Jeroen van Paridon, Gary Lupyan

Organizations: Department of Psychology, University of Wisconsin-Madison, Madison, USA

Abstract

Generic statements like "tigers are striped" and "cars have radios" communicate information that is, in general, true. However, while the first statement is true in principle, the second is true only statistically. People are exquisitely sensitive to this principled-vs-statistical distinction. It has been argued that this ability to distinguish between something being true by virtue of it being a category member versus being true because of mere statistical regularity, is a general property of people's conceptual machinery and cannot itself be learned. We investigate whether the distinction between principled and statistical properties can be learned from language itself. If so, it raises the possibility that language experience can bootstrap core conceptual distinctions and that it is possible to learn sophisticated causal models directly from language. We find that language models are all sensitive to statistical prevalence, but struggle with representing the principled-vs-statistical distinction controlling for prevalence. Until GPT-4, which succeeds.

Explore similar work

Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs
May 21, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model Performance Linguistics
Exemplars in Disguise: Pure Exemplar Models Mimic Abstraction-First Learning
Aug 1, 2026Zachary Nicholas Houghton, Vsevolod KapatsinskiAbstraction Training Language Models
Implicit Representations of Grammaticality in Language Models
May 6, 2026Yingshan Susan Wang, Linlu Qiu, Zhaofeng Wu +2Grammaticality Text Corpora

May 21, 2026 · cs.CLJ/K move · Enter open · S save

Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs

Dongxin Guo, Jikun Wu, Siu Ming Yiu

The University of Hong Kong · Hong Kong, China · Stellaris AI Limited

How do learners acquire knowledge of what is unacceptable without negative evidence? Construction Grammar proposes statistical preemption: exposure to a conventional form (e.g., "donated the books to the library") preempts structurally possible but unattested alternatives ("*donated the library the books"). We present a computational study that, for the first time, directly dissociates statistical preemption from the competing entrenchment hypothesis in large language models within a single converging design. Across four experiments spanning 120 English verb-construction pairings (dative, causative, locative), we show that (1) LLM surprisal patterns correlate strongly with human acceptability judgments (

r = 0.79

), validated against three independent behavioral datasets; (2) these patterns are driven by competing-form frequency rather than overall verb frequency, confirmed by non-circular partial correlations; (3) preemption sensitivity scales as a power law with model size; and (4) a controlled fine-tuning intervention causally demonstrates that manipulating competing-form frequencies shifts preemption behavior in the predicted direction, with reverse-direction controls ruling out frequency-sensitivity confounds. These results provide converging evidence that neural language models acquire negative linguistic knowledge through distributional competition, the core mechanism posited by Construction Grammar.

Large Language Model Performance Linguistics Hypothesis Construction

Open paper Ask about it PDF

Abstract

Explore similar work

Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs

Exemplars in Disguise: Pure Exemplar Models Mimic Abstraction-First Learning

Implicit Representations of Grammaticality in Language Models