cs.CLAug 9, 2024

A systematic framework for generating novel experimental hypotheses from language models

Authors: Kanishka Misra, Najoung Kim

Organizations: Department of Linguistics, The University of Texas at Austin, 305 E. 23rd Street #4.428, Austin, TX 78712, USA · Department of Linguistics, Boston University, 111 Cummington Mall #138P, Boston, MA 02215, USA

Abstract

Neural language models (LMs) have been shown to capture complex linguistic patterns, yet their utility in understanding human language and more broadly, human cognition, remains debated. While existing work in this area often evaluates human-machine alignment, few studies attempt to translate findings from this enterprise into novel insights about humans. To this end, we propose a systematic framework for hypothesis generation that uses LMs to simulate outcomes of experiments that do not yet exist in the literature. We instantiate this framework in the context of a specific research question in child language development: dative verb acquisition and cross-structural generalization. Through this instantiation, we derive two novel, untested hypotheses: (1) the alignment between argument ordering and discourse prominence features of exposure contexts modulates how children generalize new verbs to unobserved structures; and (2) that this alignment in the first postverbal argument of the exposure context has a more noteworthy effect. Additionally, we also design a set of experiments that can test these hypotheses in the lab with children. This work contributes both a domain-general framework for systematic hypothesis generation via simulated learners and domain-specific, lab-testable hypotheses for child language acquisition research.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Measuring Form and Function in Language Models

    May 27, 2026Héctor Javier Vázquez Martínez, Charles YangLLM EvaluationSyntactic Generalization in Language Models

  2. Child-directed speech facilitates production, not comprehension, in BabyLMs

    May 31, 2026Bastian Bunzeck, Sina ZarrießLanguage Model PretrainingLLM Evaluation