cs.CLFeb 23, 2026

BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop

Authors: Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, +2 more

Organizations: IBM Research, MIT · ETH Zürich · Cornell University · University of Groningen · NYU · Boston University · University of Cambridge · Georgia Tech · UC San Diego · Georgetown University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The goal of the BabyLM is to stimulate new research connections between cognitive modeling and language model pretraining. We invite contributions in this vein to the BabyLM Workshop, which will also include the 4th iteration of the BabyLM Challenge. As in previous years, the challenge features two ``standard'' tracks (Strict and Strict-Small), in which participants must train language models on under 100M or 10M words of data, respectively. This year, we move beyond our previous English-only pretraining datasets with a new Multilingual track, focusing on English, Dutch, and Chinese. For the workshop, we call for papers related to the overall theme of BabyLM, which includes training efficiency, small-scale training datasets, cognitive modeling, model evaluation, and architecture innovation.

Figures & tables

DatasetDescription# Words (multimodal data)# Words (strict data)# Images
Localized Narratives Pont-Tuset et al. (2020)Image Caption27M–0.6M
Conceptual Captions 3M Sharma et al. (2018)Image Caption23M–2.3M
CHILDES ( MacWhinney, 2000 )Child-directed speech15M28.4M–
British National Corpus (BNC), dialogue portionDialogue4M7.6M–
Project Gutenberg (children’s stories) ( Gerlach and Font-Clos, 2018 )Written English13M25.5M–
OpenSubtitles ( Lison and Tiedemann, 2016 )Movie subtitles10M21.8M–
Table 1: Datasets for the strict track of the BabyLM competition. We release a text-only and multimodal dataset; either is allowed, as are outside corpora, so long as the total word count of the training corpus is less than 100M. Datasets have been detoxified following the procedure of Trhlik et al. (2026) (§ 4.3 ).

Explore similar work

CardsList

  1. The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese

    Jul 12, 2026Siyuan Song, Zhiheng Qian, Yunhao Zhang +11Language Model PretrainingLLM Evaluation

    Princeton University · Shanghai Jiao Tong University · Chinese Academy of Sciences +5

  2. Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training

    Oct 14, 2026JiWoo Hwang, Lukas Edman, Alexander FraserLanguage Model PretrainingMultilingual Language Models

    School of Computation, Information and Technology, TU Munich · Munich Center for Machine Learning · Munich Data Science Institute

  3. Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text

    Oct 13, 2026Lukas Edman, Alexander FraserLanguage ModelingLanguage Model Pretraining

    School of Computation, Information and Technology, TU Munich · Munich Center for Machine Learning · Munich Data Science Institute