We present "Testimole-conversational" a massive collection of discussion boards messages in the Italian language. The large size of the corpus, more than 30B word-tokens (1996-2024), renders it an ideal dataset for native Italian Large Language Models'pre-training. Furthermore, discussion boards' messages are a relevant resource for linguistic as well as sociological analysis. The corpus captures a rich variety of computer-mediated communication, offering insights into informal written Italian, discourse dynamics, and online social interaction in wide time span. Beyond its relevance for NLP applications such as language modelling, domain adaptation, and conversational analysis, it also support investigations of language variation and social phenomena in digital communication. The resource will be made freely available to the research community.
Figures & tables
Figure 1: Total corpus size (in MB) per year. Forum overtakes Usenet around 2004.
Figure 2: Usenet - Number of tokens per year
Figure 3: Forums - Number of tokens per year
Figure 4: Top 50 newsgroups by total character count (all periods combined).
Figure 6: Normalized frequencies of six words across time in the TestiMole-Conversational corpus.
Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics, Malostranské náměstí 25, 118 00 Praha, Czech Republic