Monolingual Model

Monolingual models, trained exclusively on a single language's data, offer a counterpoint to multilingual models in natural language processing. Research currently focuses on comparing their performance against multilingual counterparts across various tasks, including speech recognition, sentiment analysis, and named entity recognition, often employing transformer-based architectures like BERT and its variants. This comparative approach aims to determine the optimal model type for specific languages and tasks, considering factors like resource availability and the need to mitigate biases or security vulnerabilities. The findings inform the development of more effective and ethical NLP systems for diverse languages and applications.

Papers

March 25, 2022

L3Cube-MahaHate: A Tweet-based Marathi Hate Speech Detection Dataset and BERT models
Abhishek Velankar, Hrushikesh Patil, Amol Gore, Shubham Salunke, Raviraj Joshi
BERT Model Hate Speech Monolingual Model L3Cube MahaSocialNER

March 7, 2022

IT5: Text-to-text Pretraining for Italian Language Understanding and Generation
Gabriele Sarti, Malvina Nissim
Faithful Generation Monolingual Model Italian Language Encoder Decoder Transformer Model

January 11, 2022

The GINCO Training Dataset for Web Genre Identification of Documents Out in the Wild
Taja Kuzman, Peter Rupnik, Nikola Ljubešić
Training Data Wild Challenge Multilingual Model Document Relevance Monolingual Model Multilingual Transformer Genre Identification

December 20, 2021

Training dataset and dictionary sizes matter in BERT models: the case of Baltic languages
Matej Ulčar, Marko Robnik-Šikonja
Training Data BERT Model Multilingual Model Case Relevance Monolingual Model Multilingual BERT Multilingual Model XLM BERT Like Vocabulary Size

November 18, 2021

RoBERTuito: a pre-trained language model for social media text in Spanish
Juan Manuel Pérez, Damián A. Furman, Laura Alonso Alemany, Franco Luque
Pre Trained Language Model Social Medium Transformer Language Model Monolingual Model Language Understanding Task Continuous Spanish

November 8, 2021

Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models
Angel Felipe Magnossão de Paula, Roberto Fray da Silva, Ipek Baris Schlicht
Ensemble Model Monolingual Model Multilingual BERT Monolingual Text English Tweet Monolingual BERT Model Sexism Identification

Monolingual Model

Papers

L3Cube-MahaHate: A Tweet-based Marathi Hate Speech Detection Dataset and BERT models

IT5: Text-to-text Pretraining for Italian Language Understanding and Generation

The GINCO Training Dataset for Web Genre Identification of Documents Out in the Wild

Training dataset and dictionary sizes matter in BERT models: the case of Baltic languages

RoBERTuito: a pre-trained language model for social media text in Spanish

Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models