Multilingual Language Model

Multilingual language models (MLLMs) aim to create AI systems capable of understanding and generating text across multiple languages, overcoming the limitations of English-centric models. Current research focuses on improving MLLM performance in low-resource languages, mitigating biases towards dominant languages, and developing techniques for efficient knowledge editing and unlearning to address privacy and ethical concerns. These advancements are crucial for broadening access to AI-powered tools and fostering more equitable and inclusive natural language processing applications globally.

Papers

May 25, 2022

Discovering Language-neutral Sub-networks in Multilingual Language Models
Negar Foroutan, Mohammadreza Banaei, Remi Lebret, Antoine Bosselut, Karl Aberer
Multilingual Model Cross Lingual Transfer Multilingual Language Model Multilingual Pre Trained Language Model Language Specific Language Agnostic

May 24, 2022

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models
Terra Blevins, Hila Gonen, Luke Zettlemoyer
Multilingual Model Cross Lingual Transfer Cross Lingual Multilingual Language Model Multilingual Pretraining Cross Lingual Ability

May 22, 2022

The Geometry of Multilingual Language Model Representations
Tyler A. Chang, Zhuowen Tu, Benjamin K. Bergen
Multilingual Language Model Geometric Analysis Cross Lingual Transfer Learning Multilingual Representation

May 21, 2022

Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese
Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, Claudia Borg
Large Corpus Low Resource Language Multilingual Language Model Data Quality Multilingual BERT Wikidata Statement Monolingual Data Monolingual BERT Model Low Resource Maltese

May 19, 2022

Automatic Spoken Language Identification using a Time-Delay Neural Network
Benjamin Kepecs, Homayoon Beigi
Automatic Speech Recognition Multilingual Language Model Multilingual Dataset Language Identification Time Delay Neural Network

May 12, 2022

April 27, 2022

A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer
Ayan Sengupta, Tharun Suresh, Md Shad Akhtar, Tanmoy Chakraborty
Multilingual Language Model Hierarchical Transformer Deep Understanding Code Mixed Code Semantics Morphologically Rich Language

April 20, 2022

Analyzing Gender Representation in Multilingual Models
Hila Gonen, Shauli Ravfogel, Yoav Goldberg
Multilingual Model Multilingual Language Model Language Specific Gender Identity Gender Mobility

April 11, 2022

April 8, 2022

Marvelous Agglutinative Language Effect on Cross Lingual Transfer Learning
Wooyoung Kim, Chaerin Jo, Minjung Kim, Wooju Kim
Multilingual Language Model Cross Lingual Transfer Learning

April 7, 2022

BERTuit: Understanding Spanish language in Twitter through a native transformer
Javier Huertas-Tato, Alejandro Martin, David Camacho
Language Model Natural Language Processing Multilingual Language Model Twitter Resource Multilingual Transformer Continuous Spanish English Tweet

March 22, 2022

Factual Consistency of Multilingual Pretrained Language Models
Constanza Fierro, Anders Søgaard
Pretrained Language Model Multilingual Language Model Multilingual Pre Trained Language Model Factual Knowledge Factual Consistency Monolingual Language Model

March 19, 2022

Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models
Ryokan Ri, Yoshimasa Tsuruoka
Language Model Knowledge Transfer Multilingual Language Model Cross Lingual Transferability Structural Knowledge

March 18, 2022

Do Multilingual Language Models Capture Differing Moral Norms?
Katharina Hämmerl, Björn Deiseroth, Patrick Schramowski, Jindřich Libovický, Alexander Fraser, Kristian Kersting
Large Corpus Multilingual Language Model Zero Shot Cross Lingual Transfer Multilingual Lm Multilingual Model XLM Multilingual Sentence

March 3, 2022

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages
Vaidehi Patil, Partha Talukdar, Sunita Sarawagi
Low Resource Language Cross Lingual Transfer Multilingual Language Model Zero Shot Cross Lingual Transfer Different Language Monolingual Corpus Low Resourced Language Tongue Twister

February 14, 2022

Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?
Boshko Koloski, Senja Pollak, Blaž Škrlj, Matej Martinc
Unsupervised Setting Multilingual Language Model Multilingual Corpus Keyword Extraction Thin Air Keyword Localisation

January 29, 2022

Does Transliteration Help Multilingual Language Modeling?
Ibraheem Muhammad Moosa, Mahmud Elahi Akhter, Ashfia Binte Habib
Multilingual Language Model High Resource Language Cyrillic Latin Script Transliteration Machine Transliteration Script Knowledge

January 9, 2022

An Ensemble Approach to Acronym Extraction using Transformers
Prashant Sharma, Hadeel Saadany, Leonardo Zilio, Diptesh Kanojia, Constantin Orăsan
Transformer Megatron Decepticons Machine Translation Multilingual Language Model Natural Language Processing Task Automatic Extraction Ensemble Approach Acronym Disambiguation

Multilingual Language Model

Papers

Discovering Language-neutral Sub-networks in Multilingual Language Models

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models

The Geometry of Multilingual Language Model Representations

Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese

Automatic Spoken Language Identification using a Time-Delay Neural Network

Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages

On the Economics of Multilingual Few-shot Learning: Modeling the Cost-Performance Trade-offs of Machine Translated and Manual Data

A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer

Analyzing Gender Representation in Multilingual Models

Team \'UFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models

Adapting BigScience Multilingual Model to Unseen Languages

Marvelous Agglutinative Language Effect on Cross Lingual Transfer Learning

BERTuit: Understanding Spanish language in Twitter through a native transformer

Factual Consistency of Multilingual Pretrained Language Models

Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models

Do Multilingual Language Models Capture Differing Moral Norms?

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages

Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?

Does Transliteration Help Multilingual Language Modeling?

An Ensemble Approach to Acronym Extraction using Transformers