Multilingual Model
Multilingual models aim to process and generate text across multiple languages, overcoming limitations of monolingual approaches and expanding access to natural language processing (NLP) for low-resource languages. Current research focuses on improving the performance of these models, particularly for low-resource languages, using architectures like transformer-based models (e.g., BERT, mT5) and exploring techniques such as instruction tuning, knowledge distillation, and targeted multilingual adaptation. This work is significant because it addresses biases inherent in predominantly English-centric models and enables broader access to NLP tools and applications across diverse linguistic communities.
Papers
What Language Model to Train if You Have One Million GPU Hours?
Teven Le Scao, Thomas Wang, Daniel Hesslow, Lucile Saulnier, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Models
Harshita Diddee, Sandipan Dandapat, Monojit Choudhury, Tanuja Ganu, Kalika Bali
Scaling Up Deliberation for Multilingual ASR
Ke Hu, Bo Li, Tara N. Sainath
Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models
Isabel Papadimitriou, Kezia Lopez, Dan Jurafsky
Are Pretrained Multilingual Models Equally Fair Across Languages?
Laura Cabello Piqueras, Anders Søgaard