Multilingual Language Model Evaluation

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 68% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

MKJ at SemEval-2026 Task 9: A Comparative Study of Generalist, Specialist, and Ensemble Strategies for Multilingual Polarization
Apr 23, 2026Maziar Kianimoghadam JouneghaniMultilingual Language Model Evaluation Language Model Ensembles
University of Turin
RespondeoQA: a Benchmark for Bilingual Latin-English Question Answering
Apr 22, 2026Marisa Hudspeth, Patrick J. Burns, Brendan O'ConnorMultilingual Language Model Evaluation Multilingual QA
Manning College of Information & Computer Sciences, University of Massachusetts Amherst · Institute for the Study of the Ancient World, New York University
The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models
Apr 22, 2026Yilun Liu, Chunguang Zhao, Mengyao Piao +14Multilingual Language Model Evaluation LLM Evaluation
Huawei, China · Fudan University, China
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs
Apr 21, 2026Guy Mor-Lan, Omer Goldman, Matan Eyal +6Multilingual Language Models Multilingual Language Model Evaluation
GGoogle Research · BBar-Ilan University · CUniversity of Cambridge +1
CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks
Apr 21, 2026Peiqin Lin, Chenyang Lyu, Wenjiang Luo +22Multilingual Language Model Evaluation LLM Evaluation
Alibaba Group · Beijing Language and Culture University · LMU Munich +6
MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation
Apr 20, 2026Mehul Agarwal, Aditya Aggarwal, Arnav Goel +2Multilingual Language Model Evaluation Text Generation
SBILab, Indraprastha Institute of Information Technology Delhi
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
Apr 20, 2026Shaden Alshammari, Kevin Wen, Abrar Zainal +5Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
MIT · HUMAIN · Individual Researcher +1
Do LLMs Use Cultural Knowledge Without Being Told? A Multilingual Evaluation of Implicit Pragmatic Adaptation
Apr 20, 2026Mehwish Nasim, Sanjeevan Selvaganapathy, Neel Ganapathi Sabhahit +6Multilingual Language Models Multilingual Language Model Evaluation
1Network Analysis and Social Influence Modelling (NASIM) Lab · 3Computational Social Science & Systems Analysis University of Münster · School of Computing Macquarie University
BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories
Apr 18, 2026Yuxuan Ouyang, yingfeng luo, JingBo Zhu +1Multilingual Language Model Evaluation Social Bias in Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs
Apr 18, 2026Wei-Chi Wu, Sheng-Lun Wei, Hen-Hsen Huang +1Multilingual Language Model Evaluation LLM Prompting
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan
No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus
Apr 17, 2026Hitesh Mehta, Arjit Saxena, Garima Chhikara +1Multilingual Language Model Evaluation LLM Evaluation
Department of Computer Science Engineering, Delhi Technological University, New Delhi, India · Department of Electronics and Communication Engineering, Delhi Technological University, New Delhi, India
Optimizing Korean-Centric LLMs via Token Pruning
Apr 17, 2026Hoyeol Kim, Hyeonwoo KimMultilingual Language Model Evaluation LLM Pruning
College of Computing · Independent Researcher
Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences
Apr 16, 2026Jingnong Qu, Ashvin Ranjan, Shane Steinert-ThrelkeldMultilingual Language Model Evaluation Language Modeling
University of Washington
Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation
Apr 13, 2026Lester James V. Miranda, Ivan Vulić, Anna KorhonenSupervised Fine-Tuning Multilingual Language Models
Language Technology Lab, University of Cambridge
What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features
Apr 6, 2026Dayeon Ki, Kevin Duh, Marine CarpuatMultilingual Language Model Evaluation Cross-Lingual Reasoning in Language Models
University of Maryland · Johns Hopkins University
Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation
Apr 6, 2026Alhasan Mahmood, Samir Abdaljalil, Hasan KurbanMultilingual Language Model Evaluation LLM-as-a-Judge
Hamad Bin Khalifa University, Doha, Qatar · Texas A&M University, College Station, TX USA
SalamahBench: Dialect and Category Level Safety Evaluation of Arabic Language Models
Feb 3, 2026Omar Abdelnasser, Fatemah Alharbi, Khaled Khasawneh +2LLM Safety Benchmarks Multilingual Language Model Evaluation
Compumacy for Artificial Intelligence Solutions, Cairo, Egypt · Computer Science Department, College of Computer Science and Engineering, Taibah University, Yanbu 46522, KSA · Electrical and Computer Engineering Department, George Mason University, VA, USA +2
MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
Jan 29, 2026Tianyi Xu, Kosei Uemura, Alfred Malengo Kondoro +6Numerical Reasoning in Language Models Mathematical Reasoning Benchmarks
McGill University · Mila-Quebec AI Institute · University of Toronto +7
The Effect of Scripts and Formats on LLM Numeracy
Jan 21, 2026Varshini Reddy, Craig W. Schmidt, Seth Ebner +3Numerical Reasoning in Language Models Multilingual Language Model Evaluation
Kensho Technologies · Ben-Gurion University · MIT Cambridge, MA
The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents
Jan 17, 2026Oleg SmirnovMultilingual Language Model Evaluation Political Bias in Language Models
Microsoft / Stockholm, Sweden
Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection
Jan 17, 2026Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoLLM Quantization Multilingual Language Model Evaluation
Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence
Untangling Input Language from Reasoning Language: A Diagnostic Framework for Cross-Lingual Moral Alignment in LLMs
Jan 15, 2026Nan Li, Bo Kang, Tijl De BieMultilingual Language Model Evaluation LLM Alignment
IDLab, Department of Electronics and Information Systems Ghent University, Belgium
BizFinBench.v2: Towards Reliable LLMs in Finance via Real-User Data and Offline/Online Bilingual Evaluation
Jan 10, 2026Xin Guo, Rongjunchen Zhang, Guilong Lu +4Financial Services Multilingual Language Model Evaluation
HiThink Research · Shanghai University of Finance and Economics
What Matters When Building Universal Multilingual Named Entity Recognition Models?
Jan 9, 2026Jonas Golde, Patrick Haller, Alan AkbikNamed Entity Recognition Multilingual Language Models
Humboldt Universität zu Berlin
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
Jan 8, 2026Zheng Luo, T Pranav Kutralingam, Ogochukwu N Okoani +3Multilingual Language Model Evaluation LLM Tool Use
University of Southern California · 2Arizona State University
The Language of Bargaining: Linguistic Effects in LLM Negotiations
Jan 7, 2026Stuti Sinha, Himanshu Kumar, Aryan Raju Mandapati +2Multilingual Language Model Evaluation Multi-Agent Negotiation
BITS Pilani
Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Results
Nov 7, 2025Jan-Thorsten Peter, David Vilar, Tobias Domhan +2Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
Google
MultiZebraLogic: A Multilingual Logical Reasoning Benchmark
Nov 5, 2025Sofie Helene Bruun, Dan Saattrup SmartLogical Reasoning Multilingual Language Model Evaluation
The Alexandra Institute Rued Langgaards Vej 7, 5D, 2300 Copenhagen S, Denmark
MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts
Nov 1, 2025Naoto Iwase, Hiroki Okuyama, Junichiro IwasawaMultilingual Language Model Evaluation Healthcare
Preferred Networks, Inc., Tokyo, Japan · School of Medicine, Nagoya University, Nagoya, Japan
Zoom In Disparities in Healthcare LLM Q&A
Oct 20, 2025Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao +5Multilingual Language Model Evaluation Knowledge Augmentation for Language Models
Universitat Politècnica de València, Valencia, Spain · University of Trento, Trento, Italy · University of Sheffield, Sheffield, UK +5

Apr 23, 2026, cs.CLJ/K move · Enter open · S save

MKJ at SemEval-2026 Task 9: A Comparative Study of Generalist, Specialist, and Ensemble Strategies for Multilingual Polarization

Maziar Kianimoghadam Jouneghani

University of Turin

Multilingual Language Model Evaluation Language Model Ensembles Cross-Lingual Transfer

Open paper AskPDF arXiv

We present a systematic study of multilingual polarization detection across 22 languages for SemEval-2026 Task 9 (Subtask 1), contrasting multilingual generalists with language-specific specialists and hybrid ensembles. While a standard generalist like XLM-RoBERTa suffices when its tokenizer aligns with the target text, it may struggle with distinct scripts (e.g., Khmer, Odia) where monolingual specialists yield significant gains. Rather than enforcing a single universal architecture, we adopt a language-adaptive framework that switches between multilingual generalists, language-specific specialists, and hybrid ensembles based on development performance. Additionally, cross-lingual augmentation via NLLB-200 yielded mixed results, often underperforming native architecture selection and degrading morphologically rich tracks. Our final system achieves an overall macro-averaged F1 score of 0.796 and an average accuracy of 0.826 across all 22 tracks. Code and final test predictions are publicly available at: https://github.com/Maziarkiani/SemEval2026-Task9-Subtask1-Polarization.

Momentum

Related topics

Latest papers 309