Multilingual Language Model Evaluation

Ask about this topic

Momentum

37 papers in the last four weeks, up 68% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 309

All topics
CardsList

  1. Document-Level Text Simplification in Estonian Using Large Language Models

    Oct 7, 2026Meeri-Ly Muru, Eduard BarbuLow-Resource Language ProcessingLanguage Model Generation Evaluation

    National Library of Estonia · Institute of Computer Science, University of Tartu Tartu, Estonia

  2. Which Language Should a Skeleton Speak? Language Choices in Multilingual Reasoning

    Oct 7, 2026HyeonSeok Lim, SeungWoo Song, Inho Won +3LLM ReasoningCross-Lingual Reasoning in Language Models

    ETRI · KAIST · Department of Artificial Intelligence, Dankook University

  3. sk-bench: A Native-First Benchmark for Evaluating Large Language Models in Slovak

    Oct 6, 2026Marek Šuppa, Ivan Vykopal, Andrej Ridzik +9LLM EvaluationMultilingual Language Model Evaluation

    Comenius University in Bratislava, Slovakia · Cisco Systems · Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia +2

  4. The Missing Minimal Pair: Stereotype Evaluation in LLMs

    Oct 6, 2026Nataliya Stepanova, Ivan Titov, Emily Allaway +1Multilingual Language Model EvaluationSocial Bias in Language Models

    University of Edinburgh · University of Amsterdam

  5. Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions

    Oct 6, 2026Dennis Fucci, Andrea Bacciu, Dong Liu +2Multilingual Language Model EvaluationLarge Language Model-Based Role-Play Simulation

    Amazon

  6. Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation

    Oct 6, 2026Shu-Kai Hsieh, Da-Chen LianMultilingual Language Model EvaluationLLM Evaluation

    National Taiwan University

  7. Are Language Models Script-Aware?

    Oct 6, 2026David Kletz, Sandra Mitrović, Ljiljana Dolamić +1Multilingual Language ModelsMultilingual Language Model Evaluation

    SUPSI, IDSIA, Switzerland · armasuisse, Science & Technology, Switzerland

  8. Multilingual GSM-Symbolic: What determines capability transfer across languages?

    Oct 2, 2026Kenneth Enevoldsen, Riley Herchert, Sofie Mosegaard +22Mathematical Reasoning BenchmarksMultilingual Language Model Evaluation

    Aarhus University · Danish Foundation Models · University of Alabama +13

  9. Do Multilingual Encoders Produce Language-Consistent Semantic IDs?

    Oct 1, 2026Abhinav Bohra, Anuj BohraMultilingual Language Model EvaluationSemantic ID Recommendation

    Rutgers University

  10. Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities

    Sep 30, 2026Dayeon Ki, Ruochen Zhang, Silviu Cucerzan +2Multilingual Language Model EvaluationKnowledge Conflicts in Language Models

    University of Maryland · Microsoft

  11. Assessing the Impact of Language Disparity on Multilingual Linguistic Ability in Large Language Models

    Sep 30, 2026Zhanyu Chen, Jaap JumeletMultilingual Language Model EvaluationLanguage Modeling

    University of Groningen

  12. Evaluating and Benchmarking the System One Model Jev

    Sep 29, 2026Tobias Deußer, Lorenz Sparrenberg, Rafet SifaMultilingual Language Model EvaluationLLM Evaluation

    University of Bonn, Bonn, Germany · Lamarr-Institute for Machine Learning and Artificial Intelligence, Bonn, Germany · Fraunhofer IAIS, Sankt Augustin, Germany

  13. RunyaNER: Auxiliary Language Selection for Runyankore NER

    Sep 29, 2026Prosper Arineitwe Asiimwe, Francois Meyer, Jan BuysNamed Entity RecognitionMultilingual Language Model Evaluation

    Department of Computer Science, University of Cape Town

  14. What Does Post-Training Change in Multilingual Reasoning?

    Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +2Multilingual Language Model EvaluationCross-Lingual Reasoning in Language Models

    University of Luxembourg · Seafill Open-Source Community

  15. Calibrated to Whom? Persona and Language Effects on Cultural Values in JEV

    Sep 28, 2026Bushra Asseri, Abdulaziz AsseriMultilingual Language Model EvaluationCross-Cultural Language Model Evaluation

    College of Engineering and Advanced Computing, Alfaisal University, Riyadh, Saudi Arabia · Proxa.sa, Riyadh, Saudi Arabia

  16. CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles

    Sep 28, 2026Mir Tafseer Nayeem, Susmoy Chakraborty, Davood RafieiMultilingual Language Model EvaluationLong-Context Language Model Evaluation

    University of Alberta · Independent Researcher

  17. Pass or Fail? Evaluating LLMs on Two Greek Examination Benchmarks

    Sep 28, 2026Panagiota Kyriazi, Eleni Kasoura, Prokopis ProkopidisMultilingual Language Model EvaluationPrompt Sensitivity

    Institute for Language and Speech Processing / Athena RC

  18. Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh

    Sep 28, 2026Muhammad Ahmad, Fatemeh Seyedin, Adrian Weller +2Multilingual Language Model EvaluationAutomated Fact-Checking

    Department of Computer Science, Shifa Tameer-e-Millat University, Islamabad, Pakistan · BRAINS, Brandenburg Research Center for Applied Intelligent Systems, Potsdam, Germany · Max Planck Institute for Security and Privacy, Bochum, Germany +3

  19. Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure

    Sep 24, 2026Fardeen Sadab, Adib SakhawatMultilingual Language Model EvaluationLanguage Model Generation Evaluation

    Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh

  20. Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER

    Sep 24, 2026Rakib Abdullah, Md. Maruful Islam MarufNamed Entity RecognitionMultilingual Language Model Evaluation

    Department of Artificial Intelligence and Data Science Green University of Bangladesh Dhaka, Bangladesh

  21. Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints

    Sep 23, 2026Imtiaz Ul Hassan, Öykü Akbulut, Onur Kaya +4Multilingual Language Model EvaluationLLM Evaluation

    Edge Hill University, United Kingdom · Ermetal Otomotiv ve E¸sya San. Tic. A.¸S., Türkiye

  22. MWE-ECL: Recoverable Long-Range Context Does Not Always Override Local Lexical Priors

    Sep 23, 2026Wei He, Aline Villavicencio, Rodrigo Wilkens +1Multilingual Language Model EvaluationLong-Context Language Model Evaluation

    University of Exeter · University of Cambridge

  23. The Second MLC-SLM Challenge: Multilingual Conversational Speech Diarization, Recognition, and Understanding

    Sep 23, 2026Bingshen Mu, Mingchen Shao, Zhennan Lin +7Spoken Language UnderstandingMultilingual Language Model Evaluation

    School of Intelligence Science and Technology, Nanjing University · College of Computing and Data Science, Nanyang Technological University, Singapore · Huawei Technologies, China +1

  24. Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms

    Sep 22, 2026Sahil Pardasani, Madhusudan SinghMultilingual Language Model EvaluationLanguage Model Generation Evaluation

    Blockchain Data Intelligence Lab The Pennsylvania State University University Park, PA, USA

  25. FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

    Sep 21, 2026Yusser Al Ghussin, Eva Gavaller, Cristina España-Bonet +2Multilingual Language Model EvaluationLLM Auditing

    Saarland University · German Research Center for Artificial Intelligence (DFKI) · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain

  26. BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents

    Sep 20, 2026Peng Kuang, Yuchun Fan, Jiangnan Li +7Multilingual Language Model EvaluationLLM Agent Evaluation

    Alibaba Token Hub, Alibaba Group · University of Illinois Urbana-Champaign · Northeastern University +2

  27. HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication

    Sep 17, 2026Hassan Saeed Hassan Albattra, Mazen Mohammed Bahgat, Rahatara Ferdousi +2Multilingual Language Model EvaluationHealthcare

    SD-AI ERA, School of Computing, Queen’s University, Kingston, Ontario, Canada.

  28. MTVA-Bench: Evaluating the Language Model Inside Cascaded Voice Agents

    Sep 17, 2026Pritish Mishra, Ishaan Kumar, Akshat Mandoli +1Voice Agent EvaluationMultilingual Language Model Evaluation

    Smallest AI

  29. Geopolitical Divisions Across Languages in Large Language Models

    Sep 17, 2026Maxim ChupilkinMultilingual Language Model EvaluationPolitical Bias in Language Models

    Department of Politics and International Relations, University of Oxford, Oxford, UK

  30. Cultural Competence in Context: A Large Language Model Passes the Turing Test in Finland

    Sep 16, 2026Otto Segersven, Pentti HenttonenMultilingual Language Model EvaluationLLM Evaluation

    Department of Computer Science, University of Helsinki · Faculty of Social Sciences, University of Tampere