Multilingual VLM Evaluation

VLM: Vision-Language Model

Overview

This field tests whether vision-language models can combine visual and linguistic evidence across languages, scripts, and cultural settings. Benchmarks cover tasks such as OCR and document parsing, visual question answering, reasoning, captioning, translation, and safety, using translated or parallel examples as well as native, human-verified data. Evaluations increasingly compare performance across languages and scripts, use text-only or image-only controls to check modality reliance, and measure consistency, grounding, and hallucinations rather than relying on aggregate accuracy alone. A central challenge is that strong English scores or gains from multilingual training do not guarantee reliable performance in low-resource languages, unfamiliar scripts, or culturally specific contexts.

Latest papers

CardsList

61 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. Benchmarking Multilingual Document Understanding in Vision-Language Models

    Nov 10, 2026Matteo Gabburo, Daniela Baiamonte, Stefano Simonazzi +3Multilingual VLM Evaluation

  2. Multimodal Models Can See, But Can They Think? A Study of Turkish Visual Reasoning

    Nov 7, 2026Berke Bunyamin Sule, Yavuz Selim Aygan, Ahmed Zeer +6Reasoning for VLMsVLM Reasoning

  3. MultiInpaintCOCO: Fine-Grained Visual Understanding in Multiple Languages

    Nov 2, 2026Andrei-Alexandru Manea, Jindřich Libovický, Philipp J. RöschMultilingual VLM EvaluationVision-Language Retrieval

  4. Vroom-Vroom at SHROOM-Visions: A Multi-Judge Committee for Detecting Hallucinated Spans in Vision-Language Outputs

    Sep 15, 2026Toqeer Ehsan, Nico Penttilä, Richard Schmidt +2VLM EvaluationHallucination Detection in VLMs

  5. ChitraMiti: Benchmarking Visual Grounding and Modality Reliance in Bengali Geometric Reasoning

    Sep 11, 2026Khan Raiyan Ibne Reza, Sanjana Aktar Maria, Sumaiya Tabassum Nimi +1Mathematical Reasoning BenchmarksMultilingual VLM Evaluation

  6. BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset

    Sep 7, 2026Md. Sadman Sakib, Zisan Mahmud, Md. Fahim Arefin +1Figurative Language UnderstandingMultilingual VLM Evaluation

  7. MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models

    Sep 4, 2026Changming Xiao, Zhenliang Ni, Jinhui He +2VLM RobustnessAdversarial Attacks on VLMs

  8. IchthyoNoma: Nomenclature and Context Sensitivity of Zero-Shot Biological Vision--Language Models for Bangladeshi Freshwater Fish Recognition

    Sep 3, 2026Nazim-E-Alam, Tarek Rahman, Md Kishor MorolVLM RobustnessPrompt Sensitivity

  9. CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

    Sep 3, 2026Bo Zeng, Linfeng Gao, Peiqin Lin +9Multimodal ReasoningMultilingual VLM Evaluation

  10. MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models

    Sep 1, 2026Tawsif Tashwar Dipto, Mehedi Ahamed, Radib Bin Kabir +5VLM EvaluationMultilingual VLM Evaluation

  11. ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation

    Aug 31, 2026Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti +11Hallucination DetectionT2I Generation

  12. Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts

    Aug 31, 2026Jaehee Kim, Ji Hoon Chung, Seoyoon Park +5Multimodal QAMultilingual VLM Evaluation

  13. What Can Low Resource Languages Learn From Each Other?

    Aug 27, 2026Achyuth P, Kahaan Shah, Chetan AroraOptical Character RecognitionVLM Adaptation

  14. Said Aloud, Read Different: Cross-Modal Instability in Multimodal Models

    Aug 27, 2026Basel Mousi, Fahim Dalvi, Shammur Chowdhury +2Multimodal Model EvaluationMultimodal Robustness

  15. Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

    Aug 26, 2026Raúl Vázquez, Aman Sinha, Chuyuan Li +12Hallucination Detection in VLMsHallucination Detection

  16. Analyzing and Mitigating Cross-Lingual Degradation in Multilingual Medical VQA

    Aug 23, 2026Jingbo Wang, Sendong Zhao, Haochun Wang +2Medical Visual Question AnsweringMedical VLMs

  17. BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

    Aug 18, 2026Liubov Chubarova, Alexandra Kuleshova, Daniil Volkov +2Multimodal ReasoningMultilingual VLM Evaluation

  18. Why Vision Fails as a Universal Bridge: Rectifying Modality Asynchrony in Multilingual MLLMs

    Aug 15, 2026Yihang Du, Juhao Liang, Zhengzhao Lai +2Multimodal Large Language ModelsVision-Language Alignment

  19. NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video

    Aug 13, 2026Yuheng Huang, Jianlang Chen, Jiayang Song +6Video UnderstandingVideo-Language Model Evaluation

  20. Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence

    Aug 13, 2026Jakub Pokrywka, Łukasz Grzybowski, Antoni Lasik +3Medical Visual Question AnsweringMultilingual VLM Evaluation

  21. Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

    Aug 7, 2026Anna Kołos, Grzegorz Statkiewicz, Karolina Seweryn +3VLM EvaluationVisual Question Answering

  22. Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

    Jul 23, 2026Abu Tyeb Azad, Fahim Ahmed, Ishita Sur Apan +7Document ChunkingMultilingual VLM Evaluation

  23. Test-Time Scaling for Small VLMs on Multilingual Visual MCQ

    Jul 10, 2026Spiros Baxevanakis, Peng-Jian YangVisual Question AnsweringTest-Time Scaling for VLMs

  24. Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models

    Jul 8, 2026Kaito Watanabe, Taisei Yamamoto, Tomoki Doi +1Visual Spatial ReasoningMultilingual VLM Evaluation