cs.AINov 7, 2026

Multimodal Models Can See, But Can They Think? A Study of Turkish Visual Reasoning

Authors: Berke Bunyamin Sule, Yavuz Selim Aygan, Ahmed Zeer, Elif Ince, Ali Mert Temizsoy, Ozgur Agar, Himmet Toprak Kesgin, Muzaffer Kaan Yuce, 

Organizations: Department of Computer Engineering, Yildiz Technical University, Istanbul, Turkiye

Abstract

Vision-Language Models have had massive leaps in recent years. Yet, when tested on high-level cognitive tasks like pattern completion, abstract logic, and relational reasoning, they still fall short. This limitation stems from the models' lack of ability to find out underlying rules and maintain constant intermediate steps, rather than merely deciding on the correct option. To research these shortcomings, this study analyzes visual reasoning performance throughout a multilingual evaluation framework encompassing the BİLSEM, MM-IQ, Visual Puzzles, and PuzzleVQA datasets. We evaluate various data mixing strategies and supervised fine-tuning (SFT) methods. We observed that a correct answer does not always mean the reasoning chain is valid. Because of this, we adopt an LLM-as-a-Judge approach to assess the quality of reasoning beyond final answer accuracy. Our experimental results reveal that baseline models often exhibit more stable reasoning compared to fine-tuned counterparts. Most importantly, we observe that SFT can lead to model collapse, manifesting as infinite text loops or empty outputs. The study concludes that current VLMs still struggle with abstract visual reasoning.

Figures & tables

Explore similar work

CardsList
  1. Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap

    Apr 17, 2026Yige Xu, Yongjie Wang, Zizhuo Wu +3Vision-Language ModelsVLM Adaptation

  2. What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning

    Jun 1, 2025Zhaotian Weng, Haoxuan Li, Xin Eric Wang +2VLM EvaluationVLM Reasoning

  3. Investigating Relational Reasoning in VLMs

    Aug 24, 2026Adhithya Laxman Ravi Shankar Geetha, Aulia Kharis Rakhmasari, Haleema Ramzan +1Reasoning for VLMsVision-Language Models