VLM Reasoning

VLM: Vision-Language Model

This field studies how models connect images or video with language to solve tasks involving spatial relations, time, causality, diagrams, and action. Common approaches include chain-of-thought prompting, visual tools and retrieval, structured representations such as scene graphs, and fine-tuning with process-level rewards. Benchmarks increasingly test whether answers and intermediate steps are grounded in visual evidence, including in multi-image, video, scientific, and embodied settings. A central tension is that longer reasoning can help on complex tasks but can also weaken visual grounding, amplify language-based shortcuts, or produce explanations that do not reflect how an answer was reached.

Latest papers

853 papers96 papers in the last four weeks, up 48% on the four weeks before. 0.6% of all new papers.

Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding
Apr 14, 2026Eun Woo Im, Dhruv Madhwal, Vivek GuptaCompositional Reasoning in VLMs VLM Reasoning
Arizona State University
V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance
Apr 14, 2026Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc +2Visual Instruction Tuning Large Vision-Language Models
Sorbonne Universite, CNRS, ISIR, F-75005 Paris, France · Institut universitaire de France (IUF)
Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
Apr 14, 2026Muhammad Kamran Janjua, Hugo Silva, Di Niu +1Tool Use in VLMs VLM Reasoning
Huawei Technologies, Canada · University of Alberta, Canada
DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding
Apr 14, 2026Hao Yan, Yuliang Liu, Xingchen Liu +5Document Understanding Long-Document VQA
Huazhong University of Science and Technology · Huawei Inc.
Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding
Apr 14, 2026Jiwan Kim, Kibum Kim, Wonjoong Kim +2Visual Token Pruning VLM Reasoning
Korea Advanced Institute of Science and Technology (KAIST) · NVIDIA
Beyond Perception Errors: Semantic Fixation in Large Vision-Language Models
Apr 13, 2026Md Tanvirul AlamVLM Reasoning VLM Evaluation
Rochester Institute of Technology, Rochester, NY, USA
Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games
Apr 13, 2026Keyang Zhong, Junlin Xie, Hefeng Wu +2VLM Reasoning Multimodal Reasoning
Sun Yat-sen University · Shenzhen Loop Area Institute · The Chinese University of Hong Kong, Shenzhen
CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity
Apr 13, 2026Xuefeng Wei, Zhixuan Wang, Xuan Zhou +5VLM Evaluation VLM Reasoning
Nara Institute of Science and Technology · Liaoning Normal University
Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding
Apr 13, 2026Shivam Sharma, Sankalp Nagaonkar, Ashish Choithani +1VLM Reasoning VLM Evaluation
VideoDB∗
Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models
Apr 13, 2026Sameera Horawalavithana, Lauren Phillips, Ian Stewart +2Vision-Language Models VLM Reasoning
Pacific Northwest National Laboratory, Richland, WA 99352, USA
BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs
Apr 12, 2026Aaditya Baranwal, Vishal Yadav, Abhishek RajoraVLM Evaluation VLM Reasoning
(Independent Researcher) · University of Calgary
A Progressive Training Strategy for Embodied Vision-Language Models to Mitigate Spatio-Temporal Hallucinations
Apr 12, 2026Xiaoda Yang, Shuai Yang, Can Wang +9VLM Adaptation VLM Hallucination
The Hong Kong University of Science and Technology · The Chinese University of Hong Kong
SignReasoner: Compositional Reasoning for Complex Traffic Sign Understanding via Functional Structure Units
Apr 12, 2026Ruibin Wang, Zhenyu Lin, Xinhai ZhaoTraffic Sign Detection Compositional Reasoning in VLMs
Huawei Noah’s Ark Lab
Improving Medical VQA through Trajectory-Aware Process Supervision
Apr 10, 2026Halil Ibrahim Gulluk, Olivier GevaertMedical VQA Medical VLMs
Electrical Engineering, Stanford University · Stanford Center For Biomedical Informatics Research
From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping
Apr 10, 2026Yu Wu, Guangzeng Han, Ibra Niang Niang +6Plant Phenotyping VLM Evaluation
Computer Science, University of Memphis, Memphis, 38111, TN, United States · Fisher Delta Research Extension and Education Center, University of Missouri, Portageville, 63873, MO, United States · University of Arkansas Division of Agriculture, Batesville, 72501, AR, United States +1
Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Models
Apr 10, 2026Oliver McLaughlin, Daniel Shubin, Carsten Eickhoff +3Medical VLMs VLM Reasoning
Brown University · University of Washington · University of Tübingen +1
Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents
Apr 10, 2026Sangwon Baik, Gunhee Kim, Mingi Choi +1Language-Conditioned Robot Manipulation Object Pose Estimation
Seoul National University · RLWRLD
EgoTL: Egocentric Think-Aloud Chains for Long-Horizon Tasks
Apr 10, 2026Lulin Liu, Dayou Li, Yiqing Liang +8Long-Horizon Planning VLM Reasoning
UMN · TAMU · Brown University +5
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Apr 10, 2026Wenyi Xiao, Xinchi Xu, Leilei GanVLM Calibration Confidence Estimation for VLMs
Zhejiang University
RIRF: Reasoning Image Restoration Framework
Apr 10, 2026Wending Yan, Rongkai Zhang, Kaihua Tang +2Geometry-Guided Image Restoration Image Restoration
Southwest Jiaotong University, Chengdu, China · Nanyang Technological University, Singapore · Tongji University, Shanghai, China +2
Do Vision Language Models Need to Process Image Tokens?
Apr 10, 2026Sambit Ghosh, R. Venkatesh Babu, Chirag AgarwalVision-Language Models VLM Reasoning
IBM · Indian Institute of Science · University of Virginia
Visually-Guided Policy Optimization for Multimodal Reasoning
Apr 10, 2026Zengbin Wang, Feng Xiong, Liang Lin +5VLM Reasoning RL for VLMs
AMAP, Alibaba Group · SYSU · BUPT
FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval
Apr 10, 2026François Gardères, Camille-Sovanneary Gauthier, Jean Ponce +1Composed Image Retrieval VLM Reasoning
Louis Vuitton · Inria, École normale supérieure, CNRS, PSL Research University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University
Skill-Conditioned Visual Geolocation for Vision-Language Models
Apr 10, 2026Chenjie Yang, Yutian Jiang, Yutong Deng +1VLM Reasoning Vision-Language Models
Southwest Jiaotong University Chengdu, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Zhejiang University Hangzhou, China
Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework
Apr 10, 2026Xinyu Zhang, Zurong Mai, Qingmei Li +13Multimodal Large Language Models VLM Reasoning
Sun Yat-sen University · Tsinghua Shenzhen International Graduate School · Southwest University +3
Hidden in Plain Sight: Visual-to-Symbolic Analytical Solution Inference from Field Visualizations
Apr 10, 2026Pengze Li, Jiaquan Zhang, Yunbo Long +12Symbolic Reasoning Visually Grounded Reasoning
Fudan University, China · Shanghai Artificial Intelligence Laboratory, China · University of Cambridge, UK +3
Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Models
Apr 9, 2026Sumra Khan, Sagar Chhabriya, Aizan Zafar +5VLM Hallucination Mitigation Medical VLMs
Department of Computer Science, Salim Habib University, Karachi, Pakistan · Computer Science, Institute of Business Administration Sukkur, Pakistan · Center for Research in Computer Vision, University of Central Florida, USA +2
Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Apr 9, 2026Haolei Xu, Haiwen Hong, Hongxing Li +7VLM Reasoning Expert Routing
Zhejiang University · Alibaba Group
MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
Apr 9, 2026Zheng Jiang, Heng Guo, Chengyu Fang +4VLM Reasoning Visual Reasoning
Tsinghua University · DAMO Academy, Alibaba Group · Hupan Lab, 310023, Hangzhou, China +2
Multimodal Latent Reasoning via Predictive Embeddings
Apr 9, 2026Ashutosh Adhikari, Mirella LapataVLM Reasoning Multimodal Reasoning
School of Informatics University of Edimburgh

Apr 14, 2026J/K move · Enter open · S save

Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding

Eun Woo Im, Dhruv Madhwal, Vivek Gupta

Arizona State University

Compositional Reasoning in VLMs VLM Reasoning Contrastive Learning Compositional Representation Learning Vision-Language Alignment Hard Negative Mining

Open paper AskPDF arXiv

Vision-Language Models demonstrate remarkable capabilities but often struggle with compositional reasoning, exhibiting vulnerabilities regarding word order and attribute binding. This limitation arises from a scarcity of informative samples needed to differentiate subtle semantic variations during contrastive pretraining. Although hard negative mining offers a promising remedy, existing methods lack explicit mechanisms to dictate which linguistic elements undergo modification. Instead of engineering generative architectures, this study establishes lexical concreteness as a fundamental determinant of negative sample efficacy. Modifying highly concrete terms generates more pronounced structural and visual discrepancies, providing a substantially stronger learning signal. Leveraging this principle, ConcretePlant is proposed to systematically isolate and manipulate perceptually grounded concepts. Analyses of the InfoNCE further reveals a severe gradient imbalance, where easily distinguishable pairs disproportionately overwhelm the optimization process and restrict the bandwidth available for nuanced learning. To resolve this degradation, the Cement loss is formulated utilizing a margin-based approach. By correlating psycholinguistic scores with sample difficulty, this objective dynamically calibrates the penalization applied to individual training pairs. Comprehensive evaluations substantiate these theoretical claims. The integrated framework, designated as Slipform, achieves state-of-the-art accuracy across diverse compositional evaluation benchmarks, general cross-modal retrieval, single and multi label linear probing.