VLM Reasoning

VLM: Vision-Language Model

This field studies how models connect images or video with language to solve tasks involving spatial relations, time, causality, diagrams, and action. Common approaches include chain-of-thought prompting, visual tools and retrieval, structured representations such as scene graphs, and fine-tuning with process-level rewards. Benchmarks increasingly test whether answers and intermediate steps are grounded in visual evidence, including in multi-image, video, scientific, and embodied settings. A central tension is that longer reasoning can help on complex tasks but can also weaken visual grounding, amplify language-based shortcuts, or produce explanations that do not reflect how an answer was reached.

Latest papers

853 papers96 papers in the last four weeks, up 48% on the four weeks before. 0.6% of all new papers.

Contrastive vision-language learning with paraphrasing and negation
Nov 20, 2025Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend +1VLM Robustness VLM Reasoning
Fujitsu Research of Europe · United Kingdom · City St George’s, University of London
Skin-R1: Clinical Knowledge-Guided Dermatological Diagnosis Using Vision-Language Models
Nov 18, 2025Zehao Liu, Weijieying Ren, Jipeng Zhang +4VLM Robustness Medical Diagnosis
Pennsylvania State University, University Park, PA 16802, USA · Stanford University, Stanford, CA 94305, USA · Hong Kong University of Science and Technology, Hong Kong, China
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
Nov 15, 2025Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan +4VLM Reasoning Multimodal CoT Reasoning
Seoul National University · Human-Computer Interaction Institute, Carnegie Mellon University · Language Technologies Institute, Carnegie Mellon University
From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answering
Nov 14, 2025Yu Zhao, Ying Zhang, Xuhui Sui +4Visual Question Answering CoT Distillation
College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1
AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models
Nov 3, 2025Sarthak Mishra, Rishabh Dev Yadav, Avirup Das +3Aerial Robotics Structured Prompting
Robotics Research Center, IIIT Hyderabad, India · Department of Computer Science, University of Manchester, UK · Newcastle University, UK
Decomposing Visual Classification: Assessing Tree-Based Reasoning in VLMs
Sep 10, 2025Sary Elmansoury, Islam Mesabah, Gerrit Großmann +3Hierarchical Classification VLM Reasoning
Department of Data Science and its Applications, German Research Centre for Artificial Intelligence (DFKI) · Department of Computer Science, University of Darmstadt · Department of Computer Science, University of Kaiserslautern–Landau (RPTU) +1
Explain Before You Answer: A Survey on Compositional Visual Reasoning
Aug 24, 2025Fucai Ke, Joy Hsu, Zhixi Cai +10Compositional Reasoning Visual Reasoning
Monash University · Stanford University · University of Washington +3
Mitigating Cross-Image Information Leakage in Multi-Image Understanding with Large Vision-Language Models
Aug 19, 2025Yeji Park, Minyoung Lee, Sanghyuk Chun +1Large Vision-Language Models VLM Reasoning
Sogang University · Princeton University · NAVER AI Lab
TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding
Aug 11, 2025Chaohong Guo, Xun Mo, Yongwei Nie +3Temporal Video Grounding Vision-Language Models
South China University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Bytedance Inc.
Position: Reasoning After Perception Means Reasoning Without Vision
Jul 21, 2025Hongcheng Gao, Zihao Huang, Jingyi Tang +12Multimodal Understanding Visual Reasoning
Tsinghua University · BIT · Peking University +6
What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning
Jun 1, 2025Zhaotian Weng, Haoxuan Li, Xin Eric Wang +2VLM Evaluation VLM Reasoning
University of Southern California · University of California, Santa Barbara · Texas A&M University
REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection
Feb 20, 2025Yun-Yun Tsai, Qingyuan Liu, Ruijian Zha +4Tool Use in VLMs Vision-Language Models
Columbia University · Rutgers University
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Visually Grounded Reasoning VLM Reasoning
University of Cambridge

Nov 20, 2025J/K move · Enter open · S save

Contrastive vision-language learning with paraphrasing and negation

Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend, Artur d'Avila Garcez

Fujitsu Research of Europe · United Kingdom · City St George’s, University of London

VLM Robustness VLM Reasoning Vision-Language Retrieval Vision-Language Alignment Multimodal Contrastive Learning Contrastive Learning for VLMs

Open paper AskPDF arXiv

Contrastive vision-language models continue to be the dominant approach for image-text retrieval. Contrastive Language-Image Pre-training (CLIP) trains two neural networks to align their image and text embeddings in a shared latent space. As a challenging case-study for neurosymbolic AI, recent results evaluating CLIP on negated or paraphrased text have shown mixed performance as these are difficult to define formally for text data. Negation produces the opposite meaning using various possible but small lexical changes. Paraphrasing may use very different textual expressions to denote essentially the same thing. As a result, learning of paraphrasing and negation together poses a significant challenge because of the above mismatch between changes in syntax and intended meaning expected to be captured by distances in embedding space. This paper proposes a new CLIP contrastive loss function capable of balancing the requirements of having both paraphrasing and negation. It applies training triplets consisting of original, paraphrased and negated text generated by multiple large language models to the evaluation of CLIP models. The approach, called SemCLIP, aims to learn semantically-relevant and simple embeddings, placing paraphrased captions nearer to the original image embeddings while at the same time pushing negated captions farther away. Empirically, SemCLIP is shown to be capable of preserving roughly the same performance as CLIP augmented with either negation or paraphrasing. Although direct comparisons are difficult to make because the problem of learning with both negation and paraphrasing is different, an expected benefit of SemCLIP should be robustness when applied zero-shot to downstream image classification tasks. Our experiments confirm such robustness as measured by difference in accuracy (mean-accuracy delta) between original and negated captions on five downstream datasets.