Vision-Language Models

Also known as VLM

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

738 papers77 papers in the last four weeks, up 20% on the four weeks before. 0.4% of all new papers.

SelfThinker: Self-Evolving Reasoning via Reliability-Verified Intrinsic Visual Thoughts
Oct 29, 2026Zhangquan Chen, Ruqi HuangReasoning for VLMs Vision-Language Models
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, Guangdong, China.
Evaluating Cultural Knowledge Editing in Vision-Language Models
Oct 29, 2026Malvina Nikandrou, Georgios Pantazopoulos, Ioannis Konstas +1Vision-Language Models Knowledge Editing
University of Edinburgh · Heriot-Watt University
PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions
Oct 27, 2026Tongkun Guan, Zhibo Yang, Mingkun Yang +3Vision-Language Models RL for VLMs
MoE Key Laboratory of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · Qwen
PhysEvolve: Verifier-grounded Self-evolution for Physical Reasoning with Executable Worlds
Oct 26, 2026Jinyuan Hu, Yilei Jiang, Chen Tang +3Reasoning for VLMs Causal Reasoning
MMLab, The Chinese University of Hong Kong
Does the Outfit Change the Price? Context Halo in Vision-Language Model Valuation of an Identical Low-Cost Necklace
Oct 25, 2026Brianne LeeFraming Effects in Language Models Vision-Language Models
Independent Researcher
Recursive CLIP: Vision-Language Distillation with Looped Transformer Blocks
Oct 25, 2026Szymon Mazurek, Alberto Presta, Michał ByraVision-Language Models Recurrent Transformers
Samsung AI Center Warsaw, Poland · AGH University of Krakow, Poland · ACC Cyfronet AGH, Poland +1
Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?
Oct 25, 2026Tianchen Guo, Tengyue Zhang, Chen Liu +2Spatial Reasoning Benchmarks Multimodal Spatial Reasoning
The University of Queensland · Follow Me AI Pty LTD · Australian Institute for Machine Learning, Adelaide University +1
What Sketches Tell Us about LVLMs: Conventions, Grounding, and Localisation
Oct 25, 2026Chaitat Utintu, Ahmed Bourouis, Arkaprabha Basu +1Vision-Language Models VLM Interpretability
SketchX, Surrey Institute for People-Centred Artificial Intelligence (PAI), University of Surrey · UKRI AI: CDT in AI for Digital Media Inclusion
Reading Between the Layers: Cross-Modal Alignment and Representational Hierarchy in Vision Transformers
Oct 23, 2026Sugam Karki, Raúl Vázquez, Yova KementchedjhievaVision-Language Models Vision-Language Alignment
MBZUAI · University of Helsinki
Separating Angular Alignment from Radial Ordering in Hyperbolic Vision-Language Models
Oct 23, 2026Mingyang Gao, Yuwei Wu, Shiyu Luo +8Vision-Language Models Hyperbolic Representation Learning
Beijing Institute of Technology · Shenzhen MSU-BIT University
MAGIC: Margin-gated Contrastive Debiasing with Evidence Restoration for VLM Test-time Robustness Scaling
Oct 23, 2026Shunshun Gu, Enze Zhang, Ming Pei +2VLM Robustness Test-Time Scaling for VLMs
University of Wisconsin–Madison · University of North Carolina at Chapel Hill
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Oct 21, 2026Wen Ye, Ailin Deng, Zirui Wang +8Reasoning for VLMs Vision-Language Models
Meta Superintelligence Labs · University of Southern California · University of California, Berkeley +2
Rethinking Post-hoc Debiasing of Large Pretrained Models
Oct 21, 2026Divyaansh Mertia, Chirag Bhutra, Himanshu Shekhar +2Debiased ML Language Model Debiasing
Indian Institute of Technology Jodhpur
Physical Operating-Condition-Guided Vision-Language Matching for Cavitation Diagnosis in Industrial Pipeline Systems
Oct 20, 2026Yu Sha, Junqi Tao, Deyu Wang +4Fault Diagnosis Industrial Fault Diagnosis
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, 518172, China · Frankfurt Institute for Advanced Studies, Frankfurt am Main, 60438, Germany · Institut für Theoretische Physik, Goethe Universität Frankfurt, Frankfurt am Main, 60314, Germany +3
Give the Model a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models
Oct 20, 2026Yingxin Jia, Hui Geng, Wenxin Yang +1Vision-Language Models VLM Evaluation
Independent Researcher
From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models
Oct 18, 2026Junzhe Shi, Yuan Gan, Shida JiangVision-Language Models Visual Grounding
Systems Engineering, University of California, Berkeley, Berkeley, CA, USA · Department of AI, Quotr AI, Emeryville, CA, USA
A Language-Informed Loss for Visual Generation
Oct 16, 2026Suchir Agarwal, Keshigeyan Chandrasegaran, Ziyu Chen +4T2I Generation Vision-Language Models
Stanford University
How Murky Before a VLM Knows It Cannot See? Accuracy, Confidence and Abstention on a Physical Visibility Axis
Oct 16, 2026Mochamad Rizal Fauzan, Irgi SuryaVLM Evaluation VLM Robustness
National Taipei University of Technology, Taipei, Taiwan
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Oct 15, 2026Madhav Iyengar, Andreas Geiger, Hilde KühneVision-Language Models VLM Adaptation
University of Tübingen, Tübingen AI Center · KE:SAI
Seeing is Believing: Benchmarking and Mitigating the Interference of Language Priors on Visual Perception in MLLMs
Oct 15, 2026Li Ma, Yan Zhuang, Jiawen Deng +1Vision-Language Models VLM Evaluation
School of Computer Science and Engineering University of Electronic Science and Technology of China
Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs
Oct 15, 2026Zeyuan Yang, Zhuohao Chen, Jiaben Chen +3Attention-Guided Distillation RL for VLMs
UMass Amherst · UIUC · MIT
Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation
Oct 14, 2026Yuheng Chen, Yufan Chen, Zhuojun Cai +8Document Parsing Benchmarks Document Parsing
Karlsruhe Institute of Technology · University of Alberta · Hunan University
Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers
Oct 14, 2026Josh Barua, Kyle Mahowald, William RudmanVision-Language Models Modality Gap in VLMs
The University of Texas at Austin
Segment Where You Look: Attention as the Interface Between Language Models and Segmenters
Oct 13, 2026Eslam Abdelrahman, Hamza Omar Alkaf, Yuheng Li +5Vision-Language Models Referring Expression Segmentation
King Abdullah University of Science and Technology (KAUST) · Adobe Research
FrameLite: Decision-Oriented Aesthetic Cropping with a Compact Vision-Language Model
Oct 13, 2026Jianze Li, Kai Liu, Ziqing Zhang +4Image Editing Vision-Language Models
Shanghai Jiao Tong University · Institute of Media Technology and Experience Design, Huawei
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
Oct 12, 2026DoYoung Kim, Changkyu Choi, Adín Ramírez Rivera +2Vision-Language Models VLM Reasoning
Sungkyunkwan University, South Korea · University of Oslo, Norway
Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models
Oct 12, 2026Huang Fei, Boyi Chen, Hao Xue +3Vision-Language Models Geospatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou), China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Revisiting Natural Language Prompts from Large Language Models for Open-Vocabulary Video Recognition
Oct 12, 2026Haichen He, Weibin Liu, Weiwei XingPrompt Learning for VLMs Vision-Language Models
School of Computer Science and Technology, Beijing Jiaotong University · Visual Intelligence +X International Cooperation Joint Laboratory of MOE · School of Software Engineering, Beijing Jiaotong University
Teaching Vision-Language Models to Decline: Support Completion Against Mirage
Oct 12, 2026Alfaj Uddin Ahmed, Md. Shaown Miah, Md Arid Hasan +2Abstention in VLMs VLM Hallucination Mitigation
Department of Biomedical Engineering, Purdue University, West Lafayette, IN, USA · Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University Indianapolis, Indianapolis, IN, USA · Department of Computer Science, University of Toronto, Toronto, ON, Canada
RealityBench: Simulated Environments to Evaluate a Vision Language Model's Ability to Document Human Physical Work
Oct 9, 2026Andrew Perrault, Anmol Takiar, Dominik Winecki +1VLM Evaluation Vision-Language Models
Relantic, Inc., Columbus, OH, USA

Oct 29, 2026J/K move · Enter open · S save

SelfThinker: Self-Evolving Reasoning via Reliability-Verified Intrinsic Visual Thoughts

Zhangquan Chen, Ruqi Huang

Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, Guangdong, China.

Reasoning for VLMs Vision-Language Models Multimodal CoT Reasoning

Open paper AskPDF arXiv

Unified multimodal models can both generate and understand images within a single set of weights, offering an opportunity to use image synthesis as training supervision for visual comprehension. However, existing visual chain-of-thought methods rely on human-annotated helper images during training and require slow image generation at test time. We present SelfThinker, the first self-evolving framework that turns a unified model's self-generated visual thoughts into stronger understanding without human image annotations. Given a visual problem, SelfThinker samples candidate helper images from its generation pathway and uses its frozen understanding pathway to score each candidate via conditional pointwise mutual information. High-utility visual thoughts then refine the generator via flow matching and serve as privileged teacher context for on-policy self-distillation. At inference, SelfThinker reasons directly from the original image in text without generating intermediate images. Although trained only on small synthetic visual-search and navigation datasets, SelfThinker transfers broadly to natural-image and spatial benchmarks, raising the seven-benchmark average of ThinkMorph-7B from 52.7 to 64.0 (+11.3 points). Controlled image-perturbation probes further show that SelfThinker surpasses ThinkMorph-7B by up to 40.4 points and its pretrained BAGEL-7B backbone by up to 23.2 points across fine-grained and spatial perception tasks. Our code is available at https://github.com/zhangquanchen/SelfThinker.