Vision-Language Models

Also known as VLM

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

738 papers77 papers in the last four weeks, up 20% on the four weeks before. 0.4% of all new papers.

OCR-Agent: Agentic OCR with Capability and Memory Reflection
Feb 24, 2026Shimin Wen, Zeyu Zhang, Xingdou Bian +5Self-Correction in VLMs Vision-Language Models
Southwest Minzu University · AI Geeks
CrystaL: Spontaneous Emergence of Visual Latents in MLLMs
Feb 24, 2026Yang Zhang, Danyang Li, Yuxuan Li +4Vision-Language Models Large Vision-Language Models
VCIP, School of Computer Science, Nankai University · Xiamen University · Nankai International Advanced Research Institute (Shenzhen Futian)
HeatPrompt: Zero-Shot Vision-Language Modeling of Urban Heat Demand from Satellite Images
Feb 23, 2026Kundan Thota, Xuanhao Mu, Thorsten Schlachter +1Building Energy Management Vision-Language Models
Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology (KIT), Germany
Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness
Feb 23, 2026Xin Hu, Haomiao Ni, Yunbei Zhang +3Vision-Language Models VLM Reasoning
Department of Computer Science, Tulane University · Department of Computer Science, University of Memphis
TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translation
Feb 23, 2026Dong-Guw Lee, Tai Hyoung Rhee, Hyunsoo Jang +3Image-to-Image Translation Thermal Imaging
KENTECH
FUSAR-GPT : A Spatiotemporal Feature-Embedded and Two-Stage Decoupled Visual Language Model for SAR Imagery
Feb 22, 2026Xiaokun Zhang, Yi Yang, Ziqi Ye +6Vision-Language Models Remote Sensing Image Understanding
Discipline and Technology Center of Microwave Vision Intelligent Sensing, Fudan University · Shanghai Innovation Institute
Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification
Feb 21, 2026Haoyu Zhang, Yuwei Wu, Pengxiang Li +6Self-Correction in VLMs Vision-Language Models
Beijing Institute of Technology · Shenzhen MSU-BIT University
MiSCHiEF: A Benchmark in Minimal-Pairs of Safety and Culture for Holistic Evaluation of Fine-Grained Image-Caption Alignment
Feb 21, 2026Sagarika Banerjee, Tangatar Madi, Advait Swaminathan +4Cultural Safety Evaluation of VLMs VLM Evaluation
Algoverse AI Research · PocketFM
Tuning Qwen2.5-VL to Improve Its Web Interaction Skills
Feb 20, 2026Alexandra Yakovleva, Henrik Pärssinen, Harri Valpola +2PEFT of VLMs Web Agents
Aalto University Espoo, Finland · System 2 AI Helsinki, Finland · University of Oulu Oulu, Finland
Focusing Where Vision Matters: Selective Training for Large Vision Language Models via Visual Information Gain
Feb 19, 2026Seulbi Lee, Sangheum HwangVision-Language Models Large Vision-Language Models
Department of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea. · Currently with Tomocube, Daejeon, Republic of Korea.
DODO: Discrete OCR Diffusion Models
Feb 18, 2026Sean Man, Gilad Deutch, Roy Ganz +4Optical Character Recognition Diffusion Language Model Decoding
Technion - Israel Institute of Technology, Haifa, Israel. · Amazon Web Services.
MedProbCLIP: Probabilistic Adaptation of Vision-Language Foundation Model for Reliable Radiograph-Report Retrieval
Feb 17, 2026Ahmad Elallaf, Yu Zhang, Yuktha Priya Masupalli +4Radiology VLMs Vision-Language Models
Texas A&M University-San Antonio · Boise State University
Can Vision-Language Models See Squares? Text-Recognition Mediates Spatial Reasoning Across Three Model Families
Feb 17, 2026Yuval LeventalVision-Language Models VLM Reasoning
Rochester Institute of Technology
How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoning
Feb 17, 2026Hongxuan Wu, Yukun Zhang, Xueqing ZhouVision-Language Models VLM Interpretability
Duke kunshan University, Duke University · The Chinese University of Hong Kong, Hong Kong, China · Fudan University, Shanghai, China
Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs
Feb 17, 2026Guangtao Lyu, Qi Liu, Chenghao Xu +5Attention Intervention in VLMs VLM Hallucination Mitigation
School of Electronic Engineering, Xidian University, Xi’an, China · College of Computer and Information, Hohai University, Nanjing, China · School of Computer Science and Technology, Xidian University, Xi’an, China +1
How to Train Your Long-Context Visual Document Model
Feb 16, 2026Austin VeselkaVisual Question Answering Vision-Language Models
LightOn
Hierarchical Vision-Language Interaction for Facial Action Unit Detection
Feb 16, 2026Yong Li, Yi Ren, Yizhe Zhang +5Facial Action Unit Detection Vision-Language Models
the School of Computer Science and Engineering, and the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, Nanjing 210096, China. · the School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China. · Key Laboratory of Child Development and Learning Science (Ministry of Education), School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. +1
Robustness of Vision Language Models Against Split-Image Harmful Input Attacks
Feb 8, 2026Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu +1VLM Robustness Adversarial Attacks on VLMs
Pennsylvania State University · Bangladesh University of Engineering and Technology
Same Answer, Different Representations: Hidden instability in VLMs
Feb 6, 2026Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6VLM Evaluation VLM Robustness
Sapienza University of Rome · University of Edinburgh · University of Oxford +5
Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance
Feb 1, 2026Wei Yang, Hong Xie, Tao Tan +3VLM Evaluation Vision-Language Models
School of AI and Data Science USTC
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Jan 30, 2026Yanlong Chen, Amirhossein Habibian, Luca Benini +1Cross-Modal Knowledge Distillation Vision-Language Models
Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich, Switzerland · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Qualcomm AI Research, Amsterdam, the Netherlands +1
LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
Jan 20, 2026Said Taghadouini, Adrien Cavaillès, Baptiste AubertinVision-Language Models Multilingual OCR
LightOn
From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion
Jan 15, 2026Cheng Chen, Yuyu Guo, Pengpeng Zeng +4Vision-Language Models VLM Adaptation
Tongji University, China · Ant Group, China · Shanghai Innovation Institute, China +1
Zero-Shot Distracted Driver Detection via Vision Language Models with Double Decoupling
Jan 13, 2026Takamichi Miyata, Sumiko Miyata, Andrew MorrisVision-Language Models Autonomous Driving Perception
Chiba Institute of Technology · Chiba, Japan · Institute of Science Tokyo +3
CoMa: Contextual Massing Generation with Vision-Language Models
Jan 13, 2026Evgenii Maslov, Alexandra Vabnits, Vladimir Vorona +6Vision-Language Models Multimodal Generation
FusionBrain Lab, Artificial Intelligence Research Institute, 123112 Moscow, Russia · Research Center of the Artificial Intelligence Institute, Innopolis University, 420500 Innopolis, Russia · ITMO University, Institute of Design and Urban Studies, 197101 Saint Petersburg, Russia +2
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Jan 7, 2026Zhexiao Xiong, Xin Ye, Burhan Yaman +5Vision-Language Models World Model Learning
Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2
Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
Dec 16, 2025George-Andrei Dima, Răzvan-Alexandru Smădu, Dumitru-Clementin CercelVLM Adaptation Image Captioning
National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
Dec 11, 2025Peizheng Li, Zhenghao Zhang, David Holtz +6Vision-Language Models VLMs for Autonomous Driving
Mercedes-Benz AG · TU Munich · Karlsruhe Institute of Technology +4
How (Mis)calibrated is your Federated CLIP and what to do about it?
Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language Models VLM Adaptation
University of Trento, Italy · ´Ecole de technologie sup´erieure, QC, Canada · AMIAD, Pˆole Recherche, Palaiseau, France +2

Feb 24, 2026J/K move · Enter open · S save

OCR-Agent: Agentic OCR with Capability and Memory Reflection

Shimin Wen, Zeyu Zhang, Xingdou Bian, Hongjie Zhu+4

Southwest Minzu University · AI Geeks

Self-Correction in VLMs Vision-Language Models VLM Reasoning Optical Character Recognition Memory-Augmented VLMs

Open paper AskPDF arXiv

Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs' reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.