VLM Reasoning

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

70 papers in the last four weeks, up 250% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 541

All topics

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning
Jul 2, 2026Liyan Tang, Fangcong Yin, Greg DurrettVLM Robustness Vision-Language Models
The University of Texas at Austin · New York University
Show Me Examples: Inferring Visual Concepts from Image Sets
Jul 2, 2026Nick Stracke, Kolja Bauer, Stefan Andreas Baumann +3Vision-Language Models Conditional Image Generation
Munich Center for Machine Learning · Apple
LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
Jul 2, 2026Bowen Yuan, Zijian Wang, Yadan Luo +2Vision-Language Models Large Vision-Language Models
The University of Queensland
Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning
Jul 1, 2026Hongxing Li, Xiufeng Huang, Dingming Li +11Visual Reasoning VLM Reasoning
1Zhejiang University · 2Alibaba Group
GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Jul 1, 2026Dianyu Wang, Peirong Zhang, Xuyang Li +2Vision-Language Grounding VLM Reasoning
Aerospace Information Research Institute, the Key Laboratory of Network Information System Technology (NIST), and the Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China
ESC: Emotional Self-Correction for Reliable Vision-Language Models
Jul 1, 2026Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy +9LLM Self-Correction VLM Reasoning
GenAI4E Lab · University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam +10
Latent Visual Cache for Video Reasoning
Jul 1, 2026Yongheng Zhang, Zhipeng Xu, Hao Wu +4Memory-Augmented VLMs Vision-Language Models
1Tencent Youtu Lab · 2Tsinghua University · University of Illinois at Chicago
OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
Jul 1, 2026Xudong Li, Mengdan Zhang, Peixian Chen +7Visual Spatial Reasoning 3D Spatial Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology
M2Note: Continual Evolution of Vision Language Models via Mistake Notebook Learning
Jul 1, 2026Haiwen Li, Jing Tang, Rui Chen +2Memory-Augmented VLMs VLM Reasoning
AMAP, Alibaba Group
DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding
Jul 1, 2026Zhengbo Zhang, Mark He Huang, Zhigang Tu +1Temporal Video Grounding VLM Reasoning
Singapore University of Technology and Design · Wuhan University · University of California, Merced
GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine
Jul 1, 2026Yanan Wang, Wen Li, Yibin Ying +1Text-Guided Image Segmentation VLM Reasoning
College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China · ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou 311215, China
What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models
Jul 1, 2026Amirhosein Chahe, Tyler Naes, Jovin D'sa +4VLMs for Autonomous Driving Occlusion Reasoning
Honda Research Institute (HRI), San Jose, CA 95134, USA. · Drexel University, Philadelphia, PA 19104, USA.
Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting
Jun 30, 2026Qian Ma, S M Rayeed, Charles V. Stewart +2VLM Evaluation Visual Question Answering
Rensselaer Polytechnic Institute, Troy NY 12047, USA · AT&T Chief Data Office, Bedminster NJ 07921, USA
PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking
Jun 30, 2026Dengxian Gong, Yuanzheng Wu, Haobo Yuan +11Vision-Language Grounding Tool-Using Vision-Language Agents
Wuhan University · UC Merced · UTS +3
Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision
Jun 30, 2026Xianda Zheng, Huan Gao, Meng-Fen Chiang +3LLM Alignment Medical VLMs
School of Computer Science, University of Auckland · Sapiens · National Yang Ming Chiao Tung University +3
Technical Report of RoboSpatial Challenge at CVPR 2026: Selective Reasoning Activation and Reference-Frame Disambiguation for Embodied Spatial Reasoning
Jun 30, 2026Yuxiang Xie, Qi Lv, Jianming Xing +4Visual Spatial Reasoning VLM Reasoning
1Harbin Institute of Technology, Shenzhen · 2Ruoyu Technology
SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search
Jun 30, 2026Ming Dai, Zhihong Lu, Jinjie Gu +5Tool-Augmented Language Model Agents Agentic Search
Southeast University · Ant Group
Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding
Jun 30, 2026Ziqi Li, Zijian Chen, Tingzhu Chen +1VLM Evaluation Document Understanding
Nanjing University of Science and Technology, Nanjing 210094, China · Shanghai Jiao Tong University, Shanghai 200240, China · Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China
Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving
Jun 30, 2026Anh Dung Dinh, Simon Khan, Flora SalimEfficient VLM Inference Speculative Decoding
School of Computer Science and Engineering UNSW Sydney · Air Force Research Laboratory
Rethinking Foundation Model Collaboration: Enhancing Specialized Models through Proxy Task Reasoning
Jun 30, 2026Hongyi Lin, Yang Liu, Jinhua Zhao +1VLM Reasoning Multimodal Foundation Models
School of Vehicle and Mobility, Tsinghua University, Beijing, China · Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA, USA
Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
Jun 29, 2026Yutao Sun, Yanting Miao, Hao-Xuan Ma +8Tool Use in VLMs VLM Adaptation
1Qwen Large Model Application Team, Alibaba · 2Zhejiang University · University of Waterloo +3
Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
Jun 29, 2026Kai Jiang, Ruishu Zhu, Siqi Huang +2Visual Question Answering Multimodal Large Language Models
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University · Institute of Artificial Intelligence, China Telecom (TeleAI) · 3Fudan University +1
Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
Jun 29, 2026Peng, Lee, Yin Zhang +9Vision-Language Grounding VLM Reasoning
GMLab · Hong Kong Polytechnic University · XPENG Robotics
H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoning
Jun 29, 2026Eric Peh, Debaditya Roy, Basura FernandoReinforcement Learning Visual Reasoning
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India +1
StrucTab: A Structured Optimization Framework for Table Parsing
Jun 29, 2026Gengluo Li, Shangpin Peng, Chengquan Zhang +10Table Structure Recognition VLM Reasoning
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, China · LLM Department, Tencent, China +3
Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation
Jun 29, 2026Wentong Tian, Jiyuan Zhao, Tianliang Yao +4RL for Robotics Robotic Endovascular Intervention
Department of Control Science and Engineering, College of Electronic and Information Engineering, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China. · Department of Electronic Engineering, Faculty of Engineering, The Chinese University of Hong Kong, Hong Kong SAR 999077, China. · Shanghai Operation Robot Co., Ltd., Shanghai 201318, China.
ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models
Jun 28, 2026Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +2Visual Spatial Reasoning VLM Adaptation
Northeastern University, Boston, USA · EmbodyX Inc., USA · Zhejiang University, China
MIRROR: Aligning Semantic Relations from Language to Image via Gromov--Wasserstein
Jun 28, 2026Hong-Han Wang, Yuntao Wang, Hu DingLarge Vision-Language Models VLM Reasoning
University of Science and Technology of China, Hefei, China
EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Control
Jun 27, 2026Yuxin Liu, Zihan Chen, Haoyu Wang +3VLMs for Autonomous Driving End-to-End Autonomous Driving
College of Computer Science and Technology Zhejiang University Hangzhou, Zhejiang, China
On Test-Time Scaling for Vision-Language Models
Jun 27, 2026Fawaz Sammani, Tzoulio Chamiti, Nikos DeligiannisVLM Evaluation Test-Time Scaling for VLMs
ETRO Department, Vrije Universiteit Brussel, Belgium · imec, Kapeldreef 75, B-3001 Leuven, Belgium

Jul 2, 2026, cs.CLJ/K move · Enter open · S save

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning

Liyan Tang, Fangcong Yin, Greg Durrett

The University of Texas at Austin · New York University

VLM Robustness Vision-Language Models VLM Reasoning Visually Grounded Reasoning Visual Grounding RL for VLMs

Open paper AskPDF arXiv

Large vision-language models can reason over multimodal inputs by generating textual chains of thought (CoT). A key capability exhibited in CoT reasoning is self-reflection: revisiting earlier decisions and correcting previous errors. However, existing LVLMs often fail to properly attend to visual inputs during reflection, limiting their ability to translate feedback into grounded corrections, especially for out-of-distribution images. To address this issue, we propose a novel reinforcement learning training framework VRRL, with two components explicitly designed to elicit visually grounded self-reflection. First, we randomly mask trajectory prefixes during training to emphasize recovery from incorrect intermediate predictions rather than making early mistakes. Second, we introduce buffered roll-ins from an experience replay buffer to expose the model to diverse failure states that it must learn to correct. We evaluate our approach on visual grounding tasks involving tables and charts, as well as spatial navigation benchmarks. While off-the-shelf and conventionally fine-tuned models degrade substantially under distribution shift, our method substantially improves average out-of-distribution accuracy over standard RL and reflection-oriented fine-tuning baselines by using self-reflection effectively.

Momentum

Related topics

Latest papers 541