Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 480

All topics

Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?
Oct 8, 2026Minhao Fan, Yinyi Liu, Jiayu Zhao +3Tool Use in VLMs Vision-Language Models
College of Computing and Data Science, Nanyang Technological University · School of Microelectronics, University of Science and Technology of China
Why VLMs Miss Small Objects, and When Zooming In Is Safe
Oct 7, 2026Junzhe Shi, Yuan Gan, Shida JiangVision-Language Models VLM Inference
Systems Engineering University of California, Berkeley Berkeley, CA, USA · Department of AI Quotr AI Emeryville, CA, USA
SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning
Oct 6, 2026Hairong Yin, Huangying Zhan, Shin-Fang Chng +2Vision-Language Models Spatiotemporal Reasoning
Department of Computer Science, Purdue University, USA · Goertek Alpha Labs, USA
Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models
Oct 6, 2026Ziyuan Yang, Wenxuan Ding, Shangbin Feng +1Vision-Language Models Safety Filtering
University of Washington · New York University
Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning
Oct 5, 2026Suguru Onda, Matthew Bailey, Ryan FarrellVision-Language Models VLM Adaptation
Brigham Young University · Carnegie Mellon University
Improving Proactive AI Assistance with Hierarchical Procedural Understanding
Oct 5, 2026Jin-Seop Lee, TaeYeon Won, SeongJun Jung +5Vision-Language Models Proactive Assistance
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea · College of Computing and Data Science, Nanyang Technological University, Singapore
Fitting Vision Adapters at Frontier Scales
Oct 5, 2026Jaehoon Lee, Harry Partridge, Mudith Jayasekara +3Vision-Language Models VLM Adaptation
Base Labs
SPACE-CLIPv2: Decoding Local Geometry from Frozen CLIP for Monocular Depth Estimation
Oct 4, 2026Hyun Song, Taewan Cho, Kangmin Kim +1Vision-Language Models Vision Foundation Models
School of Computing, iRASC Lab., Gachon University
LightVLN: Efficient Aerial Vision-and-Language Navigation with Compact Memory and History-Guided Local Aggregation
Oct 4, 2026Yiming Zhao, Tianshun Li, Jingle He +2Memory-Augmented VLMs Vision-Language Models
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Models
Oct 1, 2026Abhishek Basu, Fahad Shamshad, Karthik NandakumarVLM Robustness Vision-Language Models
MBZUAI, UAE · Michigan State University, USA
Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference
Oct 1, 2026Xinye Zhao, Yunkai Dang, Yunchen Wu +1Vision-Language Models Efficient VLM Inference
School of Intelligence Science and Technology, Nanjing University
Fusing Visual and Textual Representations via Multi-layer Fusing Transformers for Vietnamese Visual Question Answering
Oct 1, 2026Cong Phu Nguyen, Huy Tien Nguyen, Tung LeVisual Question Answering Vision-Language Models
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
AiSearch: Interactive Multi-Modal Search with VLMs
Oct 1, 2026Ali Koksal, Mei Chee Leong, Vicky Sintunata +2Multimodal IR Vision-Language Models
Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
FlashBack: Knowing When to Remember in Streaming Vision-Language Models
Oct 1, 2026Yi Chen, MingMing Yu, Rui-Qi Wang +5Memory-Augmented VLMs Vision-Language Models
Rightly Robotics, A4X
Geometric Similarity in VLM Low-Level Vision Representations
Oct 1, 2026Shao-Jun Xia, Huixin Zhang, Zhen Lei +3Vision-Language Models Representational Similarity Analysis
Florida State University · UNC Chapel Hill
Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation
Sep 30, 2026Siyuan Liu, Kanghui Tian, Yue Duan +4Vision-Language Models Visual Reasoning
Nanjing University · Fudan University · Nanjing University of Posts and Telecommunications
CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models
Sep 30, 2026Chunzheng Zhu, Jiaqi Zeng, Hongbo Zhao +3Vision-Language Models Medical VQA
Hunan University · University of Hong Kong
LoopVL: Recurrent Visual Intelligence
Sep 29, 2026Zhe Qian, Ziyang Gong, Zhongxing Xu +9Vision-Language Models Large Vision-Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Baidu · Shanghai Jiao Tong University +5
Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs
Sep 29, 2026Hung-Jen Chen, Yu-Heng Ho, Ting-Yao Huang +4Vision-Language Models Few-Shot Domain Adaptation
National Tsing Hua University, Hsinchu, Taiwan · National Taiwan University, Taipei, Taiwan
It Takes Little to Rewrite Perception: Targeted Semantic Substitution in Vision-Language Models at $ε\leq 4/255$
Sep 29, 2026Binchi Zhang, Atrisha Sarkar, Apurva NarayanVLM Robustness Adversarial Attacks on VLMs
University of Western Ontario
From Routing Signals to Selective Review: Visual regrounding in MoE VLMs
Sep 29, 2026Hongzhu Guo, Mohsen Fayyaz, Nanyun PengHallucination Detection in VLMs Vision-Language Models
University of California, Los Angeles · Peking University
Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment
Sep 29, 2026Haotian Deng, Wenbin Xing, Gang Xu +5VLM Robustness Vision-Language Models
Southern University of Science and Technology · Sun Yat-sen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) +5
Are In-Context Images Worth 10 Dimensions?
Sep 29, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Vision-Language Models Multimodal ICL
Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, Paris, France · Pôle recherche de l’AMIAD, Palaiseau, France · Institut Universitaire de France, Paris, France
Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Model
Sep 29, 2026Van Bach Nguyen, Jörg Schlötterer, Christin SeiferVision-Language Models VLM Interpretability
Marburg University, Germany
Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery
Sep 29, 2026Siqi Lu, Suo Wei, Yongbin Zheng +3Vision-Language Models Efficient VLM Inference
College of Intelligence Science and Technology, National University of Defense Technology, China · School of Computer Science, Northwestern Polytechnical University, China · Ningbo Institute, Northwestern Polytechnical University, China +1
Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry
Sep 29, 2026Woosang Jeon, Jiwon Yang, Soo Chung +1Vision-Language Models Vision-Language Grounding
Seoul National University
Exploring In-Context Learning for Handwritten Text Recognition
Sep 29, 2026Eric Ayllon, Abel Gandia, Jorge Calvo-ZaragozaIn-Context Example Selection Vision-Language Models
DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing
Sep 29, 2026Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon ParkVision-Language Models Large Vision-Language Models
Korea University, Seoul, Republic of Korea
Xiaomi-OCR-0 Technical Report
Sep 28, 2026Xin Chen, Anan Du, Feng Feng +7Vision-Language Models Document Parsing
SeerRay Team
Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment
Sep 28, 2026Shunchang Liu, Lukas Fluri, Xin Chen +1Vision-Language Models Fine-Tuning
ETH Zürich · EPFL · Aalto University +1
When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Models
Sep 28, 2026Yizhou Fang, Siyue Chen, Zimo Qi +3VLM Evaluation Vision-Language Models
University of Waterloo · Independent Researcher · Johns Hopkins University +3
AnswerMap: Faithful Spatial Interpretability of VLMs from Answer Posteriors
Sep 28, 2026Mohamed Eltahir, Fardows Adam, Duaa M. Tahir +6Vision-Language Models VLM Interpretability
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England
ControlTrace: Recovering Control Fields for Hidden-Content Recognition
Sep 28, 2026Zijian Liu, Yaoguang Chen, Liwei Liu +4VLM Evaluation Vision-Language Models
Shanghai Jiao Tong University · Ant Group
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2VLM Evaluation Vision-Language Models
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?
Sep 28, 2026Bohao Xing, Xin Liu, Kaishen Yuan +5Cross-Modal Learning Vision-Language Models
Lappeenranta-Lahti University of Technology LUT · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology (Guangzhou) +3
Program-Verified Self-Evolution for Vision-Language Models
Sep 27, 2026Ahmed Heakl, Sungik Choi, Moontae Lee +1Visual Question Answering Vision-Language Models
LG AI Research · MBZUAI · University of Illinois at Chicago +1
Seeing and Solving Are Not Enough for Vision-Language Models
Sep 27, 2026Ziheng Wang, Mingxuan Xie, Yilin Liu +3VLM Evaluation Visual Question Answering
Sun Yat-sen University · Zhejiang University · The Hong Kong University of Science and Technology +2
ViCoR: Reliable Molecular Structure Extraction via Spatially Aligned Verification and Executable Revision
Sep 27, 2026Yujian Yuan, Xin Cai, Yufan Chen +5Vision-Language Models Optical Character Recognition
The Hong Kong University of Science and Technology · The Chinese University of Hong Kong · Institute of Computing Technology, Chinese Academy of Sciences.
CoViST: Visual Token Compression via Composable States
Sep 27, 2026Qi Zhang, Xiandong Meng, Ronggang Wang +1Vision-Language Models Efficient VLM Inference
Peng Cheng Laboratory Shenzhen, China · Peking University Shenzhen Graduate School Peking University Beijing, China
Where Hallucinations Live: A Cross-Architecture Circuit in VQ-Tokenized Vision-Language Models
Sep 24, 2026Shamanthak Hegde, Xiangrui Liu, Maitreya Patel +1Vision-Language Models Object Hallucination in VLMs
Arizona State University · Adobe
DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis
Sep 23, 2026Xiangyu Yin, Shiqi Wang, Abrar Alamri +4Vision-Language Models Gait Analysis
University of Pittsburgh
Small yet Assistive: Spatially-Aware Post-Training for Low Vision
Sep 23, 2026Rishabh Choudhary, Shreyansh Raj, Umesh Goyal +6Vision-Language Models VLM Adaptation
Indian Institute of Technology Mandi · Mohamed bin Zayed University of Artificial Intelligence
Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judges
Sep 22, 2026Amit Jadhav, Shaurya Beriwala, Beomjin KimVLM Evaluation Vision-Language Models
Purdue University Fort Wayne
What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
Sep 21, 2026Débora Oliveira Makowski, Samiran Gode, Abhijeet Nayak +4Vision-Language Models VLM Interpretability
University of Technology Nuremberg. · ETH Zürich. · Inria, Ecole Normale Sup´erieure, CNRS, PSL Research University.
Hierarchical Prompt Learning for Hyperbolic Vision-Language Models
Sep 21, 2026Andro Erdelez, Pascal Mettes, Behzad BozorgtabarVision-Language Models Prompt Learning
Aarhus University · University of Amsterdam
SKstars at SHROOM: Visions Agreement-Guided Ensembling of Zero-Shot and LoRA-Adapted Vision--Language Models
Sep 21, 2026Ali Athar, Imran Ahsan, Joon-Yong JungHallucination Detection in VLMs Vision-Language Models
Department of Radiology, Seoul St. Mary’s Hospital, The Catholic University of Korea · Department of Smart City, Chung-Ang University
BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring
Sep 20, 2026Liuyang Song, Yi Zhang, Zhongyi Deng +2Vision-Language Models Vision-Language Grounding
Peking University · Dongguan University of Technology · Sichuan Agricultural University
Spatial-Semantic Uncertainty in VLM-Based Target Search: Balancing Exploration and Identification
Sep 17, 2026Alkesh K. Srivastava, Jonathan Diller, Vijay Kumar +1Belief-Space Planning Vision-Language Models
Temple University, Philadelphia, PA, 19122 USA · University of Pennsylvania, Philadelphia, PA, 19104 USA
Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraint
Sep 17, 2026Zhiyun Jiang, Hanyong Wang, Binbin Liang +3Image Captioning Vision-Language Models
Sichuan University, Chengdu, China · Beijing Institute of Technology, Beijing, China
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Sep 15, 2026Zhenbin Wang, Lei Zhang, Lituan Wang +3Vision-Language Models Efficient VLM Inference
Sichuan University
ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models
Sep 15, 2026Zhipeng Zhao, Zhaoqiang Wei, Peishun Liu +2Vision-Language Models Large Vision-Language Models
Ocean University of China
ProtoLIP: From Sentence-Level to Object-Level Evidence Disentanglement
Sep 14, 2026Yan Zhu, Yongbo Chen, Zhengming Ding +1Vision-Language Models Prototype Learning
Department of Computer Science Tulane University New Orleans, LA, USA
RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs
Sep 14, 2026Maëlic NeauVision-Language Models Visual Reasoning
Independent Researcher
ExpertHTR: Unified Handwritten Text Recognition with Multi-Task Learning and Sparse Mixture-of-Experts
Sep 14, 2026Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu +1Vision-Language Models Sparse Mixture-of-Experts
University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam · AJ Technologies, Nagoya, Japan +1
Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions
Sep 12, 2026Dieuwertje Alblas, Alma M. Liezenga, Jan Erik van Woerden +3Vision-Language Models
Intelligent Imaging, Defense, Safety & Security, TNO, The Hague, The Netherlands
HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA
Sep 12, 2026Syed Mohaiminul Hoque, Md Sakhawat HossainVLM Evaluation Hallucination Detection in VLMs
Independent Author
Can Edge-Deployable Vision-Language Models Identify Species?
Sep 12, 2026William Zhou, Mayukha Siripuram, Xiao Yan +2VLM Evaluation VLM Robustness
Plano West Senior High School
VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models
Sep 10, 2026Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain +6VLM Evaluation Vision-Language Models
NVIDIA · Clemson University
Two-Token Features and Small-Large Ensembles for VLM Hallucination Detection
Sep 9, 2026Eli SchwartzHallucination Detection in VLMs Vision-Language Models
IBM Research
From Pixels to Hierarchical Sequences: Quadtree Mask Encoding for Vision-Language Binary Change Detection
Sep 9, 2026Xiao An, Ruikang Zhang, Chen Zhong +4Vision-Language Models Remote Sensing Change Detection
Wuhan University · Peking University · Shanghai Artificial Intelligence Laboratory

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?

Minhao Fan, Yinyi Liu, Jiayu Zhao, Zihan Teng+2

College of Computing and Data Science, Nanyang Technological University · School of Microelectronics, University of Science and Technology of China

Tool Use in VLMs Vision-Language Models Visual Question Answering

Open paper AskPDF arXiv

Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. Deployment requires neither weight updates nor teacher calls. Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. Recompilation for a new student model helps when the transferred interface no longer fits the student. With 100 practice items, HC exceeds answer-only LoRA on three tasks. Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. These findings support allocating work from measured student behavior rather than uniformly removing decisions.

Momentum

Related topics

Latest papers 480