Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 480

All topics

Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?
Oct 8, 2026Minhao Fan, Yinyi Liu, Jiayu Zhao +3Tool Use in VLMs Vision-Language Models
College of Computing and Data Science, Nanyang Technological University · School of Microelectronics, University of Science and Technology of China
Why VLMs Miss Small Objects, and When Zooming In Is Safe
Oct 7, 2026Junzhe Shi, Yuan Gan, Shida JiangVision-Language Models VLM Inference
Systems Engineering University of California, Berkeley Berkeley, CA, USA · Department of AI Quotr AI Emeryville, CA, USA
SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning
Oct 6, 2026Hairong Yin, Huangying Zhan, Shin-Fang Chng +2Vision-Language Models Spatiotemporal Reasoning
Department of Computer Science, Purdue University, USA · Goertek Alpha Labs, USA
Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models
Oct 6, 2026Ziyuan Yang, Wenxuan Ding, Shangbin Feng +1Vision-Language Models Safety Filtering
University of Washington · New York University
Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning
Oct 5, 2026Suguru Onda, Matthew Bailey, Ryan FarrellVision-Language Models VLM Adaptation
Brigham Young University · Carnegie Mellon University
Improving Proactive AI Assistance with Hierarchical Procedural Understanding
Oct 5, 2026Jin-Seop Lee, TaeYeon Won, SeongJun Jung +5Vision-Language Models Proactive Assistance
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea · College of Computing and Data Science, Nanyang Technological University, Singapore
Fitting Vision Adapters at Frontier Scales
Oct 5, 2026Jaehoon Lee, Harry Partridge, Mudith Jayasekara +3Vision-Language Models VLM Adaptation
Base Labs
SPACE-CLIPv2: Decoding Local Geometry from Frozen CLIP for Monocular Depth Estimation
Oct 4, 2026Hyun Song, Taewan Cho, Kangmin Kim +1Vision-Language Models Vision Foundation Models
School of Computing, iRASC Lab., Gachon University
LightVLN: Efficient Aerial Vision-and-Language Navigation with Compact Memory and History-Guided Local Aggregation
Oct 4, 2026Yiming Zhao, Tianshun Li, Jingle He +2Memory-Augmented VLMs Vision-Language Models
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Models
Oct 1, 2026Abhishek Basu, Fahad Shamshad, Karthik NandakumarVLM Robustness Vision-Language Models
MBZUAI, UAE · Michigan State University, USA
Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference
Oct 1, 2026Xinye Zhao, Yunkai Dang, Yunchen Wu +1Vision-Language Models Efficient VLM Inference
School of Intelligence Science and Technology, Nanjing University
Fusing Visual and Textual Representations via Multi-layer Fusing Transformers for Vietnamese Visual Question Answering
Oct 1, 2026Cong Phu Nguyen, Huy Tien Nguyen, Tung LeVisual Question Answering Vision-Language Models
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
AiSearch: Interactive Multi-Modal Search with VLMs
Oct 1, 2026Ali Koksal, Mei Chee Leong, Vicky Sintunata +2Multimodal IR Vision-Language Models
Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
FlashBack: Knowing When to Remember in Streaming Vision-Language Models
Oct 1, 2026Yi Chen, MingMing Yu, Rui-Qi Wang +5Memory-Augmented VLMs Vision-Language Models
Rightly Robotics, A4X
Geometric Similarity in VLM Low-Level Vision Representations
Oct 1, 2026Shao-Jun Xia, Huixin Zhang, Zhen Lei +3Vision-Language Models Representational Similarity Analysis
Florida State University · UNC Chapel Hill
Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation
Sep 30, 2026Siyuan Liu, Kanghui Tian, Yue Duan +4Vision-Language Models Visual Reasoning
Nanjing University · Fudan University · Nanjing University of Posts and Telecommunications
CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models
Sep 30, 2026Chunzheng Zhu, Jiaqi Zeng, Hongbo Zhao +3Vision-Language Models Medical VQA
Hunan University · University of Hong Kong
LoopVL: Recurrent Visual Intelligence
Sep 29, 2026Zhe Qian, Ziyang Gong, Zhongxing Xu +9Vision-Language Models Large Vision-Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Baidu · Shanghai Jiao Tong University +5
Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs
Sep 29, 2026Hung-Jen Chen, Yu-Heng Ho, Ting-Yao Huang +4Vision-Language Models Few-Shot Domain Adaptation
National Tsing Hua University, Hsinchu, Taiwan · National Taiwan University, Taipei, Taiwan
It Takes Little to Rewrite Perception: Targeted Semantic Substitution in Vision-Language Models at $ε\leq 4/255$
Sep 29, 2026Binchi Zhang, Atrisha Sarkar, Apurva NarayanVLM Robustness Adversarial Attacks on VLMs
University of Western Ontario
From Routing Signals to Selective Review: Visual regrounding in MoE VLMs
Sep 29, 2026Hongzhu Guo, Mohsen Fayyaz, Nanyun PengHallucination Detection in VLMs Vision-Language Models
University of California, Los Angeles · Peking University
Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment
Sep 29, 2026Haotian Deng, Wenbin Xing, Gang Xu +5VLM Robustness Vision-Language Models
Southern University of Science and Technology · Sun Yat-sen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) +5
Are In-Context Images Worth 10 Dimensions?
Sep 29, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Vision-Language Models Multimodal ICL
Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, Paris, France · Pôle recherche de l’AMIAD, Palaiseau, France · Institut Universitaire de France, Paris, France
Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Model
Sep 29, 2026Van Bach Nguyen, Jörg Schlötterer, Christin SeiferVision-Language Models VLM Interpretability
Marburg University, Germany
Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery
Sep 29, 2026Siqi Lu, Suo Wei, Yongbin Zheng +3Vision-Language Models Efficient VLM Inference
College of Intelligence Science and Technology, National University of Defense Technology, China · School of Computer Science, Northwestern Polytechnical University, China · Ningbo Institute, Northwestern Polytechnical University, China +1
Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry
Sep 29, 2026Woosang Jeon, Jiwon Yang, Soo Chung +1Vision-Language Models Vision-Language Grounding
Seoul National University
Exploring In-Context Learning for Handwritten Text Recognition
Sep 29, 2026Eric Ayllon, Abel Gandia, Jorge Calvo-ZaragozaIn-Context Example Selection Vision-Language Models
DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing
Sep 29, 2026Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon ParkVision-Language Models Large Vision-Language Models
Korea University, Seoul, Republic of Korea
Xiaomi-OCR-0 Technical Report
Sep 28, 2026Xin Chen, Anan Du, Feng Feng +7Vision-Language Models Document Parsing
SeerRay Team
Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment
Sep 28, 2026Shunchang Liu, Lukas Fluri, Xin Chen +1Vision-Language Models Fine-Tuning
ETH Zürich · EPFL · Aalto University +1

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?

Minhao Fan, Yinyi Liu, Jiayu Zhao, Zihan Teng+2

College of Computing and Data Science, Nanyang Technological University · School of Microelectronics, University of Science and Technology of China

Tool Use in VLMs Vision-Language Models Visual Question Answering

Open paper AskPDF arXiv

Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. Deployment requires neither weight updates nor teacher calls. Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. Recompilation for a new student model helps when the transferred interface no longer fits the student. With 100 practice items, HC exceeds answer-only LoRA on three tasks. Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. These findings support allocating work from measured student behavior rather than uniformly removing decisions.

Momentum

Related topics

Latest papers 480