Large Vision-Language Models

Also known as LVLM

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 36% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 253

All topics

Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions
Oct 7, 2026Bojun Yang, Haochen Zhou, Zhifang Zhang +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · Zhejiang University
Mixture of Layers: Dynamic Layer Routing for Visual Reasoning
Oct 7, 2026Jeonghwan Kim, Sofia Stoica, Jiwan Chung +5VLM Reasoning Large Vision-Language Models
University of Illinois Urbana-Champaign · Yonsei University · Meta Reality Labs
Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning
Oct 5, 2026Suguru Onda, Matthew Bailey, Ryan FarrellVision-Language Models VLM Adaptation
Brigham Young University · Carnegie Mellon University
What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization
Oct 5, 2026Ayesh Abu Lehyeh, Jay Hwasung Jung, Safwan WshahCross-View Geo-Localization Large Vision-Language Models
Department of Computer Science, University of Vermont
Fitting Vision Adapters at Frontier Scales
Oct 5, 2026Jaehoon Lee, Harry Partridge, Mudith Jayasekara +3Vision-Language Models VLM Adaptation
Base Labs
IRSTD-Agent: Agentic Infrared Small Target Detection via Zoom-Guided Interaction Learning
Oct 4, 2026Jiawen Xi, Yu Zhang, Tianyi Zhao +3Tool Use in VLMs Infrared Small Target Detection
Beihang University · Dalian University of Technology
MoLE: Mixture of Latent Experts for Complementary Visual Reasoning
Oct 1, 2026Yingcheng Liu, Tianyi Jiang, Yujuan Ding +5Large Vision-Language Models Latent Visual Reasoning
Tongji University, China · Hong Kong Polytechnic University, Hong Kong · Alibaba Group, China +1
VETO: Video Efficient Token Optimization for Vision Language Models
Oct 1, 2026Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen +1Efficient VLM Inference Large Vision-Language Models
National Taiwan University · NVIDIA
Architectural Sampling: Test-Time Scaling via Computational Diversity in Frozen Vision-Language Models
Oct 1, 2026Akshit Singh, Shyam Marjit, Wei Lin +2Test-Time Scaling for VLMs Large Vision-Language Models
Dartmouth College · IISc Bangalore · Xero
Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting
Sep 30, 2026Chang Liu, Yu Tian, Rui XieFDR Control Large Vision-Language Models
School of Data, Mathematical, and Statistical Sciences Institute of Artificial Intelligence University of Central Florida
SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Models
Sep 30, 2026Rafi Ibn Sultan, Xiangyu Zhou, Md. Sajid Alam Chowdhury +4Visual Spatial Reasoning Large Vision-Language Models
Department of Computer Science, Wayne State University · Department of Radiation Oncology, Henry Ford Health · Department of Electrical and Computer Engineering, The Ohio State University +1
LoopVL: Recurrent Visual Intelligence
Sep 29, 2026Zhe Qian, Ziyang Gong, Zhongxing Xu +9Vision-Language Models Large Vision-Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Baidu · Shanghai Jiao Tong University +5
Selective Channel Restoration for Backdoored Vision-Language Models
Sep 29, 2026Shuming Liu, Zhifang Zhang, Suqin Yuan +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · University of Sydney +1
Are In-Context Images Worth 10 Dimensions?
Sep 29, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Vision-Language Models Multimodal ICL
Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, Paris, France · Pôle recherche de l’AMIAD, Palaiseau, France · Institut Universitaire de France, Paris, France
Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning
Sep 29, 2026Bowen Yuan, Danny Wang, Ruihong Qiu +2Large Vision-Language Models VLM Interpretability
The University of Queensland
TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning
Sep 29, 2026Jing Wang, Zhiping Wu, Dongdong Ren +3Efficient VLM Inference Large Vision-Language Models
School of Intelligence Science and Technology, Nanjing University · School of Electronic Science and Engineering, Nanjing University · Geely Automobile Research Institute (Ningbo) Co., Ltd., 315000 +1
Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery
Sep 29, 2026Siqi Lu, Suo Wei, Yongbin Zheng +3Vision-Language Models Efficient VLM Inference
College of Intelligence Science and Technology, National University of Defense Technology, China · School of Computer Science, Northwestern Polytechnical University, China · Ningbo Institute, Northwestern Polytechnical University, China +1
DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing
Sep 29, 2026Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon ParkVision-Language Models Large Vision-Language Models
Korea University, Seoul, Republic of Korea
Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision
Sep 28, 2026Hanoona Rasheed, Mohammed Irfan Kurpath, Bin Ren +3Vision Foundation Models Large Vision-Language Models
Mohamed bin Zayed University of Artificial Intelligence · Apertix
Beyond Selection: Token Parameterization for Extreme Visual Token Compression
Sep 28, 2026Rui Zhong, Yu Li, Zheyu Yan +1Efficient VLM Inference Large Vision-Language Models
Zhejiang University
ReSight-SMC: Two-Stage Power Sampling via Island SMC with Visual Scouts
Sep 28, 2026Yaowen Zhang, Xiangyu Qiu, Junyi Hu +5Markov Chain Monte Carlo Large Vision-Language Models
Independent Researcher · University of Electronic Science and Technology of China · Tsinghua University
Beyond Reconstruction Loss in Post-Training Quantization: Balanced Fitting for Large Vision-Language Models
Sep 28, 2026Minchan Kang, Kyeonghye Park, Seungyeon Sa +3VLM Quantization Large Vision-Language Models
Korea Advanced Institute of Science and Technology (KAIST) · Hanbat National University
Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos
Sep 28, 2026Fangzhou Ma, Ivo Alexander Ban, Eren Homburg +5Temporal Video Grounding Visual Spatial Reasoning
ETH Zurich · Microsoft Spatial AI Lab · Bocconi University
Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
Sep 28, 2026Xi Xiao, Tianchen Zhao, Youngeun Kim +10Large Vision-Language Models Latent Visual Reasoning
University of Alabama at Birmingham · Amazon AGI · Work done during an internship at Amazon AGI.
ACPruner: Visual Token Pruning as Biased Attention Coverage Maximization in LVLMs
Sep 28, 2026Xu Li, Yuxuan Liang, Yi Zheng +6Efficient VLM Inference Large Vision-Language Models
College of Computer Science and Artificial Intelligence Fudan University, Shanghai, China
Distilling Visual Reasoning into Text Space
Sep 28, 2026Wenhan Yang, Nilay Naharas, Ali Payani +1CoT Distillation VLM Distillation
University of California, Los Angeles · Cisco Research
MaLiang-Harness: A Programmable Path to Image and Video Generation
Sep 28, 2026Haoyu Zhao, Zihao Zhang, Xudong Wang +4Image Generation Large Vision-Language Models
National University of Singapore · Fudan University · Tencent
Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
Sep 26, 2026Junxian Li, Ruixuan Yang, Tianao Zhang +3Efficient VLM Inference VLM Distillation
Shanghai Jiao Tong University · Xi’an Jiaotong University · University of Cambridge +1
RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Sep 21, 2026Zhiping Wu, Dongdong Ren, Yangchengyu Zhou +4VLM Quantization Large Vision-Language Models
State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China
VPRune: Efficient Training-free Pre-LLM Visual Token Pruning
Sep 21, 2026Guangchuan Lv, Dianxing Shi, Dingjie FuEfficient VLM Inference Large Vision-Language Models
Northeastern University · Beihang University · Huazhong University of Science and Technology

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions

Bojun Yang, Haochen Zhou, Zhifang Zhang, Haobo Wang+2

Southeast University · The University of Queensland · Zhejiang University

VLM Robustness Large Vision-Language Models Backdoor Attacks on VLMs

Open paper AskPDF arXiv

Large vision-language models (LVLMs) are increasingly deployed in safety-critical applications, yet they remain vulnerable to backdoor attacks. Defending against such attacks remains costly, as existing methods require either extensive retraining on clean data or per-query intervention at inference time. To address this limitation, we propose OrthoPurify, a more efficient method to purify backdoored model weights via one-step orthogonal projection. Specifically, through structural analysis of backdoor weight updates, we find that the backdoor is encoded by diverting a small number of weight update directions from task adaptation to backdoor shortcut encoding, a phenomenon we term direction hijacking. However, identifying these hijacked directions requires a benign reference model, which is typically inaccessible to the defender. We show that a pseudo-benign model, obtained by fine-tuning the pretrained weights on only a small set of clean samples, provides a sufficient approximation, as the dominant update directions stabilize within the first few gradient steps. OrthoPurify uses this pseudo-benign reference to isolate the hijacked directions and removes them through a single projection on the weight update. Extensive experiments show that OrthoPurify reduces the attack success rate to near zero while preserving the original performance across diverse benchmarks, without retraining the backdoored model or introducing inference-time overhead. Our code is publicly available at https://github.com/womeimingzi/OrthoPurify.

Momentum

Related topics

Latest papers 253