Large Vision-Language Models

Also known as LVLM

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 36% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 253

All topics

AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model
Jun 17, 2026Diogo Glória-Silva, João Cardeira, Manuel Letras da Luz +8Vision-Language Models Large Vision-Language Models
1NOVA School of Science and Technology · 2NOVA LINCS
Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding
Jun 15, 2026Yifan Wang, Peiming Li, Shiyu Li +5Efficient VLM Inference Large Vision-Language Models
1Tencent BAC · School of Electronic and Computer Engineering, Peking University · School of Mathematics and Statistics, University of Glasgow +1
Comparing Human Gaze and Vision-Language Model Attention in Safety-Relevant Environments
Jun 13, 2026Marta Vallejo, Siwen WangVLM Evaluation Large Vision-Language Models
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, EH14 4AS, United Kingdom
Understanding Cross-Modal Contributions in Continual Vision-Language Models: A Theoretical Perspective
Jun 12, 2026Salimeh Sekeh, Mary WisellContinual Learning Large Vision-Language Models
San Diego State University
One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs
Jun 12, 2026Yongru Chen, Kai Zhang, Zeliang Zong +4Efficient VLM Inference Large Vision-Language Models
Hikvision Research Institute · East China Normal University · Peking University
ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation
Jun 10, 2026Jiangtao Kong, Peijun Zhao, Chun-Fu Chen +4Large Vision-Language Models Incremental Learning
Department of Computer Science, William & Mary, Virginia, USA · Global Technology Applied Research, JPMorgan-Chase, USA · Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi
Density Ridge Selective Prediction for LLM and VLM Hallucination Detection under Calibration Label Scarcity
Jun 8, 2026Nina I. ShamsiHallucination Detection in VLMs Vision-Language Models
Northeastern University Boston, United States
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Jun 5, 2026Lujun Li, Lama Sleem, Niccolo Gentile +4VLM Evaluation Large Vision-Language Models
University of Luxembourg · Foyer S.A. · Université Paris-Saclay
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
Jun 5, 2026Shizhe Xiang, Ke An, Wenlong Yu +4Large Vision-Language Models Multimodal Reasoning
1Tianjin University · 2Beijing Institute of Technology · 3Singapore Management University +2
Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation
Jun 4, 2026Haocheng Luo, Jiahui Liu, Ruicheng Zhang +6Cross-Modal Knowledge Distillation Visual Spatial Reasoning
1Tsinghua University · 2The Hong Kong University of Science and Technology
Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
Jun 4, 2026Tianxiang Jiang, Linquan Wu, Sheng Xia +5Large Vision-Language Models Latent Visual Reasoning
University of Science and Technology of China · Shanghai AI Laboratory · City University of Hong Kong +4
Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs
Jun 2, 2026Wei Ding, Yudong Zhang, Ruobing Xie +3VLM Evaluation Vision-Language Models
Tsinghua University Beijing, China · Tsinghua University, Tencent Beijing, China · University of Science and Technology Beijing Beijing, China
Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation
Jun 2, 2026Ruipeng Zhang, Zhihao Li, C. L. Philip Chen +1VLM Hallucination Large Vision-Language Models
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, School of Computer Science & Engineering, South China University of Technology, Guangzhou, China · Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou, China · Pazhou Lab, Guangzhou, China.
BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks
Jun 1, 2026Ivan Sabolić, Marin Oršić, Josip Šarić +1Supervised Fine-Tuning VLM Robustness
Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia.
ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning
Jun 1, 2026Yu-Cheng Shi, Zhen-Hao Xie, Jun-Tao Tang +1Large Vision-Language Models Adapter Tuning
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models
Jun 1, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
Harbin Institute of Technology · Xiaohongshu · Fudan University
Self-Improving Small Object Grounding in LVLMs
Jun 1, 2026Tianze Yang, Yucheng Shi, Ruitong Sun +2Large Vision-Language Models VLM Interpretability
University of Georgia
An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation
May 31, 2026Bingyu Li, Da Zhang, Tao Huo +3Large Vision-Language Models Referring Image Segmentation
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence (TeleAI), China Telecom, China · School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China
ETC: Extreme Token Compression via Task-aware Visual Information Distillation in VLMs
May 30, 2026Yiling Gao, Hongchen Wei, Zhenzhong ChenEfficient VLM Inference Large Vision-Language Models
School of Remote Sensing and Information Engineering, Wuhan University
Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models
May 29, 2026Zijie Zhou, Dandan Zhu, Hangxiangpan Wang +3Vision-Language Models Large Vision-Language Models
China University of Petroleum (Beijing) · Hainan Institute of China University of Petroleum (Beijing) · South China Normal University
SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
May 29, 2026Olaf Dünkel, Basavaraj Sunagad, Haoran Wang +3VLM Evaluation Semantic Correspondence
Max Planck Institute for Informatics, SIC · CISPA Helmholtz Center for Information Security · University of Freiburg
Personalize Your Large Vision-language Models With In-context Prompt Tuning
May 29, 2026Yanshu Li, Jiaqian Li, Kuai Yu +4Vision-Language Models Large Vision-Language Models
Brown University · Columbia University · University of Alabama at Birmingham +2
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
May 29, 2026Chang-Bin Zhang, Yujie Zhong, Qiang Zhang +1Large Vision-Language Models Vision-Language Grounding
Visual AI Lab, The University of Hong Kong · Independent Researcher · University of Science and Technology of China
Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness
May 29, 2026Xiang Fang, Wanlong Fang, Wei JiVLM Robustness Large Vision-Language Models
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · Nanjing University
ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models
May 28, 2026Zihu Wang, Karthik Somayaji N. S, Peng LiVisual Reasoning Large Vision-Language Models
University of California, Santa Barbara
PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding
May 28, 2026Selim Kuzucu, Alessio Tonioni, Vasile Lup +3Image Tokenization Efficient VLM Inference
Max Planck Institute for Informatics, SIC · Google · Technical University of Munich
Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering
May 28, 2026Soumyadeep Jana, Pulkit Mittal, Sanasam Ranbir SinghHallucination in Language Models Large Vision-Language Models
Indian Institute of Technology Guwahati, India
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
May 28, 2026Geng Li, Guohao Chen, Ting Chen +6Efficient VLM Inference Large Vision-Language Models
1Nanyang Technological University (NTU)
Self-Prophetic Decoding to Unlock Visual Search in LVLMs
May 27, 2026Zhendong He, Qiyuan Dai, Guanbin Li +2Large Vision-Language Models VLM Reasoning
School of Computer Science and Engineering, Sun Yat-sen University · ShanghaiTech University
Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models
May 27, 2026Yuang Huang, Yafeng Zhang, Yu ZilanVision-Language Models Hallucination in Language Models
Shanghai Jiao Tong University · iFLYTEK · Tsinghua University
Structure-Guided Visual Perturbation Neutralization for LVLMs
May 27, 2026Yuanhe Zhang, Xueting Wang, YanBin Ren +6VLM Robustness Adversarial Attacks on VLMs
1Beijing University of Posts and Telecommunications · University of Science and Technology of China · 3JIUTIAN Research +2
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
May 26, 2026Joseph Hoche, David Brellmann, Gianni FranchiVision-Language Models Large Vision-Language Models
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · ENSTA Paris
LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
May 25, 2026Xiang An, Yin Xie, Feilong Tang +27VLM Evaluation Temporal Video Grounding
Glint Lab · AIM for Health Lab · MVP Lab
Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?
May 25, 2026Jing Huang, Duanchu Wang, Junjie Yang +5VLM Evaluation Remote Sensing Image Understanding
School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Shenyang Institute of Automation, University of Chinese Academy of Sciences, Shenyang, China · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University
Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation
May 25, 2026Ruoxi Cheng, Haoxuan Ma, Zhengfei Hai +6Adversarial Representation Learning Vision-Language Models
Fudan University · Tencent · Nanjing University +3
VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation
May 23, 2026Bo Li, Ronghao Chen, Ningyuan Deng +3VLM Adaptation Large Vision-Language Models
Tsinghua University Baidu Inc. Beijing, China · QuantaAlpha Beijing, China · The Hong Kong University of Science and the Technology Hong Kong, China +1
PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs
May 22, 2026Rim Assouel, Amir Bar, Michal Drozdzal +1Visual Spatial Reasoning Synthetic Data Generation
1Mila - Québec AI Institute · 4McGill University · 3FAIR at Meta Superintelligence Labs +2
CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
May 22, 2026Liupeng Li, Haoqian Kang, Zhenyu Lu +4Efficient VLM Inference Large Vision-Language Models
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
May 21, 2026Xiang Chen, Yuxian Dong, Chao Li +6VLM Robustness Adversarial Patch Attacks
Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
May 20, 2026Meng Shen, Minghao Wu, Deepu RajanVision-Language Models Hallucination in Language Models
Nanyang Technological University · Monash University
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
May 20, 2026Yutong Xie, Zhenglin Hua, Ran Wang +3Vision-Language Models Hallucination in Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · National Engineering Laboratory for Big Data Systems Computing Technology, Shenzhen University, Shenzhen, China +3
Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
May 19, 2026Yi Zhong, Haotong Qin, Xindong Zhang +2Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · D-ITET, ETH Zürich · OPPO Research Institute +1
Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
May 18, 2026Shravan Murlidaran, Ziqi Wen, Sana Shehabi +1Visual Attention Large Vision-Language Models
Psychological & Brain Sciences, University of California, Santa Barbara & 93106, USA. · Computer Science, University of California, Santa Barbara & 93106, USA. · Electrical and Computer Engineering, University of California, Santa Barbara & 93106, USA.
When a Zero-Shooter Cheats: Improving Age Estimation via Activation Steering
May 17, 2026Erik Imgrund, Pia Hanfeld, Klim Kireev +1VLM Robustness Large Vision-Language Models
BIFOLD & TU Berlin
DriveSafe: A Framework for Risk Detection and Safety Suggestions in Driving Scenarios
May 16, 2026Sainithin Artham, Shankar Gangisetty, Avijit Dasgupta +1VLMs for Autonomous Driving Autonomous Driving Safety Evaluation
IIIT-Hyderabad, India
A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
May 15, 2026Hao Yang, Zhuo Ma, Yang Liu +3VLM Robustness Adversarial Attacks on VLMs
Xidian University Xi’an, Shaanxi, China
EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
May 15, 2026Xuanyu Ge, Zhongqi Wang, Jie Zhang +2Large Vision-Language Models Backdoor Detection
China University of Geosciences · University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University
AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models
May 15, 2026Zhiwei Li, Jiacheng Xue, Weining Wang +4VLM Robustness Adversarial Robustness
1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · School of Computer Science and Engineering, Central South University · University of Chinese Academy of Sciences
Deep Pre-Alignment for VLMs
May 14, 2026Tianyu Yu, Kechen Fang, Zihao Wan +5Vision-Language Models Large Vision-Language Models
1Tsinghua University · 2Taobao & Tmall Group of Alibaba · 3Shanghai Qi Zhi Institute.
ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both
May 14, 2026Ziyu Guo, Rain Liu, Xinyan Chen +1Visual Reasoning Large Vision-Language Models
Meta AI · The Chinese University of Hong Kong
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
May 14, 2026Wei Ding, Yilin Li, Yudong Zhang +4Vision-Language Models LLM Hallucination Mitigation
Tsinghua University Beijing, China · Tsinghua University, Tencent Beijing, China · Tencent Beijing, China +2
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding
May 14, 2026Ailar Mahdizadeh, Puria Azadi, Muchen Li +2Streaming Video Understanding KV Caching
University of British Columbia · Vector Institute
To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model
May 14, 2026Chengshuai Zhao, Zhen Tan, Dawei Li +2VLM Robustness Multimodal Robustness
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA.
Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
May 13, 2026Zhaowei Wang, Lishu Luo, Haodong Duan +9Visual Question Answering Vision-Language Models
1CSE Department, HKUST · 2ByteDance Seed · *Work done at ByteDance Seed
Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models
May 12, 2026Furkan Mumcu, Michael J. Jones, Anoop Cherian +1Large Vision-Language Models Anomaly Localization
University of South Florida · Mitsubishi Electric Research Laboratories (MERL)
Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Models
May 12, 2026Runhe Lai, Xinhua Lu, Yanqi Wu +3Hallucination Detection in VLMs Hallucination Detection
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Peng Cheng Laboratory, Shenzhen, China · Key Laboratory of Machine Intelligence and Advanced Computing, MOE, Guangzhou, China.
VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority
May 12, 2026Chenhao Qiu, Yechao Zhang, Xin Luo +2Large Vision-Language Models Video QA
Mango TV · Mango TV, Changsha, China · Nanyang Technological University +1

Jun 17, 2026, cs.CVJ/K move · Enter open · S save

AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

Diogo Glória-Silva, João Cardeira, Manuel Letras da Luz, Afonso Simplício+7

1NOVA School of Science and Technology · 2NOVA LINCS

Vision-Language Models Large Vision-Language Models

Open paper AskPDF arXiv

Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-source multimodal models, which either conflate it with Brazilian Portuguese or severely under-represent it in their training data mixes. We introduce AMALIA-VL, the first open-source instruction-tuned LVLM built natively for pt-PT, pairing a high-resolution vision encoder with dynamic image tiling and a fully open pt-PT-optimized language model via a learned connector. We contribute with a purposefully designed three-stage training process - vision-language alignment, general visual instruction tuning, and preference optimization - together with a pt-PT-centric multimodal data mix combining curated and translated public datasets with novel datasets that address the near-total absence of European Portuguese multimodal resources. Our evaluation shows that AMALIA-VL establishes a strong baseline for open-source pt-PT LVLMs. We will release model weights, training data, and construction pipelines along with machine-translated pt-PT evaluation benchmarks to help democratize pt-PT LVLM development.

Momentum

Related topics

Latest papers 253