Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 491

All topics

MAGIC: Multimodal Alignment & Grounding-aware Instruction Coreset for Vision-Language Models
May 25, 2026Shristi Das Biswas, Kaushik RoyVision-Language Models Coreset Selection
Purdue University
MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning
May 25, 2026Aritra Dutta, Somak AdityaVision-Language Models Efficient VLM Inference
Indian Institute of Technology, Kharagpur
Binding Visual Features Point by Point
May 25, 2026Udith Haputhanthri, Declan Campbell, Rim Assouel +2Vision-Language Models VLM Interpretability
Princeton University · Mila – Quebec AI Institute · Université de Montréal
Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation
May 25, 2026Ruoxi Cheng, Haoxuan Ma, Zhengfei Hai +6Adversarial Representation Learning Vision-Language Models
Fudan University · Tencent · Nanjing University +3
Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation
May 24, 2026Yangneng Chen, Jing LiVision-Language Models VLM Adaptation
1Harbin Institute of Technology, Shenzhen, China.
Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
May 24, 2026Yuanzhi Xu, Qian Gao, Jun Fan +4VLM Robustness Vision-Language Models
Qilu University of Technology (Shandong Academy of Sciences) Jinan, China · China Telecom Digital Intelligence Technology Co, Ltd Jinan, China · Shenyang Aerospace University Shenyang, China +1
From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks
May 23, 2026Bruce Changlong Xu, Jose James, Alexander RyuVision-Language Models Medical Image Classification
Department of Computer Science, Stanford University, Stanford, CA, USA · Mayo Clinic, Rochester, MN, USA.
VectorArk: Learning Practical Image Vectorization with Rounded Polygon Representation
May 23, 2026Tarun Gehlaut, Difan Liu, Charu Bansal +5Vision-Language Models
Adobe
Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems
May 22, 2026Alan Gerson Contreras Montanares, Luis Valenzuela, Luis Martí +1Vision-Language Models Vision Foundation Models
Inria Chile Research Center · Inria — Institut national de recherche en sciences et technologies du numérique · Av. Apoquindo 2827, Las Condes 7550268 Chile.
DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving
May 22, 2026Hao Vo, Khoa Vo, Phu Loc Nguyen +10VLM Evaluation Autonomous Driving Datasets
University of Arkansas, USA · Google Research, Google · University of Liverpool, UK +1
Cambrian-P: Pose-Grounded Video Understanding
May 21, 2026Jihan Yang, Zifan Zhao, Xichen Pan +5Vision-Language Models Camera Pose Estimation
New York University · UC Berkeley · Meta FAIR
Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs
May 21, 2026Shanshan Wang, Fengying Ye, Hanjia Lyu +6Vision-Language Models AI-Generated Text Detection
Department of Computer and Information Science, University of Macau · University of Rochester · Sichuan University +1
HyLoVQA: Dynamic Hypernetwork-Generated Low-Rank Adaptation for Continual Visual Question Answering
May 21, 2026Yiran Wang, Chenyi Xiong, Ziyue Qin +3Visual Question Answering Vision-Language Models
School of Computer Science, Hubei University, Wuhan 430062, China · Hubei Key Laboratory of Big Data Intelligent Analysis and Application (Hubei University), Wuhan 430062, China · Key Laboratory of Intelligent Sensing System and Security (Hubei University), Ministry of Education, Wuhan 430062, China
EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models
May 21, 2026Shiqi Huang, Ziyue Wang, Zhongrong Zuo +3Vision-Language Models Video Reasoning
School of Electrical and Electronic Engineering, Nanyang Technological University · ByteDance
Visual-Advantage On-Policy Distillation for Vision-Language Models
May 21, 2026Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8Vision-Language Models VLM Distillation
Institute of Automation, CAS · School of Advanced Interdisciplinary Sciences, UCAS · 3Hello +1
SDGBiasBench: Benchmarking and Mitigating Vision--Language Models' Biases in Sustainable Development Goals
May 21, 2026Zihang Lin, Huaiyuan Qin, Muli Yang +1VLM Evaluation Vision-Language Models
1Nanyang Technological University · Institute for Infocomm Research (I2R), A*STAR, Singapore
Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception
May 21, 2026Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape +1Thermal Imaging Visual Question Answering
Department of Electrical & Computer Engineering, Johns Hopkins University, Baltimore, MD 21218 USA
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
May 20, 2026Gonçalo Gomes, Bruno Martins, Chrysoula ZervaVLM Evaluation Image Captioning
Instituto Superior Técnico, University of Lisbon · INESC-ID · Instituto de Telecomunicações
Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
May 20, 2026Meng Shen, Minghao Wu, Deepu RajanVision-Language Models Hallucination in Language Models
Nanyang Technological University · Monash University
Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
May 20, 2026Yutong Xie, Zhenglin Hua, Ran Wang +3Vision-Language Models Hallucination in Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · National Engineering Laboratory for Big Data Systems Computing Technology, Shenzhen University, Shenzhen, China +3
Focus-then-Context: Subject-Centric Progressive Visual Token Reduction for Vision-Language Models
May 20, 2026Yulin Zhao, Zheng ZhangVision-Language Models Efficient VLM Inference
Harbin Institute of Technology, Shenzhen, China · ShenZhen Loop Area Institute
RISE: Reliable Improvement in Self-Evolving Vision-Language Models
May 20, 2026Chaoran Xu, Yingmao Miao, Pengfei Zhang +3Vision-Language Models VLM Adaptation
AMAP, Alibaba Group
MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization
May 20, 2026Zhong Wang, Zukang Xu, Xing Hu +1Mixed-Precision Quantization Vision-Language Models
Bauman Moscow State Technical University · Houmo AI
A Human-in-the-Loop Framework for Efficient Prompt Selection in Microscopy Vision-Language Models
May 19, 2026Abhiram Kandiyana, Ankur Mali, Lawrence O. Hall +2Vision-Language Models Human-in-the-Loop Annotation
University of South Florida, USA · SRC Biosciences, USA
HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation
May 19, 2026Haoyu Wang, Zitong LiVision-Language Models Audio-Language Model Hallucination Detection
Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
FullFlow: Upgrading Text-to-Image Flow Matching Models for Bidirectional Vision--Language Generation
May 19, 2026Eric Tillmann Bill, Enis Simsar, Alessio Tonioni +1Flow Matching Image Captioning
ETH Zurich · Google
From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
May 19, 2026Juncheng Wu, Hardy Chen, Haoqin Tu +6Vision-Language Models VLM Reasoning
Amazon · UC Santa Cruz · University of Waterloo +1
Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
May 19, 2026Yi Zhong, Haotong Qin, Xindong Zhang +2Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · D-ITET, ETH Zürich · OPPO Research Institute +1
Mechanisms of Object Localization in Vision-Language Models
May 19, 2026Timothy Schaumlöffel, Martina G. Vilas, Gemma RoigVision-Language Models VLM Interpretability
Goethe University Frankfurt, Germany · The Hessian Center for AI, Germany
Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay
May 19, 2026Subba Reddy Oota, Anant Khandelwal, Khushbu Pahwa +4Vision-Language Models Neural Encoding
Independent · Microsoft Research, Bangalore, India · AWS AI Labs, Amazon +4

May 25, 2026, cs.CVJ/K move · Enter open · S save

MAGIC: Multimodal Alignment & Grounding-aware Instruction Coreset for Vision-Language Models

Shristi Das Biswas, Kaushik Roy

Purdue University

Vision-Language Models Coreset Selection Multimodal Foundation Models Multimodal Learning Multimodal Instruction Tuning

Open paper AskPDF arXiv

Instruction tuning of large vision-language models (LVLMs) increasingly depends on massive multimodal corpora, yet these datasets contain samples with substantial redundancy, low visual dependency, and highly imbalanced coverage of multimodal reasoning behaviors. As a result, uniform subsampling or naive score-based selection often yields suboptimal training subsets. We introduce MAGIC, a training-free, forward-only coreset selection method designed to construct compact yet behaviorally faithful subsets for multimodal instruction tuning. MAGIC is built on three intrinsic signals extracted from a pretrained VLM: Multimodal Gain, which measures the likelihood improvement obtained from visual input; Bridging Relevance, which captures the sharpness of answer-token grounding over visual tokens; and Skill-Neuron Signatures, which characterize the functional computation elicited by each sample via top-activated feed-forward neurons. MAGIC combines these signals in a three-stage pipeline: filtering low-gain examples, ranking candidates by a normalized quality objective, and performing bucket-wise budget allocation over discrete neuron signatures to preserve latent multimodal skill coverage. This formulation avoids backpropagation, auxiliary selector training, and expensive clustering in continuous activation spaces, while remaining efficient and easily deployable in existing VLMs. Across LLaVA-665K and Vision-Flan datasets, and transfer settings to large target models, LLaVA-1.5-7B and -13B, MAGIC consistently improves over strong baselines under matched 20% budgets: it achieves 100.3% relative performance to full finetuning on LLaVA-665K and 101.6% relative performance on Vision-Flan-186K, while yielding a 73.7% reduction in wall-clock run time.

Momentum

Related topics

Latest papers 491