cs.CVOct 23, 2026

MiD-VLM: Detecting and Grounding Global Semantic Misinformation in Image-Caption Pairs

Authors: Tarunya Prasad, Bharatesh Chakravarthi

Organizations: Arizona State University

Abstract

Semantic misinformation can arise when an authentic image is paired with a caption that changes or distorts its narrative. Because the image itself may contain no visible signs of manipulation, methods designed to detect pixel-level artifacts can miss these misleading image-caption pairs. We study this problem using DGM4+, a dataset designed to evaluate the detection of global scene and semantic inconsistencies. We propose a multimodal model that combines image-text representations from SigLIP with visual scene features from DINOv2 through a projection MLP, and adapts Qwen2-VL-7B using LoRA. The model jointly predicts whether an image-caption pair is authentic, identifies the manipulation type, and grounds the misleading caption tokens. On the DGM4+ validation split, the model achieves an F1 score of 0.96790.9679 for binary authenticity classification. These results suggest that parameter-efficient adaptation of vision-language models is promising for detecting semantic inconsistencies, while also motivating further work on precise token-level grounding.

Explore similar work

CardsList
  1. GAVEL: Grounded Caption Error Verification and Localization

    Jun 25, 2026Zixian Gao, Atsushi Hashimoto, Kuniaki SaitoVLM EvaluationText-Image Alignment

  2. When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations

    Mar 23, 2026Harsh Nishant Lalai, Raj Sanjay Shah, Hanspeter Pfister +2VLM EvaluationChart Understanding

  3. ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

    Jun 23, 2026Chenhao Dang, Dantong Zhu, Jun Yang +2Multimodal AgentsMultimodal Model Evaluation