cs.CLNov 10, 2026

Sckwoky at SHROOM-Visions: Character-Level Hallucination-Span Detection in Image-Conditioned VLMs via Dual-Prompted LoRA Decoders

Authors: Mikhail Kulakov, Ruslan Solikhov, Ivan Bondarenko

Organizations: Novosibirsk State University

Abstract

We present our system for the SHROOM-visions 2026 shared task on character-level hallucination span detection in image-conditioned vision-language model (VLM) responses, evaluated across English, French, Italian, and Chinese. The detector is built on a frozen Gemma-4-12B-it backbone with trainable LoRA adapters (approximately 10.7M parameters). Because the backbone is a causal decoder, hidden states read from a single copy of an answer cannot attend to subsequent tokens; we therefore repeat the candidate answer within one prompt and extract features from its second occurrence, giving every position approximate bidirectional context in a single non-autoregressive pass. A character-level locator combines intermediate hidden states to predict hallucination probabilities, BIO boundaries, and a response-level abstention score, while separate language-specific classifiers assign one of five hallucination types to each predicted span. On the closed test set, our system reaches a macro-averaged character IoU of 0.4404, more than doubling the official baseline (0.1854), and ranks 5th-6th on IoU across the four languages. On the English tuning split, controlled ablations attribute a gain to the repeated answer that is comparable to that of LoRA adaptation (its held-out gain is smaller and concentrated on hallucinated answers), and they show that the detector relies on the image-answer pairing: shuffling images across examples drives IoU down to the level of the predict-nothing floor or below it in every language. In deployments where the same Gemma-4-12B-it model also generates answers, its base weights can be reused, making the approach practical for local question-answering systems that cannot run a separate verifier.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

    Aug 26, 2026Raúl Vázquez, Aman Sinha, Chuyuan Li +12Hallucination Detection in VLMsHallucination Detection

  2. Vroom-Vroom at SHROOM-Visions: A Multi-Judge Committee for Detecting Hallucinated Spans in Vision-Language Outputs

    Sep 15, 2026Toqeer Ehsan, Nico Penttilä, Richard Schmidt +2VLM EvaluationHallucination Detection in VLMs

  3. SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models

    Aug 30, 2026Amanuel Gizachew Abebe, Yasmin MoslemHallucination Detection in VLMsHallucination Detection