alkhder at ImageEval 2026 Shared Tasks: Constrained Selection and Rotation-Averaged Logit Scoring for Multimodal Hallucination Detection

Maher Pasa; Maria Hanini; Amro Najjar; Hasan Alkhder

cs.CLOct 15, 2026

alkhder at ImageEval 2026 Shared Tasks: Constrained Selection and Rotation-Averaged Logit Scoring for Multimodal Hallucination Detection

Authors: Maher Pasa, Maria Hanini, Amro Najjar, Hasan Alkhder

Organizations: Altınbaş University, Türkiye · Imperial College London · Luxembourg Institute of Science and Technology · Sakarya University, Türkiye

Abstract

We describe our submission to Subtask~1b of ImageEval 2026. The task consists of selecting the single visually valid statement from three candidates paired with an image, and is in both English and Modern Standard Arabic (MSA). Our system uses a pre-trained 7B vision--language model in a zero-shot setting, without task-specific training or fine-tuning. We formulate the task as a constrained selection problem and choose the answer using next-token log-probabilities. To reduce answer-position bias, we apply a Latin-square rotation to the candidate statements. Our system achieves CI scores of 0.0510 on the English test set and 0.0960 on the MSA test set, reducing CI relative to the official 3B baselines reported on devtest by 83.7% and 80.4%, respectively.

Figures & tables

Panel of Figure 1 — Figure 1: System overview.

System	CI $\downarrow$	Comb. $\uparrow$	$Q^{+}\uparrow$	$Q^{-}\uparrow$
English track ( test , $n{=}1000$ )
Baseline 3B †	0.3133	0.6840	0.9400	0.8390
Rank 1 ( miles_ma )	0.0290	0.9710	0.9710	0.9855
Ours (rank 5)	0.0510	0.9490	0.9490	0.9745
MSA track ( test , $n{=}1000$ )
Baseline 3B †	0.4900	0.5080	0.9400	0.8390

Table 1: Official Subtask 1b results. † Baseline figures are reported on devtest .

Configuration	Comb. acc	CI $\downarrow$
Pointwise only	0.9440	0.0560
Multiple-choice only	0.9500	0.0500
Both, 1:1 ( submitted )	0.9500	0.0500
Both, swept $W_{\text{PW}}{=}0.25$	0.9520	0.0480

Table 2: View ablation on English dev (

n{=}500

Team	EN	MSA	$\Delta$
NYUAD	0.9590	0.9530	0.6
myamin66	0.9670	0.9440	2.3
keslerady	0.8870	0.8580	2.9
md_faisal	0.9330	0.8990	3.4
alkhder (ours)	0.9490	0.9040	4.5

Table 3: Combined accuracy per track and the gap in points, for teams appearing on both EN and MSA tracks.

Category	$n$	EN acc.	MSA acc.
Objects & Clothing	94	0.9681	0.9043
Vehicles & Transport	148	0.9662	0.9189
People & Society	185	0.9568	0.9351
Culture & Arts	175	0.9543	0.9029
Buildings & Landmarks	53	0.9434	0.9434
History & Identity	64	0.9375	0.9219

Table 4: Combined accuracy by cultural category.

Country	$n$	EN acc. [95% CI]	MSA acc. [95% CI]
Morocco	85	0.9882 [0.9363, 0.9979]	0.9412 [0.8696, 0.9746]
UAE	81	0.9877 [0.9333, 0.9978]	0.9136 [0.8322, 0.9575]
Sudan	80	0.9875 [0.9325, 0.9978]	0.9125 [0.8302, 0.9570]
Palestine	84	0.9643 [0.9002, 0.9878]	0.8929 [0.8088, 0.9426]
Kuwait	91	0.9560 [0.8924, 0.9828]	0.9231 [0.8496, 0.9622]
Syria	83	0.9518 [0.8825, 0.9811]	0.8795 [0.7922, 0.9332]

Table 5: Combined accuracy by country of origin with 95% confidence intervals.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Model	Qwen2.5-VL-7B-Instruct
Precision	bf16 (not quantised)
MAX_PIXELS	$1024\times 28\times 28$
Training	none (zero-shot)
Views	pointwise $+$ multiple-choice
Rotations	$(0,1,2),(1,2,0),(2,0,1)$

Appendix

Table 6: Submitted system configuration.

Item	Grounded statement (gold)	Model selection	Result
UAE c8cfb559 Traditional Cuisines	“The dish shown in the image is likely Middle Eastern cuisine.” (distractor: Italian, Indian)	same as gold statement	✓ (EN & MSA)
Lebanon 4ff7e8f3 Rituals & Ceremonies	“The patterned rugs are likely used to provide a decorative cultural element for the gathering.” (culturally expected distractor: “seating area for guests”)	same as gold statement	✓ (EN & MSA)
Iraq c409e355 Healthcare & Well-being	“The primary focus of the soldiers in the image is providing medical assistance.”	“The primary focus of the soldiers in the image is engaging in combat.”	× (EN & MSA)
Syria 7395c814 Heritage & History	“The occasion depicted in the image is a religious foot-washing ritual.” (MSA gold mistranslated as “strange religious rituals”)	“The occasion depicted in the image is a baptism event.”	✓ EN, × MSA

Appendix

Table 7: Qualitative examples from the released test labels, verified against the images. Statements are quoted from the released English data; the MSA track used the released Arabic translations.

Position Bias in VLMs

See all 9 papers

ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation
Aug 31, 2026Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti +11Hallucination Detection T2I Generation
Texas A&M University · Qatar Computing Research Institute, Qatar · Birzeit University, Palestine +2
HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA
Sep 10, 2026Syed Mohaiminul Hoque, Md Sakhawat HossainVLM Evaluation Hallucination Detection in VLMs
Independent Author
FAU at ImageCLEF 2026 Task on Multimodal Reasoning Robust Candidate Scoring and Concise Multilingual Visual Answering
Aug 3, 2026Mohamed Basem, Vincent ChristleinVisual Question Answering Multimodal Reasoning
Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

Aug 31, 2026J/K move · Enter open · S save

ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation

Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti, Farina Amir+10

Texas A&M University · Qatar Computing Research Institute, Qatar · Birzeit University, Palestine+2

Hallucination Detection T2I Generation Multimodal QA Multilingual VLM Evaluation Multimodal Model Evaluation T2I Generation Evaluation

Open paper AskPDF arXiv

We present an overview of the ImageEval 2026 shared task on culturally grounded Arabic multimodal evaluation. It includes two tasks: (i) AynVQA, covering spoken visual question answering and image-grounded hallucination detection in English and Modern Standard Arabic (MSA), and (ii) CRAI-Bench, evaluating the cultural accuracy of text-to-image generation. A total of 14 teams participated in the test phase, with 12 teams submitting system description papers. Participating systems used a range of approaches, including zero-shot prompting, fine-tuning of vision-language models, speech-recognition pipelines, ensembling, and score calibration. We describe the task setup, datasets, evaluation procedure, and participating systems, and summarize the main results across the different tracks. All datasets and evaluation scripts from the shared task are released to the research community. The shared task highlights the challenges of culturally grounded multimodal evaluation, particularly for Arabic speech and image-text reasoning.

Abstract

Position Bias in VLMs

Explore similar work

ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation

HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA

FAU at ImageCLEF 2026 Task on Multimodal Reasoning Robust Candidate Scoring and Concise Multilingual Visual Answering