Vision-language models (VLMs) are increasingly used to interpret religious, educational, and culturally sensitive visual content, yet their reliability on citation verification remains underexplored. We introduce Visual Sanad, a controlled benchmark for testing whether VLMs falsely accept incorrect Qur'anic verse attributions when the same quote-reference pair is rendered under different visual framings. The benchmark contains 2,400 rendered images built from 100 English Qur'an verses, four factual states per verse, and six visual conditions ranging from plain text to decorative and authority-framed layouts. All quote-reference pairs were source-checked and human-audited before rendering. Across 24 full model-prompt runs, we observe a sharp capability split. Proprietary API-served VLMs range from near-perfect citation verification to moderate false acceptance, whereas recent open-weight VLMs often show weak discrimination and default to VALID on most images. De-biased prompting can reduce false acceptance, but often by shifting the decision threshold toward INVALID rather than by producing strong citation discrimination. Visual Sanad therefore exposes a culturally grounded multimodal factuality failure: fluent VLMs may treat displayed religious citations as self-authenticating unless they have strong citation-verification ability.
Figures & tables
Model
Natural false accept ↓
Natural discrimination ↑
Natural says VALID
De-biased false accept ↓
De-biased discrimination ↑
De-biased says VALID
Closed/API VLMs
GPT-4o
14.89
83.44
35.8
13.50
84.50
34.6
GPT-4.1
15.94
83.73
36.9
14.39
85.44
35.8
GPT-5.6-Terra
4.72
95.11
28.5
4.72
95.11
28.5
GPT-5.5
0.00
99.83
25.0
0.00
99.67
24.9
Open-weight VLMs
Table 1: Overall invalid-citation false acceptance, discrimination, and response bias on the locked 2,400-image dataset. Values are percentages. Discrimination is 100−falseaccept−falsereject , so higher is better; false accept is lower better. Says VALID is diagnostic and has no preferred direction.
Figure 1: Invalid-citation false acceptance. (a) Natural-prompt failures show a sharp closed/open split. (b) De-biased prompting reduces false acceptance for some models but does not replace verification.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Example Visual Sanad renderings: valid plain item; invalid plain item; invalid item with decorative Islamic-authority framing.
Model
Prompt
Rows
Errors
Unparsed
Accuracy ↑
False accept ↓
False reject ↓
Discrimination ↑
Says VALID
GPT-4o
natural
2400
0
0
88.42
14.89
1.67
83.44
35.8
GPT-4o
de-biased
2400
0
0
89.38
13.50
2.00
84.50
34.6
GPT-4.1
natural
2400
0
0
87.96
15.94
0.33
83.73
36.9
GPT-4.1
de-biased
2400
0
0
89.17
14.39
0.17
85.44
35.8
GPT-5.5
natural
2400
0
0
99.96
0.00
0.17
99.83
25.0
GPT-5.5
de-biased
2400
0
0
99.92
0.00
0.33
99.67
24.9
Appendix
Table 2: Full model–prompt summary. Accuracy, invalid-citation false accept, valid-citation false reject, discrimination, and VALID response rate are percentages. Discrimination is 100−falseaccept−falsereject ; arrows mark the preferred direction for metric columns. Says VALID is diagnostic and has no preferred direction.
Model
Near-neighbor ↓
Same-surah far ↓
Cross-surah lexical ↓
GPT-4o
17.67
2.83
24.17
GPT-4.1
20.17
3.00
24.67
GPT-5.6 Terra
10.50
1.67
2.00
GPT-5.5
0.00
0.00
0.00
Qwen3-VL-8B
85.00
86.50
78.83
Qwen3-VL-32B
61.17
56.67
38.67
Appendix
Table 3: Natural-prompt invalid false-accept rate by foil type. Values are percentages; lower is better.
Figure 3: Condition-wise invalid false-accept rate under the natural prompt for selected models. Visual framing effects vary by model and are smaller than the overall capability gap, but authority and decoration can still shift false acceptance.
Model
Prompt
Plain ↓
Decorative ↓
Islamic authority ↓
Decorative + Islamic ↓
Neutral authority ↓
Decorative + Neutral ↓
GPT-4o
natural
13.00
16.00
14.67
15.67
12.67
17.33
GPT-4o
de-biased
13.33
15.00
11.33
15.67
12.67
13.00
GPT-4.1
natural
15.00
16.00
16.33
16.00
15.00
17.33
GPT-4.1
de-biased
13.33
13.33
15.00
14.33
14.33
16.00
GPT-5.5
natural
0.00
0.00
0.00
0.00
0.00
0.00
GPT-5.5
de-biased
0.00
0.00
0.00
0.00
0.00
0.00
Appendix
Table 4: Invalid false-accept rate by visual condition. Values are percentages; lower is better. Bold marks the best value in each condition column.
Model
Cue comparison
Net induced
McNemar p↓
GPT-4o
decorative + neutral
13
0.00235
Qwen3-VL-8B
Islamic authority
-34
5.65×10−8
Qwen3-VL-8B
neutral authority
-42
1.28×10−9
Qwen3-VL-32B
decorative
45
6.27×10−8
Qwen3-VL-32B
Islamic authority
-52
5.75×10−11
InternVL3.5-38B
Islamic authority
62
8.47×10−16
Appendix
Table 5: Selected paired visual-cue effects under the natural prompt. Net induced is cue-only minus plain-only false accepts over the same 300 invalid items; positive values increase false acceptance and negative values reduce it. Bold marks the largest shifts.
Model
Invalid pairs
Natural-only false accept ↓
De-biased-only false accept ↓
Net rescue ↑
McNemar p↓
GPT-4.1
1800
50
22
28
0.00129
GPT-4o
1800
57
32
25
0.01055
GPT-5.5
1800
0
0
0
1.00000
GPT-5.6 Terra
1800
51
51
0
1.00000
Gemma 4 12B-IT
1800
36
0
36
2.91×10−11
InternVL3.5-38B
1800
603
0
603
<10−10
Appendix
Table 6: Paired prompt effect on invalid-citation false accepts. Positive net rescue means the de-biased prompt corrected false accepts made under the natural prompt. Lower McNemar p indicates stronger evidence of asymmetric paired flips.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.