Diagnosing Dense Same-Class Attribute Misbinding in Large Vision-Language Models
Authors: Yuanzhi Xu, Qian Gao, Jun Fan, Guohui Ding, Zhenyu Yang, Yuteng Xiao, Sixue Lin
Organizations: Qilu University of Technology (Shandong Academy of Sciences), Jinan, China · China Telecom Digital Intelligence Technology Co., Ltd., Jinan, China · Shenyang Aerospace University, Shenyang, China · University of Nottingham Ningbo China, Ningbo, China
Large vision-language models can recognize the objects and attributes in a crowded scene yet assign an attribute to the wrong same-class instance. Generic visual-question-answering accuracy marks the response as wrong, while object-hallucination metrics may regard both the object and attribute as image-supported; neither reveals the transfer. This study formalizes this blind spot as Dense Same-Class Attribute Misbinding (DSCAM) and presents InstaBind-Lite, a controlled benchmark that makes it directly measurable. Its 524 images contain 529 curated groups of 3-6 same-class entities, 1773 boxed instances, ordered neighbors, distinguishable color-like attributes, and four complementary question levels, yielding 9580 deterministically evaluated questions. Unlike existing protocols, source-instance annotations separate unsupported generation and recognition failure from an attribute copied from another visible entity. Binding-specific metrics further quantify transfer frequency, adjacency, ordinal distance, and intervention effects. Across five open-source and two commercial/API models, the open-source systems average 19.84% Misbinding Rate and the API systems 7.55%; these errors are hidden by aggregate accuracy. Among identifiable transfers, 80.70% and 81.51%, respectively, originate from adjacent instances. Localization and instance-first interventions help selected models but are not universal remedies. InstaBind-Lite therefore turns previously undifferentiated wrong answers into source-identifiable failure categories and tests a reliability dimension that conventional benchmarks cannot determine: whether a model knows not only what is visible, but which instance owns each attribute.
Figures & tables
Evaluation family
Primary diagnostic target
Controlled same-class groups
Wrong-source attribution
Spatial misbinding metrics
Object hallucination
Unsupported objects or attributes
No
No
No
General VQA / multimodal
End-answer correctness and broad capability
Partial
No
No
Referring / grounding
Phrase-to-region localization
Partial
No
No
Compositionality
Attribute–object and relation sensitivity
Partial
No
No
InstaBind-Lite
Same-class instance–attribute binding
Yes
Yes
Yes
TABLE I: Comparison by diagnostic objective. “Partial” indicates that same-class instances may occur naturally, but are not jointly controlled with source-instance attribution for misbinding analysis.
Item
Value
Images
524
Same-class groups
529
Instances
1773
Questions
9580
Person groups
36.86%
Non-person groups
63.14%
TABLE II: Dataset statistics of InstaBind-Lite v0.4.
Model
Type
Acc.
MBR
Err-MBR
A-MBR
Out-set
Invalid
Qwen2.5-VL-7B
Open
75.89
13.65
56.62
82.65
7.07
0.0
InternVL3-8B
Open
76.69
14.01
60.1
84.65
7.36
0.0
LLaVA-1.5-7B
Open
54.37
34.01
74.54
78.7
7.89
0.0
LLaVA-OneVision-7B
Open
72.3
18.58
67.07
76.97
6.88
0.0
MiniCPM-V-2.6
Open
71.18
18.96
65.77
80.51
7.13
0.0
Gemini-3.5-Flash
API
80.13
5.79
29.15
78.38
9.66
1.42
TABLE III: Full-image performance and misbinding metrics on InstaBind-Lite.
Fig. 1: Full-image accuracy, MBR, and out-of-set hallucination across seven LVLMs. Aggregate accuracy and object-level hallucination do not reveal the visible wrong-instance transfers measured by MBR.
Fig. 2: Full-image error decomposition. Adjacent misbinding forms a substantial error component in open-source models and remains structurally visible in API models.
Fig. 3: Ordinal Distance-MBR. Most identifiable transfers originate from the immediately adjacent same-class instance ( d=1 ), while errors rapidly decline for more distant instances.
Model
Accuracy
MBR
A-MBR
Qwen2.5-VL-7B
75.89 [74.50, 77.50]
13.65 [12.50, 14.83]
82.65 [80.08, 85.36]
InternVL3-8B
76.69 [75.28, 78.12]
14.01 [12.94, 15.03]
84.65 [82.33, 86.97]
LLaVA-1.5-7B
54.37 [53.05, 55.78]
34.01 [32.70, 35.24]
78.70 [77.15, 80.54]
LLaVA-OneVision-7B
72.30 [70.80, 73.73]
18.58 [17.41, 19.85]
76.97 [75.03, 78.95]
MiniCPM-V-2.6
71.18 [69.69, 72.70]
18.96 [17.77, 20.13]
80.51 [78.66, 82.38]
Gemini-3.5-Flash
80.13 [78.57, 81.54]
5.79 [5.02, 6.53]
78.38 [73.93, 82.85]
TABLE IV: Image-cluster bootstrap 95% confidence intervals for full-image evaluation. All intervals are computed by resampling images with replacement, preserving all questions from each sampled image.
Fig. 4: Six manually selected, intervention-confirmed adjacent misbindings. The queried target is outlined in red and the source of the full-image answer in orange. In every panel, the full-image prediction is a visible attribute of the adjacent source instance, whereas both crop and context-crop evaluation recover the gold answer. The first case is from LLaVA-1.5 and the remaining cases are from InternVL3.
Model
Full Acc.
Full MBR
Crop Acc.
Crop MBR
Ctx Acc.
Ctx MBR
Qwen2.5-VL-7B
69.56
14.57
69.44
12.72
73.36
11.24
InternVL3-8B
69.87
13.59
66.7
13.42
70.52
12.27
LLaVA-1.5-7B
47.92
34.33
68.06
15.21
66.86
18.21
LLaVA-OneVision-7B
63.98
20.56
69.56
15.61
69.54
15.84
MiniCPM-V-2.6
62.22
21.76
71.44
11.19
72.8
11.92
Gemini-3.5-Flash
71.46
7.16
71.53
8.97
70.55
9.18
TABLE V: Intervention results on the L1/L3 subset. Crop and context crops are compared against full images.
Model
Crop
Context
Qwen2.5-VL
+1.85 [-0.12, +3.69]
+3.33 [+1.38, +5.18]
InternVL3
+0.16 [-1.68, +2.06]
+1.31 [-0.38, +2.93]
LLaVA-1.5
+19.13 [+16.73, +21.41]
+16.12 [+13.62, +18.40]
LLaVA-OV
+4.95 [+2.89, +6.94]
+4.72 [+2.73, +6.64]
MiniCPM-V
+10.56 [+8.64, +12.45]
+9.83 [+7.87, +11.78]
Gemini
-1.81 [-3.18, -0.23]
-2.02 [-3.52, -0.54]
TABLE VI: Image-cluster bootstrap 95% confidence intervals for MBR reduction on L1/L3. Positive values indicate that the intervention reduces MBR relative to full-image evaluation.
Fig. 5: Intervention effect on L1/L3 MBR. Localization sharply reduces source-attributable misbinding for several open-source models but can remove useful context or shift the input distribution for stronger API systems.
Model
Full Acc.
IF Acc.
Acc. Gap
Full MBR
IF MBR
MBR Gap
Qwen2.5-VL-7B
75.89
73.19
-2.69
13.65
12.76
-0.9
InternVL3-8B
76.69
78.24
1.54
14.01
9.65
-4.36
LLaVA-1.5-7B
54.37
31.8
-22.58
34.01
35.16
1.15
LLaVA-OneVision-7B
72.3
70.66
-1.64
18.58
19.83
1.25
MiniCPM-V-2.6
71.18
73.35
2.17
18.96
15.5
-3.46
TABLE VII: Instance-first prompting on open-source LVLMs.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
Department of Computer Science, Pontificia Universidad Católica, Santiago, Chile · 2Centro Nacional de Inteligencia Artificial (CENIA), Santiago, Chile · 3Instituto Milenio en Ingeniería e Inteligencia Artificial para la Salud (iHEALTH), Santiago, Chile.