When Depth Lies: Benchmarking Vision-Language Models on Mirror-Induced RGB-D Ambiguity
Authors: Hao Yin, Tianchen Guo, Heming Du, Yan Ke, Yanbin Liu, Xin Yu
Organizations: The University of Queensland, Australia · Australian Artificial Intelligence Institute University of Technology Sydney, Australia · Australian Institute for Machine Learning Adelaide University, Australia
Vision-language models (VLMs) have made significant progress in spatial reasoning, particularly by integrating RGB images with depth data, which is often assumed to faithfully reflect physical distances. However, mirrors violate this assumption: they distort depth readings while reproducing object appearances, making reflections difficult to distinguish from real objects. As existing benchmarks rarely consider such reflective scenarios, VLM performance in mirror scenes remains largely unexplored. In this work, we introduce a mirror-centric RGB-D dataset from reflective scenes to evaluate the spatial reasoning capabilities of state-of-the-art VLMs in these scenarios. Specifically, we collect 1,011 RGB-D images from 525 real-world scenes, spanning diverse indoor and outdoor environments, six reflective surface types, and both daytime and nighttime conditions. All images are annotated with reflective surface region masks and mask annotations for both directly observed and mirror-reflected objects. Then, we curate a benchmark named MIRA-Bench (Mirror-Induced Reasoning Ambiguity Benchmark), comprising 14,140 QA pairs across eight sub-tasks organized into three cognitive levels: Reflection-aware Perception, Spatial Reasoning, and Scene-level Decision-making. We evaluate state-of-the-art VLMs on MIRA-Bench and find consistent failures across tasks: models misclassify whether objects are directly observed or reflected, fail to match reflected instances to their real-world counterparts, and produce erroneous spatial inferences. Remarkably, providing depth data actually degrades reasoning performance, implying that VLMs cannot identify incorrect depth in mirror scenes. The code and data are available at https://github.com/EvaYin-cloud/MIRA-Bench
Figures & tables
Figure 1: Overview of the MIRA-Bench construction pipeline . Left: data collection from public datasets and newly captured scenes, followed by filtering and selection. Right: annotation pipeline combining reflective surface mask labeling, object instance polygon annotation, and QA pair generation with multi-stage quality control.
Figure 2: Statistics of MIRA-Bench. (a) Distribution of QA pairs across three cognitive levels and eight subtasks. (b) Distribution of images by data source and scenes by lighting condition (daytime versus nighttime).
Refl.-aware Perception
Spatial Reasoning
Scene-level Decision-making
Model
VC
MC
W-Avg
MR
CT
MPAC
W-Avg
RE
AS
CR
W-Avg
Overall
Baselines
Random Guess
12.3
15.0
13.1
3.7
2.5
25.0
8.6
33.3
16.7
33.3
24.1
14.1
Human (RGB)
85.1
82.3
84.2
79.3
78.0
90.8
81.7
92.2
91.5
90.2
91.7
85.0
Human (RGB-D)
92.3
90.5
91.7
85.0
80.2
92.1
85.2
94.4
94.9
91.5
94.5
90.7
Proprietary Models
Table 1: Evaluation results on MIRA-Bench. We report exact accuracy (%) for each sub-task. VC: Visibility Classification (joint); MC: Mirror Correspondence; MR: Mirror To Real Spatial Inference; CT: Counting; MPAC: Mirror Provided Attribute Completion; RE: Reachability; AS: Next Step Action Selection; CR: Collision Risk. W-Avg: weighted average within each cognitive level, weighted by the number of questions per sub-task. Overall: weighted average across all sub-tasks.
Refl.-aware Perception
Spatial Reasoning
Scene-level Decision-making
Input Mode
VC
MC
W-Avg
MR
CT
MPAC
W-Avg
RE
AS
CR
W-Avg
Overall
Qwen3.5-9B
RGB
32.2
39.8
34.6
2.1
20.9
52.7
20.6
57.2
8.1
52.6
29.8
30.3
RGB-D
31.4
37.7
33.4
2.7
20.6
50.4
20.2
61.4
7.9
36.0
30.2
29.6
RGB-M
32.6
42.7
35.8
3.2
19.7
49.8
20.0
62.1
10.3
45.7
32.4
31.3
RGB-DM
32.5
42.4
35.6
2.9
17.2
49.8
19.0
56.0
9.8
38.3
29.3
30.4
Table 2: Impact of input modality on mirror-aware spatial reasoning. We report exact accuracy (%) for each sub-task across four input configurations. RGB: raw image only; RGB-M: image with mirror region mask overlaid; RGB-D: image paired with depth map; RGB-DM: image with both depth map and mirror masks.
Referring Protocol
Overall Acc.
Text-BBox
28.0
Text-BBox+Desc
31.6
Visual-BBox
33.0
Visual-Mask
27.9
Visual-BBox+Desc
33.0
Table 3: Impact of object-referencing protocols on overall accuracy (%). Results shown for Qwen3.5-9B with RGB-M input. Desc denotes textual object descriptions.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Gradio-based QA annotation interface. The interface displays the annotated RGB image and corresponding depth map, together with controls for task selection, instance referencing, question preview, and answer annotation. The example shows a mirror-to-real spatial-inference question with horizontal, vertical, and depth relation labels.
ID
Question template and response specification
Visibility classification
vc_001
Is the object {X} visible in a mirror reflection but not directly in this image? Format: yes_no . Options: yes; no.
vc_002
Is the object {Y} directly visible but not visible in any mirror or other reflective surface in this image? Format: yes_no . Options: yes; no.
vc_003
Is the object {X/Y} visible both directly and in a mirror reflection in this image? Format: yes_no . Options: yes; no.
Mirror correspondence
mc_001
Which real-world instance corresponds to mirror instance {X}? Format: single_choice . Options: candidate real-world instance IDs.
Appendix
Table 4: QA templates for reflection-aware perception. Braced placeholders denote instance identifiers. {X} and {Y} denote objects visible in reflection or directly. Candidate instances for the multiple-choice questions are drawn from the same scene.
ID
Question template and response specification
Mirror-to-real spatial inference
mr_001
Where is the real object corresponding to mirror instance {X} relative to the real object {Y} in real-world space? Format: spatial_3d_axes . Options: horizontal: left, aligned, right; vertical: above, aligned, below; depth: front, same_depth, behind.
mr_003
Where is the real object corresponding to mirror instance {X1} relative to the real object corresponding to mirror instance {X2} in real-world space? Format: spatial_3d_axes . Options: horizontal: left, aligned, right; vertical: above, aligned, below; depth: front, same_depth, behind.
Counting-based spatial reasoning
ct_001
How many objects are there in the real world to the {DIR} of the real object corresponding to instance {X/Y}? (Count only polygon-annotated objects and the mirror itself. Do not count unannotated objects.) Format: count ; integer response. DIR options: left; right; front; behind; above; below.
Mirror-provided attribute completion
Appendix
Table 5: QA templates for spatial reasoning. Braced placeholders denote instance identifiers or spatial directions. {X} and {Y} denote objects visible in reflection or directly. For counting questions, annotators select {DIR} from the six listed directions. Each template specifies the answer format and, where applicable, the response options and text field requirements.
ID
Question template and response specification
Reachability
ac_001
Is real-world instance {X/Y} reachable from the current camera position? Format: yes_no . Options: yes; no; not sure.
Next-step action selection
ac_002
What is the best next action for the camera to move toward real-world instance {X/Y}? Format: single_choice . Options: move left; move right; move forward; move backward; rotate left; rotate right; turn around; stop; not sure; none.
ac_003
How should the camera move to better observe the real object corresponding to mirror instance {X}? Format: single_choice . Options: move left; move right; move forward; move backward; rotate left; rotate right; turn around; stop; not sure; none.
Collision-risk awareness
Appendix
Table 6: QA templates for scene-level decision-making. {X} and {Y} denote objects visible in reflection or directly. Braced placeholders denote instance identifiers or directions. For collision-risk questions, annotators select {DIR} from the six listed directions. The response format and options are provided below each question template.
Figure 4: Examples of the five object-referencing protocols in MIRA-Bench. Each surrounding panel presents the same underlying visibility classification question for the same target object using a different referring protocol. The central panel identifies the target instance. RGB-DM inputs are shown for illustration.
Figure 5: More visualization examples of MIRA-Bench.
Model
RGB
RGB-D
Δ
GPT-5.4
62.01
61.27
−0.74
Gemini-2.5-Flash
62.50
61.98
−0.52
Qwen3.5-9B
66.17
65.62
−0.55
LLaVA-OV-7B
57.30
56.98
−0.32
Appendix
Table 7: Accuracy (%) on 10,000 non-reflective RefSpatial [ 56 ] questions. Δ denotes the accuracy difference (RGB-D minus RGB), measured in percentage points.
Lower-level outcome
GPT-5.4
Gemini-2.5-Flash
Qwen3.5-9B
Perception fails
68.3
65.8
73.4
Reasoning fails
63.0
60.4
64.4
At least one fails
63.9
59.4
65.9
Both correct
45.3
40.8
56.5
Appendix
Table 8: Scene-level failure rate (%) grouped by the outcomes of linked lower-level questions. The groups overlap except for “both correct.”
Model
RGB
RGB-D
Δ
GPT-5.4
54.47
55.04
+0.57
Qwen3.5-9B
49.96
48.38
−1.58
LLaVA-OV-7B
29.77
29.16
−0.61
SpatialRGPT
37.48
39.27
+1.79
Appendix
Table 9: Accuracy (%) on the common single-region question set. Δ denotes the accuracy difference (RGB-D minus RGB), measured in percentage points.