Vision-language models (VLMs) increasingly process multiple views of the same scene, but it remains unclear whether they integrate these observations into a coherent scene-level representation. Existing multi-image evaluations do not isolate this capability, as tasks may admit correct solutions through independent per-view processing without requiring cross-view correspondence. We introduce OneScene-Bench, a controlled benchmark designed to explicitly distinguish when cross-view integration is necessary. Our benchmark evaluates three complementary capabilities on real-world multi-view scenes: unique object counting, cross-view consistency, and spatial reasoning. For counting, we control object overlap across views to separate cases solvable by simple per-view aggregation from those requiring instance-level correspondence. For consistency, we introduce localized edits in shared and view-specific regions, testing whether models distinguish genuine scene contradictions from differences caused by viewpoint and visibility. For spatial reasoning, we construct object relations that require combining spatial evidence across viewpoints to compare physical proximity. We evaluate twelve state-of-the-art open-weight and proprietary VLMs and find that counting accuracy drops substantially on complementary views, ranging from 4.5-50.0%, compared with 45.5-81.5% on complete-overlap examples. Moreover, cross-view consistency remains around 50% across models, including recent proprietary VLMs, despite substantially stronger performance on spatial reasoning. Our results reveal a substantial gap between processing multiple images and integrating evidence across them, highlighting the need for evaluations that explicitly test scene-level multi-view understanding. Code and data are available at https://github.com/swathisuhas/OneScene-Bench.
Figures & tables
Figure 1: Overview of OneScene-Bench. OneScene-Bench evaluates whether VLMs correctly integrate and reason across views of the same physical scene through three tasks: unique object counting under complete-overlap, subset-overlap, and complementary-view configurations; cross-view consistency; and spatial reasoning. Representative image pairs, questions, and ground-truth answers are shown for each task.
Figure 2: Properties and Composition of OneScene-Bench. (a) Comparison with representative multi-image and multi-view benchmarks. Multi-view denotes multiple observations of the same physical scene; Free-form denotes unrestricted responses; Diagnostic denotes controlled behavioral analyses; and Cross-view consistency denotes explicit consistency evaluation. Counting types denotes different counting scenarios covering different levels of view overlap. Green ticks, orange tildes, and red crosses indicate full, partial, and no coverage, respectively. (b) Composition of the 1,300-question core two-view benchmark. Multi-view counting contains 700 questions: 200 complete-overlap, 300 subset-overlap, and 200 complementary-view examples. Cross-view consistency and multi-view spatial reasoning contain 300 questions each.
Model
Counting ↑
Consistency ↑
Spatial
Complete
Subset
Complementary
Reasoning ↑
Open
Qwen3-VL-32B-Instruct [ 7 ]
70.0
70.7
34.0
49.3
85.3
Gemma-4-31B-IT [ 31 ]
61.0
63.3
35.0
50.3
86.3
InternVL3.5-38B-HF [ 35 ]
72.0
57.3
14.5
50.7
80.7
LLaVA-OneVision-2-8B-Instruct [ 3 ]
55.5
53.0
18.0
50.0
77.0
MiniCPM-V-4.5 [ 39 ]
67.0
51.7
20.5
53.3
72.7
Table 1: VLMs on OneScene-Bench. Accuracy (%) of twelve open-weight and proprietary VLMs on OneScene-Bench. The best-performing model in each group is shown in bold for every task. Across models, counting performance drops substantially on complementary views, where object correspondence and deduplication across viewpoints are required. Cross-view consistency remains around 50%, whereas spatial reasoning is substantially stronger, revealing a marked difference in model capability across forms of multi-view integration.
Counting
Consistency ↑
Spatial
Model
Complete ↑
Subset ↑
Complementary ↑
Reasoning ↑
Qwen3-VL-32B-Instruct [ 7 ]
37.5 (70.0)
26.7 (70.7)
7.0 (34.0)
50.0 (49.3)
63.7 (85.3)
Gemma-4-31B-IT [ 31 ]
34.0 (61.0)
28.3 (63.3)
13.5 (35.0)
0.0 (50.3)
1.7 (86.3)
InternVL3.5-38B-HF [ 35 ]
41.0 (72.0)
14.3 (57.3)
1.5 (14.5)
50.0 (50.7)
55.3 (80.7)
LLaVA-OneVision-2-8B-Instruct [ 3 ]
40.0 (55.5)
17.7 (53.0)
3.5 (18.0)
50.0 (50.0)
54.7 (77.0)
Table 2: No-image bias. Accuracy (%) when models receive the task instructions but no images; performance with images is shown in gray in parentheses. Models achieve non-trivial accuracy without visual input, indicating response biases and task priors, while visual evidence substantially improves counting and spatial reasoning performance.
Figure 3: Diagnostic analyses of reasoning and number of views. (a) Counting accuracy of instruction-tuned and reasoning-enabled variants. Reasoning generally reduces performance, indicating that explicit reasoning does not improve multi-view counting. (b) Complete-overlap counting accuracy with N=2 – 6 input views. Performance initially improves with additional views but peaks before N=6 for all models, indicating that more views do not yield monotonic gains.
Complete overlap
Subset overlap
Complementary views
Model
Acc. ↑
Sum agr.
Max agr.
Acc. ↑
Sum agr.
Max agr.
Acc. ↑
Sum agr.
Max agr.
Qwen3-VL-32B-Instruct [ 7 ]
70.0
8.0
82.0
70.7
23.0
77.7
34.0
18.0
39.0
InternVL3.5-38B-HF [ 35 ]
72.0
4.0
77.0
57.3
15.0
75.3
14.5
11.0
51.0
LLaVA-OneVision-2-8B-Instruct [ 3 ]
55.5
13.5
68.0
53.0
10.3
64.3
18.0
4.5
62.0
Gemma-4-31B-IT [ 31 ]
61.0
8.5
69.5
63.3
21.7
70.3
35.0
17.5
34.0
Table 3: Aggregation Heuristics. Comparison of joint and single-view counting predictions. C1 and C2 denote individual-view predictions and C3 the joint prediction. Acc. is ground-truth accuracy, shown in gray for reference. Sum agr. and Max agr. report the percentage of questions satisfying C3=C1+C2 and C3=max(C1,C2) , respectively, without using ground truth. Joint predictions often agree with the maximum in complete- and subset-overlap settings; in complementary views, maximum agreement remains higher than sum agreement despite being insufficient for scene-level counting.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Counting
Consistency ↓
Spatial
Model
Complete ↓
Subset ↓
Complementary ↓
Reasoning ↓
Qwen3-VL-32B-Instruct [ 7 ]
0.5
2.4
1.5
4.0
2.7
Gemma-4-31B-IT [ 31 ]
6.0
1.6
4.0
0.3
0.3
InternVL3.5-38B-HF [ 35 ]
3.5
1.0
1.5
2.3
1.0
LLaVA-OneVision-2-8B-Instruct [ 3 ]
1.5
0.3
0.5
0.0
9.0
Appendix
Table 4: Image-Order Sensitivity. Absolute difference in accuracy (percentage points) between the original and reversed input-image orders, ∣Accoriginal−Accreverse∣ . Overall, 18 of 20 model–task combinations change by at most four points, indicating broad robustness to image order, with larger sensitivity for Gemma-4 on complete-overlap counting and LLaVA-OneVision-2 on spatial reasoning.
Counting
Consistency
Spatial
Model
Complete
Subset
Complementary
Reasoning
(n=200)
(n=300)
(n=200)
(n=300)
(n=300)
Qwen3-VL-32B-Instruct [ 7 ]
0
0
0
0
0
InternVL3.5-38B-HF [ 35 ]
0
0
0
0
0
LLaVA-OneVision-2-8B-Instruct [ 3 ]
0
0
0
0
0
Gemma-4-31B-IT [ 31 ]
6
3
12
300
293
Appendix
Table 5: No-image refusal count. Number of explicit refusals in the no-image setting. A refusal is counted only when a model explicitly states that the question cannot be answered without visual input; incorrect or guessed answers are not counted.
Figure 4: Screenshot of the manual counting review interface . The interface displays the selected multi-view image pair, target object category, reviewed scene-level count, and manually assigned overlap type. Reviewers can also skip ambiguous image pairs before finalizing the annotation.
Figure 5: Screenshot of the consistency review tool . The interface displays the original and edited views, ground-truth label, edit type, and edited object. Reviewers can confirm the example, correct an incorrect ground-truth label, or permanently remove an invalid or ambiguous example.
Figure 6: Screenshot of the spatial-reasoning verification interface . The tool displays the two views, the three referenced objects ( A , B , and C ), the generated question, and the reviewer controls for assigning the ground-truth proximity relationship or marking the example for removal.
Figure 7: Representative examples from the five tasks in OneScene-Bench : multi-view counting under complete-overlap, subset-overlap, and complementary-view configurations, cross-view consistency, and multi-view spatial reasoning. Questions are slightly shortened for visual presentation while preserving their original meaning. Model predictions are shown using the corresponding model logos, with green checkmarks and red crosses indicating correct and incorrect predictions, respectively.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University · Ministry of Education Key Laboratory of Intelligent Networks and Network Security