Multimodal Large Language Models (MLLMs) have shown impressive abilities in understanding and reasoning over conventional images. However, their perception of 360° images remains largely underexplored. Unlike conventional images, 360° images capture the entire surrounding environment, enabling holistic spatial reasoning but introducing challenges such as geometric distortion and complex spatial relations. To comprehensively assess MLLMs' capabilities to perceive 360° images, we introduce 360Bench, a Visual Question Answering (VQA) benchmark featuring 7K-resolution 360° images, seven representative (sub)tasks with annotations carefully curated by human annotators. Using 360Bench, we systematically evaluate seven MLLMs and six enhancement methods, revealing their shortcomings in 360° image perception. To address these challenges, we propose Free360, a training-free scene-graph-based framework for high-resolution 360° VQA. Free360 decomposes the reasoning process into modular steps, applies adaptive spherical image transformations to 360° images tailored to each step, and seamlessly integrates the resulting information into a unified graph representation for answer generation. Experiments show that Free360 consistently improves its base MLLM and provides a strong training-free solution for 360° VQA tasks. The source code and dataset will be publicly released upon acceptance.
Figures & tables
Figure 1: (a) Example of an image from 360Bench in two common projection formats with relevant entities highlighted. (b) Responses of GPT-4o and Free360 (Ours) with correct answers highlighted in red. GPT-4o struggles with spatial reasoning, whereas Free360 correctly infers spatial relations. 360° image source: [ 15 ] .
Dataset
Type
Res.
Scene
Annotation
#Samples
#Tasks
FP-IR
FP-IC
PP-IR
PP-IC
SR-Os
SR-OV
DG
Avail.
HR-Bench [ 36 ]
CI
8K
Diverse
Manual
200 (200)
2
✓
✗
✗
✗
✓
✗
✗
✗
Pano-AVQA [ 45 ]
360V
N/A
Diverse
Temp. (7)
51.7K (5.3K)
2
✗
✗
✗
✗
✓
✗
✗
✗
CFpano [ 47 ]
c360I
1K
Indoor
Temp. (36)
8094 (1619)
17
✗
✗
✗
✗
✗
✗
✗
✗
VQA360 [ 8 ]
360I
1K
Indoor
Temp. (17)
16,945 (6962)
3
✗
✗
✗
✗
✓
✓
✗
✗
OmniVQA [ 48 ]
360I
1K
Indoor
Temp. (6)
5652 (800)
2
✗
✗
✓
✗
✓
✗
✗
✗
ODI-Bench [ 42 ]
360I
Mixed
Diverse
Auto&Manual
4254 (4254)
10
✗
✗
✓
✗
✓
✓
✗
✗
Table 1: Comparison between 360Bench and existing VQA datasets. CI , 360I , 360V , and c360I denote conventional images, 360° images, 360° videos, and cross-frame correlated 360° images, respectively. Res. refers to image resolution. Temp. (#) indicates template-based annotations, with the number of templates in parentheses. Auto refers to annotations generated automatically using vision models. Manual denotes human-crafted annotations. #Samples (#) indicates the number of unique samples, with test samples shown in parentheses. Avail. indicates whether the dataset is publicly accessible at the time of writing.
Figure 2: Statistics of 360Bench: (a) Task breakdown, (b) Image distribution, (c) Spatial distribution of annotated bounding boxes.
Figure 3: Illustration of tasks included in 360Bench. Correct answers and relevant entities are highlighted in red. 360° image sources: [ 28 , 12 ] .
Figure 4: Overview of Free360. The model performs question answering via scene graph generation through four steps (Sec. 4.1 ): (1) Entity Identification, detecting entities relevant to the question Q from the CMP image I ; (2) Attribute Extraction, deriving descriptive attributes from each entity crop; (3) Inter-Entity Relation Detection, capturing spatial relations among entities; and (4) Entity-View Relation Detection, modeling spatial relations between entities and camera views. The resulting scene graph, represented in textual form, is then fed into the MLLM to generate the final answer and reasoning analysis (Sec. 4.2 ). 360° image source: [ 15 ] .
Label
Attribute
left view
Left side of the scene and to the left of the viewer.
right view
Right side of the scene and to the right of the viewer.
front view
Front of the scene and front of the viewer.
behind view
Back of the scene and behind the viewer.
top view
Top of the scene and above the viewer.
bottom view
Bottom of the scene and below the viewer.
Table 2: Labels and attributes of the six view nodes.
Figure 5: Example of a scene graph in the serialized textual form.
Model
Projection format
FP (Acc.)
PP (Acc.)
SR (Acc.)
DG
Overall
Inf. Time
IR
IC
Avg.
IR
IC
Avg.
Os
OV
Avg.
(Acc.)
(Acc.)
(seconds)
Random guess
—
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
—
Human
—
96.0
82.0
89.0
86.0
78.0
82.0
84.0
90.0
87.0
88.0
86.3
28.9
MLLMs
GPT-4o
ERP
48.6
32.0
40.3
54.2
38.8
46.5
37.7
35.7
36.7
38.5
40.7
1.5
CMP
45.5
30.8
38.1
63.0
44.0
53.5
37.4
34.0
35.7
35.3
41.3
1.6
Table 3: Human and model evaluation results. The highest score for each task is highlighted in bold. Acc. denotes per-task accuracy; Difference indicates the difference between Free360 and Qwen2.5-VL-7B in accuracy and inference time.
(a) Impact of SGG components Setting Acc. Inf. Time Full 45.3 22.5 w/o Crop 42.7 23.8 w/o Rotate 44.7 21.0 w/o EVR 42.8 21.1
(b) Effectiveness across model scales Model Qwen2.5-VL Free360 Acc. Gain scale ERP CMP Hybrid ERP CMP 3B 35.2 37.2 40.8 + 5.5 + 3.5 7B 37.3 38.1 45.3 + 8.1 + 7.3 32B 37.8 39.7 45.0 + 7.2 + 5.3
The Hong Kong Polytechnic University Hong Kong, China · Guangzhou University Guangzhou, China · Queen Mary, University of London London, United Kingdom +1