Think in 360°: Benchmarking and Bootstrapping Panoramic Video Reasoning
Organizations: Nanjing University · NVIDIA
Abstract
A panoramic video captures all directions around the camera, but relevant evidence can appear in different directions and at different times. Understanding it requires models to connect observations across the viewing sphere and time. We introduce PanoVideoBench, the first manually annotated benchmark for comprehensive panoramic video understanding. Its 1,000 questions span five categories and 15 question types, with diagnostic annotations for target search and panoramic capabilities. Evaluation of 11 proprietary, open-source, and panoramic-specialized models identifies cross-view tracking and sphere-time evidence integration as their weakest capabilities. To strengthen these capabilities, we construct PanoVideoCoT, the first reasoning dataset grounded in real panoramic video, through sphere-time-grounded synthesis that converts verified panoramic observations and successful agentic explorations into 13,465 QA-CoT pairs. Starting from a general video MLLM, PanoVideoReasoner learns from this corpus through Panoramic CoT SFT and Evidence-Privileged On-Policy Self-Distillation, a panoramic instantiation of Vision-OPD's OPSD in which the teacher uses verified perspective views to guide student-generated trajectories. It reaches 55.7% on PanoVideoBench, improving the backbone by 12.5 points and the previous SOTA by 2.6. The gains concentrate on the same spatial, cross-view, and temporal capabilities exposed by the benchmark, showing that grounded sphere-time supervision can equip a general video MLLM with panoramic reasoning.
Figures & tables
| Benchmark | Input | #Eval | Manual | APP | SPA | ACT | MOT | TEM | AV |
|---|---|---|---|---|---|---|---|---|---|
| VQA 360 ∘ Chou et al. (2020) | 360I | 7.0K | – | ✓ | ✓ | – | – | – | – |
| 360Bench Tran et al. (2026) | 360I | 1.5K | ✓ | ✓ | ✓ | – | – | – | – |
| CFpano Zhang et al. (2025a) | C360I | 1.6K | – | ✓ | ✓ | – | – | – | – |
| OmniCoT-B He et al. (2026) | 360I | 6.7K | – | – | ✓ | – | – | – | – |
| Pano-AVQA Yun et al. (2021) | 360V+A | 5.3K | – | – | ✓ | – | – | – | ✓ |
| ST-OmniQA Zeng et al. (2026) | 360V+FOA | 4.0K | – | – | ✓ | – | ✓ | ✓ | ✓ |
| Data basis | Supervision | Construction | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Dataset | #Vis. | #Train | 360V | Real | CoT | VG | TG | E-QG | ReSolve |
| Pano-AVQA Yun et al. (2021) | 4.5K clips | 42.8K QA | ✓ | ✓ | – | ✓ | – | ✓ | – |
| OmniVQA Zhang et al. (2025b) | 1.2K images | 4.9K QA | – | ✓ | ✓ | – | – | – | – |
| Dense360 Zhou et al. (2025) | 160K images | 5M ent. cap. | – | ✓ | – | ✓ | – | – | – |
| PanoEnv-QA Lin and Zheng (2026) | 0.4K images | 10.3K QA | – | – | – | ✓ | – | ✓ | – |
| OmniCoT-T He et al. (2026) | 2.8K images | 14.3K QA–CoT | – | – | ✓ | ✓ | – | ✓ | – |
Video Understanding
Video understanding develops methods for recognizing actions, tracking events, grounding evidence in time, and reasoning across visual and audio streams.
See all 122 papers