Organizations: University of California, Irvine, USA · University of California, San Diego, USA · City University of Hong Kong, Hong Kong, China · University of Pennsylvania, USA · Adobe Research, USA · Adobe Research, UK
Modeling scenes using video generation models has garnered growing research interest in recent years. However, most existing approaches rely on perspective video models that synthesize only limited observations of a scene, leading to issues of completeness and global consistency. We propose OmniRoam, a controllable panoramic video generation framework that exploits the rich per-frame scene coverage and inherent long-term spatial and temporal consistency of panoramic representation, enabling long-horizon scene wandering. Our framework begins with a preview stage, where a trajectory-controlled video generation model creates a quick overview of the scene from a given input image or video. Then, in the refine stage, this video is temporally extended and spatially upsampled to produce long-range, high-resolution videos, thus enabling high-fidelity world wandering. To train our model, we introduce two panoramic video datasets that incorporate both synthetic and real-world captured videos. Experiments show that our framework consistently outperforms state-of-the-art methods in terms of visual quality, controllability, and long-term scene consistency, both qualitatively and quantitatively. We further showcase several extensions of this framework, including real-time video generation and 3D reconstruction. Code is available at https://github.com/yuhengliu02/OmniRoam.
Figures & tables
Figure 1. Two-stage Long-horizon Panoramic Scene Wandering . Starting from an input frame/video, OmniRoam first produces a coarse 480 × 960 preview with accelerated playback speed to quickly traverse the scene along the user-specified camera trajectory (left). It then upscales this video to a high-quality 720 × 1440 resolution (middle), jointly improving spatial details and temporal coherence, creating an immersive exploration experience. The resulting wandering path is visually represented as a series of viewpoints within the 3D scene (right).
Figure 2. OmniRoam Pipeline consists of (1) a preview stage that takes an input image/video and a user-specified camera trajectory, and generates an accelerated, mid-resolution panoramic video via decomposed trajectory conditioning (flow + scale), and (2) a refine stage that performs scale alignment and segment-wise diffusion to produce the final high-resolution long-horizon panoramic video. The framework further supports real-time autoregressive preview and downstream 3D reconstruction. Both models in preview and refine stage are finetuned from a pretrained Diffusion Transformers.
Figure 3. Dataset Visualization. We curate a hybrid training set containing real-world handheld panoramic videos captured across diverse scenes (top) and synthetic panoramic sequences rendered from 3DGS scenes with synthesized wide-ranging, physically feasible trajectories (bottom).
Method
Resolution
No. of Frames
Visual Quality
Trajectory Controllability
Loop Consistency
FAED ↓
SSIM ↑
LPIPS ↓
PSNR’25 ↑
PSNR’55 ↑
PSNR’75 ↑
Matrix-3D
480p
81
8.64
0.63
0.37
19.05
17.35
17.04
1.38
Imagine360
480p
81
23.35
0.33
0.61
–
–
–
–
Ours
480p
81
5.27
0.70
0.18
23.06
20.69
19.87
2.34
Matrix-3D
720p
81
9.76
0.66
0.36
19.06
17.63
17.12
1.41
Ours
720p
641
5.07
0.66
0.33
19.75
18.59
18.24
1.96
Table 1. Quantitative Comparison. Our method consistently outperforms prior approaches ( Yang et al., 2025 ; Tan et al., 2024 ) across all evaluated metrics, achieving superior visual quality, stronger trajectory controllability, and higher loop consistency. FAED, SSIM, and LPIPS are evaluated on 81 frames, while loop consistency is reported on the full sequences.
Figure 4. Qualitative Comparisons. Our method generates panoramic videos that closely follow the input trajectory and preserve sharper details from the input images, while prior methods often produce blurry artifacts or semantically inconsistent content. Zoomed-in perspective crops are shown in the red inset.
Method
No. of Frames
Visual Quality
Trajectory Controllability
Loop Consistency
FAED ↓
SSIM ↑
LPIPS ↓
PSNR’25 ↑
PSNR’55 ↑
PSNR’615 ↑
PSNR’635 ↑
Perspective (Preview)
81
16.90
0.62
0.44
17.72
16.06
–
–
1.70
Perspective (Refine)
641
15.48
0.63
0.57
16.60
14.94
13.76
14.07
1.42
Autoregressive
641
16.04
0.33
0.64
15.44
14.84
10.14
10.11
0.89
Ours
641
7.70
0.58
0.44
19.75
18.59
15.55
15.63
1.96
Table 2. Design Analysis. We analyze key design choices, including video representation (ours: panoramic vs. perspective) and generation strategy (ours: global-to-local vs. direct autoregressive). We report FAED, SSIM, LPIPS, and loop consistency over the full video sequences. For long videos (641 frames), we additionally report the average PSNR over extended temporal windows, specifically frames 610–615 (PSNR’615) and 630–635 (PSNR’635).
Figure 5. CLIP Similarity over Loop Trajectories. We show temporal CLIP similarity to the first frame under loop trajectories for long-horizon generation (641 frames). Our method shows the expected trend: similarity decreases as the camera moves away and recovers as the trajectory closes the loop; whereas (a) the autoregressive variant exhibits a largely monotonic decline (drift from the initial view) and (b) the perspective-video variant shows weaker recovery with noticeable structural degradation. Representative frames are shown alongside the curves.
Figure 6. Additional Qualitative Comparisons. Our method maintains better geometric stability and finer textures during scene exploration.
Figure 7. Results from Real-Time Previewer-based Two-Stage Generation. Videos are generated in real time at 480p and subsequently refined to 720p .
Figure 8. Long Video Generation with Loop Trajectory. We present 641-frame generated video sequences that traverse the scene in a closed loop, where the last frames closely resemble the initial frame, demonstrating strong long-term consistency.
Figure 9. Visualization of Real-time Previewer and 3DGS. (a) demonstrates trajectory-following panoramic wandering with our real-time previewer. (b) shows the reconstructed 3D Gaussian splatting results and corresponding panoramic samples.
We present PanoWorld, a panoramic video world model that generates geometry-consistent 360° video from a single image and a caption. Existing panoramic video methods optimize primarily for visual realism and do not explicitly constrain the underlying 3D scene state, producing outputs that appear plausible yet exhibit inconsistent depth, broken correspondences, and implausible motion across the spherical surface. We address this gap by framing panoramic video generation as a geometry- and dynamics-consistent latent state modeling problem rather than pure visual synthesis. Building on a pre-trained perspective video world model, we introduce two lightweight regularizers: a depth consistency loss against pseudo ground-truth panoramic depth, and a trajectory consistency loss that supervises the 3D world-frame positions of tracked points across time. We further apply spherical-geometry-aware adaptation to the conditioning and positional encoding. We additionally introduce PanoGeo, a unified geometry-aware panoramic video dataset with consistent depth, trajectory, and prompt annotations across diverse real and synthetic sources, used for both training and stratified evaluation. Experiments show that PanoWorld improves geometric consistency over prior panoramic generation methods while maintaining competitive visual realism, establishing that panoramic video generation must be treated as a geometric modeling problem to support the holistic spatial understanding requirements of embodied AI applications. Code is available at https://github.com/ostadabbas/PanoWorld.
World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation.
Tianxing Xu, Zixuan Wang, Guangyuan Wang +5
Tsinghua University China · Tongyi Lab, Alibaba China · University of Science and Technology of China China
Interactive panoramic video generation aims to synthesize immersive 360\textdegree{} videos that remain visually coherent while following user-specified camera trajectories during exploration. However, progress is limited by a coupled data-and-model gap: existing panoramic video datasets are often short, weakly annotated, or lack camera trajectories, while existing camera-controlled video generation models are designed for perspective videos and do not directly support panoramic geometry. In this paper, we introduce MUGEN and Wan360 to address these limitations. MUGEN is a large-scale real-world panoramic video dataset tailored to interactive 360-degree world exploration, comprising over 1,300 hours of at least 4K panoramic videos with rich semantic and geometric annotations. Built on MUGEN, we further present Wan360, a camera-controllable interactive panoramic video generation model. Panoramic videos are commonly represented by EquiRectangular Projection (ERP), which unfolds a spherical 360-degree view into a rectangular frame with cyclic longitude seams and pole distortions. To this end, Wan360 introduces three parameter-free ERP-aware components: periodic longitude RoPE for seam-consistent positional encoding, ERP-aware padding for reducing boundary artifacts, and random roll yaw for consistent learning. For camera control, Wan360 uses a panoramic Plücker embedding that represents camera motion with ERP rays rather than perspective pinhole rays. Experiments show that MUGEN serves as a data foundation for panoramic world exploration, and that Wan360 enables high-quality, temporally coherent, camera-controllable 360-degree video generation.
Jiaming Tan, Zhen Li, Shuwei Shi +6
Alaya Lab · Beijing Institute of Technology · Shanghai Innovation Institute