Multi-View Video Generation

Overview

Multi-view video generation studies how to create videos that remain coherent across camera viewpoints and over time, enabling novel-view rendering, 3D reconstruction, and interactive simulation. Common approaches extend diffusion or autoregressive video models with camera and pose controls, cross-view attention, shared spatial representations, or geometric cues such as depth, point clouds, and 3D memory. Work spans dynamic human scenes, robot manipulation, and driving world models, with benchmarks including DNA-Rendering, ActorsHQ, Ego-Exo4D, and nuScenes. A central challenge is balancing visual realism and controllability with geometric consistency across large viewpoint changes and long sequences, especially when training data provides only sparse views.

Latest papers

61 papers8 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

Aug 11, 2025

S^2VG: 3D Stereoscopic and Spatial Video Generation via Denoising Frame Matrix

While video generation models excel at producing high-quality monocular videos, generating 3D stereoscopic and spatial videos for immersive applications remains an underexplored challenge. We present a pose-free and training-free method that leverages an off-the-shelf monocular video generation model to produce immersive 3D videos. Our approach first warps the generated monocular video into pre-defined camera viewpoints using estimated depth information, then applies a novel frame matrix inpainting framework. This framework utilizes the original video generation model to synthesize missing content across different viewpoints and timestamps, ensuring spatial and temporal consistency without requiring additional model fine-tuning. Moreover, we develop a \dualupdate~scheme that further improves the quality of video inpainting by alleviating the negative effects propagated from disoccluded areas in the latent space. The resulting multi-view videos are then adapted into stereoscopic pairs or optimized into 4D Gaussians for spatial video synthesis. We validate the efficacy of our proposed method by conducting experiments on videos from various generative models, such as Sora, Lumiere, WALT, and Zeroscope. The experiments demonstrate that our method has a significant improvement over previous methods. Project page at: https://daipengwa.github.io/S-2VG_ProjectPage/