Creating dynamic, view-consistent videos of customized subjects is highly sought after for a wide range of emerging applications, including immersive VR/AR, virtual production, and next-generation e-commerce. However, despite rapid progress in subject-driven video generation, existing methods predominantly treat subjects as 2D entities, focusing on transferring identity through single-view visual features or textual prompts. Because real-world subjects are inherently 3D, applying these 2D-centric approaches to 3D object customization reveals a fundamental limitation: they lack the comprehensive spatial priors necessary to reconstruct the 3D geometry. Consequently, when synthesizing novel views, they must rely on generating plausible but arbitrary details for unseen regions, rather than preserving the true 3D identity. Achieving genuine 3D-aware customization remains challenging due to the scarcity of multi-view video datasets. While one might attempt to fine-tune models on limited video sequences, this often leads to temporal overfitting. To resolve these issues, we introduce a novel framework for 3D-aware video customization, comprising 3DreamBooth and 3Dapter. 3DreamBooth decouples spatial geometry from temporal motion through a 1-frame optimization paradigm. By restricting updates to spatial representations, it effectively bakes a robust 3D prior into the model without the need for exhaustive video-based training. To enhance fine-grained textures and accelerate convergence, we incorporate 3Dapter, a visual conditioning module. Following single-view pre-training, 3Dapter undergoes multi-view joint optimization with the main generation branch via an asymmetrical conditioning strategy. This design allows the module to act as a dynamic selective router, querying view-specific geometric hints from a minimal reference set. Project page: https://ko-lani.github.io/3DreamBooth/
Figures & tables
Figure 1: 3D-aware video customization using our proposed framework. Given a few multi-view reference images (left) and a text prompt, our approach generates high-fidelity, view-consistent videos that seamlessly integrate customized 3D subjects into dynamic environments.
Figure 2: Overview of the 3DreamBooth training pipeline. (Left) From multi-view images, one is selected as the target, while a sampled subset serves as reference conditions alongside a global prompt with a unique identifier V . (Right) The text and noisy target latents pass through the main branch (3DB LoRA), while reference latents pass through a shared 3Dapter . Their features are concatenated for Multi-view Joint Attention. This 1-frame optimization decouples spatial geometry from temporal dynamics to efficiently learn a 3D prior.
Figure 3: Convergence Analysis and Detail Preservation. (A) Reconstruction Loss: Integrating 3Dapter (blue) drastically accelerates convergence compared to the 3DreamBooth baseline (gray). (B) Qualitative Comparison: 3DreamBooth alone (purple dot) struggles with high-frequency details due to the information bottleneck. In contrast, 3Dapter + 3DreamBooth (yellow dot) perfectly preserves intricate textures (e.g., “RIO” typography) much earlier, demonstrating the efficacy of explicit visual priors.
Figure 4: Detailed architecture of our two-stage conditioning mechanism. (A) Single-view Pre-training: The visual adapter ( 3Dapter ) is pre-trained using single-view references and fused via Single-view Joint Attention. (B) Multi-view Joint Optimization: A trainable 3DreamBooth LoRA is added to the main branch. A minimal set of multi-view reference images is processed in parallel by the shared 3Dapter . The Multi-view Joint Attention acts as a dynamic selective router, querying relevant view-specific geometric hints to reconstruct the target view.
Figure 5: Visualization of the Dynamic Selective Router mechanism. (Left) Generated frames and four multi-view conditions provided to 3Dapter . The generated poses align with View 2 (red box). (Right) Cross-attention heatmaps across diffusion timesteps ( t=0,20,40 ). The network selectively assigns higher attention weights to the relevant view (View 2) to extract specific geometric features, rather than uniformly aggregating all conditions.
Figure 6: Qualitative Comparisons. We compare our proposed framework against state-of-the-art video customization baselines, VACE and Phantom. Here, “Ours” denotes our full synergistic model ( 3Dapter + 3DB ). While existing baselines struggle to reconstruct complex structures and fine-grained details (e.g., product tags and small accessories) from unseen viewpoints, our method faithfully maintains the subject’s precise 3D geometry and intricate textures during dynamic 360∘ rotations.
Feature-based
GPT-4o
Method
Views
CLIP-I ↑
DINO-I ↑
Shape ↑
Color ↑
Detail ↑
Overall ↑
VACE [ 18 ]
S
0.8964
0.7395
4.39±0.05
4.09±0.09
3.35±0.15
3.95±0.11
Phantom [ 24 ]
S
0.8576
0.5861
3.48±0.12
3.94±0.13
3.03±0.16
3.31±0.15
3Dapter
S
0.8647
0.5899
3.06±0.03
3.09±0.06
2.28±0.08
2.67±0.07
3DreamBooth
M
0.8382
0.6530
4.18±0.06
3.63±0.09
3.14±0.11
3.53±0.07
3Dapter+3DB
M
0.8871
0.7420
4.80±0.03
4.53±0.04
4.04±0.13
4.57±0.04
Table 1: Multi-View Subject Fidelity. We compare our framework against state-of-the-art subject-driven video generation models and our ablation variants. ‘S’ and ‘M’ denote Single-view and Multi-view conditioning, respectively.
Method
Views
Accuracy ↓
Completeness ↓
CD ↓
VACE [ 18 ]
S
0.0278
0.0427
0.0353
Phantom [ 24 ]
S
0.0289
0.0388
0.0338
3Dapter
S
0.0315
0.0659
0.0487
3DreamBooth
M
0.0156
0.0322
0.0239
3Dapter+3DB
M
0.0182
0.0172
0.0177
Table 2: 3D Geometric Fidelity. We compare Chamfer Distance between generated frames and ground-truth multi-view images from our 3D-CustomBench .
Method
Type
Aesthetic Quality ↑
Imaging Quality ↑
Motion Smoothness ↑
ViCLIP ↑
VACE [ 18 ]
S
0.5915
70.84
0.9916
0.2663
Phantom [ 24 ]
S
0.5798
70.58
0.9934
0.2634
3Dapter
S
0.6283
71.65
0.9944
0.2048
3DreamBooth
M
0.5245
73.34
0.9928
0.2415
3Dapter + 3DB
M
0.5920
74.33
0.9918
0.2388
Table 3: Video Quality and Text Alignment. We evaluate the intrinsic video quality using VBench [ 15 ] and text-video alignment using ViCLIP Score (ViCLIP-L/14) [ 42 ] . ‘S’ and ‘M’ denote Single-view and Multi-view conditioning, respectively.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Qualitative results across diverse subjects. Each row presents a distinct object, demonstrating accurate 3D identity preservation in dynamic contexts.
Figure A2 : Additional qualitative results across diverse subjects. Each row presents a distinct object, further demonstrating the generalizability of our framework in preserving 3D identity across varied object categories and dynamic contexts.
Figure A3 : Single-subject generation across diverse prompts. Given a single subject (top-left), we generate videos using multiple prompts with varying scenarios. Our framework consistently preserves the subject’s identity across complex scenes, including human-object interactions and richly detailed environments.
Figure A4 : Effect of 3Dapter pre-training on joint optimization. Randomly initialized 3Dapter (yellow borders) versus pre-trained 3Dapter (blue borders) across different 3DreamBooth joint training iterations. Without pre-training, the model suffers optimization collapse even at 400 iterations. In contrast, the pre-trained 3Dapter enables accurate multi-view alignment and high-fidelity 3D customization within a short optimization period.
Figure A5 : Comprehensive qualitative ablation across three configurations. Comparing 3Dapter only, 3DreamBooth only (at 400 and 1600 iterations), and our full framework at 400 iterations ( 3Dapter + 3DreamBooth ). Compared to our full model ( 3Dapter + 3DreamBooth ), 3Dapter alone fails to maintain 3D geometric volume and view consistency, while 3DreamBooth alone lacks structural precision and texture fidelity even with 4× more iterations.
Figure A6 : Step-by-step training dynamics of the 3DreamBooth -only baseline. Generated frames at 0, 400, 800, 1200, 1600, and 2000 training iterations illustrate the convergence dynamics of the 3DreamBooth -only baseline. While the coarse shape and color of the reference plushie are visible from early iterations, accurate subject identity and view-consistent geometry emerge only with prolonged optimization.
Figure A7 : Extension to DiT-based Video Models. We demonstrate the extensibility of 3DreamBooth to state-of-the-art Diffusion Transformer (DiT) architectures by applying our fine-tuning paradigm to the Wan 2.1 text-to-video model. Even without explicit spatial conditioning, the generated sequences faithfully preserve the intricate 3D identity and complex geometric structures of the subjects across diverse contexts.
Figure A8 : Visual comparison and convergence analysis. (Left) Ablation study on the components of our framework. Single-view baselines (VACE, Phantom) and individual modules (3Dapter, 3DreamBooth) struggle to maintain fine-grained identity details, whereas the synergistic combination of 3Dapter + 3DreamBooth successfully achieves accurate identity preservation. (Right) Training loss curve and qualitative progression of 3DreamBooth across optimization steps, demonstrating that the model requires a prolonged optimization process to fully converge and refine subject details.
Figure A9 : Pipeline of the 3D geometric fidelity evaluation. Generated frames and ground-truth images are independently lifted to 3D point clouds using depth estimation and foreground masking. The reconstructed point clouds are then aligned to compute the Chamfer Distance.
Figure A10 : Qualitative comparison of reconstructed 3D point clouds. Using the evaluation pipeline described above, we lift each method’s generated 360∘ rotation video into colored point clouds and visualize them from four different viewpoints. The leftmost column shows the reference condition used for each method: for ours, a representative subset of the 30 multi-view images is displayed, while single-view baselines (Phantom [ 24 ] and VACE [ 18 ] ) rely on a single reference image. Our framework produces a geometrically coherent 3D structure, while Phantom [ 24 ] yields over-smoothed geometry and VACE [ 18 ] exhibits fragmented surfaces with structural artifacts.
Generating coherent videos grounded in a unified underlying world remains a major challenge, particularly under large viewpoint changes, where existing methods often struggle to maintain stable representations of individual objects. To this end, we introduce Totem, an object-centric 3D memory framework for world-consistent video generation. Totem first represents each selected object as a persistent 3D memory aligned to a shared world frame, assigning it a dedicated state. Based on these memories, we design a dual-stream 3D conditioning architecture that combines object memories with scene geometry for target-view synthesis. Moreover, these object memories constrain the model's attention with 3D structural information, thereby preserving object surfaces and relative spatial layouts. Extensive experiments on world-generation and object-centric benchmarks demonstrate that Totem generates videos depicting a more object-consistent 3D world under challenging camera trajectories, while supporting versatile applications.
Zichen Wan, Tianyu Huang, Fan Li +3
Harbin Institute of Technology · Huawei Technologies Ltd.
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via feed-forward reconstruction techniques. This generative reconstruction approach combines the visual fidelity and creative capacity of video models with 3D outputs ready for real-time rendering and simulation. Scaling to large, complex environments requires 3D-consistent video generation over long camera trajectories with large viewpoint changes and location revisits, a setting where current video models degrade quickly. Existing methods for long-horizon generation are fundamentally limited by two forms of degradation: spatial forgetting and temporal drifting. As exploration proceeds, previously observed regions fall outside the model's temporal context, forcing the model to hallucinate structures when revisited. Meanwhile, autoregressive generation accumulates small synthesis errors over time, gradually distorting scene appearance and geometry. We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale. To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing -- retrieving relevant past frames and establishing dense correspondences with the target viewpoints -- while relying on the generative prior for appearance synthesis. To address temporal drifting, we train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it. Together, these enable substantially longer and 3D-consistent video trajectories, which we leverage to fine-tune feed-forward reconstruction models that reliably recover high-quality 3D scenes.
Recent video diffusion models achieve strong visual quality and temporal coherence, but still lack coordinated control over scene layout, customized subject identity, and camera/subject motion. We study controllable video generation from three prompts: a first-frame scene image, 3D-aware multi-view subject references, and a motion-driving signal. This setting is challenging because background regions are often trackable under camera motion, while foreground subjects can rotate, self-occlude, and reveal new regions that require identity-consistent appearance. We introduce Tri-Prompting, a video diffusion framework that combines multi-view subject conditioning with dual-conditioned motion control. Tri-Prompting uses XYZ tracking points for visible background motion and low-resolution RGB proxies for foreground subject pose, allowing the model to recover fine appearance from multi-view references while retaining flexibility for plausible subject-scene interactions. An inference-time ControlNet scale schedule further balances motion controllability and visual realism. We evaluate Tri-Prompting against specialized baselines: DaS for motion reconstruction and Phantom for subject-driven generation. Tri-Prompting achieves competitive or improved reconstruction quality, stronger multi-view identity preservation, and better 3D consistency, while enabling 3D-aware subject insertion and in-scene manipulation.