The creation of high-fidelity, customizable 3D indoor scene textures remains a significant challenge. While text-driven methods offer flexibility, they lack the precision for fine-grained, instance-level control, and often produce textures with insufficient quality, artifacts, and baked-in shading. To overcome these limitations, we introduce CustomTex, a novel framework for instance-level, high-fidelity scene texturing driven by reference images. CustomTex takes an untextured 3D scene and a set of reference images specifying the desired appearance for each object instance, and generates a unified, high-resolution texture map. The core of our method is a dual-distillation approach that separates semantic control from pixel-level enhancement. We employ semantic-level distillation, equipped with an instance cross-attention, to ensure semantic plausibility and ``reference-instance'' alignment, and pixel-level distillation to enforce high visual fidelity. Both are unified within a Variational Score Distillation (VSD) optimization framework. Experiments demonstrate that CustomTex achieves precise instance-level consistency with reference images and produces textures with superior sharpness, reduced artifacts, and minimal baked-in shading compared to state-of-the-art methods. Our work establishes a more direct and user-friendly path to high-quality, customizable 3D scene appearance editing.
Figures & tables
Figure 2: Pipeline of CustomTex. CustomTex textures a complete 3D indoor scene by optimizing a texture map in UV space through a dual-distillation training approach. In each iteration, the 3D scene with optimized texture is rendered from a random viewpoint, producing an RGB image, a depth map and instance masks. Instance masks are used to align each reference image’s features with the correct object instance in the rendered RGB image via a specialized cross-attention. The Variational Score Distillation gradient and the Super-Resolution gradient are computed based on the well-aligned reference images condition to update the texture field.
Figure 3: Qualitative comparison on image-to-texture generation. All generated textures are rendered by 3ds Max software at a resolution of 2000×2000 for visualization. CustomTex demonstrates instance-level consistency with the reference images, while also exhibiting greater sharpness with fewer shading effects and artifacts compared with the baselines.
Figure 4: Qualitative comparison on close-up texture renderings.
Figure 5: Qualitative comparison on text-to-texture generation. The text prompt is: “ The Nanyang vintage-style living room equipped with walls featuring dark wood panel textures, a brown leather sofa, a round fabric stool with floral patterns, a TV stand made of dark wood with golden handles, dark brown wooden chairs and a light-color wood coffee table. ” GPT-4v is used to convert this text prompt into reference image prompts for our CustomTex. All generated textures are rendered to 768×768 resolution images for visualization. Only CustomTex demonstrates instance-level consistency with the text prompt.
Method
CLIP-I ↑
CLIP-FID ↓
Q-Align
Q-Align
IQA ↑
IAA ↑
Paint3D [ 76 ]
0.694
130.138
2.896
2.401
HY3D-2.1 [ 64 ]
0.682
134.680
2.187
1.838
SceneTex-IPA [ 12 ]
0.741
121.118
4.009
3.594
CustomTex (Ours)
0.797
106.229
4.469
3.629
Table 1: Quantitative comparison on image-to-texture generation.
Method
CLIP-T ↑
IS ↑
Q-Align
Q-Align
IQA ↑
IAA ↑
TEXture [ 58 ]
0.557
1.372
1.645
1.574
Paint3D [ 76 ]
0.734
2.330
2.442
1.792
SyncMVD [ 45 ]
0.712
2.467
2.409
2.328
HY3D-2.1 [ 64 ]
0.734
2.381
2.774
2.033
SceneTex [ 12 ]
0.639
3.009
3.824
2.681
Table 2: Quantitative comparison on text-to-texture generation.
Method
Visual Quality ↑
Prompts Consistency ↑
Paint3D [ 76 ]
2.792
2.817
HY3D-2.1 [ 64 ]
2.525
2.619
SceneTex-IPA [ 12 ]
3.842
3.617
CustomTex (Ours)
4.008
4.125
Table 3: User study results on visual quality and prompt consistency, averaged over 60 participants on a 1–5 scale. Our method achieves the highest scores across both criteria.
Figure 6: Qualitative ablation study results: post-SR vs. full model .
Figure 7: Qualitative ablation study results.
Method
CLIP-I ↑
CLIP-FID ↓
Q-Align
Q-Align
IQA ↑
IAA ↑
post-SR
0.746
114.612
2.959
2.190
w/o LSR
0.736
118.247
3.330
2.664
w/o multi-ref
0.757
109.243
4.053
3.519
w/o f-mask
0.743
111.205
3.689
3.231
full model
0.797
106.229
4.469
3.629
Table 4: Quantitative ablation study results. We validate the effectiveness of our method by testing four model variants: post-SR , w/o LSR , w/o feature-level mask and w/o multi-ref , showcasing their distinct contributions.
Figure 8: Textures before and after image super-resolution operation. These two textures have a resolution of 4,096×4,096 , and correspond to the scene shown in the first row of Fig. 6 . Black pixels in textures denote invalid regions where UV coordinates do not map to any vertex in the mesh.
Figure 9: Comparison on single-object texturing.
Figure 10: Visual comparison with InstanceTex [ 72 ] and RoomPainter [ 35 ] .
Texture resolution
Inference time (s)
1,024×1,024
0.15
2,048×2,048
0.54
3,072×3,072
1.19
4,096×4,096
2.41
8,192×8,192
9.69
12,288×12,288
21.69
Table 5: Inference time across different texture resolutions
Figure 11: Qualitative results on challenging scenes with diverse room types and complex layouts.
Figure 12: The “living room” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 13: The “living room” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 14: The “bedroom” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 15: The “bedroom” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 16: Using images with more diverse styles as the reference images.
SceneHI is a framework that lifts high-resolution, illumination-aware priors from 2D diffusion models to perform 3D texture synthesis. It is the first to demonstrate that high-resolution textures, previously limited to 2D synthesis, can be generated directly on 3D objects without model fine-tuning or optimization. Designed for complex, multi-object environments, SceneHI uniquely combines 3D-consistency, high-resolution fidelity, and physically plausible baked shadows within a single generative pipeline. To enforce strict geometric coherence, we introduce an exact analytical pixel-to-texel mapping that aligns diffusion trajectories across multiple viewpoints. We utilize High-Resolution Latent Textures (HRLTs) as a persistent canvas for gradually denoised textures, while camera views perform the denoising steps in latent pixel space. This ensures a shared base texture that can be subsequently refined to high resolution without compromising multi-view consistency. Finally, a light-aware generative pass embeds realistic geometry-consistent shadows directly into the atlases, bridging the gap to production workflows. SceneHI achieves high visual fidelity while reducing generation time by 80% compared to existing scene-level methods.
Athanasios Tragakis, Marco Aversa, Daniela Ivanova +4
University of Glasgow, Glasgow, United Kingdom · Independent Researcher, Switzerland
This paper presents Native3D, the first end-to-end 3D scene generation framework that completely bypasses 2D intermediate representations. Traditional approaches typically require adapting 3D representations to the 2D domain to leverage pre-trained diffusion models, which inevitably introduces domain adaptation issues including geometric structural distortion and texture detail degradation. To address these limitations, we design a unified mesh-texture joint representation that simultaneously models both geometric structures and texture features through a Transformer-based scene encoder, effectively maintaining spatial relationships and visual consistency among objects within scenes. We further propose the 3D Representation Alignment Loss (3D REPA Loss), which employs an improved contrastive learning mechanism to align multi-level semantic representations in the latent space, significantly enhancing geometric and textural fidelity. Experimental results demonstrate that Native3D outperforms existing methods in both generation quality and editing flexibility, providing a novel solution for 3D scene editing.
High-quality 3D scene reconstruction has recently advanced toward generalizable feed-forward architectures, enabling the generation of complex environments in a single forward pass. However, despite their strong performance in static scene perception, these models remain limited in responding to dynamic human instructions, which restricts their use in interactive applications. Existing editing methods typically rely on a 2D-lifting strategy, where individual views are edited independently and then lifted back into 3D space. This indirect pipeline often leads to blurry textures and inconsistent geometry, as 2D editors lack the spatial awareness required to preserve structure across viewpoints. To address these limitations, we propose VGGT-Edit, a feed-forward framework for text-conditioned native 3D scene editing. VGGT-Edit introduces depth-synchronized text injection to align semantic guidance with the backbone's spatial poses, ensuring stable instruction grounding. This semantic signal is then processed by a residual transformation head, which directly predicts 3D geometric displacements to deform the scene while preserving background stability. To ensure high-fidelity results, we supervise the framework with a multi-term objective function that enforces geometric accuracy and cross-view consistency. We also construct the DeltaScene Dataset, a large-scale dataset generated through an automated pipeline with 3D agreement filtering to ensure ground-truth quality. Experiments show that VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed. The project page is https://chriszkxxx.github.io/VGGT-Edit/.
Kaixin Zhu, Yiwen Tang, Yifan Yang +9
1Peking University · 2Tencent · 3The Chinese University of Hong Kong +4