The creation of high-fidelity, customizable 3D indoor scene textures remains a significant challenge. While text-driven methods offer flexibility, they lack the precision for fine-grained, instance-level control, and often produce textures with insufficient quality, artifacts, and baked-in shading. To overcome these limitations, we introduce CustomTex, a novel framework for instance-level, high-fidelity scene texturing driven by reference images. CustomTex takes an untextured 3D scene and a set of reference images specifying the desired appearance for each object instance, and generates a unified, high-resolution texture map. The core of our method is a dual-distillation approach that separates semantic control from pixel-level enhancement. We employ semantic-level distillation, equipped with an instance cross-attention, to ensure semantic plausibility and ``reference-instance'' alignment, and pixel-level distillation to enforce high visual fidelity. Both are unified within a Variational Score Distillation (VSD) optimization framework. Experiments demonstrate that CustomTex achieves precise instance-level consistency with reference images and produces textures with superior sharpness, reduced artifacts, and minimal baked-in shading compared to state-of-the-art methods. Our work establishes a more direct and user-friendly path to high-quality, customizable 3D scene appearance editing.
Figures & tables
Figure 2: Pipeline of CustomTex. CustomTex textures a complete 3D indoor scene by optimizing a texture map in UV space through a dual-distillation training approach. In each iteration, the 3D scene with optimized texture is rendered from a random viewpoint, producing an RGB image, a depth map and instance masks. Instance masks are used to align each reference image’s features with the correct object instance in the rendered RGB image via a specialized cross-attention. The Variational Score Distillation gradient and the Super-Resolution gradient are computed based on the well-aligned reference images condition to update the texture field.
Figure 3: Qualitative comparison on image-to-texture generation. All generated textures are rendered by 3ds Max software at a resolution of 2000×2000 for visualization. CustomTex demonstrates instance-level consistency with the reference images, while also exhibiting greater sharpness with fewer shading effects and artifacts compared with the baselines.
Figure 4: Qualitative comparison on close-up texture renderings.
Figure 5: Qualitative comparison on text-to-texture generation. The text prompt is: “ The Nanyang vintage-style living room equipped with walls featuring dark wood panel textures, a brown leather sofa, a round fabric stool with floral patterns, a TV stand made of dark wood with golden handles, dark brown wooden chairs and a light-color wood coffee table. ” GPT-4v is used to convert this text prompt into reference image prompts for our CustomTex. All generated textures are rendered to 768×768 resolution images for visualization. Only CustomTex demonstrates instance-level consistency with the text prompt.
Method
CLIP-I ↑
CLIP-FID ↓
Q-Align
Q-Align
IQA ↑
IAA ↑
Paint3D [ 76 ]
0.694
130.138
2.896
2.401
HY3D-2.1 [ 64 ]
0.682
134.680
2.187
1.838
SceneTex-IPA [ 12 ]
0.741
121.118
4.009
3.594
CustomTex (Ours)
0.797
106.229
4.469
3.629
Table 1: Quantitative comparison on image-to-texture generation.
Method
CLIP-T ↑
IS ↑
Q-Align
Q-Align
IQA ↑
IAA ↑
TEXture [ 58 ]
0.557
1.372
1.645
1.574
Paint3D [ 76 ]
0.734
2.330
2.442
1.792
SyncMVD [ 45 ]
0.712
2.467
2.409
2.328
HY3D-2.1 [ 64 ]
0.734
2.381
2.774
2.033
SceneTex [ 12 ]
0.639
3.009
3.824
2.681
Table 2: Quantitative comparison on text-to-texture generation.
Method
Visual Quality ↑
Prompts Consistency ↑
Paint3D [ 76 ]
2.792
2.817
HY3D-2.1 [ 64 ]
2.525
2.619
SceneTex-IPA [ 12 ]
3.842
3.617
CustomTex (Ours)
4.008
4.125
Table 3: User study results on visual quality and prompt consistency, averaged over 60 participants on a 1–5 scale. Our method achieves the highest scores across both criteria.
Figure 6: Qualitative ablation study results: post-SR vs. full model .
Figure 7: Qualitative ablation study results.
Method
CLIP-I ↑
CLIP-FID ↓
Q-Align
Q-Align
IQA ↑
IAA ↑
post-SR
0.746
114.612
2.959
2.190
w/o LSR
0.736
118.247
3.330
2.664
w/o multi-ref
0.757
109.243
4.053
3.519
w/o f-mask
0.743
111.205
3.689
3.231
full model
0.797
106.229
4.469
3.629
Table 4: Quantitative ablation study results. We validate the effectiveness of our method by testing four model variants: post-SR , w/o LSR , w/o feature-level mask and w/o multi-ref , showcasing their distinct contributions.
Figure 8: Textures before and after image super-resolution operation. These two textures have a resolution of 4,096×4,096 , and correspond to the scene shown in the first row of Fig. 6 . Black pixels in textures denote invalid regions where UV coordinates do not map to any vertex in the mesh.
Figure 9: Comparison on single-object texturing.
Figure 10: Visual comparison with InstanceTex [ 72 ] and RoomPainter [ 35 ] .
Texture resolution
Inference time (s)
1,024×1,024
0.15
2,048×2,048
0.54
3,072×3,072
1.19
4,096×4,096
2.41
8,192×8,192
9.69
12,288×12,288
21.69
Table 5: Inference time across different texture resolutions
Figure 11: Qualitative results on challenging scenes with diverse room types and complex layouts.
Figure 12: The “living room” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 13: The “living room” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 14: The “bedroom” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 15: The “bedroom” texture generated by our method is rendered into 2,000×2,000 resolution image.
Figure 16: Using images with more diverse styles as the reference images.