Using conditional image generators, texture artists can explore many single-view looks for an existing 3D shape. Despite impressive progress, state-of-the-art generative methods still struggle to generate a full object texture while closely adhering to fine scale geometric detail and single view references, leaving little room for artists guidance. Furthermore, current automatic models lack the flexibility for artist to explore multiple textures from varied sources in an interactive and controllable manner. Unlike methods trained on large 3D datasets that generate full object textures from global guidance, our work explores a local and less data-hungry approach to texture with explicit artist control. We leverage the geometric self-similarity and geometry-texture correlation existing in many natural and man-made shapes; and train a shape-specific local texture generation and completion model. This model learns from existing image model priors and a single 3D shape, and is guided by attending to a set of geometry-aware reference patches. The trained shape-specific network can transfer any novel reference to the full target object texture through patchwise inpainting. We show improved or comparable quality to strong image-conditioned texture generation baselines, suggesting local texturing as a promising research direction. Our model also enables local geometry-conditioned texture inpainting, guided by artist-selected references, and generalizes to PBR materials and unseen meshes for texture transfer. We piloted our novel texture fill capability as a Blender addon with several 3D texturing professionals who reported positive feedback on the model's controllability, practical usefulness, and creative affordances.
Figures & tables
Figure 1. We train a GLOSS network on many single-view generated images of one untextured 3D shape (left), allowing it to learn how to texture local shape regions by attending to local geometry (such as scales) and faithfully following conditional texture patches. Once trained, artists can use GLOSS to mix and match any number of references for interactive texture completion of the target shape (right), enabled through reference-guided local texture fill. We also show fully automated texturing, PBR channel completion, and transfer results across shapes.
Figure 2. Data and Training : we use off-the-shelf models to generate training data (§ 3.2 ) for our local texture inpainting model (§ 3.3 , § 3.4 ). Steps A to F show the patch data generation pipeline. Then we show how the generated data is used for model input as well as the training for the model.
Figure 3. Model Inference Applications: Reference brush creation from (A) a single view image or (B) any PBR texture map. The model can be used interactively to complete select faces, or fully automatically to complete the whole mesh.
Figure 4. Interactive texturing with multiple source single views on various parts of the object. We show the reference texture patch from the single view along with the normal, and where the texture brush created with that reference was applied to inpaint on the existing painted texture.
Figure 5. Interactive texturing : our model supports faithful and controllable texture fills from multiple sources.
Figure 6. Interactive applications, transfer, comparisons : our model supports a wide range of texturing applications.
Figure 7. Automatic texturing with fine-tuned models for the fish and reptile categories.
Figure 8. An example of Automatic texturing visual comparisons.
Method
Per-mesh
Fine-tune
LPIPS ↓
FID ↓
DS ↓
CMMD ↓
LPIPS ↓
FID ↓
DS ↓
CMMD ↓
TexGen
-
106.917
0.436
0.883
-
124.460
0.538
1.143
MV-Adapter
0.358
76.98
0.370
0.666
0.366
122.865
0.473
0.961
Hunyuan2.1
0.383
73.702
0.437
0.494
0.299
93.045
0.356
0.570
Trellis2
0.356
102.22
0.411
0.545
0.307
110.773
0.371
0.777
Ours
0.302
80.258
0.344
0.478
0.282
104.108
0.352
0.694
Table 1. Quantitative Results: Overall (DreamSim) and patch-level (LPIPS, FID, CMMD) metrics on automatic texture generation for per-mesh and fine-tuned models. Best in bold.
Variant
Croissant
Koi Fish
Sea Shell
Overall
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
No batch-attn.
0.4277
100.93
0.2248
57.09
0.2923
81.06
0.3262
79.70
No geom. cond.
0.3838
64.62
0.2512
79.73
0.2810
90.48
0.3175
78.27
No img. loss
0.3006
62.31
0.2027
53.25
0.2628
76.76
0.2517
64.11
No fine-tuning
0.3020
59.85
0.2011
51.77
0.2623
78.53
0.2518
63.38
Full model
0.2830
59.45
0.2009
55.54
0.2673
77.11
0.2420
64.03
Table 2. Ablations on 3 shapes and a truncated training run using our validation set suggest that batch-attention and geometric conditioning have the greatest effect on local texture quality.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9. Color leak effect from training model using nearest neighbor interpolation for latent mask downsampling. Top two rows show masked albedo and corresponding mask as model inputs; the third row is ground truth composited with green; the last row shows the model output.
Metric
Method
Overall
by Example
Brick
Cabbage
Croissant
Tire
Fire Hydrant
Gourd
Koi Fish
Barrel
Sea Shell
Turtle
LPIPS ↓
Paint3D
0.378
0.580
0.501
0.370
0.379
0.265
0.432
0.337
0.328
0.348
0.242
MV-Adapter
0.358
0.650
0.485
0.340
0.352
0.226
0.429
0.286
0.295
0.302
0.211
Hunyuan2.1
0.383
0.577
0.498
0.352
0.370
0.272
0.604
0.303
0.314
0.328
0.214
Trellis2.0
0.356
0.547
0.482
0.341
0.358
0.283
0.407
0.299
0.307
0.327
0.213
Ours
0.302
0.462
0.386
0.256
0.319
0.208
0.361
0.281
0.283
0.283
0.181
Appendix
Table 3. Quantitative Results: Overall (DreamSim) and patch-level (LPIPS, FID, CMMD) metrics on texture generation task conditioned on a single view, across 10 shapes with 50 views each, reveal competitive performance of our method, despite using no textured 3D training data for training.
Metric
Method
Overall
by Example
Cabbage1
Cabbage3
Cantaloup
Fish1
Fish3
Fish4
Toad
Tortoise
Turtle
LPIPS ↓
MV-Adapter
0.366
0.319
0.299
0.598
0.301
0.409
0.247
0.399
0.387
0.334
Hunyuan2.1
0.299
0.287
0.266
0.440
0.253
0.335
0.187
0.327
0.318
0.279
Trellis2.0
0.307
0.284
0.279
0.471
0.268
0.379
0.188
0.321
0.306
0.269
Ours
0.282
0.278
0.255
0.407
0.248
0.308
0.172
0.315
0.287
0.268
FID ↓
TexGen
124.460
116.061
84.111
290.178
130.762
120.826
107.264
52.956
113.996
103.989
Appendix
Table 4. Quantitative Results on Transfer Meshes: Overall and per-example LPIPS, FID, DreamSim, and CMMD on the texture-transfer evaluation set (9 sub-meshes spanning 3 base shapes). Best per column in bold.
Figure 10. We show a visual comparison of our model ablation experiments. We show reference albedo and geometry patches on the left and each model variant’s output.
Experiment
LPIPS ↓
DreamSim ↓
FID ↓
guidance_2 (final)
0.3031
0.3441
80.24
wo_syncmvd
0.3040
0.3393
74.81
guidance_4
0.3044
0.3514
86.64
guidance_10
0.3169
0.3829
120.97
overlap_0.4
0.3099
0.3644
97.85
wo_color_correct
0.3044
0.3497
84.41
Appendix
Table 5. Quantitative evaluation of textures generated with variations of final completion experiment settings. We report average LPIPS, DreamSim, and FID metrics over 10 objects.
Cabbages
Fishes
Turtles
Overall
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
v100–10k
0.260
75.87
0.176
111.95
0.263
76.32
0.233
88.05
v100–20k
0.253
80.78
0.169
109.63
0.253
76.33
0.225
88.91
v100–50k
0.255
80.16
0.170
104.27
0.268
88.24
0.231
90.89
Appendix
Table 6. Step-count ablation on the 100 view variant. LPIPS / FID at 10k, 20k, and 50k fine-tuning steps. We show that finetuning results are best at an early stage, enabling faster convergence.
Cabbages
Fishes
Turtles
Overall
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
50 views
0.255
72.80
0.170
108.66
0.257
78.77
0.227
86.74
100 views
0.255
80.16
0.170
104.27
0.268
88.24
0.231
90.89
200 views
0.255
83.40
0.172
110.26
0.268
82.68
0.232
92.11
Appendix
Table 7. View-count ablation comparing v50, v100, and v200 fine-tunes. We show that we can finetune with a small number of views.
Figure 11. Batch multi-attention visualization comparing two batches with the same reference images but a different target image (last in the batch). The overlaid heatmap shows how the center patch token in the target image attends to all patch tokens across the batch.
Figure 12. On the left we show examples of semantic texture failure. On the right we show examples of failure cases due to low geometry and texture correlation.
Figure 13. Additional result on texturing applications.
Figure 15. Additional results on the baseline comparison, both global view and zoomed-in views.
Figure 16. Additional results on the baseline comparison, both global view and zoomed-in views.
Figure 17. We show more automatic texture completion results with GLOSS. We show views of the 3D model that are opposite to the conditional view. We also show the textured 3D model from a view other than these two.
Recent 3D generative models can synthesize high-quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large-scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large-scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white-mesh geometry using an off-the-shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit-scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large-scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.
Yue Han, Chong Li, Zhening Liu +5
ZGCA & ZGCI · Zhejiang University · Microsoft Research +1
Texture map production is an important part of 3D modeling and determines the rendering quality. Recently, diffusion-based methods have opened a new way for texture generation. However, restricted control flexibility and limited prompt modalities may prevent creators from producing desired results. Furthermore, inconsistencies between generated multi-view images often lead to poor texture generation quality. To address these issues, we introduce FlexPainter, a novel texture generation pipeline that enables flexible multi-modal conditional guidance and achieves highly consistent texture generation. A shared conditional embedding space is constructed to perform flexible aggregation between different input modalities. Utilizing such embedding space, we present an image-based CFG method to decompose structural and style information, achieving reference image-based stylization. Leveraging the 3D knowledge within the image diffusion prior, we first generate multi-view images simultaneously using a grid representation to enhance global understanding. Meanwhile, we propose a view synchronization and adaptive weighting module during diffusion sampling to further ensure local consistency. Finally, a 3D-aware texture completion model combined with a texture enhancement model is used to generate seamless, high-resolution texture maps. Comprehensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.
Single image 3D generation aims to dress a given 3D mesh with relightable materials while faithfully preserving the visual details of a reference image. Existing methods either introduce artifacts when fusing inconsistent multi-view predictions or struggle to preserve fine details through implicit image-feature conditioning. To resolve these challenges, we introduce SUPRA-Tex, built on the principle of decoupling visible-material estimation, surface alignment, and observation-guided 3D completion. This division of labor allows the 3D generator, trained on limited 3D data, to focus its capacity on preserving visible evidence and completing unobserved surfaces. Specifically, we first train an MLLM-based image editing model to predict lighting-separated PBR maps in the reference view. We then project these maps onto the visible surface and encode them with the pretrained material VAE encoder into native observation latents that share the generation target's latent space. An interleaved flow transformer holds these latents as fixed, read-only anchors while alternating unobserved-region and full-surface updates to generate complete material latents. This design achieves super-alignment with the reference, delivering What You See Is What You Get fidelity on visible regions and coherent completion on unobserved surfaces. Trained solely on public datasets, SUPRA-Tex achieves SOTA performance in both geometry-conditioned texturing and end-to-end asset generation, matching or even exceeding leading commercial systems on in-the-wild images. Project page: https://supra-3d.github.io/supratex/
Chen Yang, Jiemin Fang, Fanpeng Meng +5
Huawei Inc. · Shanghai Jiao Tong University · Huazhong University of Science and Technology