Generating high-quality textures for 3D assets is a challenging task. Existing multiview texture generation methods suffer from the multiview inconsistency and missing textures on unseen parts, while UV inpainting texture methods do not generalize well due to insufficient UV data and cannot well utilize 2D image diffusion priors. In this paper, we propose a new method called MV2UV that combines 2D generative priors from multiview generation and the inpainting ability of UV refinement to get high-quality texture maps. Our key idea is to adopt a UV space generative model that simultaneously inpaints unseen parts of multiview images while resolving the inconsistency of multiview images. Experiments show that our method enables a better texture generation quality than existing methods, especially in unseen occluded and multiview-inconsistent parts.
Figures & tables
Figure 1: The Overview of our MV2UV framework. We first generate multiview images (MVs) using multiview diffusion (MVD). Then, we treat the generated MVs as semantic prompts to guide texture generation on the UV map. The generation process is based on a diffusion U-Net, which primarily consists of Self Attention, UV Self Attention, and Reference Attention modules.
Figure 2: We apply the 3D coordinates as the positional encoding within MV-UV cross attention, enabling the UV map to attend to geometrically corresponding regions on multiview images for improved inpainting and inconsistency resolution.
Figure 3: The geometric position encoding. We first render the object mesh to obtain normal and position maps, which are processed in two separate branches: one in UV space and the other in view space. In each branch, the corresponding geometric maps are transformed into positional embeddings and further encoded by a learnable position encoder to produce geometric features.
Figure 4: The UV Islands. UV self attention enables the model to establish correlations between UV regions (e.g., PA and PB ) that are adjacent in 3D space, even when they are disconnected in the UV layout.
Figure 5: Comparisons with texture generation methods from a single image input. Our method enables automatic completion of textures in occluded UV regions while avoiding inconsistencies induced by direct projection.
Figure 6: Qualitative comparison on occluded regions. The occlusion mask (left), MV-Adapter results (middle), and our results (right) demonstrate our method’s capability in resolving view inconsistencies and producing more semantically coherent results.
Method
GSO Dataset
DTC Dataset
FID ( ↓ )
KID ( ↓ )
FID ( ↓ )
KID ( ↓ )
TEXGen
75.2
283.3
41.1
118.6
Hunyuan3D 2.1
30.8
72.5
41.4
49.9
UniTEX
39.2
101.5
39.7
43.3
MV-Adapter
24.7
47.5
28.4
41.8
Ours
24.4
43.6
26.4
28.7
Table 1: Performance comparison of material generation methods. Our approach achieves the best FID and KID scores on both datasets, demonstrating superior performance.
Method
FID ( ↓ )
KID ( ↓ )
MV-Adapter
123.6
157.9
Ours
67.5
47.6
Table 2: Quantitative evaluation on occluded regions. FID and KID are computed between rendered images and the ground truth forced on occluded part. Our method achieves superior performance over the MV-Adapter baseline.
Figure 7: The generated results with rendered views.
Dataset
Metrics
PSNR ↑
SSIM ↑
FID ↓
KID ↓
GSO
25.7
0.855
3.32
8.49
GSOc
25.5
0.852
4.03
10.6
DTC
28.8
0.886
5.44
14.9
DTCc
28.6
0.885
5.90
16.4
Table 3: Ablation study on View-inconsistency Resolution.
Figure 8: Ablation study results
Configuration
GSO Dataset
DTC Dataset
PSNR ↑
SSIM ↑
PSNR ↑
SSIM ↑
w/o Pos Enc
13.6
0.688
13.9
0.689
w/o Ref Attn
18.9
0.805
22.6
0.861
w/o UV Attn
21.5
0.814
23.9
0.863
Full Model
25.7
0.855
28.8
0.886
Table 4: Ablation study on GSO and DTC datasets. Configurations: without geometric position encoding (w/o Pos Enc), reference attention (w/o Ref Attn), UV self attention (w/o UV Attn).
Figure 9: Detail of Learnable Position Encoder.
Figure 10: MV2UV can generate diverse, high-quality textures from different prompts, enabling flexible and controllable texture synthesis.
High-quality texture generation is essential for creating realistic and production-ready 3D assets. Recent multi-view diffusion methods have shown promising results for image-guided 3D texturing, but they are typically constrained to low operating resolutions such as 512 or 768, making it difficult to preserve high-frequency details from high-resolution reference images. Scaling this paradigm to 2048 resolution is computationally prohibitive, as the unified multi-view sequence exceeds 212K tokens and incurs excessive memory and latency. In this paper, we present UltraTex, an efficient end-to-end framework for high-resolution multi-view diffusion-based 3D texturing. Our key observation is that object-centric multi-view renderings contain two major sources of redundancy: background-induced sequence redundancy and sparse token interactions within the foreground. To address them, we introduce Background Token Dropping, which removes background tokens before the DiT backbone, and Block-Sparse Attention, which reduces attention computation over the retained foreground sequence. To enable efficient foreground-only inference while avoiding reconstruction artifacts, we further design Foreground-Aware VAE Decoding to ensure the quality of the final high-resolution views. To satisfy the demanding data requirements of 2K-resolution multi-view diffusion training, we construct G-buffer TexVerse, a large-scale, ultra-high-resolution multi-view rendering dataset covering over 268,000 3D assets. Extensive experiments show that UltraTex generates visually faithful textures with rich fine-grained details, while substantially improving efficiency, achieving 20.6×--91.1× training speedup and 22.3×--74.6× end-to-end inference speedup over the baseline on common samples in our dataset. Code and data is at https://yiboz2001.github.io/UltraTex.
Yibo Zhang, Ze Yuan, Nan Cao +4
Jilin University, Changchun, China · Shanghai Innovation Institute, Shanghai, China · The University of Hong Kong, Hong Kong, China +3
Generating high-quality UV textures for 3D meshes remains challenging. Multi-view projection pipelines suffer from occlusion and view inconsistency, and recent methods that generate textures directly in UV space still rely on auxiliary modules to supply 3D information, leaving the attention mechanism tied to UV-grid positions rather than to the underlying surface geometry. This mismatch limits coherence across seams and disconnected UV islands. We propose DirectUV, an image-conditioned UV texture diffusion framework that operates in the latent UV space of a pretrained image VAE, in which a Diffusion Transformer denoises the UV latent given a single input image and a coarse UV map. At its core, Surface-Aware Positional Encoding (SAPE) replaces the standard 2D-grid positional encoding with encodings derived from per-token 3D surface coordinates obtained via UV-to-surface correspondence. As positional encodings define the distance metric used by attention, SAPE enables tokens to interact according to 3D positional proximity derived from surface correspondence rather than UV-grid distance, restoring coherence across seams and disconnected islands. A multi-level extension further assigns different attention heads to progressively finer subdivisions of the same latent UV patch, allowing the model to reason about surface structure at multiple granularities. Experiments show that DirectUV produces sharper and more globally consistent textures than other baselines, with the largest improvements in occluded and view-unseen regions where projection-based methods leave gaps or stretched textures.
Jiantao Lin, Yingjie Xu, Mingzhi Sheng +3
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Texture map production is an important part of 3D modeling and determines the rendering quality. Recently, diffusion-based methods have opened a new way for texture generation. However, restricted control flexibility and limited prompt modalities may prevent creators from producing desired results. Furthermore, inconsistencies between generated multi-view images often lead to poor texture generation quality. To address these issues, we introduce FlexPainter, a novel texture generation pipeline that enables flexible multi-modal conditional guidance and achieves highly consistent texture generation. A shared conditional embedding space is constructed to perform flexible aggregation between different input modalities. Utilizing such embedding space, we present an image-based CFG method to decompose structural and style information, achieving reference image-based stylization. Leveraging the 3D knowledge within the image diffusion prior, we first generate multi-view images simultaneously using a grid representation to enhance global understanding. Meanwhile, we propose a view synchronization and adaptive weighting module during diffusion sampling to further ensure local consistency. Finally, a 3D-aware texture completion model combined with a texture enhancement model is used to generate seamless, high-resolution texture maps. Comprehensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.