UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing
Authors: Yibo Zhang, Ze Yuan, Nan Cao, Li Zhang, Yan-Pei Cao, Yuan-Chen Guo, Rui Ma
Organizations: Jilin University, Changchun, China · Shanghai Innovation Institute, Shanghai, China · The University of Hong Kong, Hong Kong, China · Tongji University, Shanghai, China · Fudan University, Shanghai, China · VAST, Beijing, China
High-quality texture generation is essential for creating realistic and production-ready 3D assets. Recent multi-view diffusion methods have shown promising results for image-guided 3D texturing, but they are typically constrained to low operating resolutions such as 512 or 768, making it difficult to preserve high-frequency details from high-resolution reference images. Scaling this paradigm to 2048 resolution is computationally prohibitive, as the unified multi-view sequence exceeds 212K tokens and incurs excessive memory and latency. In this paper, we present UltraTex, an efficient end-to-end framework for high-resolution multi-view diffusion-based 3D texturing. Our key observation is that object-centric multi-view renderings contain two major sources of redundancy: background-induced sequence redundancy and sparse token interactions within the foreground. To address them, we introduce Background Token Dropping, which removes background tokens before the DiT backbone, and Block-Sparse Attention, which reduces attention computation over the retained foreground sequence. To enable efficient foreground-only inference while avoiding reconstruction artifacts, we further design Foreground-Aware VAE Decoding to ensure the quality of the final high-resolution views. To satisfy the demanding data requirements of 2K-resolution multi-view diffusion training, we construct G-buffer TexVerse, a large-scale, ultra-high-resolution multi-view rendering dataset covering over 268,000 3D assets. Extensive experiments show that UltraTex generates visually faithful textures with rich fine-grained details, while substantially improving efficiency, achieving 20.6×--91.1× training speedup and 22.3×--74.6× end-to-end inference speedup over the baseline on common samples in our dataset. Code and data is at https://yiboz2001.github.io/UltraTex.
Figures & tables
Figure 1. A gallery of diverse 3D assets textured by UltraTex, showcasing high-fidelity results with rich fine-grained details.
Figure 2. Overview of UltraTex. UltraTex generates geometry-aligned 2048-resolution multi-view texture images for 3D texturing. UltraTex first removes all background tokens before the DiT blocks, and then applies block-sparse attention on the remaining foreground sequence. During inference, foreground-only denoising and foreground-aware VAE decoding generate multi-view results, which are reprojected onto the input mesh to obtain the final textured 3D asset.
Figure 3. Examples of foreground sparsity in object-centric multi-view renderings. Each image shows a six-view layout, where only the object foreground corresponds to surface regions requiring texture generation. The two examples contain only 24.9% and 7.9% foreground pixels, respectively, highlighting the substantial background redundancy.
Figure 4. Preliminary token-dropping experiment with pretrained FLUX. We remove background tokens according to the foreground masks in the multi-view layout while preserving the original positional indices of the retained tokens. The model can still denoise the resulting discontinuous partial sequence and produce spatially plausible foreground results.
Figure 5. Attention sparsity over the retained foreground sequence after Background Token Dropping. We visualize attention maps from a lower-resolution full-attention model trained with Background Token Dropping. Both Double-Stream (DS) and Single-Stream (SS) DiT block attention maps exhibit sparse patterns over the retained foreground tokens.
Unshaded
Shaded
Relighting
Method
FID ↓
CLIP-FID ↓
CMMD ↓
CLIP-I ↑
LPIPS ↓
FID ↓
CLIP-FID ↓
CMMD ↓
CLIP-I ↑
LPIPS ↓
FID ↓
CLIP-FID ↓
CMMD ↓
CLIP-I ↑
LPIPS ↓
Step1X-3D
–
–
–
–
–
160.3
19.265
0.613
0.900
0.154
160.7
19.189
0.600
0.901
0.154
UniTEX
–
–
–
–
–
133.8
16.750
0.524
0.918
0.116
133.4
15.697
0.458
0.923
0.113
Hunyuan3D 2.1
153.48
18.995
1.150
0.908
0.136
123.1
14.354
0.432
0.929
0.119
126.0
14.117
0.409
0.930
0.117
LumiTex
147.31
18.251
1.223
0.914
0.138
110.6
12.265
0.351
0.939
0.107
114.5
11.982
0.316
0.941
0.105
UltraTex (Ours)
125.13
15.180
0.810
0.929
0.082
97.6
12.239
0.368
0.941
0.064
96.4
11.583
0.333
0.944
0.062
Table 1. Quantitative comparison with baseline methods. ↓ indicates lower is better, and ↑ indicates higher is better.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FLUX VAE (Full Image)
47.65
0.9937
0.0023
FLUX VAE (Noise Background)
29.66
0.9555
0.0263
Ours (w/o finetune)
37.81
0.9818
0.0084
Ours (w/ finetune)
50.26
0.9956
0.0038
Table 2. Quantitative comparison of different VAE reconstruction settings.
Figure 6. Qualitative comparison with state-of-the-art PBR-based methods (* denotes a commercial model). Our method consistently recovers finer texture details and achieves high visual quality across diverse inputs, including artist-created assets (first four examples) and in-the-wild images (last three examples).
Figure 7. Efficiency analysis of UltraTex at 2048 resolution. Background Token Dropping (BTD) reduces training/inference cost and peak GPU memory by dropping background tokens and shortening the effective sequence, while Block-Sparse Attention further improves efficiency over the retained foreground tokens. With the adopted retention ratio ρ=20% , the combined design achieves 20.6× – 91.1× training speedup and 22.3× – 74.6× end-to-end inference speedup within the common foreground-ratio range of our dataset.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 . Examples of albedo entropy filtering.
Figure S2 . Foreground ratio statistics of our rendered dataset. Most objects have relatively small foreground regions, with the distribution concentrated in the 5% – 30% range and only a few samples exceeding 40% .
Figure S3 . Influence of the pretrained foundation model on identity preservation. From left to right: the reference image, the result using FLUX, and the result using FLUX.2-dev. FLUX.2-dev more faithfully preserves the distinctive posters, illustrations, and table layout of the reference. Three images of a bulletin board. The left image is the reference, the middle image is the FLUX result, and the right image is the FLUX.2-dev result. The right image retains the reference's poster designs and central table more faithfully than the middle image.
Figure S4 . PBR material generation with FLUX.2-dev as the pretrained foundation model. The reference image (left) conditions the generation of six-view roughness–metallic maps (right), visualized as packed material channels. A reference image of an armored character on the left, vertically centered next to a two-row, three-column grid of generated roughness and metallic maps on the right.
Table S1 . Efficiency analysis of UltraTex at 2048 resolution under different foreground ratios.
Numerous 3D assets are discarded due to low texture resolution, while current super-resolution models ignore texture maps and focus on natural images. An efficient and generalizable texture super-resolution model can revitalize a large corpus of aging yet valuable assets across industries such as film and video games. We present Texture++, a novel framework for texture super-resolution, which enhances the low-resolution textures of assets to produce high-resolution, high-quality results. Specifically, we reformulate the task of super-resolution in UV space into performing it across multiple rendered views and merging the outputs. Firstly, to achieve more complete and continuous textures in the view space, we propose an adaptive view selection strategy to integrate textures dispersed across UV texture patches. Furthermore, we introduce a quadtree-based texture region organization method for combining super-resolved textures from different viewpoints, providing masks to distinguish regions that require improvement. Finally, we design a diffusion-based super-resolution model that enhances the texture resolution for specified masked regions, seamlessly integrating with surrounding regions. Through comprehensive evaluations, we demonstrate that our approach yields textures with substantially improved detail and coherence over existing methods.
Shuaiwei Wang, Shi Li, Jieting Xu +4
State Key Laboratory of CAD&CG, Zhejiang University, China · North China Electric Power University, China
Recent 3D generation models can produce accurate geometries while still struggling to reconstruct detailed textures. We propose a diffusion-based native 3D material generation model TaoTex, which faithfully recovers intricate textures through tailored strategies and improvements. First, we develop a data construction agent to create high-frequency textured 3D assets to bridge the data gap in public datasets. Training with these data significantly enhances the ability of TaoTex to recover challenging details such as text and patterns. Second, we design a multi-level feature fusion (MLFF) module to adaptively integrate local and global features of the conditional input, providing more complete texture cues for the diffusion model and thereby enhancing reconstruction fidelity. To alleviate VAE reconstruction errors, we adopt a latent-to-pixel space loss transition, further improving the pixel-level details and generation quality. Finally, we scale TaoTex to multi-view inputs by incorporating learnable viewpoint embeddings, achieving accurate and consistent material reconstruction across views. Extensive experiments demonstrate that our method significantly outperforms existing approaches in preserving texture details in both single- and multi-view settings.
Texture map production is an important part of 3D modeling and determines the rendering quality. Recently, diffusion-based methods have opened a new way for texture generation. However, restricted control flexibility and limited prompt modalities may prevent creators from producing desired results. Furthermore, inconsistencies between generated multi-view images often lead to poor texture generation quality. To address these issues, we introduce FlexPainter, a novel texture generation pipeline that enables flexible multi-modal conditional guidance and achieves highly consistent texture generation. A shared conditional embedding space is constructed to perform flexible aggregation between different input modalities. Utilizing such embedding space, we present an image-based CFG method to decompose structural and style information, achieving reference image-based stylization. Leveraging the 3D knowledge within the image diffusion prior, we first generate multi-view images simultaneously using a grid representation to enhance global understanding. Meanwhile, we propose a view synchronization and adaptive weighting module during diffusion sampling to further ensure local consistency. Finally, a 3D-aware texture completion model combined with a texture enhancement model is used to generate seamless, high-resolution texture maps. Comprehensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.