Single image 3D generation aims to dress a given 3D mesh with relightable materials while faithfully preserving the visual details of a reference image. Existing methods either introduce artifacts when fusing inconsistent multi-view predictions or struggle to preserve fine details through implicit image-feature conditioning. To resolve these challenges, we introduce SUPRA-Tex, built on the principle of decoupling visible-material estimation, surface alignment, and observation-guided 3D completion. This division of labor allows the 3D generator, trained on limited 3D data, to focus its capacity on preserving visible evidence and completing unobserved surfaces. Specifically, we first train an MLLM-based image editing model to predict lighting-separated PBR maps in the reference view. We then project these maps onto the visible surface and encode them with the pretrained material VAE encoder into native observation latents that share the generation target's latent space. An interleaved flow transformer holds these latents as fixed, read-only anchors while alternating unobserved-region and full-surface updates to generate complete material latents. This design achieves super-alignment with the reference, delivering What You See Is What You Get fidelity on visible regions and coherent completion on unobserved surfaces. Trained solely on public datasets, SUPRA-Tex achieves SOTA performance in both geometry-conditioned texturing and end-to-end asset generation, matching or even exceeding leading commercial systems on in-the-wild images. Project page: https://supra-3d.github.io/supratex/
Figures & tables
Figure 1: Overview of SUPRA-Tex. Left: image-to-material translation. A 2D material translator estimates lighting-separated albedo, metallic, and roughness maps from a single reference image. Middle: native observation encoding. Using the mesh M and camera projection π , we project valid predictions onto visible surface voxels and omit unobserved voxels from the partial PBR input. The frozen material encoder Emat maps these partial PBR attributes into the generation target’s latent space. These latents, together with coverage and boundary metadata, form the observation representation Co . Right: observation-guided completion. Conditioned on geometry latents G , the interleaved flow transformer generates complete material latents Z0 from noise Z1 . In the inset, the fixed, read-only stream O supplies keys and values, while the generation stream X alternates updates over cells without observations ( U ) and the full surface ( S ). The frozen decoder Dmat converts Z0 into surface PBR materials, lit from the left and right.
Method
Shaded
PBR
PSNR ↑
PSNR fg ↑
SSIM ↑
LPIPS ↓
CLIP-I ↑
FD DINOv3 ↓
A. PSNR ↑
A. PSNR fg ↑
A. SSIM ↑
A. LPIPS ↓
Met. ↓
Rough. ↓
DTC-Tex
Step1X-3D
23.12
14.21
0.914
0.083
0.851
33.30
–
–
–
–
–
–
MV-Adapter
24.64
15.73
0.917
0.077
0.858
33.17
–
–
–
–
–
–
Ink3D
25.82
17.08
0.921
0.076
0.859
35.01
–
–
–
–
–
–
UniTEX
25.42
16.54
0.925
0.071
0.877
29.24
–
–
–
–
–
–
Table 1: PBR texture generation with given geometry. Per-view metrics are averaged over 16 views; fg denotes the GT foreground. A.: albedo; Met./Rough.: metallic/roughness MAE.
Figure 2: Qualitative comparison with given geometry. Enlarged crops compare reference-view details; novel views show the completed appearance.
Table 2: Reference/novel-view fidelity on the valid objects per method and metric in Table 1 . PSNR fg uses the GT mask.
Table 3: End-to-end single-image-to-3D generation by dataset; column-wise best results are bold. Reconstruction uses four views except for reference-view CLIP-I; distribution metrics use 16-view rendered sets.
Figure 3: Comparison with commercial systems: each row shows the input and reference-view/back-view renderings from commercial systems and SUPRA-Tex.
Table 7
Figure 4: Native observation encoding with GT geometry and materials. (a) Partial-versus-complete latent differences across valid Complex-Tex objects. (b) Decoded albedo for three cases selected by latent-difference percentile, with shared GT-selected crops. PSNR covers all retained observed GT voxels, not only the displayed crops.
Table 6: Runtime and sampling memory. Speedup is relative to Joint-update attention. Peak mean/max summarize sampling allocations; stage sum adds separately timed pipeline stages, not continuous end-to-end latency.
Figure 5: Limitations under imperfect inputs and incomplete observations. (a) The front output exhibits local motif distortion around the dragon head; the oblique view reveals a spurious protrusion in the underlying geometry. (b) Two front-oblique views show spurious protrusions in the upstream geometry that remain visible after texture refinement. (c) The glossy painted-wood appearance is assigned a high metallic value. Without material ground truth, this example illustrates ambiguity in single-image material inference rather than a verified estimation error. Grayscale maps range from black (0) to white (1). (d) Observed and unobserved base-color views show a loss of motif precision on the hidden side; boxes identify the enlarged details. The front output in (a) and both outputs in (d) use unlit base color; the oblique output in (a) and both outputs in (b) use PBR rendering under the same neutral lighting.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Quantity
Dimensions
Meaning
Geometry latents G
N×32
Geometry conditions on the full surface
Noisy material latents Zt
N×32
Full-surface material state at time t
Native observation latents {zso}s∈So
M×32
Encoded partial PBR materials
Observation metadata
M×2
Coverage and boundary distance
Generation stream X
N×1536
Hidden generation states
Observation stream O
M×1536
Read-only observation states
Appendix
Table 7: Representation dimensions. N=∣S∣ and M=∣So∣ vary across objects.
Figure 6: Interleaved flow transformer. One forward pass through 60 distinct blocks arranged in 30 alternating pairs. The fixed observation stream O (orange) supplies keys and values without queries or state updates. The generation stream X spans the full active latent-cell set S , with states at observed cells So (purple) and cells without any observation U=S∖So (teal); the subscript o denotes observed. An observed cell may still contain unseen surface voxels. Even-indexed blocks query and update XU ; odd-indexed blocks query and update all of X . Both block types read keys and values from all of O and X , and the output projection predicts velocities over all N=∣S∣ cells. Source labels denote tokens before normalization, time modulation, and attention projections. At inference, 25 Euler sampling steps each invoke a full transformer forward pass.
Figure 7: Examples from DTC-Tex and Complex-Tex. Ten selected assets from DTC-Tex (top) and Complex-Tex (bottom), shown in their designated reference views. Examples include dense text, fine patterns, and reflective surfaces.
Dataset
n
Alb. PSNR fg↑
Alb. PSNR full↑
SSIM full↑
LPIPS full↓
Alb. MAE fg↓
M MAE fg↓
R MAE fg↓
F2D (s) ↓
DTC-Tex
100
20.19
28.25
0.9678
0.0331
0.0872
0.0267
0.2540
1.682
Complex-Tex
110
19.44
28.19
0.9596
0.0427
0.1011
0.1167
0.2025
1.684
Appendix
Table 8: Standalone 2D material prediction. Metrics are averaged over objects in the benchmark PNG code domain; metallic and roughness MAE do not measure physical-linear error.
Figure 8: Dark bias on unseen surfaces with zero-filled observations. The calculator and dish are shown with their input images, followed by shaded and albedo renderings from the zero-filled model and SUPRA-Tex. The zero-filled results darken the unseen surfaces; SUPRA-Tex retains the corresponding green and red-and-white materials. Each comparison uses the same geometry and perspective view.
Variant
Material observations
Conditioning architecture
Residual injection
Predicted PBR
Observation latents added as layerwise residuals.
Joint-update attention
Predicted PBR
Observation states updated in selected layers; all generation states updated in every layer.
Read-only observations; alternating unobserved-cell and full-surface updates.
Oracle PBR (GT)
GT PBR
Reported separately as a material-quality reference.
Appendix
Table 9: Configurations for the six rows of Table 5 . All observation conditions are encoded with the pretrained material VAE. a , m , and r denote predicted albedo, metallic, and roughness.
Figure 9: Early reference-view learning on TeaPot during full-data training. Ours uses GT PBR observations; Pixal3D ⋆ uses the corresponding shaded RGB image. Each point is an evaluated checkpoint. Most of our model’s improvement occurs within the first 300 steps.
Figure 10: Reference-view appearance during early training. Renderings correspond to the experiment in Fig. 9 ; our model recovers the reference colors and floral patterns within 300 steps.
Figure 11: Twelve given-geometry comparisons. Novel views show the completed appearance.
Figure 12: Qualitative comparison of end-to-end single-image asset generation (Table 3 ) across four cases. All methods use the same reference image. Pixal3D ⋆ and SUPRA-Tex share predicted geometry; other baselines use their own geometry pipelines. For each object we show the input, the reference-view rendering, and a novel view. Reference views assess text and repeated patterns; novel views assess the continuity of the completed appearance.
Figure 13: Single-view generation, two-view fusion, and back-view editing. The first condition panel shows the input photograph (upper left) and generated front (lower right); other panels show designed back-view materials.
Figure 14: Additional comparisons with commercial models across eight cases. Each row shows the input image and front/back renderings from Hunyuan3D, Rodin, Tripo, Meshy, Hi3D, and SUPRA-Tex. Model versions and evaluation settings are given in Appendix C.4 .
Symbol
Domain / Dimensions
Physical or Mathematical Interpretation
Inputs and Geometry Foundation
I
RH×W×3
Input reference photograph in RGB space
M
3D Surface
Given or upstream-predicted object surface mesh
π
Projection operator
Camera intrinsic and extrinsic projection mapping 3D to 2D
Recent 3D generative models can synthesize high-quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large-scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large-scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white-mesh geometry using an off-the-shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit-scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large-scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.
Yue Han, Chong Li, Zhening Liu +5
ZGCA & ZGCI · Zhejiang University · Microsoft Research +1
Recent 3D generation models can produce accurate geometries while still struggling to reconstruct detailed textures. We propose a diffusion-based native 3D material generation model TaoTex, which faithfully recovers intricate textures through tailored strategies and improvements. First, we develop a data construction agent to create high-frequency textured 3D assets to bridge the data gap in public datasets. Training with these data significantly enhances the ability of TaoTex to recover challenging details such as text and patterns. Second, we design a multi-level feature fusion (MLFF) module to adaptively integrate local and global features of the conditional input, providing more complete texture cues for the diffusion model and thereby enhancing reconstruction fidelity. To alleviate VAE reconstruction errors, we adopt a latent-to-pixel space loss transition, further improving the pixel-level details and generation quality. Finally, we scale TaoTex to multi-view inputs by incorporating learnable viewpoint embeddings, achieving accurate and consistent material reconstruction across views. Extensive experiments demonstrate that our method significantly outperforms existing approaches in preserving texture details in both single- and multi-view settings.
Although recent advances have improved the quality of 3D texture generation, existing methods still struggle with incomplete texture coverage, cross-view inconsistency, and misalignment between geometry and texture. To address these limitations, we propose Hitem3D 2.0, a multi-view guided native 3D texture generation framework that enhances texture quality through the integration of 2D multi-view generation priors and native 3D texture representations. Hitem3D 2.0 comprises two key components: a multi-view synthesis framework and a native 3D texture generation model. The multi-view generation is built upon a pre-trained image editing backbone and incorporates plug-and-play modules that explicitly promote geometric alignment, cross-view consistency, and illumination uniformity, thereby enabling the synthesis of high-fidelity multi-view images. Conditioned on the generated views and 3D geometry, the native 3D texture generation model projects multi-view textures onto 3D surfaces while plausibly completing textures in unseen regions. Through the integration of multi-view consistency constraints with native 3D texture modeling, Hitem3D 2.0 significantly improves texture completeness, cross-view coherence, and geometric alignment. Experimental results demonstrate that Hitem3D 2.0 outperforms existing methods in terms of texture detail, fidelity, consistency, coherence, and alignment.