cs.CVNov 5, 2026
SaveMIKU: Material Invariance and Knowledge Unification for PBR Generation
Organizations: Nanjing University, China · DreamTech, China · Peking University, China · The University of Hong Kong, Hong Kong
Abstract
High-quality material generation for physically based rendering (PBR) remains challenging because datasets and synthesis pipelines lack standardized representations. Differences in material definitions, rendering conventions, and parameterizations lead to unstable and physically implausible results. We propose MIKU, a unified framework for standardized PBR asset representation and physically consistent material generation across diverse 3D content domains. Its core is a unified processing pipeline that standardizes the workflow from data preprocessing to final texture synthesis, maintaining consistent geometry and texture representations. A normalization scheme harmonizes heterogeneous material parameterizations in a common representation space, enabling robust cross-dataset training and inference. Building on this formulation, we train a novel physically aware diffusion model for PBR texture generation within a two-stage multi-view synthesis framework. First, geometry-aware albedo generation synthesizes multi-view albedo maps under explicit geometric guidance to ensure cross-view appearance consistency. Second, metallic-roughness generation uses the diffusion model to predict physically plausible roughness and metallic maps conditioned on the generated albedo and unified representation. Trained end-to-end on standardized data, the model learns to generate materials that respect category-level material priors and semantic part boundaries. By decoupling representation standardization from appearance synthesis, MIKU substantially improves material generation stability and physical consistency. Extensive experiments demonstrate state-of-the-art texture quality and semantic consistency across object parts, advancing automated material generation for 3D content creation.
Figures & tables
Figure 1. The entire scene is assembled from AIGC-generated meshes, where both the surface textures and PBR material maps are consistently synthesized by our method. A scene assembled from generated 3D meshes, showing consistent textures and physically based material maps under varied illumination.
Figure 2. Illustration of the discrepancy attenuation in image-supervised PBR learning. Even with fixed geometry, albedo, camera view, and illumination, substantially different MR conditions may result in only minor shading differences after rendering.
Figure 3. Visualization of our color-preserving illumination strategy. Compared with chromatic illumination, which introduces color casts and appearance ambiguity, achromatic illumination preserves intrinsic albedo colors while providing shading cues. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 4. Overview of Our Multi-Stage Pipeline for High-Quality PBR Material Dataset Construction and Refinement. This figure illustrates our end-to-end pipeline for constructing a high-quality PBR material dataset. We first collect multi-source data from the public TexVerse dataset and private asset collections. We then define physically based material properties and render multi-view observations under controlled settings. Next, we apply albedo extraction and PBR-aware filtering to remove low-quality samples. Vision-Language Models (VLMs) with SAM3 are used for material recognition and semantic segmentation to enable fine-grained decomposition. Finally, we enhance texture details via affine transformations and high-frequency boosting, followed by human validation to ensure data quality. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
| Material Class | Metallic | Roughness |
| Skin | 0.0 | 0.40–0.50 |
| Eye | 0.0 | 0.02–0.12 |
| Hair | 0.0 | 0.35–0.45 |
| Metal | 1.0 | 0.00–0.10 |
| Teeth | 0.0 | 0.28–0.34 |
| Lips | 0.0 | 0.30–0.36 |
Table 1. Metallic–roughness (MR) definitions for the 20 predefined semantic material categories and the four optional fine-grained metal subclasses under our unified PBR specification. Each category is associated with a representative MR range that serves as a consistent material prior during data standardization.
Figure 5. Overview of our conditional diffusion framework for multi-view albedo generation. Given a noisy input, a reference image, and geometric guidance, the model progressively denoises the input to synthesize consistent multi-view albedo maps. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 6. Overview of our deterministic diffusion framework for multi-view PBR material estimation. Given a packed albedo input and a shaded reference image , we formulate a conditional diffusion process to predict the target material representation . A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 7. Qualitative comparison of texture and PBR material generation across different methods. Each example is arranged in two rows: the first row shows the rendered results under a fixed illumination setting, while the second row presents the predicted material maps, including the MR map, i.e., metallic and roughness channels, and the generated albedo texture. Compared with existing methods, our approach produces sharper and more detailed albedo textures, while also yielding cleaner, more spatially coherent, and semantically consistent MR predictions across diverse samples.
Figure 8. Visualization results of PBR materials and texture maps generated for two given meshes under image guidance. The rendered results under diverse lighting environments demonstrate that the model effectively adapts to varying illumination conditions. In particular, metallic regions exhibit physically plausible reflections, accurately capturing environmental lighting and producing realistic appearance. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 9. Qualitative comparison of texture generation using geometry-based classifier-free guidance (CFG) versus alternative CFG strategies. The results show that geometry-guided CFG produces sharper textures that more closely align with the conditioning image, leading to improved structural fidelity and overall visual consistency. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 10. User-study results comparing perceptual quality and reference-image consistency across different methods for multi-view image generation. The bar chart reports the average scores for each method, with higher values indicating better perceived quality and consistency with the reference image. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 11. Visual comparison between models trained on original artist-defined PBR data and our unified normalized PBR data. The first column shows the reference image, the second column presents results from the model trained on the original artist-defined PBR data, and the third column shows results from the model trained on our unified normalized PBR data. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
| Method | FID | CMMD | LPIPS | CLIP-I | CLIP-FID | Inference Time (s) | Albedo Resolution | MR Resolution |
|---|---|---|---|---|---|---|---|---|
| MvAdapter | 165.49 | 2.2235 | 0.2383 | 0.9097 | 22.911 | 36 | 768 | None |
| UniTex | 161.27 | 2.1898 | 0.2174 | 0.9052 | 22.623 | 220 | 512 | None |
| Hunyuan3d-2.1 | 154.88 | 2.2625 | 0.2167 | 0.9132 | 20.132 | 48 | 768 | 768 |
| LumiTex | 144.61 | 1.8677 | 0.1992 | 0.9215 | 16.182 | 110+110 | 512 | 512 |
| Ours | 121.25 | 1.4652 | 0.1612 | 0.9411 | 13.358 | 40+12 | 1024 | 512 |
Table 2. Quantitative Comparison with State-of-the-Art Methods on PBR Texture Generation Quality and Efficiency.
| Method | Setting | FID | CMMD | LPIPS | CLIP-I | CLIP-FID |
|---|---|---|---|---|---|---|
| Ours | w/o CFG | 128.55 | 1.5439 | 0.1688 | 0.9342 | 15.172 |
| w/ CFG | 121.25 | 1.4652 | 0.1612 | 0.9411 | 13.358 |
Table 3. Ablation study on the effect of classifier-free guidance (CFG) in our framework. Enabling CFG significantly improves generation quality and semantic consistency across all evaluation metrics.
Figure 12. Visualization results of arbitrary image-guided texture generation and mesh stylization. The leftmost images show the reference guidance images, while the rendered results on the right are obtained by generating textures conditioned on each reference image and applying them back onto the same underlying mesh. Despite the substantial appearance differences among the guidance images, our method produces coherent and harmonious stylized textures that are well integrated with the shared geometry, demonstrating strong image-guided stylization capability.
Figure 13. User study on perceptual PBR material quality. Left: Pairwise preferences between our model trained on the curated, standardized annotations and that trained on the original annotations, where win, tie, and loss are defined from the perspective of the former. Right: Mean five-point ratings of Hunyuan3D-V2.1 under the pretrained setting and after fine-tuning on either the raw collected data or our curated data. A pipeline illustrating deterministic diffusion-based multi-view PBR material estimation with packed inputs, conditional guidance, and LoRA fine-tuning.
Figure 14. Visualization of text and logo fidelity under reference guidance. For each example, the leftmost image denotes the reference input containing distinctive textual or logo-like patterns. The two rendered results on the right show the textured mesh after back-projecting the generated maps, where the left rendering presents the frontal view and the right rendering presents the back view. The results demonstrate that our method can preserve fine-grained text and logo structures from the reference image while maintaining coherent appearance across different viewpoints.
Figure 15. Additional qualitative results on diverse inputs. Our method demonstrates strong generalization across a wide range of mesh geometries, styles, and input variations, consistently producing high-quality textures along with semantically accurate and physically plausible PBR materials.
| Training Data | PBR Consistency MSE |
|---|---|
| Artist-defined Materials | 0.0513 |
| Standardized Materials | 0.0258 |
Table 4. Ablation study on the effect of PBR material standardization. The standardized material specification improves prediction consistency under semantic-preserving appearance variations.