Pygmalion Effect in Vision: Image-to-Clay Translation for Reflective Geometry Reconstruction
Organizations: NAVER AI Lab · KAIST · SNU AIIS
Abstract
Understanding reflection remains a long-standing challenge in 3D reconstruction due to the entanglement of appearance and geometry under view-dependent reflections. In this work, we present the Pygmalion Effect in Vision, a novel framework that metaphorically "sculpts" reflective objects into clay-like forms through image-to-clay translation. We then introduce a dual-branch design in which a BRDF-based reflective branch and a clay-guided branch share the same Gaussian geometry but operate through independent rendering paths. The clay-guided branch, supervised by the synthesized clay images, provides reflection-free geometric guidance that complements the photometric supervision of the BRDF branch. Experiments on both synthetic and real datasets show consistent improvements in both geometry and photometric metrics. Beyond technical gains, our framework reveals that seeing by unshining, translating radiance into neutrality, can serve as a powerful inductive bias for reflective object geometry learning.
Figures & tables
| Method | Angel | Bell | Cat | Horse | Luyu | Potion | Tbell | Teapot | Average |
| GShader Jiang et al. (2024) | 0.0060 | 0.0078 | 0.0175 | 0.0072 | 0.0101 | 0.0382 | 0.0308 | 0.0178 | 0.0169 |
| GS-IR Liang et al. (2024) | 0.0110 | 0.1097 | 0.0566 | 0.0149 | 0.0224 | 0.0593 | 0.0989 | 0.0693 | 0.0553 |
| R3DG Gao et al. (2024) | 0.0090 | 0.0403 | 0.0326 | 0.0117 | 0.0151 | 0.0380 | 0.0472 | 0.0488 | 0.0303 |
| Ref-GS Zhang et al. (2025b) | 0.0049 | 0.0084 | 0.0170 | 0.0060 | 0.0098 | 0.0090 | 0.0064 | 0.0084 | 0.0087 |
| GS-2DGS Tong et al. (2025) | 0.0061 | 0.0037 | 0.0074 | 0.0050 | 0.0076 | 0.0088 | 0.0078 | 0.0083 | 0.0068 |
| RGS Yao et al. (2025) | 0.0050 | 0.0071 | 0.0161 | 0.0069 | 0.0093 | 0.0081 | 0.0065 | 0.0090 | 0.0085 |
| Method | Angel | Bell | Cat | Horse | Luyu | Potion | Tbell | Teapot | Average |
| RGS Yao et al. (2025) | 0.0050 | 0.0071 | 0.0161 | 0.0069 | 0.0093 | 0.0081 | 0.0065 | 0.0090 | 0.0085 |
| RGS + Clay | 0.0032 | 0.0067 | 0.0142 | 0.0042 | 0.0066 | 0.0077 | 0.0061 | 0.0110 | 0.0074 |
| MaterialRefGS Zhang et al. (2025a) | 0.0038 | 0.0087 | 0.0130 | 0.0047 | 0.0061 | 0.0067 | 0.0049 | 0.0059 | 0.0067 |
| MaterialRefGS + Clay | 0.0035 | 0.0069 | 0.0057 | 0.0047 | 0.0057 | 0.0064 | 0.0046 | 0.0057 | 0.0054 |
| MaterialRefGS + Clay - Normal | 0.0035 | 0.0047 | 0.0087 | 0.0044 | 0.0063 | 0.0064 | 0.0034 | 0.0060 | 0.0054 |
| Datasets | Shiny Blender ( Verbin et al., 2024a ) | Glossy Synthetic ( Liu et al., 2023 ) | Ref-Real ( Verbin et al., 2024a ) | Glossy-Real ( Liu et al., 2023 ) | NeRF-casting ( Verbin et al., 2024b ) | ||||||||||
| Metric | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |
| 2DGS | 29.58 | 0.946 | 0.084 | 26.07 | 0.918 | 0.088 | 24.15 | 0.661 | 0.292 | 21.20 | 0.804 | 0.243 | 24.66 | 0.822 | 0.225 |
| Ref-GS | 34.80 | 0.973 | 0.056 | 30.16 | 0.956 | 0.055 | 24.44 | 0.682 | 0.225 | 22.33 | 0.814 | 0.228 | 28.09 | 0.855 | 0.185 |
| RGS | 35.04 | 0.973 | 0.056 | 30.37 | 0.958 | 0.049 | 24.61 | 0.685 | 0.252 | 22.00 | 0.802 | 0.254 | 28.35 | 0.849 | 0.211 |
| MaterialRefGS | 34.68 | 0.975 | 0.054 | 30.60 | 0.960 | 0.048 | 24.73 | 0.698 | 0.183 | 22.65 | 0.818 | 0.186 | 28.07 | 0.845 | 0.148 |
| Ours | 35.12 | 0.976 | 0.053 | 30.68 | 0.961 | 0.047 | 24.74 | 0.699 | 0.183 | 23.04 | 0.824 | 0.184 | 28.69 | 0.853 | 0.137 |
| Translation consistency | Downstream reconstruction | ||||||
| Method | Cross-view Var | Global Var | Ratio | MAE | PSNR | SSIM | LPIPS |
| Qwen-Edit Wu et al. (2025) | 0.0214 | 0.0555 | 0.3875 | 1.53 | 34.60 | 0.9732 | 0.057 |
| LTX-2 (sorted) | 0.0181 | 0.0455 | 0.3966 | 1.39 | 34.78 | 0.9749 | 0.054 |
| LTX-2 (no anchor) | 0.0157 | 0.0398 | 0.4299 | 1.33 | 34.94 | 0.9750 | 0.054 |
| LTX-2 ( ours ) | 0.0163 | 0.0436 | 0.3661 | 1.27 | 35.12 | 0.9760 | 0.053 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Time |
| LTX-2 model loading (one-time) | 1m 16s |
| Clay translation of all training views | 4m 59s |
| MaterialRefGS optimization ( iterations) | 2h 15m 7s |
| Total | 2h 21m 22s |
| Datasets | Shiny Blender ( Verbin et al., 2024a ) | Glossy Synthetic ( Liu et al., 2023 ) | ||||||||||||||
| Scenes | Ball | Car | Coffee | Helmet | Teapot | Toaster | Avg. | Angel | Bell | Cat | Horse | Luyu | Potion | Tbell | Teapot | Avg. |
| PSNR | ||||||||||||||||
| 2DGS | 25.97 | 26.38 | 32.31 | 27.42 | 44.97 | 20.42 | 29.58 | 26.95 | 24.79 | 30.65 | 25.18 | 26.89 | 29.50 | 23.28 | 21.29 | 26.07 |
| GShader | 30.99 | 27.96 | 32.39 | 28.32 | 45.86 | 26.28 | 31.97 | 25.08 | 28.07 | 31.81 | 26.56 | 27.18 | 30.09 | 24.48 | 23.58 | 27.11 |
| R3DG | 23.64 | 25.92 | 30.10 | 25.01 | 43.15 | 18.80 | 27.77 | 24.90 | 23.51 | 27.59 | 23.37 | 24.68 | 27.29 | 21.25 | 20.47 | 24.13 |
| 3DGS-DR | 33.43 | 30.48 | 34.53 | 31.44 | 47.04 | 26.76 | 33.95 | 29.07 | 30.60 | 32.59 | 26.17 | 28.96 | 32.65 | 29.03 | 25.77 | 29.36 |
| Datasets | Ref-Real ( Verbin et al., 2024a ) | Glossy-Real ( Liu et al., 2023 ) | NeRF-casting ( Verbin et al., 2024b ) | ||||||||||||
| Scenes | Garden | Sedan | Toycar | Avg. | Bear | Bunny | Coral | Maneki | Vase | Avg. | Compact | Grinder | Hatchback | Toaster | Avg. |
| PSNR | |||||||||||||||
| 2DGS | 22.53 | 26.23 | 23.70 | 24.15 | 19.47 | 23.59 | 18.94 | 22.37 | 21.61 | 21.20 | 23.38 | 25.33 | 23.96 | 25.99 | 24.66 |
| Ref-GS | 22.48 | 26.63 | 24.20 | 24.44 | 17.17 | 26.02 | 19.09 | 24.65 | 24.70 | 22.33 | 28.19 | 30.46 | 25.27 | 28.45 | 28.09 |
| RGS | 22.97 | 26.60 | 24.27 | 24.61 | 19.89 | 24.56 | 18.83 | 23.70 | 23.01 | 22.00 | 28.32 | 30.67 | 26.12 | 28.28 | 28.35 |
| MaterialRefGS | 23.29 | 26.04 | 24.86 | 24.73 | 20.38 | 25.59 | 19.32 | 24.26 | 23.70 | 22.65 | 27.47 | 30.89 | 25.77 | 28.16 | 28.07 |
| Car | Ball | Helmet | Teapot | Toaster | Coffee | Average | |
| GS-IR | 28.31 | 25.79 | 25.58 | 15.35 | 33.51 | 15.38 | 23.99 |
| GShader | 14.05 | 7.03 | 9.33 | 7.17 | 13.08 | 14.93 | 10.93 |
| 3DGS-DR | 2.32 | 0.85 | 1.67 | 0.53 | 6.99 | 2.21 | 2.43 |
| Ref-GS | 2.02 | 1.05 | 1.99 | 0.69 | 3.92 | 3.61 | 2.21 |
| RGS | 1.77 | 1.17 | 2.07 | 0.70 | 3.90 | 3.07 | 2.11 |
| MaterialRefGS | 1.55 | 0.65 | 1.18 | 0.43 | 2.51 | 1.72 | 1.34 |
| CD | PSNR | SSIM | LPIPS | |
| Baseline (MaterialRefGS) | 0.0067 | 30.60 | 0.960 | 0.048 |
| Ours w/ S2Dnet diffuse images | 0.0229 | 28.62 | 0.943 | 0.065 |
| Ours w/ our clay images | 0.0054 | 30.68 | 0.961 | 0.047 |
| Routed parameters | Lock duration | MAE | PSNR | |||
| Position | Opacity | Rotation | Weak RGB | |||
| - | - | - | - | 10K | 1.32 | 35.01 |
| ✓ | - | - | - | 10K | 1.29 | 35.04 |
| ✓ | ✓ | - | - | 10K | 1.30 | 35.05 |
| ✓ | ✓ | ✓ | - | 10K | 1.31 | 34.94 |
| ✓ | ✓ | - | ✓ | 5K | 1.33 | 34.89 |
3D Reconstruction
3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.
See all 1,013 papers