3D Multi-View Stylization with Pose-Free Correspondences Matching for Robust 3D Geometry Preservation
Organizations: SCHOOL OF COMPUTATION, INFORMATION AND TECHNOLOGY - INFORMATICS TECHNISCHE UNIVERSITÄT MÜNCHEN
Abstract
Artistic style transfer is well studied for images and videos, but extending it to multi-view 3D scenes remains difficult because stylization can disrupt correspondences needed by geometry-aware pipelines. Independent per-view stylization often causes texture drift, warped edges, and inconsistent shading, degrading SLAM, depth prediction, and multi-view reconstruction. This thesis addresses multi-view stylization that remains usable for downstream 3D tasks without assuming camera poses or an explicit 3D representation during training. We introduce a feed-forward stylization network trained with per-scene test-time optimization under a composite objective coupling appearance transfer with geometry preservation. Stylization is driven by an AdaIN-inspired loss from a frozen VGG-19 encoder, matching channel-wise moments to a style image. To stabilize structure across viewpoints, we propose a correspondence-based consistency loss using SuperPoint and SuperGlue, constraining descriptors from a stylized anchor view to remain consistent with matched descriptors from the original multi-view set. We also impose a depth-preservation loss using MiDaS/DPT and use global color alignment to reduce depth-model domain shift. A staged weight schedule introduces geometry and depth constraints. We evaluate on Tanks and Temples and Mip-NeRF 360 using image and reconstruction metrics. Style adherence and structure retention are measured by Color Histogram Distance (CHD) and Structure Distance (DSD). For 3D consistency, we use monocular DROID-SLAM trajectories and symmetric Chamfer distance on back-projected point clouds. Across ablations, correspondence and depth regularization reduce structural distortion and improve SLAM stability and reconstructed geometry; on scenes with MuVieCAST baselines, our method yields stronger trajectory and point-cloud consistency while maintaining competitive stylization.
Figures & tables
| Method | Horse | Panther | Lighthouse | Francis | Train |
| greatwave | mosaic | abstract | abstract2 | starry | |
| CHD / DSD | CHD / DSD | CHD / DSD | CHD / DSD | CHD / DSD | |
| MuVieCAST | 0.3042 / 10.2075 | 0.1238 / 13.1499 | 0.3815 / 19.3052 | 0.1623 / 22.2035 | 0.1538 / 10.4620 |
| Ours-Res | 0.3218 / 8.6087 | 0.1747 / 9.0089 | 0.3990 / 15.7968 | 0.2404 / 17.4681 | 0.1805 / 8.0876 |
| Ours-U-Net | 0.3309 / 10.6319 | 0.1543 / 10.6255 | 0.3350 / 16.7642 | 0.1471 / 19.7963 | 0.1625 / 9.3424 |
| Method | Horse | Panther | Lighthouse | Francis | Train |
| greatwave | mosaic | abstract | abstract2 | starry | |
| ATE/RTE/ChD | ATE/RTE/ChD | ATE/RTE/ChD | ATE/RTE/ChD | ATE/RTE/ChD | |
| MuVieCAST | 0.058/ 0.469/ 126.91 | 0.005/ 1.159/ 10.31 | 1.081/ 81.156/ 588.64 | 0.752/ 27.273/ 387.54 | 0.810/ 10.017/ 287.34 |
| Ours-Res | 0.051/ 0.401 / 67.27 | 0.004/ 0.896/ 7.72 | 0.917 / 48.129 / 173.38 | 0.050 / 0.697 / 201.73 | 0.433 / 4.95 / 26.23 |
| Ours-U-Net | 0.049 / 0.423/ 89.54 | 0.003 / 0.633 / 9.29 | 1.039/ 78.886/ 400.32 | 0.061/ 1.168/ 136.66 | 0.711/ 8.63/ 46.85 |
| Method | Train style 19 CHD/DSD | Train starry CHD/DSD | Truck style 19 CHD/DSD | Truck starry CHD/DSD |
| Ours (Base) | 0.3268 /11.8940 | 0.1620 /9.2920 | 0.3060 /9.2320 | 0.2121 /8.8169 |
| Ours (Base + ) | 0.3492/9.4581 | 0.1767/8.6051 | 0.3177/8.9358 | 0.2347/8.5099 |
| Ours (Base + ) | 0.3521/8.4576 | 0.1843/8.4328 | 0.3409/8.8692 | 0.2464/7.8745 |
| Ours (Base + + ) | 0.3568/ 8.4146 | 0.1895/ 8.0876 | 0.3488/ 8.5543 | 0.2581/ 7.3641 |
| Method | Train style 19 ATE/RTE/ChD | Train starry ATE/RTE/ChD | Truck style 19 ATE/RTE/ChD | Truck starry ATE/RTE/ChD |
| Ours (Base) | 0.824/26.418/580.51 | 0.854/23.904/103.82 | 0.0375/1.065/16.20 | 0.055/1.823/9.71 |
| Ours (Base + ) | 0.809/20.273/70.14 | 0.755/22.045/66.75 | 0.029/0.625/15.25 | 0.021/0.971/8.33 |
| Ours (Base + ) | 0.526/16.697/56.71 | 0.536/17.233/44.63 | 0.027/0.609/14.21 | 0.019/0.881/7.56 |
| Ours (Base + + ) | 0.018 / 0.870 / 48.35 | 0.433 / 4.95 / 26.23 | 0.022 / 0.587 / 13.19 | 0.014 / 0.724 / 6.37 |
| Method | Garden style 19 CHD/DSD | Garden starry CHD/DSD | Counter style 19 CHD/DSD | Counter starry CHD/DSD | Counter style_2 CHD/DSD |
| Ours (Base) | 0.3215 /13.8940 | 0.5205 /11.2367 | 0.3903/8.2029 | 0.2530/9.0322 | 0.1867 /9.8447 |
| Ours (Base + ) | 0.3287/13.6059 | 0.5353/10.1142 | 0.5464/9.0010 | 0.2342 /9.0809 | 0.4229/9.1835 |
| Ours (Base + ) | 0.3351/12.7059 | 0.5424/9.8853 | 0.3872/ 7.8421 | 0.2572/9.3280 | 0.2516/8.2836 |
| Ours (Base + + ) | 0.3370/ 12.4668 | 0.5434/ 9.0919 | 0.3750 /7.9625 | 0.2564/ 8.7614 | 0.2771/ 7.3569 |
| Method | Garden style 19 ATE/RTE/ChD | Garden starry ATE/RTE/ChD | Counter style 19 ATE/RTE/ChD | Counter starry ATE/RTE/ChD | Counter style_2 ATE/RTE/ChD |
| Ours (Base) | 0.106/2.225/1.34 | 0.125/2.625/0.74 | 0.063/3.058/12.28 | 0.024/1.721/9.70 | 0.146/7.468/20.33 |
| Ours (Base + ) | 0.083/1.969/1.02 | 0.122/1.146/0.69 | 0.033/1.489/9.73 | 0.023/1.684/8.66 | 0.036/1.616/12.33 |
| Ours (Base + ) | 0.008/0.247/0.94 | 0.009/0.747/0.62 | 0.013/0.742/8.97 | 0.019/1.664/8.41 | 0.022/1.414/11.33 |
| Ours (Base + + ) | 0.001 / 0.163 / 0.73 | 0.006 / 0.546 / 0.59 | 0.011 / 0.655 / 8.40 | 0.014 / 1.179 / 7.48 | 0.016 / 0.987 / 9.75 |
| Method | style 19 CHD/DSD | starry CHD/DSD | style 15 CHD/DSD | style 41 CHD/DSD |
| Ours (Base) | 0.2495 /17.6369 | 0.2218 /12.7811 | 0.1875 /18.2556 | 0.2951/15.9439 |
| Ours (Base + ) | 0.3246/18.2966 | 0.2230/13.3792 | 0.2017/16.1685 | 0.2723/14.5357 |
| Ours (Base + ) | 0.3250/16.1209 | 0.2394/11.6278 | 0.2251/15.9774 | 0.2664/13.9665 |
| Ours (Base + + ) | 0.3384/ 15.6645 | 0.2501/ 10.4830 | 0.2423/ 14.9774 | 0.2468 / 13.0096 |
| Method | style 19 ATE/RTE/ChD | starry ATE/RTE/ChD | style 15 ATE/RTE/ChD | style 41 ATE/RTE/ChD |
| Ours (Base) | 0.288/11.986/36.00 | 0.347/29.497/18.21 | 0.111/4.997/20.29 | 0.139/6.961/15.40 |
| Ours (Base + ) | 0.173/7.856/28.92 | 0.149/6.483/17.80 | 0.100/4.702/15.33 | 0.133/5.291/9.44 |
| Ours (Base + ) | 0.1381/5.970/14.20 | 0.127/6.190/13.75 | 0.095/3.841/12.89 | 0.121/4.991/8.99 |
| Ours (Base + + ) | 0.072 / 3.502 / 12.30 | 0.088 / 3.119 / 12.36 | 0.077 / 3.058 / 12.48 | 0.111 / 4.554 / 7.44 |
Image Style Transfer
Image style transfer changes an image’s visual appearance to match an artistic, photographic, or other reference while preserving its content.
See all 61 papers