Organizations: Nanyang Technological University, Singapore, Singapore. · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China. · United Arab Emirates University, Abu Dhabi, United Arab Emirates. · Hefei University of Technology, Hefei, China. · Shenzhen University, Shenzhen, China. · University of Science and Technology of China, Hefei, China. · Cardiff University, Cardiff, United Kingdom.
Novel view synthesis (NVS) in degraded scenes aims to reconstruct 3D scenes from multi-view observations affected by various degradations, and to generate clear and photorealistic images at unseen viewpoints. Despite recent advances, existing methods often rely on degradation-specific models, limiting their generalization across diverse extreme environments. Generative models provide rich visual priors, but their outputs often suffer from pixel misalignment and color deviations, which can propagate into the reconstructed 3D scene and compromise reconstruction quality. Furthermore, the high computational and memory costs of 3D reconstruction hinder the practical application of NVS in extreme environments. To this end, we propose MLLM-GS, a multimodal large language model (MLLM) prior-guided framework for fast NVS in extreme environments. MLLM-GS consists of two branches: MLLM prior-guided view enhancement and fast 3D scene reconstruction. The former leverages the rich visual priors of a frozen MLLM to enhance views under different severe degradations and introduces a lightweight correction network to reduce pixel shifts and color distortions in MLLM outputs. With supervision from only a few paired clean views, this branch produces reliable enhanced views for all degraded observations, providing pseudo-GT views for subsequent 3D reconstruction. The latter improves 3DGS efficiency by optimizing rasterization, Gaussian memory layout, and parameter updates, reducing redundant computation, memory-access overhead, and optimization costs. Extensive experiments on multiple degraded scenes demonstrate that MLLM-GS effectively handles various severe degradations and achieves high-quality NVS with significantly lower reconstruction time and memory consumption. Project Page: https://wzlwlm13.github.io/MLLM-GS
Figures & tables
Figure 1: Illustration of three different pipelines for NVS under degraded conditions. (a) Vanilla reconstruction, (b) prior‑augmented reconstruction, and (c) the proposed MLLM prior‑guided pipeline. (d) Efficiency comparison with previous representative methods.
Figure 2: Overall pipeline of the proposed MLLM-GS framework . It consists of two branches: MLLM prior-guided view enhancement and fast 3D scene reconstruction. The former combines a frozen MLLM with a lightweight correction network trained using a few paired clean views to generate reliable pseudo-GT views for subsequent 3D reconstruction. The latter accelerates 3DGS optimization through efficient rasterization, memory layout, and parameter updates, enabling fast and high-quality NVS.
Figure 3: The view statistics of smoke and low-light scenes from the RealX3D benchmark.
Method
Metrics
Akikaze
Futaba
Hinoki
Koharu
Midori
Natsume
Shirohana
Tsubaki
Avg.
3DGS ( Kerbl et al., 2023 )
PSNR ↑
10.80
13.48
7.47
11.87
13.84
9.74
8.91
15.41
11.44
SSIM ↑
0.548
0.695
0.315
0.611
0.700
0.604
0.419
0.739
0.579
LPIPS ↓
0.659
0.619
0.683
0.573
0.548
0.633
0.749
0.583
0.631
SeaSplat ( Yang et al., 2025a )
PSNR ↑
11.21
8.11
13.23
9.98
5.04
8.49
9.21
8.95
9.28
SSIM ↑
0.497
0.497
0.422
0.500
0.330
0.463
0.347
0.507
0.445
LPIPS ↓
0.863
0.727
0.677
0.619
0.850
0.863
0.847
0.637
0.760
Table 1: Quantitative evaluation of 3D reconstruction methods on the smoke scene of the RealX3D benchmark. Best results are highlighted in dark gray bold, and second-best in light gray.
Figure 4: Qualitative comparison of NVS results under smoke scenes.
Method
Metrics
BlueHawaii
Chocolate
Cupcake
GearWorks
Laboratory
MilkCookie
Popcorn
Sculpture
Ujikintoki
Avg.
3DGS ( Kerbl et al., 2023 )
PSNR ↑
7.49
7.89
4.42
6.90
7.59
6.15
7.46
5.85
6.23
6.66
SSIM ↑
0.055
0.084
0.064
0.081
0.041
0.053
0.062
0.019
0.062
0.058
LPIPS ↓
0.671
0.646
0.625
0.658
0.610
0.653
0.650
0.733
0.683
0.659
LITA-GS ( Zhou et al., 2025 )
PSNR ↑
17.30
17.94
13.07
10.90
17.56
12.74
18.97
12.84
18.82
15.57
SSIM ↑
0.624
0.541
0.643
0.344
0.597
0.573
0.557
0.211
0.656
0.527
LPIPS ↓
0.546
0.557
0.326
0.521
0.435
0.478
0.448
0.715
0.497
0.503
Table 2: Quantitative evaluation of 3D reconstruction methods on the low-light scene of the RealX3D benchmark. Best results are highlighted in dark gray bold, and second-best in light gray.
Figure 5: User study for smoke and low‑light scenes.
Figure 6: Qualitative comparison of NVS results under low-light scenes.
Method
Training ↓
#Gs ↓
VRAM ↓
Akikaze (Smoke)
3DGS
14m44s
2.26M
8.15GiB
SeaSplat
71m59s
0.63M
7.06GiB
WaterSplatting
19m30s
0.03M
5.06GiB
I2-NeRF
64m05s
N/A
5.13GiB
SeaThruNeRF
90m59s
N/A
30.77GiB
Table 3: Efficiency comparison of 3D reconstruction methods. The training steps for all methods are 30,000. “N/A” indicates that the corresponding metric is not applicable to the method.
Figure 7: Efficiency analysis on a smoke scene (Akikaze) and a low-light scene (BlueHawaii).
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Wan2.7-Image
22.13
0.714
0.381
Qwen-Image-3.0
22.29
0.713
0.431
Seedream-5.0-Lite
22.21
0.675
0.421
GPT-Image-2
19.06
0.644
0.482
Qwen-Image-Edit
18.09
0.621
0.630
Nano Banana Pro (Ours)
27.44
0.761
0.234
Table 4: Ablation of different MLLM priors on Akikaze under smoke degradation.
Figure 8: Ablation study on the correction network.
Method
Training ↓
#Gs ↓
VRAM ↓
3DGS
14m44s
2.26M
8.15GiB
+ Rasterization
11m27s
0.49M
2.57GiB
+ Memory Layout
10m42s
0.47M
2.12GiB
+ Parameter Updates
10m06s
0.46M
1.98GiB
Table 5: Ablation of the proposed components on Akikaze under smoke degradation.
Figure 9: Ablation on the number of GT views.
Figure 10: Qualitative comparison of NVS results under strong defocus blur scenes.
Figure 11: Qualitative comparison of NVS results under strong motion blur scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Akikaze (Smoke)
+Faster-GS
24.85
0.710
0.340
MLLM-GS (Ours)
27.44
0.761
0.234
BlueHawaii (Low-Light)
+Faster-GS
20.03
0.643
0.397
MLLM-GS (Ours)
24.33
0.776
0.379
Table 6: Effectiveness of fast 3D scene reconstruction branch.
Figure 12: Qualitative comparison of NVS results under varying exposure scenes.
We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos. BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone, aiming to mitigate artifacts and recover view-consistent details at inference time. Conventional methods have developed similar diffusion-based solutions to address these challenges of novel view synthesis. Despite significant improvements, these methods typically rely on off-the-shelf pretrained diffusion priors and fine-tune only the UNet module while keeping other components frozen, which still leads to inconsistent details and artifacts even when incorporating geometry-aware regularizations like depth or semantic conditions. To address this, we investigate the latent space of the diffusion model and introduce two components: (1) temporal equivariance regularization and (2) vision foundation model-aligned representation, both applied to the variational autoencoder (VAE) module within the SVD pipeline. BetterScene integrates a feed-forward 3D Gaussian Splatting (3DGS) model to render features as inputs for the SVD enhancer and generate continuous, artifact-free, consistent novel views. We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
Yuci Han, Charles Toth, John E. Anderson +2
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
Recent work has shown that neural networks can perform 3D tasks such as Novel View Synthesis (NVS) without explicit 3D reconstruction. Even so, we argue that strong 3D inductive biases are still helpful in the design of such networks. We show this point by introducing LagerNVS, an encoder-decoder neural network for NVS that builds on `3D-aware' latent features. The encoder is initialized from a 3D reconstruction network pre-trained using explicit 3D supervision. This is paired with a lightweight decoder, and trained end-to-end with photometric losses. LagerNVS achieves state-of-the-art deterministic feed-forward Novel View Synthesis (including 31.4 PSNR on Re10k), with and without known cameras, renders in real time, generalizes to in-the-wild data, and can be paired with a diffusion decoder for generative extrapolation.
Stanislaw Szymanowicz, Minghao Chen, Jianyuan Wang +2
Visual Geometry Group, University of Oxford · Meta AI
We present NimbusGS, a unified framework for reconstructing high-quality 3D scenes from degraded multi-view inputs captured under diverse and mixed adverse weather conditions. Unlike existing methods that target specific weather types, NimbusGS addresses the broader challenge of generalization by modeling the dual nature of weather: a continuous, view-consistent medium that attenuates light, and dynamic, view-dependent particles that cause scattering and occlusion. To capture this structure, we decompose degradations into a global transmission field and per-view particulate residuals. The transmission field represents static atmospheric effects shared across views, while the residuals model transient disturbances unique to each input. To enable stable geometry learning under severe visibility degradation, we introduce a geometry-guided gradient scaling mechanism that mitigates gradient imbalance during the self-supervised optimization of 3D Gaussian representations. This physically grounded formulation allows NimbusGS to disentangle complex degradations while preserving scene structure, yielding superior geometry reconstruction and outperforming task-specific methods across diverse and challenging weather conditions. Code is available at https://github.com/lyy-ovo/NimbusGS.
Yanying Li, Jinyang Li, Shengfeng He +3
School of Computer Science and Technology, Ocean University of China · Singapore Management University · Harbin Institute of Technology (Shenzhen) +1