IGSW: Multi-Weather Urban Scene Reconstruction with Intrinsic-Decoupled Gaussian Splatting

Zhongquan Wang; Long Cao; Peijia Xie; Senlei Ma; Jie Li; Guofa Li

cs.CVNov 3, 2026

IGSW: Multi-Weather Urban Scene Reconstruction with Intrinsic-Decoupled Gaussian Splatting

Authors: Zhongquan Wang, Long Cao, Peijia Xie, Senlei Ma, Jie Li, Guofa Li

Organizations: College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing 400044, China

Abstract

High-fidelity weather editing of 4D scenes can support end-to-end training and closed-loop simulation for autonomous driving. Combining generative image editing with dynamic Gaussian representations enables multi-weather scene reconstruction, but weather-edited images often contain structural distortions. Directly fitting their appearance may propagate errors from teacher images into the reconstructed 4D scene, compromising the original scene structure and temporal continuity. To address these challenges, we propose IGSW, a two-stage framework that combines intrinsic-decoupled 4D Gaussian scene modeling with structure-preserving weather refinement. In Stage I, background Gaussian appearance is decomposed into weather-shared albedo and normals and weather-specific irradiance and residuals. Weather teacher images generated by a diffusion-based image editing model and intrinsic maps estimated by a diffusion-based inverse renderer provide feature-level soft supervision for stable structure and illumination changes. In Stage II, a structure-preservation gate combines semantic conflicts with cross-weather intrinsic discrepancies to suppress unreliable teacher regions. Per-Gaussian weather transport variables and target-weather branches then strengthen the desired weather appearance over the intrinsic-decoupled scene. Experiments on nuScenes and Waymo show that IGSW achieves the highest content similarity and semantic consistency and the lowest temporal error among the compared methods, while retaining pronounced weather-dependent illumination and atmosphere. The project page is available at https://akaxi6.github.io/IGSW/ .

Figures & tables

Fig. 1: Teaser of IGSW. Given a driving-scene sequence, target-weather teacher images, and intrinsic maps (albedo, normal, and irradiance) estimated through inverse rendering, IGSW reconstructs a dynamic multi-weather urban scene in two stages: intrinsic-decoupled Gaussian appearance modeling and intrinsic-preserving weather refinement. The resulting Gaussian scene renders target-weather views while preserving the spatial structure of the original observations.

Method	Venue	Type	Intrinsic decomp	3D/4D scene	Temporal consist.
Qwen-Image	arXiv 2025	Frame editing	$\times$	$\times$	$\times$
FRESCO	CVPR 2024	Video editing	$\times$	$\times$	$\checkmark$
IntrinsicWeather	CVPR 2026	Intrinsic image editing	$\checkmark$	$\times$	$\times$
WeatherEdit	AAAI 2026	3D/4D weather	$\times$	$\checkmark$	$\checkmark$
WeatherCity	CVPR 2026	4D weather	$\times$	$\checkmark$	$\checkmark$
IGSW (Ours)	–	Intrinsic 4DGS	$\checkmark$	$\checkmark$	$\checkmark$

TABLE I: Method categories and representation capabilities.

Fig. 2: Overview of IGSW. (a) Qwen-Image produces target-weather teachers from the original driving observations. (b) A diffusion-based inverse renderer estimates albedo, normals, and irradiance for both original and teacher images. (c) Stage I constructs an intrinsic-decoupled 4D Gaussian scene with weather-shared albedo/normals and weather-specific irradiance/residuals. (d) Stage II uses a structure-preservation gate and per-Gaussian weather transport to suppress teacher-induced structural errors and strengthen target-weather illumination and atmosphere.

Fig. 3: Qualitative rain and snow comparisons on nuScenes. Rows show the original observations, Qwen-Image, FRESCO, IntrinsicWeather, WeatherEdit, WeatherCity, and IGSW. Columns use matched camera views and weather conditions.

Method	Rain				Snow
Method	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$
Qwen-Image	0.8248	0.2090	0.6748	0.0294	0.7648	0.2692	0.6253	0.0516
FRESCO	0.8070	0.0543	0.5693	0.0174	0.7885	0.0897	0.5443	0.0172
IntrinsicWeather	0.6682	0.1380	0.5497	0.0379	0.6437	0.1563	0.4969	0.0529
WeatherEdit	0.8443	0.1122	0.6863	0.0148	0.8269	0.1412	0.6839	0.0127
WeatherCity	0.8582	0.1289	0.7129	0.0023	0.8652	0.1876	0.6772	0.0038

TABLE II: Complete-sequence quantitative comparison on nuScenes. Results are averaged equally across ten scenes; higher is better for CLIP-S, CLIP-DS, and Sem-CS, while lower is better for

E_{\mathrm{warp}}

. Best values are bold.

Fig. 4: Qualitative rain and snow comparisons on Waymo. The method order matches Fig. 3 ; viewpoints and weather conditions are aligned across methods.

Method	Rain				Snow
Method	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$
Qwen-Image	0.8450	0.1942	0.5099	0.0547	0.7971	0.2007	0.4571	0.0672
FRESCO	0.8703	0.1487	0.6376	0.0195	0.8258	0.1557	0.6710	0.0184
IntrinsicWeather	0.6046	0.1601	0.3919	0.0224	0.7317	0.1929	0.5676	0.0273
WeatherEdit	0.9015	0.2095	0.8036	0.0118	0.8532	0.2055	0.8025	0.0106
WeatherCity	0.7773	0.1375	0.7398	0.0090	0.8002	0.0937	0.6633	0.0101

TABLE III: Complete-sequence quantitative comparison on the Waymo Open Dataset. Metric directions match Table II ; best values are bold.

Fig. 5: Temporal rain comparison on nuScenes at frames 49, 54, 59, 64, and 69. The samples are fixed across methods; quantitative evaluation uses all 297 adjacent frame pairs. The nuScenes snow and Waymo sequences are provided in the supplementary material.

Method	Rain $E_{\mathrm{warp}}\downarrow$	Snow $E_{\mathrm{warp}}\downarrow$
Qwen-Image	0.0294	0.0516
FRESCO	0.0174	0.0172
IntrinsicWeather	0.0379	0.0529
WeatherEdit	0.0148	0.0127
WeatherCity	0.0023	0.0038
IGSW (Ours)	0.0021	0.0028

TABLE IV: Temporal warping error on nuScenes. Lower is better.

Fig. 6: Structural comparison at one timestamp and three views. Rows show the original observations, Qwen-Image, WeatherCity, and IGSW. Clouds in the teacher incorrectly cover visible buildings; IGSW reduces this error while adapting the scene to rainy appearance.

Method	Rain				Snow
Method	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$	CLIP-S $\uparrow$	CLIP-DS $\uparrow$	Sem-CS $\uparrow$	$E_{\mathrm{warp}}\downarrow$
WeatherCity (baseline)	0.858270	0.128999	0.712923	0.002399	0.865202	0.187677	0.677263	0.003811
Stage I only	0.858933	0.132594	0.714228	0.002265	0.866956	0.188946	0.677516	0.003186
Stage II only	0.866129	0.127921	0.898980	0.001632	0.864229	0.220130	0.877846	0.002207
IGSW (Stage I + II)	0.871860	0.159925	0.899250	0.002152	0.878254	0.273236	0.879220	0.002844

TABLE V: Module ablations using complete 100-timestamp, three-view evaluation. Best values are bold.

Fig. 7: Module ablations for rain (left) and snow (right). Rows show the original observations, Qwen teachers, WeatherCity, Stage I only, Stage II only, and complete IGSW. Each row contains three matched camera views.

Urban Scene Reconstruction

See all 16 papers

WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation
Feb 25, 2026Wenhua Wu, Huai Guan, Zhe Liu +1Urban Scene Reconstruction 3D Reconstruction
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China
AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing
Mar 27, 2026Tianyu Liu, Weitao Xiong, Kunming Luo +43D Scene Relighting Video Editing
The Hong Kong University of Science and Technology · Xiamen University · Meituan-M17, Hong Kong
GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generation
Aug 17, 2026Tianchen Deng, Xuefeng Chen, Shuang Wu +5World Models for Autonomous Driving 4D Scene Generation
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University; the State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis; and the Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai 200240, China · Tsinghua University, Shenzhen, China · Chongqing Afari Intelligent Drive Co., Ltd., China +1

Feb 25, 2026J/K move · Enter open · S save

WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation

Wenhua Wu, Huai Guan, Zhe Liu, Hesheng Wang

School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China

Urban Scene Reconstruction 3D Reconstruction Dynamic Scene Reconstruction Autonomous Driving Simulation 3D Gaussian Splatting

Open paper AskPDF arXiv

Editable high-fidelity 4D scenes are crucial for autonomous driving, as they can be applied to end-to-end training and closed-loop simulation. However, existing reconstruction methods are primarily limited to replicating observed scenes and lack the capability for diverse weather simulation. While image-level weather editing methods tend to introduce scene artifacts and offer poor controllability over the weather effects. To address these limitations, we propose WeatherCity, a novel framework for 4D urban scene reconstruction and weather editing. Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds. To tackle the challenge of multi-weather modeling, we introduce a novel weather Gaussian representation based on shared scene features and dedicated weather-specific decoders. This representation is further enhanced with a content consistency optimization, ensuring coherent modeling across different weather conditions. Additionally, we design a physics-driven model that simulates dynamic weather effects through particles and motion patterns. Extensive experiments on multiple datasets and various scenes demonstrate that WeatherCity achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing. Our framework not only enables fine-grained control over weather conditions (e.g., light rain and heavy snow) but also supports object-level manipulation within the scene. Codes are released at https://github.com/IRMVLab/WeatherCity.

Abstract

Urban Scene Reconstruction

Explore similar work

WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation

AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing

GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generation