Feed-forward 3D Gaussian Splatting methods have achieved impressive reconstruction quality for autonomous driving scenes, yet they entangle scene geometry with transient appearance properties such as lighting, weather, and time of day. This coupling prevents relighting, appearance transfer, and consistent rendering across multi-traversal data captured under varying environmental conditions. We present SpectralSplat, a method that disentangles appearance from geometry within a feed-forward Gaussian Splatting framework. Our key insight is to factor color prediction into an appearance-agnostic base stream and and appearance-conditioned adapted stream, both produced by a shared MLP conditioned on a global appearance embedding derived from DINOv2 features. To enforce disentanglement, we train with paired observations generated by a hybrid relighting pipeline that combines physics-based intrinsic decomposition with diffusion based generative refinement, and supervise with complementary consistency, reconstruction, cross-appearance, and base color losses. We further introduce an appearance-adaptable temporal history that stores appearance-agnostic features, enabling accumulated Gaussians to be re-rendered under arbitrary target appearances. Experiments demonstrate that SpectralSplat preserves the reconstruction quality of the underlying backbone while enabling controllable appearance transfer and temporally consistent relighting across driving sequences.
Figures & tables
Figure 1: Appearance-disentangled Gaussian reconstruction. Given input frames (row 1) relighted under progressively varying conditions (row 2), SpectralSplat produces Gaussians whose base colors remain consistent regardless of the input appearance (row 3), while the adapted colors faithfully capture each target lighting condition (row 4). The rendered outputs (row 5) rendered from our predicted Gaussians with swapped appearance embeddings demonstrate the appearance transfer capability.
Table 1: Method comparison. We compare along three axes: Feed-forward – inference without per-scene optimization; Disentangled – appearance is disentangled from geometry; Novel – can render under unseen target appearances ( e.g . from a reference image). SpectralSplat is the only method satisfying all three.
Figure 2: Training pipeline. Original and augmented images share geometry but produce separate features and appearance embeddings. Four losses enforce disentanglement: Linv (base invariance), Laug (augmented reconstruction), Lswap (cross-appearance), and Lbase (base color alignment).
Figure 3: Relighting pipeline. MVInverse + physics rendering is 3D-consistent but flat; IC-Light alone is photorealistic but inconsistent; our hybrid pipeline achieves both.
Figure 4: Cross-appearance results on Waymo. Rows 1–3: source ground truth, adapted render, and base color. Rows 4–6: same for the augmented condition. Base colors from both are nearly identical, confirming appearance-invariance. Rows 7–8: swapping the appearance embedding transfers appearance while preserving geometry.
Waymo
nuScenes
Recon.
NVS
Recon. + NVS
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PixelSplat [ 4 ]
—
—
—
—
—
—
21.51
0.616
0.372
MVSplat [ 5 ]
24.94
0.80
0.23
22.04
0.68
0.34
21.61
0.658
0.295
DepthSplat [ 43 ]
25.38
0.76
0.26
23.86
0.70
0.31
—
—
—
Omni-Scene [ 41 ]
—
—
—
—
—
—
24.27
0.736
0.237
Table 2: Reconstruction quality. SpectralSplat preserves reconstruction quality on both datasets while enabling appearance control. Baseline metrics reported from [ 28 ] . Best in bold , second best underlined .
Figure 5: t-SNE of appearance embeddings. Embeddings cluster by illumination type, confirming the encoder captures meaningful appearance information.
Figure 6: WildGaussians vs. SpectralSplat. Both methods are trained on augmented images and evaluated on the original source images. WildGaussians exhibits artifacts in the reconstruction, while SpectralSplat produces cleaner geometry and appearance.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Time (per 10 frames)
WildGaussians [ 18 ]
30.42
0.843
0.296
∼ 31 min
SpectralSplat (ours)
27.52
0.862
0.199
∼ 18 s
Table 4: Comparison with WildGaussians. SpectralSplat achieves competitive appearance control in a feed-forward pass, ∼ 100 × faster than per-scene optimization.
PSNR ↑
SSIM ↑
LPIPS ↓
SpectralSplat (full)
Src geom. → Aug app.
21.84
0.678
0.502
Aug geom. → Src app.
19.16
0.579
0.514
w/o Lswap
Src geom. → Aug app.
14.58
0.525
0.525
Aug geom. → Src app.
14.28
0.493
0.534
w/o Lbase
Src geom. → Aug app.
21.32
0.669
0.514
Aug geom. → Src app.
19.03
0.572
0.524
Table 5: Ablation study on Waymo (cross-appearance). We report the swapped configurations (geometry from one source, appearance embedding from the other) to isolate each loss’s contribution to disentanglement. Best in bold .
Figure 7: Appearance transfer grid. Each row renders the same scene geometry under different appearance embeddings. Column 1 : original render. Column 2 : base color ( a=0 ). Columns 3–7 : renders using the appearance embedding from the reference image shown in the top row. The appearance embedding transfers the global color tone and lighting mood while preserving scene geometry.
Figure 8: Cross-appearance results on Waymo 2. Rows 1–3: source ground truth, adapted render, and base color. Rows 4–6: same for the augmented condition. Base colors from both are nearly identical, confirming appearance-invariance. Rows 7–8: swapping the appearance embedding transfers appearance while preserving geometry.
Figure 9: Cross-appearance results on Waymo 3. Rows 1–3: source ground truth, adapted render, and base color. Rows 4–6: same for the augmented condition. Base colors from both are nearly identical, confirming appearance-invariance. Rows 7–8: swapping the appearance embedding transfers appearance while preserving geometry.
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
Feedforward 3D Gaussian Splatting (3DGS) often struggles in trajectory-based sparse-view driving scenes. Existing Gaussian repair methods mainly target optimization-based 3DGS, while diffusion-based repair is typically restricted to iterative refinement near observed viewpoints, leaving feedforward 3DGS repair underexplored. We propose ConFixGS, a plug-and-play method that learns to fix feedforward 3DGS with confidence-aware diffusion priors. Starting from a pretrained feedforward model, ConFixGS generates diffusion-enhanced local pseudo-targets and validates them through reprojection-based cross-checking against support views. The resulting dense confidence maps guide refinement, enhancing reliable details while suppressing hallucinated or inconsistent evidence. On Waymo, nuScenes, and KITTI, ConFixGS improves challenging novel view synthesis, with PSNR gains of up to 3.68 dB and FID reduced by nearly half. Our results highlight confidence-aware fusion of generative priors and support-view consistency as a key principle for robust feedforward 3D driving scene reconstruction.
Rui Song, Tianhui Cai, Markus Gross +5
University of California, Los Angeles · Technical University of Munich · University of Cambridge
We propose WildSplatter, a feed-forward 3D Gaussian Splatting (3DGS) model for unconstrained images with unknown camera parameters and varying lighting conditions. 3DGS is an effective scene representation that enables high-quality, real-time rendering; however, it typically requires iterative optimization and multi-view images captured under consistent lighting with known camera parameters. WildSplatter is trained on unconstrained photo collections and jointly learns 3D Gaussians and appearance embeddings conditioned on input images. This design enables flexible modulation of Gaussian colors to represent significant variations in lighting and appearance. Our method reconstructs 3D Gaussians from sparse input views in under one second, while also enabling appearance control under diverse lighting conditions. Experimental results demonstrate that our approach outperforms existing pose-free 3DGS methods on challenging real-world datasets with varying illumination.