RxGS: Relightable Gaussian Stylization via BRDF Decomposition
Authors: Wanrong Hong, Liu Jiang, Mukesh Prasad, Jun Li
Organizations: School of Computer Science, University of Technology Sydney, Sydney, Australia. · Department of Civil and Environmental Engineering, Stanford University, Stanford, United State. · Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, 2007, Australia.
Reference-based 3D scene stylization transfers the appearance of a reference image onto a reconstructed 3D scene while preserving scene geometry. Recent methods perform this transfer by optimizing the appearance representation of neural scene models including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). These methods optimize an RGB appearance representation, in which a single per-point colour combines the effects of material reflectance, environment illumination, together with the shading it produces, and the viewing direction. When stylization modifies this combined colour, the illumination effects present during stylization are baked into the stored appearance, so the stylized scene cannot be rendered under different illumination. To address this limitation, we present RxGS, a relightable 3D scene stylization framework built on a Bidirectional Reflectance Distribution Function (BRDF)-aware 3D Gaussian representation. Rather than modifying a per-point RGB colour, RxGS optimizes only the diffuse albedo field recovered by inverse rendering, while leaving roughness, metallicity, surface normals, and the estimated environment lighting unchanged. As a result, the stylized scene keeps all the components a physically-based renderer needs other than the edited albedo, and can therefore still be relit. RxGS optimizes the albedo field using a nearest-neighbour feature matching loss together with a spatial smoothness prior over Gaussian centres. We evaluate the method using a Geometry Consistency Cost (GCC) based on optical-flow warping error. RxGS achieves the lowest GCC on all evaluated NeRF-Synthetic scenes and remains competitive on real-captured scenes, while additionally supporting relightable stylization, which existing reference-based 3D scene stylization methods do not provide.
Figures & tables
Figure 1: Comparison between stylization on an RGB appearance representation and stylization on a BRDF-aware scene representation. Existing methods transfer appearance onto a per-point RGB colour, which already combines material reflectance and illumination. The illumination present during stylization is therefore written into the stored colour, and the result no longer changes when the lighting changes. RxGS instead modifies only the diffuse albedo and leaves the estimated lighting representation unchanged, so the stylized albedo can be rendered under different illumination conditions, producing relightable stylized scenes without re-running the stylization.
Figure 2: Overview of RxGS. Given posed multi-view images and a reference style image, a BRDF-aware Gaussian scene representation is first reconstructed with GS-IR, which separates the scene into geometry, diffuse albedo, material attributes (roughness and metallicity), surface normals, and one global environment lighting. RxGS stylizes by optimizing only the diffuse albedo field, with a nearest-neighbour feature matching (NNFM) loss and a spatial smoothness term, while leaving roughness, metallicity, surface normals, and the lighting unchanged. The stylized BRDF-aware representation therefore keeps its relightable property and can be rendered under novel illumination without further optimization.
Method
Lego
Ship
Chair
Mic
Drums
Ficus
Hotdog
Materials
ARF Zhang et al. (2022)
0.0038
0.017
0.0061
0.0024
0.0028
0.0024
0.0043
0.0036
Ref-NPR Zhang et al. (2023)
0.0045
0.0085
0.0034
0.0015
0.0035
0.0015
0.0047
0.0042
RxGS
0.0017
0.0045
0.0028
0.0004
0.0017
0.0005
0.0029
0.0011
Table 1: Geometry Consistency Cost (GCC) for stylization of the NeRF-Synthetic dataset Mildenhall et al. (2021) . Lower is better. The best result in each column is highlighted in orange. StylizedGS Zhang et al. (2025) does not report results on NeRF-Synthetic and is therefore omitted from this comparison.
Method
Fern
Fortress
Horns
Leaves
Orchids
Room
Trex
Flower
ARF Zhang et al. (2022)
0.0084
0.0168
0.0129
0.0111
0.0083
0.0093
0.0117
0.0126
Ref-NPR Zhang et al. (2023)
0.0064
0.0111
0.0086
0.0071
0.0077
0.0071
0.0085
0.0082
StylizedGS Zhang et al. (2025)
0.0056
0.0054
0.0074
0.0052
0.0051
0.0068
0.0064
0.0067
RxGS
0.0078
0.0105
0.0085
0.0084
0.0084
0.0102
0.0080
0.0113
Table 2: Geometry Consistency Cost (GCC) for stylization of the LLFF dataset Mildenhall et al. (2019) . Lower is better. The best and second-best results are highlighted in orange and peach, respectively.
Figure 3: Qualitative comparison of stylization on LLFF scenes. Each row shows a content scene and its style reference image (left), then the results of ARF, StylizedGS, Ref-NPR, and RxGS. For RxGS the stylized albedo field is shown by flat-albedo rendering. All methods receive the same content and style inputs.
Figure 4: Qualitative comparison of stylization on NeRF-Synthetic scenes. Each row shows a content scene and its style reference image (left), then the results of ARF, Ref-NPR, and RxGS. For RxGS the stylized albedo field is shown by flat-albedo rendering.
Figure 5: Relightable stylization on LLFF scenes. The same stylized representation is rendered under different illumination. The first row is albedo rendering without illumination; the remaining rows are renderings under the original environment lighting and under the environment map rotated by 90 and 180 degrees. Changing the illumination produces different shading and shadows while the stylized appearance stays consistent across viewpoints.
Figure 6: Relightable stylization on Mip-NeRF 360 under different environment lighting. For each scene the leftmost columns show the style image, the content image, and the stylized albedo rendering. The remaining columns render the same stylized scene under four environment maps ( City , Fireplace , Forest , and Sunset ). All results come from the same stylized representation without further optimization. The stylized scene responds to the illumination change, producing different lighting-dependent appearance while the stylized texture and the scene structure stay fixed.
3D Gaussian Splatting (3DGS) has established itself as a leading technique for 3D reconstruction and novel view synthesis of static scenes, achieving outstanding rendering quality and fast training. However, the method does not explicitly model the scene illumination, making it unsuitable for relighting tasks. Furthermore, 3DGS struggles to reconstruct scenes captured in the wild by unconstrained photo collections featuring changing lighting conditions. In this paper, we present R3GW, a novel method that learns a relightable 3DGS representation of an outdoor scene captured in the wild. Our approach separates the scene into a relightable foreground and a non-reflective background (the sky), using two distinct sets of Gaussians. R3GW models view-dependent lighting effects in the foreground reflections by combining Physically Based Rendering with the 3DGS scene representation in a varying illumination setting. We evaluate our method quantitatively and qualitatively on the NeRF-OSR dataset, offering state-of-the-art performance and enhanced support for physically-based relighting of unconstrained scenes. Our method synthesizes photorealistic novel views under arbitrary illumination conditions. Additionally, our representation of the sky mitigates depth reconstruction artifacts, improving rendering quality at the sky-foreground boundary
Margherita Lea Corona, Wieland Morgenstern, Peter Eisert +1
Fraunhofer Heinrich Hertz Institute, Berlin, Germany · Humboldt Universität zu Berlin, Berlin, Germany
Neural radiance fields (NeRFs) and 3D Gaussian Splatting (3DGS) encode a scene with complementary inductive biases, but existing cross-representation distillation typically fixes one representation as teacher for the entire scene. A globally fixed teacher can propagate local reconstruction errors. We present RouteBridge, a bidirectional framework that selects the teaching direction for each ray. Its reliability estimator combines photometric residuals with representation-specific geometric evidence and routes supervision from NeRF to 3DGS, from 3DGS to NeRF, or abstains. A renderer-independent interface transfers color, opacity, and normalized depth without shared features or point correspondence. On mip-NeRF 360, the NeRF and 3DGS exports reach 28.56 and 28.77 dB, respectively. The 3DGS export improves over 3DGS by 1.56 dB and over NeRF-GS by 0.45 dB while reducing LPIPS to 0.207. On static three-view DTU, RouteBridge obtains 21.12 dB. Ablations show that both adaptive routing and geometric ray targets contribute to the improvement.
Reference-guided stylization of scenes represented by 3D Gaussian Splatting (3DGS) is important for efficient and controllable 3D content creation. Existing VGG-feature-based 3D stylization methods provide stable rendered-view optimization, but often under-represent expressive reference style cues; diffusion models offer stronger image priors, yet direct per-view or score-based diffusion guidance can lead to view drift, local artifacts, and hard-to-control appearance updates. We present DReSG, a 3D-grounded residual-feedback framework for stylized Gaussian splatting. DReSG represents attention-guided diffusion proposals as residual targets relative to the current render, and progressively absorbs these residuals into a shared Gaussian scene through multi-view Gaussian feedback. To make this feedback stable and controllable, DReSG modulates residual strength during target construction and combines coverage-aware view selection with conflict-filtered color updates during multi-view fitting. Extensive experiments demonstrate that DReSG achieves competitive reference-guided stylization while better preserving scene structure and cross-view stability. Our project page is available at https://vpx-ecnu.github.io/DReSG-website/.
Zhongliang Liu, Wenjie Liu, Yang Li
School of Software Engineering, East China Normal University, Shanghai, China · School of Computer Science and Technology, East China Normal University, Shanghai, China · School of Intelligent Interaction, East China Normal University, Shanghai, China