VIRGi: View-dependent Instant Recoloring of 3D Gaussians Splats

Alessio Mazzucchelli; Ivan Ojeda-Martin; Fernando Rivas-Manzaneque; Elena Garces; Adrian Penate-Sanchez; Francesc Moreno-Noguer

cs.CVMar 3, 2026

VIRGi: View-dependent Instant Recoloring of 3D Gaussians Splats

Authors: Alessio Mazzucchelli, Ivan Ojeda-Martin, Fernando Rivas-Manzaneque, Elena Garces, Adrian Penate-Sanchez, Francesc Moreno-Noguer

Organizations: Arquimea Research Center, San Cristóbal de la Laguna, Santa Cruz de Tenerife, 38320 · Universidad Polit`ecnica de Catalunya, Doctoral Degree in Automatic Control, Robotics and Vision, Carrer de Jordi Girona, 31, Les Corts, Barcelona, 08034, Spain · Volinga AI, San Cristóbal de la Laguna, Santa Cruz de Tenerife, 38320 · Universidad Polit´ecnica de Madrid, Programa de Doctorado en Autom´atica y Rob´otica, Calle de Jos´e Guti´errez Abascal 2, Madrid, 28006, Spain ·

Abstract

3D Gaussian Splatting (3DGS) has recently transformed the fields of novel view synthesis and 3D reconstruction due to its ability to accurately model complex 3D scenes and its unprecedented rendering performance. However, a significant challenge persists: the absence of an efficient and photorealistic method for editing the appearance of the scene's content. In this paper we introduce VIRGi, a novel approach for rapidly editing the color of scenes modeled by 3DGS while preserving view-dependent effects such as specular highlights. Key to our method are a novel architecture that separates color into diffuse and view-dependent components, and a multi-view training strategy that integrates image patches from multiple viewpoints. Improving over the conventional single-view batch training, our 3DGS representation provides more accurate reconstruction and serves as a solid representation for the recoloring task. For 3DGS recoloring, we then introduce a rapid scheme requiring only one manually edited image of the scene from the end-user. By fine-tuning the weights of a single MLP, alongside a module for single-shot segmentation of the editable area, the color edits are seamlessly propagated to the entire scene in just two seconds, facilitating real-time interaction and providing control over the strength of the view-dependent effects. An exhaustive validation on diverse datasets demonstrates significant quantitative and qualitative advancements over competitors based on Neural Radiance Fields representations.

Figures & tables

Panel of Figure 1 — Fig. 1: V I R Gi enables – by editing a single view – instant recolorings of one or multiple objects in scenes modeled with 3D Gaussian Splats. Our key idea of decomposing the reflectance of the scene into diffuse and specular components allows view-dependent consistent edits. Our separation can also be used to enhance or reduce material specularity.

Fig. 2: Comparison of recoloring methods with a ground truth recolor. Our method V I R Gi is the first one to propose a solution for 3DGS recoloring, outperforming one of the best methods in NeRFs recoloring [ 15 ] (IReNe). We propose several contributions that make our method robust and better than a baseline ( Vanilla- V I R Gi ) obtained by naïvely combining existing methods.

Fig. 3: Overview of V I R Gi . Our method comprises two main steps. In the initial step, we train a 3DGS architecture consisting of a Hashgrid $f$ representing the geometry and two MLPs for color modeling. We separate the observed color into a diffuse term $\textsc{MLP}_{\textrm{diff}}$ , which solely relies on hashgrid features $f$ , and a specular term $\textsc{MLP}_{\textrm{spec}}$ , which incorporates the view direction $\theta$ . Our training strategy involves leveraging multiple views of the same scene point in a single batch, as opposed to using a single image per batch, resulting in improved PSNR compared to previous methods. In the second step, given an edited 2D image, we fine-tune the last layer of the diffuse MLP using a soft-segmentation mask $\alpha$ to achieve the target color.

Original image

Monoview

Multiview

Fig. 4: Specularity Comparison: Monoview vs. Multiview. Training with a single viewpoint poses challenges in learning specularities due to limited sample diversity, multiview training facilitates an easier understanding of specular effects.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
	Mip NeRF 360 [ 71 ]			LLFF [ 75 ]			NeRF Synthetic [ 2 ]
PaletteNeRF	22.24	0.720	0.278	24.19	0.800	0.187	29.92	0.960	0.038
RecolorNeRF	-	-	-	24.52	0.736	0.333	29.00	0.943	0.048
IReNe	26.80	0.714	0.292	26.51	0.823	0.136	30.33	0.959	0.035
Vanilla- V I R Gi	29.19	0.881	0.150	28.01	0.905	0.129	32.13	0.975	0.023
V I R Gi	29.74	0.886	0.146	29.10	0.915	0.119	31.65	0.971	0.027

TABLE I: Quantitative color edition results. Best , Second . Results using the recolored ground truth images from [ 15 ] .

DC	MV	DS	PSNR $\uparrow$	PSNR $\uparrow$	PSNR $\uparrow$
			Mip NeRF 360	LLFF	NeRF Synthetic
✗	✗	✗	29.19	28.01	32.13
✗	✓	✗	29.45	28.54	31.90
✗	✓	✓	29.56	28.62	31.78
✓	✗	✗	28.58	27.66	30.93
✓	✓	✗	29.61	29.15	31.50

TABLE II: Ablation study. Recoloring results on all datasets of our ablated method . Best , Second . DC indicates if the output is decoupled or not. MV indicates if the model has been trained using our multi-view rasterization DS indicates if the diffuse’s network

\textsc{MLP}_{\textrm{diff}}

output has been introduced into the soft segmentation network

{\textsc{MLP}_{seg}}

Dataset (PSNR $\uparrow$ )	Mip NeRF 360	LLFF	NeRF Synthetic
Mono-view C3DGS	28.96	25.63	35.60
Multi-view C3DGS	29.47	25.87	35.84
Mono-view V I R Gi	28.43	25.73	33.87
Multi-view V I R Gi	29.30	25.97	35.76

TABLE III: Multi-View VS Mono-View novel view synthesis . Training both the original C3DGS as well as V I R Gi using the proposed multi-view approach enhances the novel view synthesis results. We underline that V I R Gi also provides a decomposition of the image into diffuse and specular.

Panel of Figure 8 — Fig. 5: Comparison between V I R Gi (upper row per case) and Gaussian Editor [ 68 ] (bottom row per case). Gaussian Editor requires the user to initialize a mask using SAM [ 76 ] before applying color edits based on provided prompts: “kitchen - make it magenta”, “bicycle - make it yellow”, and “bonsai - make it green”. Gaussian Editor exhibits several drawbacks, such as color bleeding into surrounding areas in the first two scenes and not understanding the scene structure when recoloring the third scene. Notably, Gaussian Editor takes an average of 612 seconds to process these scenes, while V I R Gi completes the task in just 2 seconds on average.

Fig. 6: Illumination editing with decoupled output. Top row: specular component reduced by applying a multiplicative factor of $0$ . Bottom row: specular component increased by applying a factor of $1.5$ .

Panel of Figure 10 — Fig. 7: Qualitative results. Results on all scenes from the MipNeRF-360 dataset [ 71 ] . This dataset shows complex real scenes with a complete 360 rotation on all scenes. We show several examples of single and multiple color editions. The first column is the original image. The second column is the edit performed by the user and used to train V I R Gi . We also show the soft segmentation predicted by V I R Gi for each recolored image.

Panel of Figure 11 — Fig. 8: Some qualitative results of the IReNe [ 15 ] dataset.

Panel of Figure 12 — Fig. 9: Limitations of diffuse/specular modeling. First row Kitchen scene, second row Counter scene. Comparison between Ground Truth, Diffuse, Specular and Combined RGB render. Our simple diffuse/specular model struggles with complex or anisotropic materials. This can be seen in the Counter (bottom) scene, while the Kitchen (top) scene, especially the table, shows better results. Still, both yield solid RGB reconstructions. Third row - Fourth Row: Comparison of original, edited, and rendered views on specular failure cases. In the case of the water jug, although we are able to apply the recoloring, the result lacks uniformity: some areas of the jug appear more red than others, and the color bleeds onto objects visible through the jug’s transparent surface. For the reflective bowl, we manage to change its color as well, but the edits are less clean compared to those on non-reflective, opaque objects. Some regions of the bowl retain the original color, with only a slight green tint appearing in the edited output

Fig. 10: Limitations in segmentation. We show examples where issues in recoloring happened. These cases occur mainly with parts of the scene that were not present in the first edited image. In the first example, the notebook has been incorrectly recolored. In the second example, the palm tree has been incorrectly recolored. In the third example, the legs of the table have not been fully recolored to red. These cases were fixed by adding a second edited image. In most cases, such issues can be solved by adding an additional edited image, but in some cases, like the last example, certain edits cannot be achieved. In this particular case, the curtain is far from the camera, and not much viewpoint change is present in the dataset from which to learn the correct model.

3D Scene Editing

3D scene editing studies how to make controlled changes to objects, appearance, layout, and motion while keeping a scene coherent across views.

See all 90 papers

Instant Colorization of Gaussian Splats
Apr 18, 2026Daniel Lieber, Alexander Mock, Nils Wandel3D Gaussian Splatting
University of Osnabrück
Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting
May 15, 2026Raushan Joshi, Jean-Yves Guillemaut3D Gaussian Splatting 3D Scene Editing
University of Surrey, Guildford, Surrey, UK
TransSplat: Unbalanced Semantic Transport for Language-Driven 3DGS Editing
Apr 21, 2026Yanhui Chen, Jiahong Li, Jingchao Wang +33D Gaussian Splatting 3D Scene Editing
Guangdong University of Technology · Peking University

Apr 18, 2026J/K move · Enter open · S save

Instant Colorization of Gaussian Splats

Daniel Lieber, Alexander Mock, Nils Wandel

University of Osnabrück

3D Gaussian Splatting

Open paper AskPDF arXiv

Gaussian Splatting has recently become one of the most popular frameworks for photorealistic 3D scene reconstruction and rendering. While current rasterizers allow for efficient mappings of 3D Gaussian splats onto 2D camera views, this work focuses on mapping 2D image information (e.g. color, neural features or segmentation masks) efficiently back onto an existing scene of Gaussian splats. This 'opposite' direction enables applications ranging from scene relighting and stylization to 3D semantic segmentation, but also introduces challenges, such as view-dependent colorization and occlusion handling. Our approach tackles these challenges using the normal equation to solve a visibility-weighted least squares problem for every Gaussian and can be implemented efficiently with existing differentiable rasterizers. We demonstrate the effectiveness of our approach on scene relighting, feature enrichment and 3D semantic segmentation tasks, achieving up to an order of magnitude speedup compared to gradient descent-based baselines.

Abstract

3D Scene Editing

Explore similar work

Instant Colorization of Gaussian Splats

Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting

TransSplat: Unbalanced Semantic Transport for Language-Driven 3DGS Editing