Gaussians Can Subtract: Neural Dissolve Fields for Hybrid Scene Rendering

Yiyan Wang; Haohan Liu; Tetsuo Shibuya

cs.CVNov 4, 2026

Gaussians Can Subtract: Neural Dissolve Fields for Hybrid Scene Rendering

Authors: Yiyan Wang, Haohan Liu, Tetsuo Shibuya

Organizations: The University of Tokyo, Japan · Fuzhou University, China

Abstract

Hybrid Mesh-Gaussian rendering combines compact textured surfaces with flexible 3D Gaussian primitives, offering an efficient alternative to purely Gaussian scene representations for structured environments. Existing scene-reconstruction hybrids commonly follow an additive formulation, where Gaussian residuals compensate for errors left by the mesh. However, not every Mesh-Gaussian conflict is caused by missing content: residual geometry errors, occlusion mismatch, texture-projection artifacts, and view-dependent appearance can make the same mesh content consistent in one view but inconsistent in another. We therefore formulate hybrid rendering as a multi-view inconsistency problem and present Gaussolve, a subtract-and-refill framework that selectively dissolves inconsistent mesh content before residual Gaussians refill the revealed regions. We introduce a lightweight Neural Dissolve Field, parameterized by scalar spherical-harmonic Gaussians, to learn where and from which viewing directions mesh content should be dissolved. A continuous transmittance fusion rule progressively exposes residual Gaussians behind the dissolved mesh, preserving their reconstruction gradients and avoiding hard visibility discontinuities. Experiments on structured and semi-structured scenes demonstrate higher visual fidelity with fewer Gaussian primitives and faster rendering. Project page: https://ooosn.github.io/gaussians-can-subtract/

Figures & tables

Figure 1: Gaussians Can Subtract. Existing hybrids primarily compensate for mesh errors by adding Gaussian residuals. Gaussolve instead introduces a subtractive Neural Dissolve Field that selectively dissolves inconsistent mesh content and lets residual Gaussians refill the revealed regions. This explicit subtract-and-refill decomposition improves visual quality while retaining a compact hybrid representation. Compared with existing methods, Gaussolve achieves a more favorable quality–compactness trade-off by allowing meshes and Gaussians to cooperate rather than compete.

Figure 2: Overview of Gaussolve. The projection-textured mesh represents broad regular content, while the Neural Dissolve Field selectively suppresses mesh content that becomes inconsistent across views. Residual Gaussians refill the revealed regions, and all components are combined through continuous transmittance fusion.

Figure 3: Hard-Z brittleness and gradient recovery. (a) Gaussians initially in front of the mesh frequently cross behind the Hard-Z visibility boundary during training. (b) For Gaussians whose reconstruction gradients are blocked by Hard-Z, Gaussolve recovers a substantial fraction of the ungated reference gradient. Curves and shading denote the four-scene mean and range.

Method	Mesh	Hybrid	ML-Hypersim-AI (Structured)					Deep Blending (Semi-Structured)
	Mesh	Hybrid	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	Aux.%	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	Aux.%
3DGS	$\times$	$\times$	34.39	0.959	0.155	0%	–	28.50	0.877	0.272	0%	–
Reduced 3DGS	$\times$	$\times$	34.04	0.957	0.153	62%	–	28.43	0.875	0.277	51%	–
3DGS-MCMC	$\times$	$\times$	35.82	0.960	0.140	-1%	–	28.56	0.875	0.268	-12%	–
SuGaR	$\checkmark$	$\times$	32.29	0.950	0.149	33%	–	27.58	0.858	0.306	19%	–
PGSR	$\checkmark$	$\times$	32.95	0.958	0.147	61%	–	28.44	0.863	0.302	28%	–

Table 1: Main comparison on structured and semi-structured scenes. The target regime contains broad regular structures where a fixed-budget textured mesh can replace redundant Gaussian primitives. Bold indicates the best performance, and cell colors highlight the top , second-best , and third-best results .

Method	Mesh	Hybrid	DrJohnson				Playroom				Average
	Mesh	Hybrid	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$
MeshGS	$\checkmark$	$\checkmark$	28.80	0.900	0.240	–	30.50	0.900	0.250	–	29.65	0.900	0.245	$\sim$ 30%
Hybrid Mesh-Gaussian	$\checkmark$	$\checkmark$	29.01	0.888	0.262	43%	30.11	0.901	0.255	30%	29.56	0.895	0.259	36%
Ours	$\checkmark$	$\checkmark$	29.69	0.894	0.240	58%	30.93	0.893	0.223	61%	30.31	0.894	0.232	59.5%

Table 2: Comparison with MeshGS [ 6 ] and Hybrid Mesh-Gaussian [ 15 ] on the Deep Blending dataset. For the baselines, we report the 3DGS-based results directly from the respective papers.

Panel of Figure 4 — Figure 4: Qualitative comparisons on novel-view synthesis. Compared with 3DGS, PGSR, 3D-GES, and MeshSplatting, Gaussolve better preserves weakly textured surfaces. The red and blue boxes indicate the corresponding enlarged regions.

Panel of Figure 5 — Figure 5: Visualization of the subtract-and-refill mechanism. Compared with the projection-textured mesh and the Hard-Z hybrid baseline, Gaussolve selectively dissolves inconsistent mesh content (Diss.) and lets residual Gaussians refill the revealed regions (Res.), producing cleaner boundaries and more accurate appearance. The red and blue boxes indicate the corresponding enlarged regions.

Panel of Figure 6 — Figure 6: Directional dissolve for view-dependent appearance. A view-invariant dissolve field can suppress some static geometry errors, but it cannot provide a directional handoff for view-specific evidence. The scalar-SH dissolve field attenuates mesh evidence only from conflict-prone viewing directions, allowing residual RGB Gaussians to recover the corresponding view-dependent details.

Panel of Figure 7 — Figure 7: Textured mesh priors and direct image projection. We compare RC, MILo, PGSR, and SuGaR mesh priors on SaintAnne and ML-Hypersim-001. The top row visualizes the reconstructed mesh geometry, and the bottom row shows enlarged textured renderings. Direct projection preserves sharper local appearance than appearance baked from reconstructed representations.

Panel of Figure 8 — Figure 8: Rendered-depth comparison. Aligned Depth Anything V2 (DA-V2) reference [ 41 ] , mesh depth, Gaussolve, and Hard-Z. Red boxes mark representative conflict regions.

	3DGS			Reduced 3DGS
Component	Storage	FPS	Mem.	Storage	FPS	Mem.
Mesh Only	0.069G	1003	0.98G	0.069G	988	0.98G
Dissolve GS Only	0.016G	1110	0.09G	0.016G	1111	0.09G
Residual GS Only	0.251G	265	0.77G	0.134G	463	0.49G
GS Baseline	0.655G	99	1.64G	0.071G	311	0.48G
Ours	0.336G	168	1.70G	0.219G	236	1.42G

Table 3: Efficiency breakdown. Average rendering efficiency, storage, and memory on the Deep Blending dataset.

Variant	Faces	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
A. Component Ablation (RC)
Mesh Only	–	27.97	0.894	0.249
Residual GS Only	–	34.25	0.960	0.153
Mesh + Residual GS (Hard-Z)	–	33.51	0.932	0.197
Mesh + NDF	–	29.76	0.896	0.241
Full	–	35.21	0.963	0.106

Table 4: Ablation study on ML-Hypersim-001. Ablations of representation components, training design, and mesh geometry/appearance.

Method	AbsRel $\downarrow$	RMSE $\downarrow$	$\delta_{1}\uparrow$	Boundary F1 $\uparrow$
Mesh	0.0787	0.0909	0.9725	0.5274
Hard-Z	0.0868	0.0845	0.9736	0.6720
Ours	0.0680	0.0785	0.9751	0.6814

Table 5: Depth consistency with the aligned DA-V2 reference. We report the standard AbsRel, RMSE, and

\delta_{1}

metrics used by Depth Anything V2 [ 41 ] , together with Boundary F1 to measure depth-discontinuity alignment. Boundary F1 uses a shared Sobel threshold and a two-pixel matching tolerance.

Method	3DGS		Reduced 3DGS		3DGS-MCMC		Mesh
Method	Time	Mem.	Time	Mem.	Time	Mem.	Mem.
Baseline	32.5min	3.67G	24.5min	1.51G	36.0min	4.11G	–
Ours	32.9min	4.91G	34.1min	4.78G	53.1min	6.95G	3.95G

Table 6: Training overhead. Training time and peak GPU memory for Gaussolve with different Gaussian backbones.

Panel of Figure 15 — Figure S1 : Surface-alpha dissolve alternatives. We compare texture-map alpha, per-vertex alpha, and the proposed volumetric NDF. The first row of each scene shows the learned mesh/dissolve representation, the second row shows the blended rendering with red and blue boxes, and the third row shows the corresponding zoom-ins. Texture-map and vertex alpha can reduce some mesh–image conflicts but often produce diffuse or fragmented masks, while the volumetric NDF localizes unreliable mesh evidence and enables cleaner residual Gaussian refill.

Panel of Figure 16 — Figure S2 : Per-scene component visualizations. The decomposition shows how the dissolve field suppresses unreliable mesh evidence and how residual Gaussians refill the corresponding regions. Here, ML-Hy denotes the ML-Hypersim-AI scene family.

Panel of Figure 17 — Figure S3 : Additional adjacent-view consistency. Consecutive rendered crops remain stable across nearby views, including regions where residual Gaussians correct unreliable mesh evidence.

Method	Mesh	Hybrid	Indoor: DrJohnson				Indoor: Playroom				Outdoor: SaintAnne				Outdoor: Bridge
Method	Mesh	Hybrid	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$
3DGS	$\times$	$\times$	29.33	0.897	0.271	0%	30.62	0.897	0.266	0%	25.54	0.838	0.280	0%	26.44	0.854	0.260	0%
Reduced 3DGS	$\times$	$\times$	29.30	0.894	0.276	53%	30.50	0.896	0.266	61%	25.50	0.835	0.287	39%	25.59	0.837	0.286	59%
3DGS-MCMC	$\times$	$\times$	29.40	0.889	0.274	1%	30.65	0.896	0.252	-36%	25.62	0.840	0.279	0%	26.46	0.856	0.257	0%
SuGaR	$\checkmark$	$\times$	28.30	0.870	0.326	59%	30.25	0.884	0.289	37%	24.20	0.820	0.301	-38%	24.50	0.791	0.320	5%
PGSR	$\checkmark$	$\times$	29.25	0.895	0.273	37%	30.65	0.868	0.318	28%	25.42	0.827	0.315	19%	25.46	0.834	0.291	44%

Table S1 : Quantitative breakdown of hybrid methods on the Deep Blending dataset. We select four representative scenes encompassing both complex indoor layouts and structured outdoor architectural environments. Bold indicates the best performance, and cell colors highlight the top , second-best , and third-best results. † Indicates catastrophic geometry collapse.

Method	Mesh	Hybrid	Outdoor: Treehill				Outdoor: Garden				Outdoor: Flowers				Outdoor: Bicycle
	Mesh	Hybrid	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\Delta$ GS $\uparrow$
3DGS	$\times$	$\times$	20.42	0.684	0.258	0%	28.55	0.883	0.123	0%	22.48	0.641	0.314	0%	26.24	0.809	0.187	0%
Hard-Z Proxy	$\checkmark$	$\checkmark$	19.21	0.655	0.290	-1%	27.41	0.866	0.119	7%	20.77	0.573	0.334	5%	24.33	0.764	0.211	13%
Ours (3DGS)	$\checkmark$	$\checkmark$	20.58	0.689	0.255	14%	29.00	0.897	0.093	5%	22.34	0.635	0.295	13%	26.21	0.808	0.175	17%

Table S2 : Irregular-scene stress test on Mip-NeRF 360. We report results on four outdoor scenes with complex geometry, vegetation, and weakly structured regions. Bold indicates the best performance, and cell colors highlight the top and second-best .

3D Scene Representation

3D scene representation studies how to encode geometry, appearance, semantics, and motion for reconstruction, rendering, generation, and interaction.

See all 249 papers

Apr 16, 2026

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

We introduce a hybrid Gaussian-hash-grid radiance representation for reconstructing 2D Gaussian scene models from multi-view images. Similar to NeST splatting, our approach reduces the entanglement between geometry and appearance common in NeRF-based models, but adds per-Gaussian latent features alongside hash-grid features to bias the optimizer toward a separation of low- and high-frequency scene components. This explicit frequency-based decomposition reduces the tendency of high-frequency texture to compensate for geometric errors. Encouraging Gaussians with hard opacity falloffs further strengthens the separation between geometry and appearance, improving both geometry reconstruction and rendering efficiency. Finally, probabilistic pruning combined with a sparsity-inducing BCE opacity loss allows redundant Gaussians to be turned off, yielding a minimal set of Gaussians sufficient to represent the scene. Using both synthetic and real-world datasets, we compare against the state of the art in Gaussian-based novel-view synthesis and demonstrate superior reconstruction fidelity with an order of magnitude fewer primitives.

Neel Kelkar, Simon Niedermayr, Klaus Engel +1

Technical University of Munich, Munich, Germany · Siemens Healthineers, Erlangen, Germany

3D Gaussian Splatting Multi-View 3D Reconstruction Representation Disentanglement Novel View Synthesis Gaussian Splatting

Ask PDF

Feb 16, 2026

Gaussian Mesh Renderer for Lightweight Differentiable Rendering

3D Gaussian Splatting (3DGS) has enabled high-fidelity virtualization with fast rendering and optimization for novel view synthesis. On the other hand, triangle mesh models still remain a popular choice for surface reconstruction but suffer from slow or heavy optimization in traditional mesh-based differentiable renderers. To address this problem, we propose a new lightweight differentiable mesh renderer leveraging the efficient rasterization process of 3DGS, named Gaussian Mesh Renderer (GMR), which tightly integrates the Gaussian and mesh representations. Each Gaussian primitive is analytically derived from the corresponding mesh triangle, preserving structural fidelity and enabling the gradient flow. Compared to the traditional mesh renderers, our method achieves smoother gradients, which especially contributes to better optimization using smaller batch sizes with limited memory. Our implementation is available in the public GitHub repository at https://github.com/huntorochi/Gaussian-Mesh-Renderer.

Xinpeng Liu, Fumio Okura

The University of Osaka

Differentiable Rendering 3D Gaussian Splatting

Ask PDF

Nov 3, 2026

GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats

Can a neural network learn to render 3D Gaussian splats on its own, with no rasterizer in the renderer? We present GaussianRenderFormer, a transformer that takes a set of Gaussians and a camera and outputs the image directly. Each Gaussian becomes one token in a view-independent encoder, and a view-dependent decoder turns small patches of camera rays into pixels by attending to those tokens. Trained on 26,820 Objaverse objects pruned to 20,000 Gaussians each, GaussianRenderFormer renders 300 unseen objects at 38.08 dB PSNR against the full, unpruned splats, while rasterizing the same pruned input reaches 44.02 dB. The gap is not one of generalization, and our diagnostics are consistent with a bottleneck in the view decoder: telling it the projected footprint of each Gaussian, which it could in principle infer, adds 1.4 dB. Under-training is not ruled out. Code, trained models and videos: https://svlwoof.github.io/gaussianrenderformer

Shahaf Valencia Levy, Sagie Benaim

The Hebrew University of Jerusalem

Neural Rendering 3D Gaussian Splatting 3D ViTs

Ask PDF

Abstract

3D Scene Representation

Explore similar work

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

Gaussian Mesh Renderer for Lightweight Differentiable Rendering

GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats