3D Gaussian Splatting (3DGS) has received tremendous popularity over the past few years due to its photorealistic visual appearance. However, 3DGS uses volumetric rendering that is not suitable for objects with non-lambertian or transparent materials. To remedy this issue, a family of Order-Independent Transparency (OIT) rendering methods propose to remove or modify the depth sorting step in the 3DGS rendering equation. However, the potential of OIT-based method is still underexplored. In this paper, we observe that the OIT modifications to the rendering equation significantly reduce the inter-independence among individual gaussian splats, resulting in very sparse variable dependencies that can be harnessed by specific optimization techniques such as active set method. To this end, we propose SparseOIT, an OIT-based 3DGS reconstruction algorithm that maintains an active set of gaussian splats and enjoys an acceleration ratio that is proportional to the potential sparsity. SparseOIT is designed by jointly considering the OIT rendering equation, the reconstruction algorithm and the geometric regularization. Through extensive experiments, we demonstrate that SparseOIT outperforms existing methods in the OIT-family by a large margin and also achieves comparable performance to the state-of-the-art 3DGS reconstruction methods based on volumetric rendering. Project page:
Figures & tables
Figure 2: Overview of SparseOIT. Our SparseOIT framework takes multi-view images as input and follows the same optimization procedure as 3DGS in the early stage. After completing the first 15,000 iterations (including the densification phase), we switch to the proposed sparsity optimization with active set method. We organize the optimization into stages separated by active set updates. Before the each stage, we update the active set and use the pre-rendered result from the previous step (corresponding to the rendering of the inactive set). We then render images using the current active set, while simultaneously updating a new pre-rendered result that will serve as the input for the next stage.
Figure 3: Gradient backpropagation. (Top) We utilize per-splat parallelization for backward propagation, which similar as Taming-3DGS [ 21 ] . (Bottom) Taming-3DGS store the pixel states for every 32nd splat in the sorted list in forward propagation, while we need not do anything. For the backward, lane 0 of Taming-3DGS fetches each pixel’s parameters and broadcasts them to the entire warp. By contrast, our design allows every lane to fetch parameters for 32 pixels concurrently and exchange them across lanes, thereby improving efficiency.
Figure 4: Comparison with baselines. We compare the SparseOIT-B (which incorporates the active set acceleration) against 3DGS [ 15 ] , Taming-3DGS [ 21 ] (named “Taming” in the figure), and SortFree [ 14 ] . For evaluation, we select indoor and outdoor scenes from the Mip-NeRF 360 dataset, as well as indoor scenes from the DeepBlending dataset. The PSNR values shown in this figure correspond to a single view.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Time ↓
N(k)
3DGS [ 15 ]
27.68
0.8214
0.1771
909
2679
Taming-3DGS [ 21 ]
27.44
0.8012
0.2193
183
665
SortFree [ 14 ]
27.33
0.8067
0.1792
2302
4314
SparseOIT-A
27.38
0.8058
0.1964
564
2082
SparseOIT-B
27.38
0.8058
0.1973
460
2094
SparseOIT-C
27.11
0.7884
0.2268
201
686
Appendix
Table 5: Quantitative results on Mip-NeRF 360 under adjusted learning rate
We address inverse rendering with 3D Gaussian Splatting (3DGS) by introducing explicit visibility modeling for relighting as well as material and light decomposition. Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions. This oracle is implemented using a point-based ray tracer accelerated by a bounding volume hierarchy (BVH) and queried sparsely within the inverse-rendering objectives, while the original 3DGS loop is retained for training efficiency, interactive visualization, and tooling compatibility. Shading follows a physically based rendering (PBR) formulation with a principled microfacet bidirectional reflectance distribution function (BRDF) and low-resolution high-dynamic-range (HDR) environment maps. We evaluate our method on the TensoIR synthetic dataset against baselines based on neural radiance fields (NeRFs), tensor representations, and 3DGS. The results show competitive performance and improvements over several earlier baselines for novel-view synthesis (NVS), albedo reconstruction, and relighting. Ablations indicate that explicit visibility modeling is an important contributor to relighting and material-illumination disentanglement, while pure PBR supervision can favor appearance fitting in NVS. Overall, the findings suggest that sparse, point-based visibility queries effectively complement rasterized splatting for inverse rendering while preserving the practical advantages of the 3DGS ecosystem.
Sven Kluge, Oliver Staadt
University of Rostock, Albert-Einstein-Straße 22, 18059 Rostock, Germany
While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction, it suffers from significant overhead due to massive redundant primitives. Existing compression methods typically rely on local sampling or fixed pruning thresholds, which often struggle to balance redundancy reduction with high-fidelity rendering. To address this, we propose a novel framework that formulates Gaussian optimization as a global geometric distribution matching problem. Specifically, our approach integrates three components: (1) we introduce a multi-view 3D Gaussian contribution ranking mechanism that filters primitives using geometric consistency instead of local heuristics; (2) we propose a global Optimal Transport (OT)-based aggregation algorithm that merges redundant primitives while preserving the underlying geometry; and (3) we design an OT-based densification operator that maintains the Gaussian's distributional properties for stable optimization. Our approach achieves state-of-the-art rendering quality with only \textbf{10%} primitives and \textbf{10×} accelerated training speeds compared to vanilla 3DGS.
Beizhen Zhao, Sicheng Yu, Ziran Yin +2
The Hong Kong University of Science and Technology (Guangzhou)
Recent progress in feed-forward 3D Gaussian Splatting (3DGS) has notably improved rendering quality. However, the spatially uniform and highly redundant 3DGS map generated by previous feed-forward 3DGS methods limits their integration into downstream reconstruction tasks. We propose SparseSplat, the first feed-forward 3DGS model that adaptively adjusts Gaussian density according to scene structure and information richness of local regions, yielding highly compact 3DGS maps. To achieve this, we propose entropy-based probabilistic sampling, generating large, sparse Gaussians in textureless areas and assigning small, dense Gaussians to regions with rich information. Additionally, we designed a specialized point cloud network that efficiently encodes local context and decodes it into 3DGS attributes, addressing the receptive field mismatch between the general 3DGS optimization pipeline and feed-forward models. Extensive experimental results demonstrate that SparseSplat can achieve state-of-the-art rendering quality with only 22% of the Gaussians and maintain reasonable rendering quality with only 1.5% of the Gaussians. Project page: https://victkk.github.io/SparseSplat-page/.