We present PaveSplat, a feed-forward Gaussian splatting method that adds a learned residual to the analytic surface-aligned covariance, keeping surfels paving the visible surface across rendering resolutions. Pixel-aligned feed-forward Gaussian splatting predicts one primitive per pixel from a few images in a single pass, and its rendering quality depends on how well these primitives cover the surface. Recent surfel-based methods compute the surface-aligned covariance analytically from predicted depth. However, this covariance has no way to correct the surfel orientation where the depth-derived frame is ill-posed. The surfels then cover the surface loosely; at the training resolution each surfel's one-pixel minimum footprint covers the gaps, but they show up as holes when the same prediction is re-rendered at a higher resolution. To address this, we keep the analytic covariance as an initialization, with its axes aligned to the surface tangents, and add an identity-initialized unit-quaternion residual that reorients a surfel only where this improves rendering, turning surfels that the analytic frame mis-aims back onto the surface. On RealEstate10K, PaveSplat achieves the best PSNR, SSIM, and LPIPS among single-Gaussian-per-pixel methods at every scale (1x-8x). The native-resolution gain is small; the benefit appears when the same prediction is rendered at higher display resolutions without re-running the network. On zero-shot DL3DV, it leads PSNR within this group at every scale and ranks first or second on SSIM/LPIPS. A coverage-hole analysis shows fewer uncovered pixels than both baselines under high-resolution re-rendering.
Figures & tables
Figure 1: Three ways to cover a surface with surfels. Per regime: the per-pixel Gaussians for one scene as covariance ellipsoids from a novel view (top) and a schematic of how surfels lie on the surface (bottom). (a) Fully-learned covariances (DepthSplat [ 18 ] ) leave scale uncontrolled, so oversized ellipsoids over-spread while small ones fail to tile. (b) The analytic prior (SurfSplat [ 7 ] ) sets the frame from depth with no pathway to correct it, leaving anisotropic surfels mis-oriented to the tangents. (c) PaveSplat makes the analytic frame per-pixel correctable ( q=qa⊗Δq ), so surfels reorient to pave the surface. This isolates surfel shape and coverage.
Figure 2: Overview of PaveSplat covariance construction. From the predicted depth, (a) the surface-aligned prior sets the analytic frame Ra=[e^1e^2n^] , supplying tangent-aligned shape and the initial orientation. (b) Two per-primitive refinements act on this prior: in-plane scale multipliers (orange) set the anisotropic radii si=sˉimi , and a residual reorientation (blue) — the only added learned covariance parameter — adapts orientation via q=qa⊗Δq . (c) The two recombine as R=RaRΔ into the final covariance Σ , rendered to the novel view.
2562 ( Standard )
5122 (HRRC)
10242 (HRRC)
20482 (HRRC)
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
pixelSplat [ 1 ]
26.073
0.864
0.136
25.262
0.855
0.208
24.378
0.864
0.275
24.003
0.898
0.280
HiSplat [ 14 ]
27.194
0.882
0.117
25.097
0.867
0.191
24.097
0.876
0.244
23.816
0.910
0.248
MVSplat [ 3 ]
26.387
0.869
0.128
20.294
0.808
0.279
17.882
0.754
0.421
17.195
0.770
0.438
TranSplat [ 23 ]
26.688
0.875
0.125
20.499
0.814
0.275
18.071
0.760
0.423
17.383
0.776
0.438
DepthSplat [ 18 ]
27.506
0.890
0.112
19.914
0.772
0.332
16.299
0.631
0.491
15.079
0.619
0.508
Table 1: Novel view synthesis on RealEstate10K across render resolutions. All cells are our measurements on the common index; multi-Gaussian methods (pixelSplat, HiSplat) are ruled off above the single-Gaussian-per-pixel group. SurfSplat ‡ : our same-recipe reproduction.
Figure 3: PaveSplat leaves fewer coverage holes under high-resolution re-rendering (RealEstate10K). The coverage-hole rate (black-hole fraction) = share of pixels rendering near-black (max RGB channel <0.02 ) where the GT has content (max channel >0.02 ). (a) vs. re-render scale (per-scene median line, IQR band, log- y ): holes open under upsampling, fewest for PaveSplat at every re-rendered scale. (b) per-scene distribution at 8× (box = median/IQR, whiskers to the 10th/90th percentile): PaveSplat < SurfSplat < DepthSplat (per-scene Wilcoxon, both p<10−10 ).
Configuration
frame
Δq
smin/smax↓
black% 8×↓
PSNR 1×↑
PSNR 8×↑
SurfSplat ‡
normal
–
0.667
1.00
27.519
23.342
+ tangent frame
tangent
–
0.580
1.11
27.544
23.850
+ residual Δq (Ours)
tangent
✓
0.585
0.78
27.686
24.266
Table 2: PaveSplat design-path ablation on RealEstate10K (identical recipe per row). Starting from the analytic baseline, we add the tangent frame and residual; columns report anisotropy, 8× black-hole fraction, and 1× / 8× PSNR.
Gaussian splatting has emerged as a flexible representation for 3D reconstruction from posed images. However, existing methods are optimized primarily using rasterization-based losses, which supervise a splat only when it contributes to sampled camera rays. Gaussians that are occluded or contribute little to the sampled view therefore receive weak or no geometric gradients and may drift away from the underlying surface, producing undesired floaters. We introduce PCAsplat, a geometry-aware regularization framework for Gaussian splatting based on differentiable local principal component analysis (PCA). Our PCA regularizer acts directly on neighborhoods of Gaussian centers and can therefore update Gaussians that do not contribute to the current training view. We regularize the PCA eigenvalues to encourage Gaussians to move to the underlying surface with isotropic tangent-plane coverage. We also align each Gaussian normal with the PCA-estimated neighborhood normal to enforce consistent orientation. Experiments on DTU, Tanks and Temples, and NeRF Synthetic show that the splats produced by PCAsplat better approximate samples of the reference surface while substantially reducing undesired floaters. These surface-aligned splats enable downstream geometry-processing tasks, including point cloud segmentation, and direct Poisson reconstruction. Additionally, PCAsplat remains competitive under conventional novel view synthesis and mesh extraction tasks. Code will be released.
Vitor Matias, Filipe Nascimento, Kiyohiro Nakayama +6
Universidade de São Paulo · Stanford University · UFRPE +1
Generalizable 3D Gaussian Splatting has recently emerged as an efficient approach for novel-view synthesis, enabling feed-forward synthesis from only a few input views. However, existing pixel-wise feed-forward methods suffer from over-bright renderings when the number of input views varies during inference, as well as insufficient supervision for accurate Gaussian scale estimation, which leads to hole artifacts, particularly in high-resolution renderings. To address these issues, we identify that the over-brightness is caused by the varying number of overlapping Gaussians and propose a simple alpha normalization strategy to maintain brightness consistency across different number of input views. In addition, we introduce an auxiliary 3D sampling-based regularizer to improve Gaussian scale estimation, thereby mitigating hole artifacts in high-resolution rendering. Experiments on benchmark datasets demonstrate that our method significantly improves baseline models under varying input-view and high-resolution rendering settings.
Hoang Chuong Nguyen, Renjie Wu, Jose M. Alvarez +1
Feed-forward 3D Gaussian Splatting reconstructs scenes from images in seconds. Pixel-aligned methods keep every Gaussian grounded in an observation, but produce redundant, potentially misaligned Gaussians across views. Token-based methods instead aggregate all views into a fixed set of learned tokens but lack explicit observation support and use a fixed reconstruction budget. We introduce ZipSplat, a method that predicts a compact, observation-grounded scene representation whose size is set at inference. It derives each scene token from an image patch rather than a learned query, and decodes it into freely placed 3D Gaussians. Tokens across views are merged into a compact set whose size is adjustable at inference. However, without geometric supervision, decoded Gaussians can drift from their source patch, and merging can mix tokens from different surfaces. We therefore introduce two geometric losses to ground and shape this representation. A cone loss ties each token's Gaussians to its source patch, and GeoCE aligns token similarity with 3D proximity so that tokens observing the same surface merge together. Building on these observation--geometry associations, we further introduce TokenBA, a token-level bundle adjustment that refines scene tokens and camera poses in a few seconds. ZipSplat requires neither camera poses nor intrinsics, yet outperforms pixel-aligned and token-based baselines on DL3DV and RealEstate10K, even those given ground-truth cameras. It further generalizes zero-shot to Mip-NeRF360 and ScanNet++. Our project page is at https://veichta.com/zipsplat.