NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement

Georgios Floros; Evangelos Sartinas; Orestis Vangelis; Maria Pateraki

cs.CVOct 9, 2026

NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement

Authors: Georgios Floros, Evangelos Sartinas, Orestis Vangelis, Maria Pateraki

Organizations: National Technical University of Athens

Abstract

6D pose estimation of novel objects without CAD models remains an open challenge: model-free methods must build the object representation from scratch during onboarding and have long trailed their model-based counterparts. We present NovaSplat, a model-free pipeline that closes this gap. From a short smartphone video captured at onboarding, we train a 3D Gaussian Splatting (3DGS) model that serves both as a template source for detection and coarse pose estimation and as a differentiable backbone for pose refinement. Our refiner optimizes the object-to-camera pose with combined photometric and geometric losses under a continuous rotation parametrization free of gradient singularities. On a household-object benchmark, NovaSplat surpasses all model-based methods. On a challenging industrial benchmark with textureless and specular surfaces, it outperforms all model-free competitors while substantially narrowing the gap to model-based ones. We further introduce NovaSplat-Lite, an inference variant that is $3.1\times$ faster at negligible accuracy cost. Code will be made available at https://github.com/POSE-Lab/novasplat.

Figures & tables

Figure 1: AP

{}_{\text{MSSD}}

( HOPEv2 ) and AR

{}_{\text{MSSD}}

( IndustryShapes , averaged over Classic and Extended splits) for model-free ( blue ) and model-based ( orange ) methods. The shaded region marks the gap between the best model-free baseline and the best model-based method. NovaSplat ( green ) surpasses all model-based methods on HOPEv2, effectively closing the gap on that benchmark. In IndustryShapes NovaSplat sits within the gap, outperforming all model-free competitors while narrowing the distance to model-based methods.

Figure 2: NovaSplat pipeline. During onboarding (top), a short smartphone video is processed through SfM, 3DGS training, and MVS to produce a 3DGS representation and a 3D mesh. Template images $\{\mathcal{I}_{\text{templ}}^{i}\}_{i=1}^{N_{\text{templ}}}$ are rendered from the 3DGS model at icosphere-sampled viewpoints, and a template point cloud $\mathcal{T}$ is sampled from the mesh via Poisson disk sampling. During inference (bottom), a query image is processed by a modified CNOS module to detect the object and produce a segmentation mask $M_{\text{seg}}$ . A FreeZe-based Coarse Pose Estimator (CPE) uses $\{\mathcal{I}_{\text{templ}}^{i}\}_{i=1}^{N_{\text{templ}}}$ and $\mathcal{T}$ for joint visual and geometric feature matching to produce $P_{\text{coarse}}$ . Our 3DGS-based refiner then optimizes $P_{\text{coarse}}$ via a photometric loss $\mathcal{L}_{\text{photo}}$ and a geometric Chamfer loss $\mathcal{L}_{\text{geo}}$ to yield the final 6D pose. Dashed arrows indicate onboarding assets reused at inference.

Method	CAD	AP $\uparrow$	AP ${}_{\text{MSSD}}$ $\uparrow$	AP ${}_{\text{MSPD}}$ $\uparrow$
GigaPose [ 32 ]	✓	0.167	0.046	0.288
OPFormer* [ 30 ]	✓	0.351	0.272	0.431
GenFlow [ 28 ]	✓	0.411	0.346	0.475
Co-op (CNOS) [ 29 ]	✓	0.477	0.405	0.549
Co-op (MUSE) [ 29 ]	✓	0.514	0.435	0.593
NeMO [ 18 ]	✗	0.302	0.291	0.312

Table 1: Quantitative results on HOPEv2 . Model-based methods use CAD models at test time. Model-free methods use only onboarding videos. Best model-based and model-free results appear in bold . *Results taken from the BOP leaderboard. [ 30 ] reports slightly higher numbers that do not affect the ranking. NovaSplat‑Lite is our fast inference variant (Sec. 4.3 ). It uses the same onboarding and object representation and differs only in inference cost.

Panel of Figure 3 — Figure 3: Qualitative comparison on the HOPEv2 dataset. NovaSplat robustly estimates poses in cluttered scenes with heavy occlusions, outperforming competing model-free methods. Bounding cuboids highlight the predicted 6D poses. Additional qualitative comparisons are provided in Sec. H .

		IndustryShapes Classic				IndustryShapes Extended
Method	CAD	AR ${}_{\text{MSPD}}$ $\uparrow$	AR ${}_{\text{MSSD}}$ $\uparrow$	AR ${}_{\text{VSD}}$ $\uparrow$	AR $\uparrow$	AR ${}_{\text{MSPD}}$ $\uparrow$	AR ${}_{\text{MSSD}}$ $\uparrow$	AR ${}_{\text{VSD}}$ $\uparrow$	AR $\uparrow$
FreeZe [ 5 ]	✓	0.66	0.72	0.54	0.64	0.75	0.67	0.42	0.61
FoundPose [ 35 ]	✓	0.31	0.39	0.22	0.30	0.41	0.25	0.18	0.28
FoundationPose [ 56 ]	✓	0.73	0.74	0.53	0.67	0.83	0.74	0.50	0.69
OnePose [ 44 ]	✗	0.08	0.08	0.06	0.07	0.04	0.01	0.01	0.02
OnePose++ [ 11 ]	✗	0.12	0.16	0.12	0.14	0.26	0.19	0.14	0.20

Table 2: Quantitative results on the IndustryShapes dataset, classic and extended splits. Methods are grouped by category, with model-based methods listed first and model-free methods listed below. Top-performing model-based and model-free scores are in bold .

Configuration	AP $\uparrow$	AP ${}_{\text{MSSD}}$ $\uparrow$	AP ${}_{\text{MSPD}}$ $\uparrow$
CPE	0.724	0.703	0.744
+ ICP	0.471	0.384	0.557
CPE + 3DGS (only $\mathcal{L}_{\text{photo}}$ )	0.724	0.706	0.743
CPE + 3DGS (only $\mathcal{L}_{\text{geo}}$ )	0.666	0.651	0.682
CPE + 3DGS (full $\mathcal{L}$ )	0.736	0.718	0.754

Table 3: Ablation study on the HOPEv2 validation set. We evaluate the effect of the refinement module and its loss components.

Panel of Figure 4 — Figure 4: Effect of pose refinement on HOPEv2 . The improvement is most pronounced on the Ketchup and Mustard objects (center), where the initial coarse estimates 4(a) are visibly misaligned, and refinement 4(b) brings the projected bounding boxes into close agreement with the observed geometry.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

	Parameter	NovaSplat	-Lite
Onboard	3DGS iterations	14k (densify $<$ 7k)
	Depth loss	enabled
	$N_{\text{templ}}$	336
	Template resolution	$480\times 480$
	$N_{\text{pts}}$	5k
Detect	Proposal model	YOLO-E

Appendix

Table 4: Implementation settings for both configurations. Entries spanning both columns are identical. Bold entries mark where NovaSplat-Lite differs.

Onboarding (per object, one-time)
SfM	10.4 s
MVS	77.7 s
3DGS training (14k iterations)	326.7 s
Poisson subsampling	11.5 s
Template rendering	14.5 s
Feature extraction	27.4 s

Appendix

Table 5: Per-stage runtime breakdown. Onboarding is a one-time per-object cost and is identical for both configurations. NovaSplat-Lite streamlines coarse pose estimation and 3DGS refinement (Sec. B.4 ) for a

3.1\times

end-to-end speed-up. Per-instance figures divide the per-image total by the

21.6

object instances per image on the HOPEv2 validation set. Detection and segmentation is an off-the-shelf component that we do not modify, and the two figures are drawn from separate profiling runs. The difference between them is not attributable to the optimizations described here, which affect only coarse pose estimation and refinement.

CPE sub-step	Total	s/image	%
Point-cloud preparation	79.6 s	0.174	1.8
GeDi descriptor extraction	4058.3 s	8.880	91.0
Visual–geometric fusion	114.9 s	0.251	2.6
Feature similarity	94.2 s	0.206	2.1
Correspondence matching	8.7 s	0.019	0.2
RANSAC	19.7 s	0.043	0.4

Appendix

Table 6: Sub-step breakdown of coarse pose estimation for NovaSplat-Lite, aggregated over the

457

images and

9{,}873

object instances of the HOPEv2 validation set. Geometric descriptor extraction dominates, consuming

91\%

of the CPE budget and

49\%

of total inference time. Every remaining sub-step costs under

0.3

s per image.

Panel of Figure 5 — Figure 5: Host and GPU memory utilization over the full HOPEv2 validation timeline. Top: $14$ k 3DGS iterations (default). Bottom: $5$ k iterations. Onboarding covers all $28$ HOPEv2 objects sequentially. Inference covers all $457$ validation images. Reducing the 3DGS budget shortens onboarding from $235$ to $140$ minutes while leaving the memory envelope essentially unchanged.

Iterations	Time	AP $\uparrow$	$\mathrm{AP}_{\mathrm{MSSD}}$ $\uparrow$	$\mathrm{AP}_{\mathrm{MSPD}}$ $\uparrow$
500	153 s	0.655	0.632	0.678
1k	165 s	0.706	0.684	0.728
5k	258 s	0.741	0.722	0.761
14k	468 s	0.736	0.718	0.754

Appendix

Table 7: Effect of the 3DGS training budget on downstream pose accuracy (HOPEv2 validation set). “Time” is the full per-object onboarding cost.

Ratio	AP $\uparrow$	AP ${}_{\text{MSSD}}$ $\uparrow$	AP ${}_{\text{MSPD}}$ $\uparrow$
100% (baseline)	0.736	0.718	0.754
25%	0.738	0.722	0.754
5%	0.655	0.635	0.674
1%	0.212	0.203	0.221

Appendix

Table 8: Effect of onboarding view sampling ratio on pose accuracy, evaluated on the HOPEv2 validation set. Accuracy is stable down to 25% of the original views, degrades moderately at 5%, and collapses at 1%.

Panel of Figure 6 — Figure 6: Qualitative comparison of rendered templates under different onboarding sampling ratios on the HOPEv2 dataset. The rendering fidelity degrades significantly at extreme sampling ratios (1%), resulting in artifacts and loss of detail, while 25% maintains comparable quality to the baseline.

	HOPEv2	IndustryShapes
	obj_000001	obj_000005
Onboarding frames	178	522
Frames registered	178 (100%)	522 (100%)
Reconstructed sub-models	1	1
Mean reprojection error (px)	0.50	0.67
Sparse 3D points	78,973	5,802

Appendix

Table 9: SfM reconstruction of the onboarding sequences, with no ground-truth poses used. Both objects register fully into a single model. The textureless object yields a sparse cloud two orders of magnitude thinner,

80\%

of which lie off the object.

Absolute rotation error ( ∘ )
	HOPEv2	IndustryShapes
	obj_000001	obj_000005
Frames compared	178	482
Mean viewing distance (mm)	—	447
median	0.32	1.36
mean	0.38	1.37

Appendix

Table 10: Estimated versus provided onboarding poses, after similarity alignment of the camera centers. Absolute errors are alignment-dependent. The relative rotation error is not and measures the local consistency that the MVS and 3DGS stages rely on.

Panel of Figure 7 — Figure 7: Onboarding camera trajectories, shown as orthographic projections onto the three coordinate planes (millimeters). The estimated trajectory (orange) is the structure-from-motion reconstruction after similarity alignment; the reference (blue) is the trajectory provided with the dataset. The two overlap throughout both sequences, including on the textureless object, where the sparse point cloud is forty times thinner per frame.

Panel of Figure 8 — Figure 8: Absolute rotation and position error per frame, after similarity alignment. Note the difference in vertical scale between the two objects. On both sequences the error is bounded and oscillates about a stable baseline rather than accumulating along the trajectory, indicating a globally consistent reconstruction. No frame fails to register on either object.

Object	AP $\uparrow$	AP ${}_{\text{MSSD}}$ $\uparrow$	AP ${}_{\text{MSPD}}$ $\uparrow$
Obj 01	0.77	0.71	0.84
Obj 02	0.83	0.75	0.92
Obj 03	0.83	0.76	0.91
Obj 04	0.48	0.38	0.58
Obj 05	0.75	0.69	0.80
Obj 06	0.65	0.60	0.69

Appendix

Table 11: Per-object Average Precision (AP) of NovaSplat on the HOPEv2 validation set, along with the MSSD and MSPD components.




Original	Ground Truth	NovaSplat (Ours)

Appendix

Figure 9: Impact of object symmetry on NovaSplat performance (HOPEv2, Object 23). Across three diverse viewpoints, the object’s near-symmetry causes NovaSplat to predict an inverted pose (

\sim 180

degrees error).

Figure 10: Example of a geometric inconsistency in HOPEv2. Left: onboarding image. Right: image from validation set. The cap orientation relative to the printed label differs between the two splits, resulting in a mismatch between the onboarding-derived reference geometry and the validation observation.

Panel of Figure 11 — Figure 11: Qualitative comparison on the HOPEv2 dataset. Bounding boxes highlight the predicted 6D poses.

Panel of Figure 12 — Figure 12: Qualitative comparison on the IndustryShapes Classic dataset. Bounding boxes highlight the predicted 6D poses.

Panel of Figure 13 — Figure 13: Qualitative comparison on the IndustryShapes Extended dataset. Bounding boxes highlight the predicted 6D poses.

Explore similar work

Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting
Sep 7, 2026Jie Lu, Hengtan Zhang, Li Gong +7Object Pose Estimation Gaussian Splatting
College of Future Information Technology, Fudan University, Shanghai, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. · Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku, 20014 Turku, Finland. +1
SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views
Sep 21, 2025Ranran Huang, Krystian Mikolajczyk3D Gaussian Splatting 3D Reconstruction
Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, United Kingdom
GS-CPE: Unified 6-Degree-of-Freedom Camera Pose Estimation via 3D Gaussian Splatting
Aug 11, 2026Huaiyuan Weng, Chul Min Yeum, Su-Min KangCamera Pose Estimation
Department of Civil and Environmental Engineering, University of Waterloo, 200 University Ave. W., Waterloo, ON N2L 3G1, Canada · School of Architecture, Soongsil University, 369 Sangdo-ro, Dongjak District, Seoul, South Korea

Sep 7, 2026, cs.ROJ/K move · Enter open · S save

Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting

Jie Lu, Hengtan Zhang, Li Gong, Pengpeng Wang+6

College of Future Information Technology, Fudan University, Shanghai, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. · Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku, 20014 Turku, Finland.+1

Object Pose Estimation Gaussian Splatting

Open paper AskPDF arXiv

Estimating the 6D pose of textureless objects without prior CAD models remains a critical challenge due to the lack of appearance features. While recent generalizable approaches alleviate the dependence on object-specific models, their performance on low-texture objects is often limited by insufficient geometric constraints in the underlying representations. In this work, we propose PG-Pose, a geometry-aware framework combining Planar-based Gaussian Splatting (PGS) reconstruction and Geometry-driven pose optimization. In the offline representation extraction stage, three distinct representations of the object are extracted from multi-view reference RGB images with known poses. PG-Pose reconstructs a 3D Gaussian representation and renders high-fidelity depth maps to generate 3D point clouds through back projection. In the online pose inference stage, the initial pose of the input image is estimated by 2D-3D correspondence matching between the input image and the reconstructed 3D point clouds, followed by a PGS-Refiner for iterative pose optimization. Evaluations on the OnePose-LowTexture datasets, PG-Pose achieves an average accuracy of 94.2% ADD(S)@0.1d, with a 2.1% improvement average accuracy compared with the state-of-the-art (SOTA) GS-based approach. To further demonstrate the effectiveness of PG-Pose for industrial robots in grasping tasks, we deploy it on a dual-arm industrial robot and successfully realize the grasping task on an unseen object.

Abstract

Explore similar work

Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting

SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views

GS-CPE: Unified 6-Degree-of-Freedom Camera Pose Estimation via 3D Gaussian Splatting