Robust estimation of object poses in robotic manipulation is often addressed using foundational general estimators, that aim to handle diverse error sources naively within a single model. Still, they struggle due to environmental uncertainties, while requiring long inference times and heavy computation. In contrast, we propose a modular, uncertainty-aware framework that attributes pose estimation errors to specific error sources and applies targeted mitigation strategies only when necessary. Instantiated with Iterative Closest Point (ICP) as a simple and lightweight pose estimator, we leverage our framework for real-world robotic grasping tasks. By decomposing pose estimation into failure detection, error attribution, and targeted recovery, we significantly improve the robustness of ICP and achieve competitive performance compared to foundation models, while relying on a substantially simpler and faster pose estimator.
Figures & tables
Fig. 1: Overview of our framework. After the scene is recorded, ICP provides an initial pose estimate. A grasp success predictor detects possible grasp failure, which queries an error attribution system. Based on the detected error, one of the mitigation strategies will be selected. In this example, the failure is attributed to structural noise in the point cloud. A point cloud reconstruction module recovers the clean point cloud, and the pose estimate is corrected, leading to a successful grasp. Since ICP is robust to surface noise, reconstructing the rough shape is sufficient in this example.
Fig. 2: Overview of the deployed framework. (A) A point cloud sampled from the object mesh is aligned to the recorded scene point cloud via ICP. (B) An MLP predicts grasp success based on the calculated transformation estimate and alignment metrics. (C) In the case of predicted grasp failure, a PointBERT [ 34 ] classifier attributes the failure to a specific source, here shown for noise. (D) Based on the classification, targeted mitigation strategies - BO-ICP [ 27 ] , the TGV-Planner [ 31 ] , and a custom point cloud reconstruction model - are applied to recover the pose estimate. In the illustrated case, the clean point cloud is reconstructed from the noisy point cloud and used for ICP alignment, which corrects the pose estimate.
Fig. 3: Point cloud reconstruction module. A transformer based encoder creates tokens from point patches of the noisy point cloud, while a DGCNN [ 20 ] encodes point-wise features of the mesh. From the point-wise mesh features, a subset is selected via FPS. The tokens from the noisy point cloud are merged with the selected point-wise mesh features by calculating a weighted sum with learned weights. An encoder predicts displacement vectors for each patch center point of the noisy point cloud, which is propagated to the surrounding points.
Fig. 4: Synthetic scenes showing the error cases created using SAPIEN [ 35 ] . For bad initialization, the initial pose hypothesis is shown in red.
Fig. 5: Overview of the real-world scenes used for evaluation and training. For each object, we show one example of each error case.
Fig. 6: Confusion matrix of the failure detection MLP evaluated on 108 grasping tasks.
Model
Mean Acc.
True Bad Init. Pred. Occlusion
True Occlusion Pred. Bad Init.
PointBERT
99.47%
1.02%
0.61%
DGCNN [ 17 ]
85.20%
20.60%
23.00%
TABLE I: Accuracy of uncertainty attribution and confusion rates of bad initialization (Bad Init.) and occlusion compared to the results in [ 17 ] .
Model
Mean Acc.
Std. Dev.
PointBERT
83.83%
2.27%
DGCNN
71.49%
2.55%
PointNet
50.64%
3.40%
TABLE II: Accuracy of error attribution on real-world scenes.
Error Case
ICP success
ICP&Mitigation success
FP success
Bad Init.
0%
55%
85%
Noise
15%
80%
30%
Occlusion
10%
70 %
70%
TABLE III: Grasp success comparison of our framework against FP [ 11 ] for each error case.
Faculty of Electrical Engineering, Czech Technical University in Prague, Czechia · Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague, Czechia