We present DeSOPE, a large-scale dataset for 6DoF deformed objects. Most 6D object pose methods assume rigid or articulated objects, an assumption that fails in practice as objects deviate from their canonical shapes due to wear, impact, or deformation. To model this, we introduce the DeSOPE dataset, which features high-fidelity 3D scans of 26 common object categories, each captured in one canonical state and three deformed configurations, with accurate 3D registration to the canonical mesh. Additionally, it features an RGB-D dataset with 133K frames across diverse scenarios and 665K pose annotations produced via a semi-automatic pipeline. We begin by annotating 2D masks for each instance, then compute initial poses using an object pose method, refine them through an object-level SLAM system, and finally perform manual verification to produce the final annotations. We evaluate several object pose methods and find that performance drops sharply with increasing deformation, suggesting that robust handling of such deformations is critical for practical applications.
Figures & tables
Figure 2: Comparison of 6D object pose datasets. The first row shows two examples from an instance-level dataset [ 4 ] , where each object instance is associated with its own 3D model, under the assumption that the object is perfectly rigid and does not deform over time. The second row depicts a category-level dataset [ 7 ] , in which multiple instances of the same category share a single 3D model (rightmost). The third row shows the canonical instance from the proposed DeSOPE dataset along with three deformed versions, while the fourth row presents their corresponding 3D meshes. We visualize meshes as black-bordered images on a white background in this figure.
Type
Categories
Instances
Images
LINEMOD [ 4 ]
Instance
-
15
19K
YCBV [ 7 ]
Instance
-
21
80K
PhoCaL [ 46 ]
Instance
8
60
3K
Wild6D [ 17 ]
Instance
5
162
10K
Objectron [ 1 ]
Instance
9
17K
4M
CO3D [ 37 ]
Instance
5
19K
1.5M
Table 1: Comparison of 6D object pose datasets. Most instance-level datasets focus on individual instances, assuming objects are perfectly rigid and treating each instance independently. Category-level datasets, on the other hand, provide a single canonical mesh representing all instances within a semantic category. The proposed DeSOPE dataset captures multiple deformation states of the same instance, offering accurate 3D registration between the deformed state and the canonical form, along with precise 6D object pose annotations in images captured across diverse scenes.
Figure 3: Overview of the dataset generation framework. The framework consists of four main steps: Object Scanning , which acquires the canonical mesh of objects along with multiple deformed states of the same instance using a high-precision 3D scanner; Model Alignment , beginning with coarse manual alignment and followed by flow-driven 3D registration [ 20 ] ; Video Capture , which records RGB-D videos of objects across diverse scenes with a stereo camera; and Pose Annotation , which performs initial object labeling and iteratively refines poses using implicit neural networks to obtain accurate annotations.
Figure 4: Example of 3D model alignment. We estimate the optimal registration between each deformed mesh and its corresponding canonical mesh through a two-stage process. We begin with a manual alignment to obtain a coarse initialization, providing a reasonable starting point for further refinement. This initial estimate is then refined based on dense 2D matching computed from six orthogonal viewpoints. The resulting error map visualizes the pixel-wise differences between the canonical mesh and the aligned deformed mesh when projected from the same viewpoints, both before and after refinement.
Figure 5: Statistical analysis of the DeSOPE dataset. We summarize the different properties of our dataset using histograms and report percentages (%) on the y-axis across all subplots. (a) Distribution of camera pose angles (x-axis: rotation angle in degrees), illustrating the coverage of pitch, roll, and yaw across all annotated frames. (b) Distribution of object-to-camera distances (x-axis: distance in cm), with values concentrated around 50-60 cm. (c) Distribution of physical dimensions for 104 object instances across 26 categories (x-axis: size in cm), demonstrating the diversity in object sizes. (d) Distribution of deformation severity (x-axis: deformation magnitude in cm) across three levels: mild for Deformed 1, moderate for Deformed 2, and severe for Deformed 3. The deformation magnitude is computed as the average point-wise 3D distance between corresponding vertices after aligning the deformed and undeformed meshes.
Figure 6: Effect of pose refinement. We visualize the predicted pose by overlaying the rendered textured mesh onto the input image according to the estimated object pose. The initial pose exhibits noticeable misalignment; after applying our pose refinement strategy, the rendered mesh aligns much more accurately with the input image.
Figure 7: Example of captured images and pose annotations. The green boundary contours represent pose projections onto the 2D plane using the corresponding mesh, as obtained by the annotation algorithm proposed in this paper. The dataset contains images captured in cluttered scenes under both human-manipulated and non-manipulated conditions.
Deformed 1
Deformed 2
Deformed 3
Init.
+Refine
Init.
+Refine
Init.
+Refine
0.473
0.261
0.938
0.440
1.177
1.010
0.657
0.191
1.050
0.511
1.306
1.253
1.922
1.129
2.399
1.358
1.291
1.091
0.731
0.297
0.967
0.577
0.980
0.843
0.855
0.498
1.098
0.576
0.672
0.690
Table 2: Results of 3D model alignment. “Init.” denotes the matching error obtained from manual alignment between deformed and undeformed objects, while “+Refine” indicates the error after refinement. The six middle rows report the alignment errors for each of the six projected views, and the final row summarizes the overall alignment error. The error is reported in centimeters after RANSAC outlier removal with a threshold of 3 cm.
SCFlow2
FoundationPose
GenPose
Canonical
0.82
0.78
0.67
Deformed 1
0.67
0.58
0.56
Deformed 2
0.43
0.38
0.36
Deformed 3
0.23
0.24
0.31
Canonical
0.77
0.72
0.61
Deformed 1
0.64
0.54
0.53
Table 3: Evaluation of state-of-the-art methods on DeSOPE. We report results in two groups: the first is evaluated on all the images without human manipulation, and the second on images with human manipulation. We report the Average Recall (AR) for three representative methods: SCFlow2 [ 47 ] , FoundationPose [ 48 ] , and GenPose [ 53 ] . Although these methods generalize well, their performance degrades significantly when the meshes in the images do not match the canonical mesh assumed by the model. The more severe the deformation, the worse their performance.
Figure 8: State-of-the-art methods on DeSOPE. Most methods achieve strong performance on images with canonical meshes (first row). However, their accuracy degrades significantly when the meshes undergo deformations that deviate from the canonical configuration (second row), as they assume the target in the image still conforms to the canonical mesh, which is not the case. Pose estimation results are projected onto the 2D plane using the corresponding mesh. Color code: green—Ground Truth; red—GenPose; pink—FoundationPose; cyan—SCFlow2.
Figure 9: Performance analysis. All plots present Average Recall (AR) across four mesh sets (Canonical and Deformed 1–3) for three methods: SCFlow2, FoundationPose (FPose), and GenPose. Key observations: (1) performance decreases as deformation severity increases across all settings; (2) greater occlusion leads to lower performance; (3) scenes with human manipulation consistently perform worse due to complex motion and occlusions during interaction.
Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/
Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy
Object pose estimation is a fundamental problem in 3D vision. Although recent state-of-the-art approaches achieve strong performance, generalization to novel categories and unseen scenes remains challenging. We propose UniPose9D, a unified model for category-agnostic 9D object pose estimation: given an instance mask/ROI and either an RGB-D observation or an RGB image with predicted depth, the model estimates rotation, translation, and metric size without category labels, CAD models, mean-shape priors, or reference views. Specifically, UniPose9D samples point pairs from the observed object geometry and uses DINOv2 and PointNet features to predict NOCS coordinates for each pair. To improve accuracy, we introduce a point-pair-based RANSAC N-hop Kabsch-Umeyama algorithm with an adaptive threshold. We further employ flow matching to address symmetric ambiguities and construct a large-scale training set by curating and aligning pose annotations from existing public datasets. Experiments across eight datasets show that a single unified model achieves competitive performance on standard benchmarks while generalizing to unseen objects, unseen categories, and in-the-wild scenarios. Our code and model are available at https://github.com/qq456cvb/UniPose9D.
Estimating the 9D pose of everyday objects from a single real-world image remains challenging. This is largely due to the lack of large-scale supervision. Most existing datasets either rely heavily on synthetic renderings or provide limited coverage of real-world objects: the largest real-world 9D pose dataset to date contains only 17K annotated objects across 9 categories. We address this gap with Every9D-21M, a dataset of 9D pose annotations for 21.8M real-world images from 109K object- centric videos spanning 700 everyday object categories - two orders of magnitude larger than prior real-world 9D pose benchmarks in both image and category count. To achieve this scale, we leverage object-centric videos by reconstructing object- level point clouds via multi-view geometry and aligning similar instances into a shared canonical coordinate frame. Canonical poses are manually annotated for only a small set of reference objects (fewer than 0.01% of all images) and propagated to the remaining instances via cross-instance alignment. All propagated canonical poses are then verified from multiple viewpoints. We further introduce cross-category orientation rules that induce category-level symmetries, enabling symmetry-aware evaluation. Beyond establishing dedicated training and evaluation splits as a benchmark for 9D pose foundation models, we show that training on Every9D-21M improves performance on ImageNet3D and PASCAL3D+, and generalizes to HANDAL substantially better than training on ImageNet3D. Data and code are available at https://github.com/GenIntel/Every9D.
Leonhard Sommer, Emil Akopyan, Adam Kortylewski
University of Freiburg · CISPA Helmholtz Center for Information Security