Smart glass is emerging as an useful device since it provides plenty of insights under hands-busy, eyes-on-task situations. To understand the context of the wearer, 6D object pose estimation in egocentric view is becoming essential. However, existing 6D object pose estimation benchmarks fail to capture the challenges of real-world egocentric applications, which are often dominated by severe motion blur, dynamic illumination, and visual obstructions. This discrepancy creates a significant gap between controlled lab data and chaotic real-world application. To bridge this gap, we introduce EgoXtreme, a new large-scale 6D pose estimation dataset captured entirely from an egocentric perspective. EgoXtreme features three challenging scenarios - industrial maintenance, sports, and emergency rescue - designed to introduce severe perceptual ambiguities through extreme lighting, heavy motion blur, and smoke. Evaluations of state-of-the-art generalizable pose estimators on EgoXtreme indicate that their generalization fails to hold in extreme conditions, especially under low light. We further demonstrate that simply applying image restoration (e.g., deblurring) offers no positive improvement for extreme conditions. While performance gain has appeared in tracking-based approach, implying using temporal information in fast-motion scenarios is meaningful. We conclude that EgoXtreme is an essential resource for developing and evaluating the next generation of pose estimation models robust enough for real-world egocentric vision. The dataset and code are available at https://taegyoun88.github.io/EgoXtreme/
Figures & tables
Figure 2: Visualization of ground truth 6D pose annotations from our dataset, arranged sequentially from top-left to bottom-right. The frames include the sports scenario (normal and middle light), the industrial maintenance scenario (low light, flashlight with smoke, and headlight), and the emergency rescue scenario (warning light, exit green light, and high light with smoke).
Extreme Condition
Dataset
Instance
Speed
Light
Smoke
Frames
Egocentric
Subjects
Objects
Annotation
LM [ 15 ]
No
-
1
No
18.2k
No
-
15
RGB-D
IC-BIN [ 46 ]
Yes
-
1
No
177
No
-
2
RGB-D
T-LESS [ 16 ]
No
-
1
No
49k
No
-
30
RGB-D
YCB-V [ 46 ]
No
Slow
1
No
0.1M
No
-
21
RGB-D
TUD-L [ 17 ]
No
-
8
No
62k
No
-
3
RGB-D
Table 1: Datasets for object pose estimations.
Scenario
Lighting condition
Smoke
Object
Speed (m/s)
Standard
Extreme
Camera
Object
normal
middle
high
low
head
flash
warning
green
Maintenance
✔
✔
✔
✔
✔
✔
✔
5
0.03
0.09
Sports
✔
✔
✔
✔
5
0.40
1.37
Emergency
✔
✔
✔
✔
✔
✔
✔
3
0.47
0.10
Table 2: EgoXtreme datasets configuration
Figure 3: 3D models. This image shows the 13 object models in the EgoXtreme dataset. From top to bottom five maintenance scenarios, five sports scenarios and three emergency scenarios.
Figure 4: Diagram for data collection.
Scenario
Light
Smoke
FoundPose [ 36 ]
GigaPose [ 34 ]
PicoPose [ 28 ]
0.1d
0.2d
0.3d
MSSD
MSPD
0.1d
0.2d
0.3d
MSSD
MSPD
0.1d
0.2d
0.3d
MSSD
MSPD
Sports
Standard
0.53
1.55
4.72
0.59
7.65
4.12
11.77
24.64
5.21
10.61
3.13
9.48
24.61
5.00
12.87
Extreme
0.18
0.78
2.42
0.23
6.87
3.11
9.19
19.04
4.15
8.31
1.80
6.61
17.86
3.07
10.10
Maintenance
Standard
21.02
30.53
37.61
12.94
18.78
33.64
48.84
62.77
15.97
24.17
39.27
62.42
76.84
24.76
33.20
Extreme
13.78
22.94
30.03
9.92
16.40
19.78
32.20
45.52
10.44
19.89
26.44
47.18
64.09
18.37
28.12
Standard
✔
14.44
22.49
30.00
8.23
13.11
23.01
37.87
52.86
12.24
19.83
26.37
46.05
59.87
18.26
25.48
Table 3: 6D object pose estimation on EgoXtreme . Performance is evaluated using ADD(S) recall at thresholds of 0.1d , 0.2d , 0.3d ( ↑ ) and spatial accuracy metrics MSSD/MSPD ( ↓ ), where d denotes the object diameter.
Figure 5: Example 6D Pose estimation results on baseline models. The red line is prediction and green is GT. (a), (b), and (c) are the industry maintenance, sports, and emergency rescue scenarios, respectively. The top row indicates standard light condition, and the bottom row indicates extreme light condition.
Scenario
Deblur
Dehaze
Light enhance
PicoPose
0.1d
0.2d
0.3d
Sports
2.81
8.80
23.02
✔
2.61
8.52
22.24
✔
2.87
9.05
22.05
✔
✔
2.58
8.22
20.99
Maintenance
28.22
48.51
63.32
Table 4: 6D object pose estimation with pre-processing for PicoPose .
Figure 6: Example 6D Pose estimation results with preprocessing. The top row shows the original, non-preprocessed images. The bottom row displays the corresponding images after applying specific preprocessing: deblurring (left), light enhancement (middle), and dehazing (right).
Object
Method
GigaPose
0.1d
0.2d
0.3d
Pingpong
Per-frame
0.53
2.97
17.63
Direct
0.05
0.35
6.58
Fusion
0.56
1.57
10.81
Hybrid
0.49
1.52
16.53
Tennis
Per-frame
6.77
34.71
50.91
Table 5: 6D object pose tracking for GigaPose . Applied to sports normal scenario.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Project Aria. RGB capture device.
Figure A2 : Test bed. Motion capture systems.
Figure B1 : Visualization of temporal dynamics in the EgoXtreme dataset.
Table C2 : Additional baseline results on the Tennis sequence.
Object
Method
GoTrack(GigaPose)
0.1d
0.2d
0.3d
Pingpong
Per-frame
1.42
3.51
17.33
Direct
0.37
0.85
4.95
Tennis
Per-frame
13.81
35.17
44.29
Direct
7.14
9.74
11.67
Bat
Per-frame
14.43
29.78
46.66
Appendix
Table D1 : 6D object pose tracking using GoTrack.
Object
Method
Lighting condition
Standard
Extreme
0.1d
0.2d
0.3d
0.1d
0.2d
0.3d
Pingpong
Per-frame
2.17
5.36
17.84
2.77
5.94
15.15
Direct
0.33
1.25
5.22
0.36
1.03
2.01
Fusion
0.76
2.61
11.43
0.92
2.30
8.34
Hybrid
2.44
4.59
19.07
3.32
5.84
16.51
Appendix
Table D2 : 6D object pose tracking for GigaPose.
Figure D1 : Visualization of pose tracking performance. The panels, arranged from left to right, showcase the results of the per-frame baseline, direct temporal, fusion temporal, and hybrid temporal methods.
Scenario
Conditions
Pre-processing
PicoPose
Light
Smoke
Deblur
Dehaze
Light enhance
0.1d
0.2d
0.3d
Sports
Standard
3.13
9.48
24.61
✔
2.77
9.12
24.42
✔
3.18
9.77
23.49
✔
✔
2.81
8.91
23.12
Sports
Extreme
1.80
6.61
17.86
Appendix
Table E1 : 6D object pose estimation with pre-processing under conditions .
Figure E1 : Visualization of 6D Pose estimation results with preprocessing on sports scenario.
Figure E2 : Visualization of 6D Pose estimation results with preprocessing on maintenance scenario.
Figure E3 : Visualization of 6D Pose estimation results with preprocessing on emergency scenario.
We present DeSOPE, a large-scale dataset for 6DoF deformed objects. Most 6D object pose methods assume rigid or articulated objects, an assumption that fails in practice as objects deviate from their canonical shapes due to wear, impact, or deformation. To model this, we introduce the DeSOPE dataset, which features high-fidelity 3D scans of 26 common object categories, each captured in one canonical state and three deformed configurations, with accurate 3D registration to the canonical mesh. Additionally, it features an RGB-D dataset with 133K frames across diverse scenarios and 665K pose annotations produced via a semi-automatic pipeline. We begin by annotating 2D masks for each instance, then compute initial poses using an object pose method, refine them through an object-level SLAM system, and finally perform manual verification to produce the final annotations. We evaluate several object pose methods and find that performance drops sharply with increasing deformation, suggesting that robust handling of such deformations is critical for practical applications.
Zhiqiang Liu, Rui Song, Duanmu Chuangqi +3
State Key Laboratory of ISN, Xidian University · MagicLeap
While current 6D pose estimation benchmarks have reached near-saturation on household objects, they often fail to capture the stochastic and optical complexities of industrial environments. We introduce XYZ-IBD, a high-precision benchmark for object detection and 6D pose estimation specifically designed for industrial bin-picking. XYZ-IBD addresses the domain gap by providing 75 multi-view real-world scenes containing approximately 273k annotated instances of metallic, symmetrical, and specular objects. Unlike existing datasets, our benchmark features high-density stochastic stacking and multi-instance ambiguity, reflecting authentic robotic manipulation challenges. We employ a rigorous multi-stage and semi-automatic annotation pipeline, ensuring sub-millimeter annotation accuracy. The annotations are validated through our designed error quantification scheme, securing the reliability of the annotation quality. In addition to real-world evaluation data, we provide a large-scale complementary synthetic training set that is rendered under a realistic bin-picking simulation. Benchmarking state-of-the-art (SOTA) methods for 2D detection and 6D pose estimation reveals a significant performance degradation compared to standard household benchmarks, highlighting the unsolved challenges of industrial vision. XYZ-IBD establishes a new frontier for robust pose estimation in complex, high-occlusion, and reflective scenarios. The dataset and benchmark are publicly available at https://xyz-ibd.github.io.
Junwen Huang, Jiaqi Hu, Peter KT Yu +3
Technical University of Munich · Munich Center for Machine Learning (MCML) · XYZ Robotics +2
In many practical 6D object pose estimation scenarios, we often have access to only a single real-world RGB-D reference view per object, typically without CAD models. Existing methods largely rely on explicit 3D models or multi-view data, which limits their scalability. To address this challenging single-reference model-free setting, we propose OneViewAll, a semantic-prior-guided framework that performs pose estimation via a novel Project-and-Compare paradigm. Instead of relying on computationally expensive CAD-based rendering, our method directly aligns reference and query observations within a projection-equivariant space. OneViewAll progressively integrates hierarchical semantic priors across three levels: (1) category- and scene-level priors for efficient hypothesis initialization; (2) object-level symmetry priors for geometry completion via mirror fusion; and (3) patch-level priors for discriminative refinement. Extensive experiments demonstrate that OneViewAll achieves 92.5% ADD-0.1 accuracy on the LINEMOD dataset using only one real reference view -- significantly outperforming the CVPR 2025 baseline One2Any (52.6%). It also yields consistent improvements on YCB-V, Real275, and Toyota-Light while maintaining low inference latency. Our results underscore the efficacy of symmetry-aware projection in handling symmetric, texture-less, and occluded objects.