Smart glass is emerging as an useful device since it provides plenty of insights under hands-busy, eyes-on-task situations. To understand the context of the wearer, 6D object pose estimation in egocentric view is becoming essential. However, existing 6D object pose estimation benchmarks fail to capture the challenges of real-world egocentric applications, which are often dominated by severe motion blur, dynamic illumination, and visual obstructions. This discrepancy creates a significant gap between controlled lab data and chaotic real-world application. To bridge this gap, we introduce EgoXtreme, a new large-scale 6D pose estimation dataset captured entirely from an egocentric perspective. EgoXtreme features three challenging scenarios - industrial maintenance, sports, and emergency rescue - designed to introduce severe perceptual ambiguities through extreme lighting, heavy motion blur, and smoke. Evaluations of state-of-the-art generalizable pose estimators on EgoXtreme indicate that their generalization fails to hold in extreme conditions, especially under low light. We further demonstrate that simply applying image restoration (e.g., deblurring) offers no positive improvement for extreme conditions. While performance gain has appeared in tracking-based approach, implying using temporal information in fast-motion scenarios is meaningful. We conclude that EgoXtreme is an essential resource for developing and evaluating the next generation of pose estimation models robust enough for real-world egocentric vision. The dataset and code are available at https://taegyoun88.github.io/EgoXtreme/
Figures & tables
Figure 2: Visualization of ground truth 6D pose annotations from our dataset, arranged sequentially from top-left to bottom-right. The frames include the sports scenario (normal and middle light), the industrial maintenance scenario (low light, flashlight with smoke, and headlight), and the emergency rescue scenario (warning light, exit green light, and high light with smoke).
Extreme Condition
Dataset
Instance
Speed
Light
Smoke
Frames
Egocentric
Subjects
Objects
Annotation
LM [ 15 ]
No
-
1
No
18.2k
No
-
15
RGB-D
IC-BIN [ 46 ]
Yes
-
1
No
177
No
-
2
RGB-D
T-LESS [ 16 ]
No
-
1
No
49k
No
-
30
RGB-D
YCB-V [ 46 ]
No
Slow
1
No
0.1M
No
-
21
RGB-D
TUD-L [ 17 ]
No
-
8
No
62k
No
-
3
RGB-D
Table 1: Datasets for object pose estimations.
Scenario
Lighting condition
Smoke
Object
Speed (m/s)
Standard
Extreme
Camera
Object
normal
middle
high
low
head
flash
warning
green
Maintenance
✔
✔
✔
✔
✔
✔
✔
5
0.03
0.09
Sports
✔
✔
✔
✔
5
0.40
1.37
Emergency
✔
✔
✔
✔
✔
✔
✔
3
0.47
0.10
Table 2: EgoXtreme datasets configuration
Figure 3: 3D models. This image shows the 13 object models in the EgoXtreme dataset. From top to bottom five maintenance scenarios, five sports scenarios and three emergency scenarios.
Figure 4: Diagram for data collection.
Scenario
Light
Smoke
FoundPose [ 36 ]
GigaPose [ 34 ]
PicoPose [ 28 ]
0.1d
0.2d
0.3d
MSSD
MSPD
0.1d
0.2d
0.3d
MSSD
MSPD
0.1d
0.2d
0.3d
MSSD
MSPD
Sports
Standard
0.53
1.55
4.72
0.59
7.65
4.12
11.77
24.64
5.21
10.61
3.13
9.48
24.61
5.00
12.87
Extreme
0.18
0.78
2.42
0.23
6.87
3.11
9.19
19.04
4.15
8.31
1.80
6.61
17.86
3.07
10.10
Maintenance
Standard
21.02
30.53
37.61
12.94
18.78
33.64
48.84
62.77
15.97
24.17
39.27
62.42
76.84
24.76
33.20
Extreme
13.78
22.94
30.03
9.92
16.40
19.78
32.20
45.52
10.44
19.89
26.44
47.18
64.09
18.37
28.12
Standard
✔
14.44
22.49
30.00
8.23
13.11
23.01
37.87
52.86
12.24
19.83
26.37
46.05
59.87
18.26
25.48
Table 3: 6D object pose estimation on EgoXtreme . Performance is evaluated using ADD(S) recall at thresholds of 0.1d , 0.2d , 0.3d ( ↑ ) and spatial accuracy metrics MSSD/MSPD ( ↓ ), where d denotes the object diameter.
Figure 5: Example 6D Pose estimation results on baseline models. The red line is prediction and green is GT. (a), (b), and (c) are the industry maintenance, sports, and emergency rescue scenarios, respectively. The top row indicates standard light condition, and the bottom row indicates extreme light condition.
Scenario
Deblur
Dehaze
Light enhance
PicoPose
0.1d
0.2d
0.3d
Sports
2.81
8.80
23.02
✔
2.61
8.52
22.24
✔
2.87
9.05
22.05
✔
✔
2.58
8.22
20.99
Maintenance
28.22
48.51
63.32
Table 4: 6D object pose estimation with pre-processing for PicoPose .
Figure 6: Example 6D Pose estimation results with preprocessing. The top row shows the original, non-preprocessed images. The bottom row displays the corresponding images after applying specific preprocessing: deblurring (left), light enhancement (middle), and dehazing (right).
Object
Method
GigaPose
0.1d
0.2d
0.3d
Pingpong
Per-frame
0.53
2.97
17.63
Direct
0.05
0.35
6.58
Fusion
0.56
1.57
10.81
Hybrid
0.49
1.52
16.53
Tennis
Per-frame
6.77
34.71
50.91
Table 5: 6D object pose tracking for GigaPose . Applied to sports normal scenario.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Project Aria. RGB capture device.
Figure A2 : Test bed. Motion capture systems.
Figure B1 : Visualization of temporal dynamics in the EgoXtreme dataset.
Table C2 : Additional baseline results on the Tennis sequence.
Object
Method
GoTrack(GigaPose)
0.1d
0.2d
0.3d
Pingpong
Per-frame
1.42
3.51
17.33
Direct
0.37
0.85
4.95
Tennis
Per-frame
13.81
35.17
44.29
Direct
7.14
9.74
11.67
Bat
Per-frame
14.43
29.78
46.66
Appendix
Table D1 : 6D object pose tracking using GoTrack.
Object
Method
Lighting condition
Standard
Extreme
0.1d
0.2d
0.3d
0.1d
0.2d
0.3d
Pingpong
Per-frame
2.17
5.36
17.84
2.77
5.94
15.15
Direct
0.33
1.25
5.22
0.36
1.03
2.01
Fusion
0.76
2.61
11.43
0.92
2.30
8.34
Hybrid
2.44
4.59
19.07
3.32
5.84
16.51
Appendix
Table D2 : 6D object pose tracking for GigaPose.
Figure D1 : Visualization of pose tracking performance. The panels, arranged from left to right, showcase the results of the per-frame baseline, direct temporal, fusion temporal, and hybrid temporal methods.
Scenario
Conditions
Pre-processing
PicoPose
Light
Smoke
Deblur
Dehaze
Light enhance
0.1d
0.2d
0.3d
Sports
Standard
3.13
9.48
24.61
✔
2.77
9.12
24.42
✔
3.18
9.77
23.49
✔
✔
2.81
8.91
23.12
Sports
Extreme
1.80
6.61
17.86
Appendix
Table E1 : 6D object pose estimation with pre-processing under conditions .
Figure E1 : Visualization of 6D Pose estimation results with preprocessing on sports scenario.
Figure E2 : Visualization of 6D Pose estimation results with preprocessing on maintenance scenario.
Figure E3 : Visualization of 6D Pose estimation results with preprocessing on emergency scenario.