Recent advances on 6D object pose estimation have achieved high performance on representative benchmarks such as LM-O, YCB-V, and T-Less. These datasets capture natural-occurring variations in illumination and camera setup but do not systematically explore the effects of controllable sensor configurations, such as exposure, gain, and depth-sensor modes. To bridge this gap, we introduce SenseShift6D, the first RGB-D object-pose benchmark that physically sweeps 13 RGB exposures, 9 RGB gains, auto-exposure, 4 depth-capture modes, and 5 illumination levels. For six common household objects, we acquire 198.8k RGB and 20.0k depth images (i.e., 795.4k RGB-D scenes), providing 1,380 unique sensor-lighting permutations per object pose. Experiments with state-of-the-art pretrained, generalizable pose estimators reveal substantial performance variation across lighting and sensor settings, despite their large-scale pretraining. Strikingly, even instance-level estimators-trained and tested on identical objects and backgrounds-exhibit pronounced sensitivity to environmental and sensor shifts. These findings establish sensor- and environment-aware robustness as an underexplored yet essential dimension of 6D pose estimation. To illustrate the opportunity enabled by this benchmark, we evaluate test-time sensor selection without retraining. An idealized controller yields remarkable gains of up to +16.2 pp for generalizable models, whereas a practical consistency-based proxy provides only limited improvements, highlighting substantial headroom and the need for future research on reliable sensor-aware adaptation. SenseShift6D is available at huggingface.co/datasets/Yegyu/SenseShift6D.
Figures & tables
Figure 1: RGB sample images under auto exposure and under all combinations of exposure and gain settings for brightness 5%, 25%, 75% and 100%. Rows indicate gain levels and columns indicate exposure levels.
Figure 2: Depth sample images under four depth capture modes: default, high accuracy, high density, and medium density.
Dataset
Environmental and sensor variations
Illumination
Exposure
Gain
Depth
Total Var.
LM [ 15 ]
×
×
×
×
×
LM-O [ 4 ]
×
×
×
×
×
T-LESS [ 16 ]
×
×
×
×
×
YCB-V [ 42 ]
×
×
×
×
×
TUD-L [ 17 ]
8 options
×
×
×
8
Table 1: Comparison of 6D pose estimation datasets.
Figure 4
Figure 4: Six household objects collected in SenseShift6D.
Table 3: AUC@[0:0.1] performance of various sensor control methods for unseen object pose estimation models
Figure 5: Overall standard deviation of AUC computed across all brightness and scenes evaluated under Baseline, Oracle-Fixed, Oracle-Dynamic.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
100.0
96.10
54.61
98.54
100.0
96.59
52.02
99.02
100.0
99.51
59.88
100.0
PBR + Train-Def w/ Aug
95.12
93.66
64.98
99.51
100.0
98.54
71.88
100.0
100.0
100.0
89.76
100.0
PBR + Train-Var
97.56
97.07
79.24
97.56
97.56
97.56
85.93
98.54
100.0
100.0
100.0
100.0
Pringles
PBR + Train-Def
90.48
84.29
43.31
95.71
69.05
60.95
35.64
89.05
100.0
99.52
58.71
100.0
Table 4: AR@5 performance of various sensor control methods under different training and test settings of SenseShift6D for instance-level pose estimation models: ZebraPose, GDRNPP, HiPose.
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Overall
74.66
77.22
82.68
64.74
84.73
(+2.44)
(+8.02)
(-9.92)
(+10.07)
Table 5: Overall AUC@[0:0.1] performance of various sensor control methods for HiPose, trained on PBR + Train-Def and tested on Test-Var.
Table 6: Performance comparison under different sensor selection strategies. Values in parentheses denote ADD-S variant. Supervised models are trained under the PBR + Train-Def setting.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Table 7: AUC@[0:0.1] performance of various sensor control methods for unseen object pose estimation models given CNOS mask
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Spray
81.17
86.05
86.84
78.71
89.06
(+4.89)
(+5.67)
(-2.46)
(+7.89)
Pringles
61.84
71.98
73.80
55.85
80.01
(+10.13)
(+11.96)
(-6.00)
(+18.17)
Tincase
63.85
75.66
75.59
60.02
81.71
(+11.81)
(+11.74)
(-3.83)
(+17.86)
Appendix
Table 8: AUC@[0:0.1] performance of SAM-6D with its Image Segmentation Model (ISM).
Augmentation Type
Activation Probability
Parameters / Range
Gaussian Blur
0.4
σ∈[0.0,3.0]
Sharpness Enhancement
0.3
factor ∈[0.0,50.0]
Contrast Enhancement
0.3
factor ∈[0.2,50.0]
Brightness Enhancement
0.5
factor ∈[0.1,6.0]
Color Enhancement
0.3
factor ∈[0.0,20.0]
Additive Intensity
0.5
value ∈[−25,25] , per-channel = 0.3
Appendix
Table 11: Augmentation operations and parameters.
Figure 6: Examples of RGB augmentations applied during training. The top-left image is the original, and the remaining images are randomly augmented samples generated using the settings summarized in Tab. 11 .
Figure 7: Sample of marker-removed test images.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
96.59
95.12
64.33
100
100
96.59
52.49
99.51
100
98.73
59.98
100
PBR + Train-Def w/ Aug
93.66
93.17
70.71
100
95.12
94.63
71.63
100
100
100
91.66
100
PBR + Train-Var
98.05
97.07
78.91
100
100
100
81.89
100
100
100
100
100
Tincase
PBR + Train-Def
94.22
86.22
55.45
99.11
64.44
58.66
35.07
84.89
97.78
84.62
56.33
97.78
Appendix
Table 12: AR@5 comparison of sensor control methods on marker-removed SenseShift6D with ZebraPose, GDRNPP, and HiPose.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
100.0
97.56
55.95
99.51
100.0
95.12
50.76
98.54
100.0
97.07
56.49
100.0
PBR + Train-Def w/ Aug
97.56
89.78
66.95
100.00
97.56
97.07
76.10
99.51
100.0
100.0
98.71
100.0
PBR + Train-Var
95.12
90.73
73.85
97.56
95.12
95.12
87.61
95.12
100.0
100.0
99.73
100.0
Tincase
PBR + Train-Def
97.78
96.10
50.89
98.67
93.33
82.22
47.60
94.67
97.33
82.49
53.82
96.44
Appendix
Table 13: AR@5 comparison of sensor control methods on marker-removed SenseShift6D with ZebraPose, GDRNPP, and HiPose trained on PBR + marker-removed Train-Def setting.
Figure 8: Average AUC by brightness level across exposure, gain, and depth capture modes for pretrained models: GigaPose, SAM-6D, and FoundationPose.
Figure 9: Average AUC by brightness level across exposure, gain, and depth capture modes for supervised models: ZebraPose, GDRNPP, and HiPose.
Figure 10: Comparison of predictions under Baseline and Oracle for GigaPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 11: Comparison of predictions under Baseline and Oracle for SAM-6D. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 12: Comparison of predictions under Baseline and Oracle for FoundationPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 13: Comparison of predictions under Baseline and Oracle for HiPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
83.17
85.07 (+1.90)
87.10 (+3.93)
86.07 (+2.90)
88.63 (+5.46)
Pringles
72.40
75.45 (+3.05)
78.60 (+6.19)
77.90 (+5.50)
81.12 (+8.71)
Tincase
32.96
43.00 (+10.04)
78.80 (+45.84)
76.69 (+43.73)
81.31 (+48.36)
Sandwich
66.85
69.85 (+3.00)
80.72 (+13.87)
78.82 (+11.97)
83.18 (+16.33)
Mouse
35.29
39.88 (+4.59)
64.85 (+29.56)
62.93 (+27.63)
69.88 (+34.59)
Duck
5.93
12.05 (+6.12)
79.16 (+73.23)
76.14 (+70.21)
81.44 (+75.51)
Appendix
Table 14: AUC@[0:0.1] performance under diverse environments of various sensor control methods for HiPose trained on PBR + Train-Def.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
65.02
71.00 (+5.98)
70.22 (+5.20)
66.85 (+1.83)
75.54 (+10.51)
Pringles
22.95
42.05 (+19.10)
42.93 (+19.98)
26.79 (+3.83)
64.60 (+41.64)
Tincase
65.98
78.67 (+12.69)
71.69 (+5.71)
74.07 (+8.09)
80.27 (+14.29)
Sandwich
59.97
69.74 (+9.77)
68.77 (+8.79)
59.97 (+0.00)
80.31 (+20.33)
Mouse
44.05
55.17 (+11.12)
62.73 (+18.68)
56.32 (+12.27)
72.78 (+28.73)
Duck
72.88
77.56 (+4.67)
75.67 (+2.79)
72.44 (-0.44)
80.30 (+7.42)
Appendix
Table 15: AUC@[0:0.1] performance under diverse environments of various sensor control methods for GigaPose.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
84.54
86.83 (+2.29)
84.22 (-0.32)
86.59 (+2.05)
89.22 (+4.68)
Pringles
50.17
71.45 (+21.28)
69.14 (+18.97)
65.76 (15.59)
80.48 (+30.31)
Tincase
47.61
66.73 (+19.12)
77.96 (+30.34)
68.61 (+21.00)
80.67 (+33.05)
Sandwich
78.36
79.00 (+0.64)
80.10 (+1.74)
78.41 (+0.05)
80.85 (+2.49)
Mouse
52.41
64.83 (+12.41)
69.12 (+16.71)
59.54 (+7.13)
74.40 (+21.99)
Duck
76.93
81.42 (+4.49)
84.56 (+7.63)
79.21 (+2.28)
87.91 (10.98)
Appendix
Table 16: AUC@[0:0.1] performance under diverse environments of various sensor control methods for SAM-6D.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
82.80
84.68 (+1.88)
85.34 (+2.54)
83.44 (+0.64)
87.17 (+4.37)
Pringles
25.07
39.45 (+14.38)
45.26 (+20.19)
32.98 (+7.91)
64.10 (+39.03)
Tincase
26.47
49.11 (+22.64)
53.64 (27.18)
38.29 (+11.82)
74.24 (+47.78)
Sandwich
81.62
84.41 (+2.79)
83.38 (+1.76)
82.77 (+1.15)
86.28 (+4.66)
Mouse
67.88
72.80 (+4.92)
75.10 (+7.22)
72.02 (+4.14)
79.54 (+11.66)
Duck
82.58
85.21 (+2.63)
87.88 (+5.30)
84.07 (+1.49)
89.74 (+7.16)
Appendix
Table 17: AUC@[0:0.1] performance under diverse environments of various sensor control methods for FoundationPose.
Table 18: Correlation analysis between ADD and ARVSD . r denotes the Pearson correlation coefficient and ρs denotes the Spearman rank correlation coefficient. For pretrained models, correlations are reported for both ADD(-S) and ARVSD .
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Spray
86.92
88.17 (+1.25)
88.79 (+1.87)
71.70 (-15.22)
89.87 (+2.95)
Pringles
73.40
76.18 (+2.78)
79.08 (+5.68)
64.23 (-9.16)
81.55 (+8.15)
Tincase
74.78
77.88 (+3.10)
85.30 (+10.52)
67.52 (-7.27)
86.84 (+12.06)
Sandwich
78.68
81.72 (+3.04)
84.06 (+5.38)
66.19 (-12.48)
86.43 (+7.75)
Mouse
63.92
66.41 (+2.49)
72.19 (+8.27)
53.25 (-10.67)
75.21 (+11.29)
Duck
70.27
72.94 (+2.67)
86.67 (+16.40)
65.54 (-4.73)
88.46 (+18.19)
Appendix
Table 19: Object-wise AUC@[0:0.1] performance of various sensor control methods for HiPose, trained on PBR + Train-Def and tested on Test-Var.
Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/
Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy
6D object pose estimation is fundamental to robotic manipulation and automation. Recent zero-shot methods have significantly improved generalization to unseen objects, but most still rely on large-scale pretrained models with substantial GPU computation and memory demands. These requirements complicate deployment on robotic platforms where perception, planning, and control share limited computational resources, while learned intermediate representations offer limited geometric interpretability for task-specific adaptation. To address these limitations, we propose G6D, a learning-free, geometry-driven RGB-D 6D pose solver. Given an RGB-D observation, an object instance mask, camera intrinsics, and a CAD model, G6D generates pose hypotheses through template-based geometric matching and refines them using silhouette and depth consistency, forming a purely geometry-driven pose estimation paradigm. This paradigm requires neither pretrained visual models nor target-specific training and preserves interpretable geometric representations throughout pose estimation. Moreover, adjustable hypothesis counts provide flexible accuracy-computation trade-offs, while a CPU-only configuration supports deployment without GPU resources. Experiments on LineMOD and five BOP19 datasets demonstrate advanced performance. Real-world pick-and-place experiments further demonstrate G6D's applicability to robotic manipulation. The complete project is publicly available at https://ai4control.github.io/G6D-Project-Page .
Smart glass is emerging as an useful device since it provides plenty of insights under hands-busy, eyes-on-task situations. To understand the context of the wearer, 6D object pose estimation in egocentric view is becoming essential. However, existing 6D object pose estimation benchmarks fail to capture the challenges of real-world egocentric applications, which are often dominated by severe motion blur, dynamic illumination, and visual obstructions. This discrepancy creates a significant gap between controlled lab data and chaotic real-world application. To bridge this gap, we introduce EgoXtreme, a new large-scale 6D pose estimation dataset captured entirely from an egocentric perspective. EgoXtreme features three challenging scenarios - industrial maintenance, sports, and emergency rescue - designed to introduce severe perceptual ambiguities through extreme lighting, heavy motion blur, and smoke. Evaluations of state-of-the-art generalizable pose estimators on EgoXtreme indicate that their generalization fails to hold in extreme conditions, especially under low light. We further demonstrate that simply applying image restoration (e.g., deblurring) offers no positive improvement for extreme conditions. While performance gain has appeared in tracking-based approach, implying using temporal information in fast-motion scenarios is meaningful. We conclude that EgoXtreme is an essential resource for developing and evaluating the next generation of pose estimation models robust enough for real-world egocentric vision. The dataset and code are available at https://taegyoun88.github.io/EgoXtreme/
Taegyoon Yoon, Yegyu Han, Seojin Ji +4
Seoul National University · VGG, University of Oxford