Recent advances on 6D object pose estimation have achieved high performance on representative benchmarks such as LM-O, YCB-V, and T-Less. These datasets capture natural-occurring variations in illumination and camera setup but do not systematically explore the effects of controllable sensor configurations, such as exposure, gain, and depth-sensor modes. To bridge this gap, we introduce SenseShift6D, the first RGB-D object-pose benchmark that physically sweeps 13 RGB exposures, 9 RGB gains, auto-exposure, 4 depth-capture modes, and 5 illumination levels. For six common household objects, we acquire 198.8k RGB and 20.0k depth images (i.e., 795.4k RGB-D scenes), providing 1,380 unique sensor-lighting permutations per object pose. Experiments with state-of-the-art pretrained, generalizable pose estimators reveal substantial performance variation across lighting and sensor settings, despite their large-scale pretraining. Strikingly, even instance-level estimators-trained and tested on identical objects and backgrounds-exhibit pronounced sensitivity to environmental and sensor shifts. These findings establish sensor- and environment-aware robustness as an underexplored yet essential dimension of 6D pose estimation. To illustrate the opportunity enabled by this benchmark, we evaluate test-time sensor selection without retraining. An idealized controller yields remarkable gains of up to +16.2 pp for generalizable models, whereas a practical consistency-based proxy provides only limited improvements, highlighting substantial headroom and the need for future research on reliable sensor-aware adaptation. SenseShift6D is available at huggingface.co/datasets/Yegyu/SenseShift6D.
Figures & tables
Figure 1: RGB sample images under auto exposure and under all combinations of exposure and gain settings for brightness 5%, 25%, 75% and 100%. Rows indicate gain levels and columns indicate exposure levels.
Figure 2: Depth sample images under four depth capture modes: default, high accuracy, high density, and medium density.
Dataset
Environmental and sensor variations
Illumination
Exposure
Gain
Depth
Total Var.
LM [ 15 ]
×
×
×
×
×
LM-O [ 4 ]
×
×
×
×
×
T-LESS [ 16 ]
×
×
×
×
×
YCB-V [ 42 ]
×
×
×
×
×
TUD-L [ 17 ]
8 options
×
×
×
8
Table 1: Comparison of 6D pose estimation datasets.
Figure 4
Figure 4: Six household objects collected in SenseShift6D.
Table 3: AUC@[0:0.1] performance of various sensor control methods for unseen object pose estimation models
Figure 5: Overall standard deviation of AUC computed across all brightness and scenes evaluated under Baseline, Oracle-Fixed, Oracle-Dynamic.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
100.0
96.10
54.61
98.54
100.0
96.59
52.02
99.02
100.0
99.51
59.88
100.0
PBR + Train-Def w/ Aug
95.12
93.66
64.98
99.51
100.0
98.54
71.88
100.0
100.0
100.0
89.76
100.0
PBR + Train-Var
97.56
97.07
79.24
97.56
97.56
97.56
85.93
98.54
100.0
100.0
100.0
100.0
Pringles
PBR + Train-Def
90.48
84.29
43.31
95.71
69.05
60.95
35.64
89.05
100.0
99.52
58.71
100.0
Table 4: AR@5 performance of various sensor control methods under different training and test settings of SenseShift6D for instance-level pose estimation models: ZebraPose, GDRNPP, HiPose.
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Overall
74.66
77.22
82.68
64.74
84.73
(+2.44)
(+8.02)
(-9.92)
(+10.07)
Table 5: Overall AUC@[0:0.1] performance of various sensor control methods for HiPose, trained on PBR + Train-Def and tested on Test-Var.
Table 6: Performance comparison under different sensor selection strategies. Values in parentheses denote ADD-S variant. Supervised models are trained under the PBR + Train-Def setting.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Table 7: AUC@[0:0.1] performance of various sensor control methods for unseen object pose estimation models given CNOS mask
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Spray
81.17
86.05
86.84
78.71
89.06
(+4.89)
(+5.67)
(-2.46)
(+7.89)
Pringles
61.84
71.98
73.80
55.85
80.01
(+10.13)
(+11.96)
(-6.00)
(+18.17)
Tincase
63.85
75.66
75.59
60.02
81.71
(+11.81)
(+11.74)
(-3.83)
(+17.86)
Appendix
Table 8: AUC@[0:0.1] performance of SAM-6D with its Image Segmentation Model (ISM).
Augmentation Type
Activation Probability
Parameters / Range
Gaussian Blur
0.4
σ∈[0.0,3.0]
Sharpness Enhancement
0.3
factor ∈[0.0,50.0]
Contrast Enhancement
0.3
factor ∈[0.2,50.0]
Brightness Enhancement
0.5
factor ∈[0.1,6.0]
Color Enhancement
0.3
factor ∈[0.0,20.0]
Additive Intensity
0.5
value ∈[−25,25] , per-channel = 0.3
Appendix
Table 11: Augmentation operations and parameters.
Figure 6: Examples of RGB augmentations applied during training. The top-left image is the original, and the remaining images are randomly augmented samples generated using the settings summarized in Tab. 11 .
Figure 7: Sample of marker-removed test images.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
96.59
95.12
64.33
100
100
96.59
52.49
99.51
100
98.73
59.98
100
PBR + Train-Def w/ Aug
93.66
93.17
70.71
100
95.12
94.63
71.63
100
100
100
91.66
100
PBR + Train-Var
98.05
97.07
78.91
100
100
100
81.89
100
100
100
100
100
Tincase
PBR + Train-Def
94.22
86.22
55.45
99.11
64.44
58.66
35.07
84.89
97.78
84.62
56.33
97.78
Appendix
Table 12: AR@5 comparison of sensor control methods on marker-removed SenseShift6D with ZebraPose, GDRNPP, and HiPose.
ZebraPose [ 35 ]
GDRNPP [ 29 ]
HiPose [ 27 ]
Test-Def
Test-Var
Test-Def
Test-Var
Test-Def
Test-Var
Object
Train
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Auto
Auto
Rand
Oracle
Spray
PBR + Train-Def
100.0
97.56
55.95
99.51
100.0
95.12
50.76
98.54
100.0
97.07
56.49
100.0
PBR + Train-Def w/ Aug
97.56
89.78
66.95
100.00
97.56
97.07
76.10
99.51
100.0
100.0
98.71
100.0
PBR + Train-Var
95.12
90.73
73.85
97.56
95.12
95.12
87.61
95.12
100.0
100.0
99.73
100.0
Tincase
PBR + Train-Def
97.78
96.10
50.89
98.67
93.33
82.22
47.60
94.67
97.33
82.49
53.82
96.44
Appendix
Table 13: AR@5 comparison of sensor control methods on marker-removed SenseShift6D with ZebraPose, GDRNPP, and HiPose trained on PBR + marker-removed Train-Def setting.
Figure 8: Average AUC by brightness level across exposure, gain, and depth capture modes for pretrained models: GigaPose, SAM-6D, and FoundationPose.
Figure 9: Average AUC by brightness level across exposure, gain, and depth capture modes for supervised models: ZebraPose, GDRNPP, and HiPose.
Figure 10: Comparison of predictions under Baseline and Oracle for GigaPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 11: Comparison of predictions under Baseline and Oracle for SAM-6D. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 12: Comparison of predictions under Baseline and Oracle for FoundationPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Figure 13: Comparison of predictions under Baseline and Oracle for HiPose. Visualized object pose on RGB images: ground truth pose in red, predicted pose in green.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
83.17
85.07 (+1.90)
87.10 (+3.93)
86.07 (+2.90)
88.63 (+5.46)
Pringles
72.40
75.45 (+3.05)
78.60 (+6.19)
77.90 (+5.50)
81.12 (+8.71)
Tincase
32.96
43.00 (+10.04)
78.80 (+45.84)
76.69 (+43.73)
81.31 (+48.36)
Sandwich
66.85
69.85 (+3.00)
80.72 (+13.87)
78.82 (+11.97)
83.18 (+16.33)
Mouse
35.29
39.88 (+4.59)
64.85 (+29.56)
62.93 (+27.63)
69.88 (+34.59)
Duck
5.93
12.05 (+6.12)
79.16 (+73.23)
76.14 (+70.21)
81.44 (+75.51)
Appendix
Table 14: AUC@[0:0.1] performance under diverse environments of various sensor control methods for HiPose trained on PBR + Train-Def.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
65.02
71.00 (+5.98)
70.22 (+5.20)
66.85 (+1.83)
75.54 (+10.51)
Pringles
22.95
42.05 (+19.10)
42.93 (+19.98)
26.79 (+3.83)
64.60 (+41.64)
Tincase
65.98
78.67 (+12.69)
71.69 (+5.71)
74.07 (+8.09)
80.27 (+14.29)
Sandwich
59.97
69.74 (+9.77)
68.77 (+8.79)
59.97 (+0.00)
80.31 (+20.33)
Mouse
44.05
55.17 (+11.12)
62.73 (+18.68)
56.32 (+12.27)
72.78 (+28.73)
Duck
72.88
77.56 (+4.67)
75.67 (+2.79)
72.44 (-0.44)
80.30 (+7.42)
Appendix
Table 15: AUC@[0:0.1] performance under diverse environments of various sensor control methods for GigaPose.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
84.54
86.83 (+2.29)
84.22 (-0.32)
86.59 (+2.05)
89.22 (+4.68)
Pringles
50.17
71.45 (+21.28)
69.14 (+18.97)
65.76 (15.59)
80.48 (+30.31)
Tincase
47.61
66.73 (+19.12)
77.96 (+30.34)
68.61 (+21.00)
80.67 (+33.05)
Sandwich
78.36
79.00 (+0.64)
80.10 (+1.74)
78.41 (+0.05)
80.85 (+2.49)
Mouse
52.41
64.83 (+12.41)
69.12 (+16.71)
59.54 (+7.13)
74.40 (+21.99)
Duck
76.93
81.42 (+4.49)
84.56 (+7.63)
79.21 (+2.28)
87.91 (10.98)
Appendix
Table 16: AUC@[0:0.1] performance under diverse environments of various sensor control methods for SAM-6D.
Env
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
B5
Spray
82.80
84.68 (+1.88)
85.34 (+2.54)
83.44 (+0.64)
87.17 (+4.37)
Pringles
25.07
39.45 (+14.38)
45.26 (+20.19)
32.98 (+7.91)
64.10 (+39.03)
Tincase
26.47
49.11 (+22.64)
53.64 (27.18)
38.29 (+11.82)
74.24 (+47.78)
Sandwich
81.62
84.41 (+2.79)
83.38 (+1.76)
82.77 (+1.15)
86.28 (+4.66)
Mouse
67.88
72.80 (+4.92)
75.10 (+7.22)
72.02 (+4.14)
79.54 (+11.66)
Duck
82.58
85.21 (+2.63)
87.88 (+5.30)
84.07 (+1.49)
89.74 (+7.16)
Appendix
Table 17: AUC@[0:0.1] performance under diverse environments of various sensor control methods for FoundationPose.
Table 18: Correlation analysis between ADD and ARVSD . r denotes the Pearson correlation coefficient and ρs denotes the Spearman rank correlation coefficient. For pretrained models, correlations are reported for both ADD(-S) and ARVSD .
Object
Baseline
Depth-Only
RGB-Only
Oracle-F
Oracle-D
Spray
86.92
88.17 (+1.25)
88.79 (+1.87)
71.70 (-15.22)
89.87 (+2.95)
Pringles
73.40
76.18 (+2.78)
79.08 (+5.68)
64.23 (-9.16)
81.55 (+8.15)
Tincase
74.78
77.88 (+3.10)
85.30 (+10.52)
67.52 (-7.27)
86.84 (+12.06)
Sandwich
78.68
81.72 (+3.04)
84.06 (+5.38)
66.19 (-12.48)
86.43 (+7.75)
Mouse
63.92
66.41 (+2.49)
72.19 (+8.27)
53.25 (-10.67)
75.21 (+11.29)
Duck
70.27
72.94 (+2.67)
86.67 (+16.40)
65.54 (-4.73)
88.46 (+18.19)
Appendix
Table 19: Object-wise AUC@[0:0.1] performance of various sensor control methods for HiPose, trained on PBR + Train-Def and tested on Test-Var.