Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In-Cabin-Pose test dataset consisting of highly distorted vehicle-interior images and our code at https://github.com/felixstillger/InCaRPose.
Figures & tables
Figure 1: Our InCaRPose predicts the relative camera pose between a reference and a target view (shown as camera frustums). Trained exclusively on synthetic images, the model generalizes to real-world cabin environments and enables camera extrinsic calibration.
Figure 2: Camera coordinate system of the standard view compared to the vehicle’s coordinate system.
Figure 3: Standard view comparison: (left) real-world image and (right) synthetic image from the simulation environment.
Figure 4: InCaRPose’s architecture overview. Two images are encoded by a frozen ViT backbone and fused by a cross-attention Transformer decoder. A prediction head outputs the relative camera pose between the views, optionally in both directions.
Model
Error
Rot. ( ∘ )
Trans. (m)
Dir. ( ∘ )
InCaRPoseSmall224
Mean
6.11
0.11
42.55
Median
4.43
0.08
37.74
InCaRPoseBase224
Mean
4.91
0.12
53.67
Median
3.55
0.09
42.45
InCaRPoseLarge224
Mean
4.15
0.10
36.17
Median
2.75
0.07
23.46
Table 1: Model Performance Comparison: Rotation and Translation Errors (Trans. = translation error in m, Dir. = translation direction error in °) on our In-Cabin-Pose real-world test set.
Figure 5: Qualitative results on real-world inference. All translation vectors are normalized to a common scale for visualization.
Model
Error
Rot. ( ∘ )
Trans. (m)
Dir. ( ∘ )
Reloc3r224 [ 17 ] 2 2 2 Trained on additional data
Mean
7.96
–
39.43
Median
2.45
–
33.51
RelPoseNet [ 35 ] 1 1 1 Values are taken from the authors’ paper
Mean
–
–
–
Median
9.30
0.21
–
Relformer [ 28 ] 1 1 1 Values are taken from the authors’ paper
Mean
–
–
–
Median
6.27
0.18
–
Table 2: Model performance: rotation and translation errors on 7-Scenes. Not available values are indicated with “-”. Bold indicates best performance.
Scope
Scene
Model
Rot. Err ( ∘ )
Trans. Err (m)
Mean
Median
Mean
Median
Overall
All
InCaRPoseSmall224
6.99
3.44
2.31
1.45
InCaRPoseBase224
6.53
3.01
2.29
1.35
InCaRPoseLarge224
6.49
2.66
2.19
1.21
Reloc3r224 [ 17 ] 2 2 footnotemark: 2
9.95
1.13
–
–
Scene
KingsCollege
InCaRPoseSmall224
2.74
2.67
1.20
1.05
Table 3: Validation results on Cambridge Landmarks.
Representation
Error
Rot. ( ∘ )
Trans. (m)
Dir. ( ∘ )
Rotation Vector
Mean
6.35
0.14
81.24
Median
4.80
0.11
70.62
Euler Intrinsic
Mean
7.61
0.14
99.14
Median
5.77
0.10
106.76
Euler Extrinsic
Mean
6.39
0.14
108.73
Median
4.67
0.11
110.92
Table 4: Output rotation representations ablation on In-Cabin-Pose with InCaRPoseBase224.
Backbone
Error
Rot. ( ∘ )
Trans. (m)
Dir. ( ∘ )
Dune-Base
Mean
4.52
0.13
55.35
Median
3.00
0.09
53.40
DUST3R-Large
Mean
5.23
0.14
81.18
Median
3.99
0.10
77.63
DINOv2-Base
Mean
5.98
0.12
45.78
Median
3.95
0.09
40.43
Table 5: Backbone ablation on In-Cabin-Pose with InCaRPoseBase224.
Backbone
Res.
Config
Latency (ms)
FPS
Speedup
Small
224
Baseline (FP32)
14.65
68.28
1.00
FP16
14.57
68.64
1.01
torch.compile
14.00
71.41
1.05
FP16 + torch.compile
14.48
69.06
1.01
Small
512
Baseline (FP32)
24.75
40.40
1.00
FP16
16.49
60.65
1.50
Table 6: Detailed inference runtime measurements on a single NVIDIA RTX 4090 GPU. We report average per-frame latency (ms), frames per second (FPS), and relative speedup with respect to the FP32 baseline at the corresponding backbone and resolution.
Backbone
Metric
Rot. Err ( ∘ )
Dir. Err ( ∘ )
Dune-Base
Mean
6.26
52.11
Median
3.88
50.90
DUST3R-Large
Mean
6.98
81.52
Median
5.29
73.93
DINOv2-Base
Mean
8.11
45.60
Median
5.30
39.58
Table 7: Backbone ablation on COLMAP ground truth (distorted images and 224 resolution).
Figure 6: Comparison of preprocessing methods. (a) and (b): images are cropped to the center. (c) and (d): images are zero-padded to a square aspect ratio. This adjustment is necessary to handle varying input resolutions within the dataset while maintaining a consistent model input.
Metric
Value
Max rotation error
19.86 ∘
Mean rotation error
3.08 ∘
Median rotation error
2.27 ∘
Max translation direction error
55.83 ∘
Mean translation direction error
5.75 ∘
Median translation direction error
4.60 ∘
Table 8: Rotation and translation direction errors. Translation direction error is only evaluated when the relative ArUco ground truth translation exceeds 0.1 m.
Figure 7: COLMAP fails to estimate translation along the y -axis. In the first view the camera is to the side of the steering wheel. In the second view the camera moved upwards (y) above the steering wheel. Translation is normalized.
Figure 8: Rotation error in degrees versus the number of parameters. Evaluated on image resolution of 224 on the In-Cabin-Pose dataset.
Figure 9: ArUco (orange) vs. COLMAP (blue) camera trajectories for intervals focused on specific transformations. Each sequence predominantly captures motion along the target axis, though residual degrees-of-freedom are also present.
Figure 10: Inference on frames with physically occluded ArUco markers. We also changed the reference image, in which a different object (a high-contrast checkerboard on the left seat) is now visible in only one image to additionally challenge the model.