ReVA-HPE: Relative 6DoF Head Pose Estimation with Synthetic Pose Supervision and Personalized Virtual Anchors
Authors: Vasiliki Vasileiou, Panagiotis P. Filntisis, Petros Maragos, Kostas Daniilidis
Organizations: HERON - Hellenic Robotics Center of Excellence, Athena Research Center, Greece · National Technical University of Athens, School of Electrical and Computer Engineering, Greece · Robotics Institute, Athena Research Center, Greece · University of Pennsylvania, USA
Monocular head pose estimation is usually formulated as absolute regression from a single face crop, yet many applications observe the same person over time and can supply an explicit anchor image. Recently, relative head pose estimation was proposed, which casts the problem as estimation of the relative geometric change between an anchor and a query image. However, existing methods still require the anchor image pose to be known at inference, significantly limiting their practical use. We first extend relative head pose estimation to full 6DoF by additionally predicting a depth-normalized relative translation, which composes with the anchor to recover metric query pose in the camera frame. To eliminate the need for a ground-truth anchor pose, we reconstruct a personalized avatar from a single unlabeled enrollment image and render a bank of virtual anchors with known poses. The bank provides robust rotation hypotheses through multi-anchor consensus, while the real enrollment image serves as the real anchor in the translation branch via landmark PnP. We also introduce an efficient variant that amortizes the bank cost once at enrollment, after which each query requires a single real-to-real relative prediction, with limited accuracy sacrifice. The relative pose estimator is trained exclusively on synthetic face renderings, without real pose supervision, pseudo-labels, domain adaptation, or fine-tuning on the evaluation sets. On real 6DoF benchmarks, ReVA-HPE achieves state-of-the-art rotation accuracy on BIWI and, given a near-frontal enrollment image, on FaMoS, while remaining competitive in metric translation despite relying solely on synthetic pose supervision. Project page and code: https://vasilikivas.github.io/ReVA-HPE/
Figures & tables
Figure 1: Absolute HPE predicts pose directly from the query and may generalize poorly across domains. VGGT-HPE uses relative prediction but requires a reference image with known pose and estimates rotation only. ReVA-HPE removes this requirement, using personalized virtual anchors for rotation and PnP on an unlabeled enrollment image for translation, and estimates full 6DoF head pose without any ground-truth pose.
Figure 2: ReVA-HPE overview. From a single unlabeled enrollment image, a personalized Gaussian avatar is reconstructed and rendered at M known poses to form the virtual anchor bank. (2A) For each query, the relative 6DoF model compares the query against all bank anchors, producing M rotation candidates that are fused by pose-space RANSAC consensus into Rqbank . (2B) In parallel, 68 landmarks detected on the enrollment image ground a metric PnP solve. The relative model then predicts the enrollment-to-query translation update, yielding tqPnP-rel . Modified VGGT-HPE denotes the frozen VGGT camera branch adapted with LoRA and trained to predict relative rotation together with the depth-normalized translation update (δx,δy,δz) from an ordered image pair. The two components are fused into the final 6DoF pose Pq=(Rqbank,tqPnP-rel) without any ground-truth anchor pose at inference.
Method
Yaw ↓
Pitch ↓
Roll ↓
MAE r ↓
Tx↓
Ty↓
Tz↓
MAE t ↓
Pose Supervision
Reported numbers
Direct 6DoF Regress [ 15 ]
16.49
14.03
5.81
12.11
62.36
85.01
366.52
171.30
R
Refined Pix2Pose [ 15 , 24 ]
5.75
5.06
11.23
7.35
16.82
21.30
255.36
97.83
R
MICA [ 39 ]
5.40
7.17
3.80
5.46
9.32
13.66
60.13
27.70
M
JMLR [ 11 ]
6.31
6.17
3.72
5.40
8.66
7.27
32.63
16.19
M
PerspNet [ 15 ]
3.10
3.37
2.38
2.95
4.15
6.43
46.69
19.09
R
Table 1: BIWI comparison. Rotation is in degrees and translation in millimeters, lower is better. Reported values follow [ 5 ] , reproduced results use the shared MTCNN protocol. “First frame” fixes each subject’s first available frame as enrollment, without pose-based selection. Bold and underlined denote the best and second-best results within each non-oracle block. The GT-anchor block is ranked independently, with its best values in bold . S, R, and M denote synthetic, real, and mixed pose supervision.
Method
Yaw ↓
Pitch ↓
Roll ↓
MAE r ↓
Tx↓
Ty↓
Tz↓
MAE t ↓
Pose Supervision
Reproduced under shared MTCNN protocol
6DRepNet [ 13 ]
4.54
8.03
5.36
5.98
–
–
–
–
M
TokenHPE-v1 [ 34 ]
8.62
5.69
4.46
6.26
–
–
–
–
M
TRG* [ 5 ]
6.18
6.26
4.32
5.59
7.52
5.06
91.40
34.66
M
SemiUHPE [ 36 ]
1.98
3.65
2.94
2.86
–
–
–
–
R
OPAL [ 6 ]
8.33
5.38
5.14
6.28
–
–
–
–
M
Table 2: FaMoS comparison under the shared MTCNN protocol. Rotation is in degrees and translation in millimeters, lower is better. “First frame” uses each subject’s first available frame as enrollment, without pose-based selection. In the non-oracle block, bold and underlined denote the best and second-best results. The GT-anchor block is ranked independently, with its best values in bold . S, R, and M denote synthetic, real, and mixed pose supervision.
M=5
M=10
Protocol
MAE r
Time
Spd.
MAE r
Time
Spd.
Direct-1r
4.30
0.683
–
2.92
1.165
–
Direct-5r
3.23
2.472
–
2.79
4.902
–
VA-fast
4.53
0.137
5.0/18.0
3.08
0.117
10.0/41.9
M=20
M=50
Direct-1r
2.69
2.185
–
2.58
5.259
–
Table 3: MAE r is in degrees and time is model-only seconds per query. VA-fast speedups are relative to Direct-1r/Direct-5r.
(a) Full-pipeline degradation
(b) Landmark jitter
(c) Landmark removal
Condition
MAE R ↓
MAE t ↓
Condition
Tz↓
MAE t ↓
Condition
Tz↓
MAE t ↓
Clean
2.53
17.10
Clean
31.74
17.10
Clean
31.74
17.10
Blur, σ=2
3.22
18.64
σ=1 px
31.96
17.21
10%
32.52
17.41
Downsample 50%
2.77
15.97
σ=2 px
31.92
17.40
25%
32.59
17.44
Occlusion 12.5%
4.46
38.46
σ=4 px
45.55
22.35
40%
33.83
17.97
Table 4: Sensitivity to enrollment-image degradation and landmark errors on BIWI.
Dataset
Enrollment pose
Coverage
ReVA-HPE MAE r ↓
SemiUHPE MAE r ↓
ReVA-HPE MAE t ↓
BIWI
α≤15∘
99.85%
2.49
3.20
18.94
15∘<α≤30∘
99.85%
3.71
3.20
18.62
30∘<α≤45∘
99.85%
4.85
3.20
19.31
α>45∘
99.85%
4.80
3.20
21.77
FaMoS
α≤15∘
84.57%
2.47
3.00
22.61
15∘<α≤30∘
73.94%
6.02
2.39
26.83
Table 5: Sensitivity to enrollment pose on BIWI and FaMoS under the shared MTCNN protocol.
Archimedes, Athena Research Center, Marousi, Greece · HERON – Hellenic Robotics Center of Excellence, Athens, Greece · School of ECE, National Technical University of Athens, Greece +2