ReVA-HPE: Relative 6DoF Head Pose Estimation with Synthetic Pose Supervision and Personalized Virtual Anchors
Authors: Vasiliki Vasileiou, Panagiotis P. Filntisis, Petros Maragos, Kostas Daniilidis
Organizations: HERON - Hellenic Robotics Center of Excellence, Athena Research Center, Greece · National Technical University of Athens, School of Electrical and Computer Engineering, Greece · Robotics Institute, Athena Research Center, Greece · University of Pennsylvania, USA
Monocular head pose estimation is usually formulated as absolute regression from a single face crop, yet many applications observe the same person over time and can supply an explicit anchor image. Recently, relative head pose estimation was proposed, which casts the problem as estimation of the relative geometric change between an anchor and a query image. However, existing methods still require the anchor image pose to be known at inference, significantly limiting their practical use. We first extend relative head pose estimation to full 6DoF by additionally predicting a depth-normalized relative translation, which composes with the anchor to recover metric query pose in the camera frame. To eliminate the need for a ground-truth anchor pose, we reconstruct a personalized avatar from a single unlabeled enrollment image and render a bank of virtual anchors with known poses. The bank provides robust rotation hypotheses through multi-anchor consensus, while the real enrollment image serves as the real anchor in the translation branch via landmark PnP. We also introduce an efficient variant that amortizes the bank cost once at enrollment, after which each query requires a single real-to-real relative prediction, with limited accuracy sacrifice. The relative pose estimator is trained exclusively on synthetic face renderings, without real pose supervision, pseudo-labels, domain adaptation, or fine-tuning on the evaluation sets. On real 6DoF benchmarks, ReVA-HPE achieves state-of-the-art rotation accuracy on BIWI and, given a near-frontal enrollment image, on FaMoS, while remaining competitive in metric translation despite relying solely on synthetic pose supervision. Project page and code: https://vasilikivas.github.io/ReVA-HPE/
Figures & tables
Figure 1: Absolute HPE predicts pose directly from the query and may generalize poorly across domains. VGGT-HPE uses relative prediction but requires a reference image with known pose and estimates rotation only. ReVA-HPE removes this requirement, using personalized virtual anchors for rotation and PnP on an unlabeled enrollment image for translation, and estimates full 6DoF head pose without any ground-truth pose.
Figure 2: ReVA-HPE overview. From a single unlabeled enrollment image, a personalized Gaussian avatar is reconstructed and rendered at M known poses to form the virtual anchor bank. (2A) For each query, the relative 6DoF model compares the query against all bank anchors, producing M rotation candidates that are fused by pose-space RANSAC consensus into Rqbank . (2B) In parallel, 68 landmarks detected on the enrollment image ground a metric PnP solve. The relative model then predicts the enrollment-to-query translation update, yielding tqPnP-rel . Modified VGGT-HPE denotes the frozen VGGT camera branch adapted with LoRA and trained to predict relative rotation together with the depth-normalized translation update (δx,δy,δz) from an ordered image pair. The two components are fused into the final 6DoF pose Pq=(Rqbank,tqPnP-rel) without any ground-truth anchor pose at inference.
Method
Yaw ↓
Pitch ↓
Roll ↓
MAE r ↓
Tx↓
Ty↓
Tz↓
MAE t ↓
Pose Supervision
Reported numbers
Direct 6DoF Regress [ 15 ]
16.49
14.03
5.81
12.11
62.36
85.01
366.52
171.30
R
Refined Pix2Pose [ 15 , 24 ]
5.75
5.06
11.23
7.35
16.82
21.30
255.36
97.83
R
MICA [ 39 ]
5.40
7.17
3.80
5.46
9.32
13.66
60.13
27.70
M
JMLR [ 11 ]
6.31
6.17
3.72
5.40
8.66
7.27
32.63
16.19
M
PerspNet [ 15 ]
3.10
3.37
2.38
2.95
4.15
6.43
46.69
19.09
R
Table 1: BIWI comparison. Rotation is in degrees and translation in millimeters, lower is better. Reported values follow [ 5 ] , reproduced results use the shared MTCNN protocol. “First frame” fixes each subject’s first available frame as enrollment, without pose-based selection. Bold and underlined denote the best and second-best results within each non-oracle block. The GT-anchor block is ranked independently, with its best values in bold . S, R, and M denote synthetic, real, and mixed pose supervision.
Method
Yaw ↓
Pitch ↓
Roll ↓
MAE r ↓
Tx↓
Ty↓
Tz↓
MAE t ↓
Pose Supervision
Reproduced under shared MTCNN protocol
6DRepNet [ 13 ]
4.54
8.03
5.36
5.98
–
–
–
–
M
TokenHPE-v1 [ 34 ]
8.62
5.69
4.46
6.26
–
–
–
–
M
TRG* [ 5 ]
6.18
6.26
4.32
5.59
7.52
5.06
91.40
34.66
M
SemiUHPE [ 36 ]
1.98
3.65
2.94
2.86
–
–
–
–
R
OPAL [ 6 ]
8.33
5.38
5.14
6.28
–
–
–
–
M
Table 2: FaMoS comparison under the shared MTCNN protocol. Rotation is in degrees and translation in millimeters, lower is better. “First frame” uses each subject’s first available frame as enrollment, without pose-based selection. In the non-oracle block, bold and underlined denote the best and second-best results. The GT-anchor block is ranked independently, with its best values in bold . S, R, and M denote synthetic, real, and mixed pose supervision.
M=5
M=10
Protocol
MAE r
Time
Spd.
MAE r
Time
Spd.
Direct-1r
4.30
0.683
–
2.92
1.165
–
Direct-5r
3.23
2.472
–
2.79
4.902
–
VA-fast
4.53
0.137
5.0/18.0
3.08
0.117
10.0/41.9
M=20
M=50
Direct-1r
2.69
2.185
–
2.58
5.259
–
Table 3: MAE r is in degrees and time is model-only seconds per query. VA-fast speedups are relative to Direct-1r/Direct-5r.
(a) Full-pipeline degradation
(b) Landmark jitter
(c) Landmark removal
Condition
MAE R ↓
MAE t ↓
Condition
Tz↓
MAE t ↓
Condition
Tz↓
MAE t ↓
Clean
2.53
17.10
Clean
31.74
17.10
Clean
31.74
17.10
Blur, σ=2
3.22
18.64
σ=1 px
31.96
17.21
10%
32.52
17.41
Downsample 50%
2.77
15.97
σ=2 px
31.92
17.40
25%
32.59
17.44
Occlusion 12.5%
4.46
38.46
σ=4 px
45.55
22.35
40%
33.83
17.97
Table 4: Sensitivity to enrollment-image degradation and landmark errors on BIWI.
Dataset
Enrollment pose
Coverage
ReVA-HPE MAE r ↓
SemiUHPE MAE r ↓
ReVA-HPE MAE t ↓
BIWI
α≤15∘
99.85%
2.49
3.20
18.94
15∘<α≤30∘
99.85%
3.71
3.20
18.62
30∘<α≤45∘
99.85%
4.85
3.20
19.31
α>45∘
99.85%
4.80
3.20
21.77
FaMoS
α≤15∘
84.57%
2.47
3.00
22.61
15∘<α≤30∘
73.94%
6.02
2.39
26.83
Table 5: Sensitivity to enrollment pose on BIWI and FaMoS under the shared MTCNN protocol.
Monocular head pose estimation is traditionally formulated as direct regression from a single image to an absolute pose. This paradigm forces the network to implicitly internalize a dataset-specific canonical reference frame. In this work, we argue that predicting the relative rigid transformation between two observed head configurations is a fundamentally easier and more robust formulation. We introduce VGGT-HPE, a relative head pose estimator built upon a general-purpose geometry foundation model. Finetuned exclusively on synthetic facial renderings, our method sidesteps the need for an implicit anchor by reducing the problem to estimating a geometric displacement from an explicitly provided anchor with a known pose. As a practical benefit, the relative formulation also allows the anchor to be chosen at test time - for instance, a near-neutral frame or a temporally adjacent one - so that the prediction difficulty can be controlled by the application. Despite zero real-world training data, VGGT-HPE achieves state-of-the-art results on the BIWI benchmark, outperforming established absolute regression methods trained on mixed and real datasets. Through controlled easy- and hard-pair benchmarks, we also systematically validate our core hypothesis: relative prediction is intrinsically more accurate than absolute regression, with the advantage scaling alongside the difficulty of the target pose. Project page and code: https://vasilikivas.github.io/VGGT-HPE
Vasiliki Vasileiou, Panagiotis P. Filntisis, Petros Maragos +1
Archimedes, Athena Research Center, Marousi, Greece · HERON – Hellenic Robotics Center of Excellence, Athens, Greece · School of ECE, National Technical University of Athens, Greece +2
Egocentric human pose estimation (HPE) using a head-mounted device is crucial for various VR and AR applications, but it faces significant challenges due to keypoint invisibility. Nevertheless, none of the existing egocentric HPE datasets provide keypoint visibility annotations, and the existing methods often overlook the invisibility problem, treating visible and invisible keypoints indiscriminately during estimation. As a result, their capacity to accurately predict visible keypoints is compromised. In this paper, we first present Eva-3M, a large-scale egocentric visibility-aware HPE dataset comprising over 3.0M frames, with 435K of them annotated with keypoint visibility labels. Additionally, we augment the existing EMHI dataset with keypoint visibility annotations to further facilitate the research in this direction. Furthermore, we propose EvaPose, a novel egocentric visibility-aware HPE method that explicitly incorporates visibility information to enhance pose estimation accuracy. Extensive experiments validate the significant value of ground-truth visibility labels in egocentric HPE settings, and demonstrate that our EvaPose achieves state-of-the-art performance in both Eva-3M and EMHI datasets.
Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/
Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy