Is Geometry Enough? An Evaluation of Landmark-Based Gaze Estimation
Authors: Daniele Agostinelli, Thomas Agostinelli, Andrea Generosi, Maura Mengoni
Organizations: Department of Industrial Engineering and Mathematical Sciences, Universit`a Politecnica delle Marche, via Brecce Bianche, 12, Ancona, 60131, Italy · Department of Science and Information Technology, Universit`a Pegaso, piazza Trieste e Trento, 48, Napoli, 80132, Italy
Appearance-based gaze estimation frequently relies on deep Convolutional Neural Networks (CNNs). These models are accurate, but computationally expensive and act as "black boxes", offering little interpretability. Geometric methods based on facial landmarks are a lightweight alternative, but their performance limits and generalization capabilities remain underexplored in modern benchmarks. In this study, we conduct a comprehensive evaluation of landmark-based gaze estimation. We introduce a standardized pipeline to extract and normalize landmarks from three large-scale datasets (Gaze360, ETH-XGaze, and GazeGene) and train lightweight regression models, specifically Extreme Gradient Boosted trees and two neural architectures: a holistic Multi-Layer Perceptron (MLP) and a siamese MLP designed to capture binocular geometry. We find that landmark-based models exhibit lower performance in within-domain evaluation, likely due to noise introduced into the datasets by the landmark detector. Nevertheless, in cross-domain evaluation, the proposed MLP architectures show generalization capabilities comparable to those of ResNet18 baselines. These findings suggest that sparse geometric features encode sufficient information for robust gaze estimation, paving the way for efficient, interpretable, and privacy-friendly edge applications. The source code and generated landmark-based datasets are available at https://github.com/daniele-agostinelli/LandmarkGaze.git.
Figures & tables
Fig. 1: Pipeline for the extraction of landmark-based dataset. Images are processed to extract landmarks and head pose, which are then used to normalize the data into a virtual camera space.
Fig. 2: Example image from the GazeGene dataset [ 38 ] : (a) original and (b) normalized images. In (b), the red arrow indicates the normalized gaze direction, cyan lines are the image principal axis, and green circles denote the N=20 landmarks extracted from the MediaPipe face mesh [ 39 ] to represent gaze and head orientation: the right iris (indices 473–477) and eye contour (263, 362, 374, 386), the left iris (468–472) and eye contour (33, 133, 145, 159), and two head anchors given by the nose tip (1) and the glabella (9).
Feature
Holistic MLP
Siamese MLP
Input representation
Global landmarks ( fG )
Local eyes ( fL,fR ), geometry ( Δc,fH )
Hidden width (D)
256
64 (per branch)
N. residual blocks (K)
3
3 (per branch)
Dropout (p)
0.1
0.1
TABLE I: Summary of input representation and specific configurations of the MLPs.
Fig. 3: Architectural comparison of gaze estimation models. (a) Holistic Multi-Layer Perceptron (MLP): facial landmarks are processed through an input projection layer, a stack of K residual blocks and a final regression head. (b) Siamese MLP: facial landmarks are split into local eye regions, processed by two independent encoders, and fused with geometric context (relative eye positions Δc and head anchors fH ) via a fusion MLP. (c) XGBoost: a gradient-boosted tree approach using a multi-output regressor for the gaze vector components, g′=(gx′,gy′,gz′) , based on global landmark features, fG . (d) Residual block used in (a) and (b), featuring a residual connection around two sets of Linear, BatchNorm, GELU, and Dropout layers.
Dataset
Original samples
Extracted samples
Retention rate
Gaze360 [ 36 ]
169935
132651
78.06 %
ETH-XGaze [ 37 ]
756540
559057
73.90 %
GazeGene [ 38 ]
1008000
832687
82.61 %
TABLE II: Dataset retention statistics. The retention rate indicates the percentage of images where MediaPipe successfully detected the facial landmarks with confidence ≥0.8 .
Fig. 4: Distributions of yaw and pitch angles (in degrees) for gaze (first row) and head pose (second row) across the three datasets - Gaze360 (a), GazeGene (b), and ETH-XGaze (c). Columns (left to right) show the distributions of the original, extracted, and excluded samples. Colors represent density values according to the reported colorbars.
Image-based
Landmark-based
Dataset
ResNet18
XGBoost
Holistic MLP
Siamese MLP
GazeGene
3.76
13.49 (8.71)
9.00 (7.10)
8.91 (7.02)
ETH-XGaze
5.73
11.63 (7.81)
8.54 (6.71)
8.50 (6.59)
Gaze360
12.23
18.81 (14.38)
14.95 (12.66)
14.68 (12.70)
TABLE III: Results of within-domain evaluation. Performance for landmark-based models is presented as Mean Angular Error and Standard Deviation (SD) in degrees ( ∘ ), calculated across all tested samples. Results for the ResNet18 are reported from the literature [ 38 ] . Bold values indicate the best performance.
Fig. 5: Examples of evident errors in face-landmark detection from different datasets (GazeGene [ 38 ] , Gaze360 [ 36 ] , ETH-XGaze [ 37 ] ). Green circles denote the N=20 landmarks extracted using MediaPipe [ 39 ] . These errors are likely due to low image resolution, face cropping, or extreme poses.
TABLE IV: Results of cross-domain evaluation. For each model architecture, rows and columns represent the training and testing datasets, respectively. Performance for landmark-based models are presented as MAE (SD) in degrees ( ∘ ), where SD is calculated across all tested samples. Results for the ResNet18 are reported from the literature [ 38 ] . Bold values indicate the best performance for each test dataset.
Fig. 6: Permutation Feature Importance (PFI) analysis of the siamese MLP trained on ETH-Xgaze, GazeGene, and Gaze360. The importance of each feature group is measured by the increase in mean angular error ( ∘ ) when the corresponding input features are permuted while keeping others constant. Higher values indicate a greater reliance of the model on that specific feature group. Error bars represent the standard deviation over 1000 permutation cycles.
While appearance-based gaze estimation has achieved significant improvements in accuracy and domain adaptation, the fairness of these systems across different demographic groups remains largely unexplored. To date, there is no comprehensive benchmark quantifying algorithmic bias in gaze estimation. This paper presents the first extensive evaluation of fairness in appearance-based gaze estimation, focusing on ethnicity and gender attributes. We establish a fairness baseline by analyzing state-of-the-art models using standard fairness metrics, revealing significant performance disparities. Furthermore, we evaluate the effectiveness of existing bias mitigation strategies when applied to the gaze domain and show that their fairness contributions are limited. We summarize key insights and open issues. Overall, our work calls for research into developing robust, equitable gaze estimators. To support future research and reproducibility, we publicly release our annotations, code, and trained models at: github.com/akgulburak/gaze-estimation-fairness
Burak Akgül, Erol Şahin, Sinan Kalkan
Dept. of Computer Eng. and ROMER Robotics Center, Middle East Technical University, Ankara, Turkey
Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher's performance while being lightweight enough for practical deployment on robots and consumer devices. The code and model checkpoints are available at our project page.
Appearance-based gaze estimation (AGE) has achieved remarkable performance in constrained settings, yet we reveal a significant generalization gap where existing AGE models often fail in practical, unconstrained scenarios, particularly those involving facial wearables and poor lighting conditions. We attribute this failure to two core factors: limited image diversity and inconsistent label fidelity across different datasets, especially along the pitch axis. To address these, we propose a robust AGE framework that enhances generalization without requiring additional human-annotated data. First, we expand the image manifold via an ensemble of augmentation techniques, including synthesis of eyeglasses, masks, and varied lighting. Second, to mitigate the impact of anisotropic inter-dataset label deviation, we reformulate gaze regression as a multi-task learning problem, incorporating multi-view supervised contrastive (SupCon) learning, discretized label classification, and eye-region segmentation as auxiliary objectives. To rigorously validate our approach, we curate new benchmark datasets designed to evaluate gaze robustness under challenging conditions, a dimension largely overlooked by existing evaluation protocols. Our MobileNet-based lightweight model achieves generalization performance competitive with the state-of-the-art (SOTA) UniGaze-H, while utilizing less than 1% of its parameters, enabling high-fidelity, real-time gaze tracking on mobile devices.
Zhenhao Li, Zheng Liu, Seunghyun Lee +2
Huawei Technologies Canada · University of Toronto