Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform

Matej Palider; Omar Eldardeer; Viktor Kocur

cs.CVSep 28, 2025

Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform

Authors: Matej Palider, Omar Eldardeer, Viktor Kocur

Organizations: Faculty of Mathematics, Physics and Informatics, Comenius University, Bratislava · CONTACT UNIT, Italian Institute of Technology, Genoa

Abstract

This paper evaluates the current gaze estimation methods within a human-robot interaction (HRI) context of a shared workspace scenario. We introduce a new, annotated dataset collected with the NICO robotic platform. We evaluate four state-of-the-art gaze estimation models. The evaluation shows that the angular errors are close to those reported on general-purpose benchmarks. However, when expressed in terms of distance in the shared workspace the best median error is 14.57~cm, quantifying the practical limitations of current methods. We conclude by discussing these limitations and offering recommendations on how to best integrate gaze estimation as a modality in HRI systems.

Figures & tables

Panel of Figure 1 — Fig. 1: a) The experimental setup. A human participant sits in front of the robot. A display is embedded within the shared workspace of the robot and the participant. In our study we consider the distance between the estimated and ground truth gaze point in the shared workspace. b) View from the robot’s left camera. c) The grid shown on the display.

Panel of Figure 2 — Fig. 2: Example of images used for stereo-calibration. Left: Wide field-of-view cameras are significantly distorted. Right: With known camera intrinsics it is possible to rectify the images.

Method	Mean Angular Error ( ∘ )	Median Distance (cm)	Distance Precision (%)
Method	Mean Angular Error ( ∘ )	Median Distance (cm)	@10cm	@20cm	@50cm
GazeTR [ 6 ]	14.69	20.21	21.27	48.57	89.84
L2CS [ 7 ]	1 9.65	15.63	33.02	66.03	90.48
3DGazeNet [ 8 ]	1 9.92	14.57	33.65	61.27	83.17
Gaze3D [ 9 ]	11.97	19.31	20.95	54.29	96.51

TABLE I: The results of the evaluated methods for the full dataset.

Panel of Figure 4 — Fig. 3: The cumulative distribution of the angular (top) and the distance (bottom) errors. The plots show the precision achieved (y-axis) given an error threshold (x-axis).

Fig. 4: The distributions of the yaw and pitch angles for the ground truth and predicted gaze. For ground truth, we consider the direction from the head position estimated using RetinaFace [ 18 ] to the center of the target square. For visualization, we allow the pitch angles to go below

-90^{\circ}

Gaze Estimation

Gaze estimation predicts where a person is looking from eye, face, head-pose, and scene cues.

See all 66 papers

Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction
May 6, 2026Berk Sezer, Ali Görkem Küçük, Erol Şahin +1Zero-Shot Learning Gaze Estimation
Dept. of Computer Eng. and ROMER (Robotics Center), Middle East Technical University, Ankara, Türkiye
PAGE: Towards Practical Human-level Gaze Target Estimation
Jul 6, 2026Zhoutong Ye, Chengwen Zhang, Zhaibin Cui +10Gaze Estimation
1Tsinghua University · 2Jinan University
Gaze-Aware Task Progression Detection Framework for Human-Robot Interaction Using RGB Cameras
Mar 16, 2026Linlin Cheng, Koen Hindriks, Artem V. BelopolskyHuman-Robot Interaction
Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands. · Department of Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.

May 6, 2026J/K move · Enter open · S save

Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction

Berk Sezer, Ali Görkem Küçük, Erol Şahin, Sinan Kalkan

Dept. of Computer Eng. and ROMER (Robotics Center), Middle East Technical University, Ankara, Türkiye

Zero-Shot Learning Gaze Estimation Human-Robot Interaction

Open paper AskPDF arXiv

While zero-shot appearance-based 3D gaze estimation offers significant cost-efficiency by directly mapping RGB images to gaze vectors, its reliability in Human-Robot Interaction (HRI) settings remains uncertain. Existing benchmarks frequently overlook fundamental HRI conditions, such as dynamic camera viewpoints and moving targets in video. Furthermore, current cross-dataset evaluations often suffer from a complexity gap, where methods trained on diverse datasets are tested on significantly smaller and less varied sets, failing to assess true robustness. To bridge these gaps, we introduce Gaze4HRI, a large-scale dataset (50+ subjects, 3,000+ videos, 600,000+ frames) designed to evaluate state-of-the-art performance against critical HRI variables: illumination, head-gaze conflict, as well as the motion of camera and gaze target in video. Our benchmark reveals that all evaluated methods fail in at least one condition, identifying steeply-downward gaze as a universal failure point. Notably, PureGaze trained on the ETH-X-Gaze dataset uniquely maintains resilience across all other conditions. These results challenge the recent focus in the literature on complex spatial-temporal modeling and Transformer-based architectures. Instead, our findings suggest that extensive data diversity, as exemplified by the ETH-X-Gaze dataset, serves as the primary driver of zero-shot robustness in unconstrained environments, while resilience-enhancing frameworks, such as PureGaze's self-adversarial loss for gaze feature purification, provide a substantial further improvement. Ultimately, this study establishes a rigorous benchmark that provides practical guidelines for practitioners as well as reshaping future research. The dataset and codes are available at https://gazeforhri.github.io.

Abstract

Gaze Estimation

Explore similar work

Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction

PAGE: Towards Practical Human-level Gaze Target Estimation

Gaze-Aware Task Progression Detection Framework for Human-Robot Interaction Using RGB Cameras