Quantifying the Impact of Motion on 2D Gaze Estimation in Real-World Mobile Interactions
Authors: Yaxiong Lei, Yuheng Wang, Fergus Buchanan, Mingyue Zhao, Yusuke Sugano, Shijing He, Mohamed Khamis, Juan Ye
Organizations: University of St Andrews, St Andrews, Fife, UK, KY16 9SX · University of Tokyo, Tokyo, Japan · King’s College London, London, UK · University of Glasgow, Glasgow, UK
Mobile gaze tracking on handheld devices commonly relies on appearance-based 2D gaze estimation from the device's front-facing RGB camera. Although this approach makes eye tracking more accessible by avoiding dedicated infrared hardware, maintaining accuracy remains difficult because the spatial relationship among the user's face, the camera, and the screen changes continuously during handheld use. This paper examines how motion-related geometry change affects a representative front-camera RGB 2D gaze-estimation pipeline. We report two complementary within-subject user studies (n=10) that capture handheld behaviour, calibration data, and gaze-estimation error during everyday-style mobile tasks across conditions ranging from lying and sitting to walking and maze navigation. Rather than treating motion as a single binary factor, we analyse within-task versus task-switch behaviour, calibration-test mismatch, and condition-specific motion signatures. Behavioural results show that head-to-camera distance is comparatively stable during sustained within-task use, while task switches introduce much larger geometry fluctuations, with head-distance standard deviations of approximately 7.42--10.10,cm across conditions. In the gaze-evaluation study, one-off calibration reduced average RMSE from 4.49,cm to 3.44,cm; under this regime, the global effect of motion condition on RMSE was not statistically reliable, whereas clearer patterns appeared in task-specific and motion-specific calibration analyses, where better calibration-test match reduced error.
Figures & tables
Figure 1: Overview of the experimental mobile eye-tracking system used to synchronise front-camera RGB input, calibration and evaluation events, and motion-sensing data.
Figure 2: Head-to-camera distance estimation from the relationship between physical interpupillary distance and image-frame interpupillary scale.
Sensor Events
Frequency
Dimension
Description
UserAccelerometer
50Hz
X,Y,Z
Acceleration (m/s 2 ) without the effects of gravity.
Accelerometer
50Hz
X,Y,Z
Acceleration (m/s 2 ) with the effects of gravity.
Gyroscope
50Hz
X,Y,Z
Device rotation.
Magnetometer
50Hz
X,Y,Z
Ambient magnetic field surrounding the device.
Head distance
30Hz
distance
Estimated distance between the eye region and the front camera.
Head pose
30Hz
Pitch, Yaw
Head orientation relative to the camera.
Table 1: Sensor data collected or estimated by the Motion Sensing Module.
Figure 3: Schematic overview of the user study design. User Study 1 characterises handheld geometry and motion patterns during everyday-style tasks. User Study 2 evaluates 2D gaze-estimation performance under task-elicited semi-controlled handheld conditions.
Task
Cue Phrase
Document editing
“Create a new document using your preferred document editing app and write today’s diary entry.”
Social chatting
“Open your preferred social media app, such as WhatsApp, and respond to any unread messages.”
Social media browsing
“Open a social media app like Instagram to browse content or like posts.”
Video calling
“Use your preferred video calling app to either simulate a video call or engage in a brief call with a friend.”
News reading
“Open your preferred news app to read some articles.”
Table 2: Cue phrases for everyday-style task prompts.
Table 3: Distribution of data samples across motion conditions and sensor types in User Study 1.
Table 4: Distribution of data samples across motion conditions, sensor types, and experimental phases in User Study 2.
Table 5: Summary of extracted features used in the motion analysis and regression-based factor identification. The full feature set contains 552 dimensions aggregated across raw channels and feature families.
Figure 4: Box plots of sensor data under each motion condition: Ly for lying, St for sitting, Ws for walking slowly and Wm for walking in maze.
Figure 5: t-SNE plots of sensor features under each motion condition
Figure 6: Comparison of the number of motion clusters under different motion conditions
Figure 7: GMM clustering mapped to time-series sensor data on Participant 1 under static conditions
Figure 8: GMM clustering mapped to time-series sensor data on Participant 1 under dynamic conditions
Table 6: Comparison of mean and standard deviation of motion data for in-task and switch-task periods under static conditions.
Table 7: Comparison of mean and standard deviation of motion data for in-task and switch-task periods under dynamic conditions.
Figure 9: Box plots of motion data under each motion condition: Ly for lying, St for sitting, Sd for standing, Wk for walking and Wm for walking in maze.
Figure 10: Cross-comparison t-SNE plots for User Study 1 and 2
Figure 11: Overview of participant-specific calibration and test regimes at different behavioural granularities.
Table 8: Comparison of RMSE for the base model and one-off calibration. Values are mean (standard deviation) in cm.
Figure 12: Held-out RMSE as a function of elapsed time within the test sequence (four equal temporal bins after one-off calibration), per motion condition. Error bars indicate ±1 standard error across participants. Within the present dataset there is no monotonic post-calibration drift across the four bins.
Figure 13: Within-task and between-task RMSE under task-specific calibration.
Figure 14: t-SNE visualisation of calibration and test motion-feature distributions under each motion condition.
Figure 15: Within-motion and between-motion RMSE under motion-specific calibration.
Table 9: Train-test RMSE matrix for the motion-specific calibration setting. Values are mean (standard deviation) in cm.
Table 10: Impact factors identified between and within motion conditions
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Model
R2
MAE (cm)
Top factor groups
Ordinary Least Squares
0.32±0.29
0.66±0.20
distance, orientation, head movement
Ridge
0.32±0.29
0.66±0.20
distance, orientation, head movement
Elastic Net
−0.07±0.15
0.85±0.13
orientation, head movement
Random Forest
0.16±0.29
0.74±0.12
distance, orientation, head movement
Lasso
−0.09±0.21
0.87±0.16
head movement, orientation
Appendix
Table 11: Supplementary comparison of regression models used for factor identification, evaluated by 5-fold cross-validation on participant-by-motion-condition aggregated features ( n=50 cells, 477 standardised features). OLS, Ridge, and Random Forest recover broadly similar geometry-related groups; Lasso is retained for the main analysis because it yields the sparsest coefficient profile.
Motion condition
Head distance
Head movement
Device orientation
Lying
≈ 0%
≈ 0%
≈ 0%
Sitting
≈ 0%
≈ 15%
≈ 20%
Standing
≈ 33%
≈ 18%
≈ 43%
Walking
≈ 0%
≈ 100%
≈ 0%
Walking in maze
≈ 0%
≈ 100%
≈ 0%
Appendix
Table 12: Per-condition decomposition of Lasso coefficient mass into three broad factor groups (head-to-camera distance, head movement, device-orientation change). Percentages represent the share of ∑∣β∣ within that condition. Methodological notes (rounding, residual “other” group, and the degenerate behaviour of the lying and walking conditions under the small per-condition sample size) are given in B .