Deployment-Oriented Session-wise Meta-Calibration for Landmark-Based Webcam Gaze Tracking

Chenkai Zhang

cs.CVMar 12, 2026

Deployment-Oriented Session-wise Meta-Calibration for Landmark-Based Webcam Gaze Tracking

Authors: Chenkai Zhang

Organizations: Independent Researcher, Wenzhou, Zhejiang 325000, China

Abstract

Practical webcam gaze tracking is constrained not only by error, but also by calibration burden, robustness to head motion and session drift, runtime footprint, and browser use. We therefore target a deployment-oriented operating point rather than the image large-backbone regime. We cast landmark-based point-of-regard estimation as session-wise adaptation: a shared geometric encoder produces embeddings that can be aligned to a new session from a small calibration set. We present Equivariant Meta-Calibrated Gaze (EMC-Gaze), a lightweight landmark-only method combining an E(3)-equivariant landmark-graph encoder, local eye geometry, binocular emphasis, auxiliary 3D gaze-direction supervision, and a closed-form ridge calibrator differentiated through episodic meta-training. To reduce pose leakage, we use a two-view canonicalization consistency loss. The deployed predictor uses only facial landmarks and fits a per-session ridge head from brief calibration. In a fixation-style interactive evaluation over 33 sessions at 100 cm, EMC-Gaze achieves 5.79 +/- 1.81 deg RMSE after 9-point calibration versus 6.68 +/- 2.34 deg for Elastic Net; the gain is larger on still-head queries (2.92 +/- 0.75 deg vs. 4.45 +/- 0.30 deg). Across three subject holdouts of 10 subjects each, EMC-Gaze retains an advantage (5.66 +/- 0.19 deg vs. 6.49 +/- 0.33 deg). On MPIIFaceGaze with short per-session calibration, the eye-focused model reaches 8.82 +/- 1.21 deg at 16-shot calibration, ties Elastic Net at 1-shot, and outperforms it from 3-shot onward. The exported eye-focused encoder has 944,423 parameters, is 4.76 MB in ONNX, and supports calibrated browser prediction in 12.58/12.58/12.90 ms per sample (mean/median/p90) in Chromium 145 with ONNX Runtime Web. These results position EMC-Gaze as a calibration-friendly operating point rather than a universal state-of-the-art claim against heavier appearance-based systems.

Figures & tables

Figure 1: EMC-Gaze overview. Landmark-only webcam gaze tracking is decomposed into a shared E(3)-equivariant landmark-graph encoder, a closed-form per-session ridge calibration head fitted on a short support set, and low-capacity deployment refinements. During meta-training, the encoder is optimized through the ridge solution together with canonicalization-consistency and pursuit-continuity regularization.

Method	Still ( ∘ )	Pose ( ∘ )	Overall ( ∘ )
EMC-Gaze (ours, pursuit-enhanced)	2.92 $\pm$ 0.75	6.42 $\pm$ 1.89	5.79 $\pm$ 1.81
Elastic Net	4.45 $\pm$ 0.30	7.11 $\pm$ 2.49	6.68 $\pm$ 2.34
Ridge	3.78 $\pm$ 2.20	31.07 $\pm$ 13.35	27.28 $\pm$ 11.27
Meta GNN	3.68 $\pm$ 2.20	29.72 $\pm$ 11.98	26.41 $\pm$ 10.18

Table 1: Interactive evaluation across 33 runs at an approximate viewing distance of

100\text{\,}\mathrm{c}\mathrm{m}

. Values report angular RMSE (degrees; mean

\pm

std) on the query split.

Method	Held-out overall ( ∘ )
EMC-Gaze (ours, pursuit-enhanced)	5.66 $\pm$ 0.19
Elastic Net	6.49 $\pm$ 0.33
Ridge	27.36 $\pm$ 4.03
Meta GNN	33.23 $\pm$ 3.33

Table 2: Primary retrospective subject-disjoint evaluation over 35 reconstructed subject groups. We sample three randomized held-out sets of 10 March 5 subjects (10 sessions) and retrain encoder-based models on the remaining 50 sessions for each split. Values report overall angular RMSE (degrees; mean

\pm

std across split means).

Method	Calibration shots
	1	3	5	9	16
EMC-Gaze (ours, eye-focused, MPII-only)	14.95 $\pm$ 1.67	10.61 $\pm$ 1.42	9.73 $\pm$ 1.31	9.24 $\pm$ 1.27	8.82 $\pm$ 1.21
Elastic Net	14.95 $\pm$ 1.67	11.72 $\pm$ 1.41	11.15 $\pm$ 1.31	10.95 $\pm$ 1.31	10.83 $\pm$ 1.31
Ridge	14.95 $\pm$ 1.67	15.40 $\pm$ 2.75	14.52 $\pm$ 2.76	13.67 $\pm$ 2.48	12.85 $\pm$ 2.09

Table 3: Leave-one-person-out MPIIFaceGaze benchmark with k-shot per-session calibration. Values report subject-macro angular RMSE (degrees; mean

\pm

std across the 15 held-out subjects).

Method	Trajectory RMSE ( ∘ )	Delta RMSE ( ∘ )	Std ratio (x/y)
EMC-Gaze (ours, pursuit-enhanced)	2.78 $\pm$ 0.25	0.93 $\pm$ 0.11	0.95 / 0.64
Elastic Net	4.86 $\pm$ 0.08	0.57 $\pm$ 0.02	0.22 / 0.20
Ridge	3.07 $\pm$ 0.36	0.99 $\pm$ 0.16	1.08 / 0.99
Meta GNN	3.15 $\pm$ 0.53	1.08 $\pm$ 0.15	1.04 / 1.03

Table 4: Smooth-pursuit evaluation across 10 pursuit sessions at an approximate viewing distance of

100\text{\,}\mathrm{c}\mathrm{m}

. Values report mean

\pm

std; lower is better except for the standard-deviation ratio, where values far below 1 indicate a collapsed trajectory.

Variant	Still ( ∘ )	Pose ( ∘ )	Overall ( ∘ )
EMC-Gaze core (without pursuit extension)	3.15 $\pm$ 0.96	6.05 $\pm$ 1.51	5.48 $\pm$ 1.38
No canonicalization consistency loss ( $\lambda_{\mathrm{cons}}{=}0$ )	3.15 $\pm$ 1.00	6.19 $\pm$ 1.62	5.59 $\pm$ 1.45
No iris-distance invariant features	5.19 $\pm$ 0.04	5.93 $\pm$ 0.05	5.72 $\pm$ 0.04
No pose residual head	3.13 $\pm$ 0.95	6.01 $\pm$ 1.59	5.45 $\pm$ 1.44
Replace equivariant encoder with MLP encoder	5.14 $\pm$ 0.05	5.90 $\pm$ 0.08	5.69 $\pm$ 0.06

Table 5: Ablations on the core fixation-style EMC model across 33 interactive runs. Values report angular RMSE (degrees; mean

\pm

std) on the query split.

Figure 2: Accuracy. (Left) Distribution of overall angular RMSE across the 33 interactive evaluation runs. Each point is one run; boxes summarize the run distribution. (Right) Mean angular error per target location (degrees) for EMC-Gaze on the representative run.

Figure 3: Efficient calibration. Aggregate angular RMSE (degrees) versus number of calibration targets

k

across the 33 interactive evaluation runs. Curves show mean performance; shaded bands show

\pm 1

standard deviation. The top axis reports approximate average calibration time for the EMC-Gaze prefix curve.

Figure 4: Robustness to head pose. Mean per-phase angular RMSE (degrees) across the 33 interactive evaluation runs under yaw/pitch/roll pose-hold blocks (still-head calibration support only). Error bars denote

\pm 1

standard deviation across runs.

Figure 5: Stability. CDF of within-target prediction jitter (degrees) on the representative run, computed as the distance from each prediction to that target’s mean prediction. Lower is more stable.

Characteristic	Value
Exported encoder	944,423 parameters; $4.76\text{\,}\mathrm{M}\mathrm{B}$ ONNX artifact
Deployed calibrated core	944,521 learned parameters including the 98-parameter pose residual
Runtime calibration head	Closed-form ridge regressor fit per session at runtime
Calibration burden	9 targets, 45 still-head support samples in the main interactive protocol
Browser demo calibration	User-facing anchor-plus-pursuit route with optional click-based online updates
Runtime stack	MediaPipe Tasks, ONNX Runtime Web, client-side ridge calibration/inference

Table 6: Deployment summary for the flagship eye-focused EMC-Gaze runtime. Values correspond to the exported browser-capable configuration discussed in the text.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Overall accuracy comparison. Mean overall angular RMSE (degrees) across the 33 interactive evaluation runs, with error bars indicating

\pm 1

standard deviation.

Figure 7: Calibration efficiency (prefix vs random). Representative-run comparison in which prefix order uses the first

k

calibration targets and random averages over random subsets (mean

\pm

std).

Gaze Estimation

Gaze estimation predicts where a person is looking from eye, face, head-pose, and scene cues.

See all 66 papers

Mar 25, 2026

Is Geometry Enough? An Evaluation of Landmark-Based Gaze Estimation

Appearance-based gaze estimation frequently relies on deep Convolutional Neural Networks (CNNs). These models are accurate, but computationally expensive and act as "black boxes", offering little interpretability. Geometric methods based on facial landmarks are a lightweight alternative, but their performance limits and generalization capabilities remain underexplored in modern benchmarks. In this study, we conduct a comprehensive evaluation of landmark-based gaze estimation. We introduce a standardized pipeline to extract and normalize landmarks from three large-scale datasets (Gaze360, ETH-XGaze, and GazeGene) and train lightweight regression models, specifically Extreme Gradient Boosted trees and two neural architectures: a holistic Multi-Layer Perceptron (MLP) and a siamese MLP designed to capture binocular geometry. We find that landmark-based models exhibit lower performance in within-domain evaluation, likely due to noise introduced into the datasets by the landmark detector. Nevertheless, in cross-domain evaluation, the proposed MLP architectures show generalization capabilities comparable to those of ResNet18 baselines. These findings suggest that sparse geometric features encode sufficient information for robust gaze estimation, paving the way for efficient, interpretable, and privacy-friendly edge applications. The source code and generated landmark-based datasets are available at https://github.com/daniele-agostinelli/LandmarkGaze.git.

Daniele Agostinelli, Thomas Agostinelli, Andrea Generosi +1

Department of Industrial Engineering and Mathematical Sciences, Universit`a Politecnica delle Marche, via Brecce Bianche, 12, Ancona, 60131, Italy · Department of Science and Information Technology, Universit`a Pegaso, piazza Trieste e Trento, 48, Napoli, 80132, Italy

Gaze Estimation Domain Generalization

Ask PDF

May 8, 2026

PicoEyes: Unified Gaze Estimation Framework for Mixed Reality with a Large-Scale Multi-View Dataset

We present PicoEyes, a unified gaze estimation framework that directly predicts all key attributes of gaze, including 3D eye parameters, eye-region segmentation, optical axis, visual axis, and depth maps, from either monocular or binocular inputs. The framework simultaneously addresses calibration, gaze forecasting, and varying device postures, while also supporting 3D eye reconstruction via joint estimation of eye parameters and depth maps in an end-to-end manner. In addition, we introduce a large-scale multi-view near-eye dataset containing comprehensive 2D and 3D annotations under diverse conditions, including train, test, rewear-test, and calibration sessions. Extensive experiments demonstrate that PicoEyes achieves state-ofthe-art performance, consistently outperforming both academic and industrial gaze tracking methods across nocalibration, calibration, rewear-after-calibration, and forecasting settings. This work establishes a practical, end-toend paradigm for robust and generalizable gaze estimation in mixed reality (MR) applications.

Fuxin Duan, Hui Wang

Pico, Bytedance

Gaze Estimation Mixed Reality Eye Tracking Gaze Forecasting

Ask PDF

Sep 9, 2026

Marker-free eye-gaze estimation using a single image and depth from defocus

This paper presents a marker-free eye-gaze estimation approach using a single 2D camera, such as an integrated laptop webcam. The gaze-related features are estimated from iris localization and head pose estimated by using depth from defocus. A variational Bayesian multinomial logistic regression framework is used as mapping from the estimated features to the position of regard, based on an 8-dimensional feature vector of head-pose and iris-displacement parameters. No external marker is needed. Experiments were conducted by estimating the gaze of people watching a computer screen at different distances and compared against five existing methods. The obtained scores demonstrate the effectiveness of the proposed approach.

David Hurtubise-Martin, Feriel Fass, Djemel Ziou +1

D´epartement d’informatique, Facult´e des sciences, Universit´e de Sherbrooke, Qu´ebec, Canada

Gaze Estimation Eye Tracking

Ask PDF

Abstract

Gaze Estimation

Explore similar work

Is Geometry Enough? An Evaluation of Landmark-Based Gaze Estimation

PicoEyes: Unified Gaze Estimation Framework for Mixed Reality with a Large-Scale Multi-View Dataset

Marker-free eye-gaze estimation using a single image and depth from defocus