Diffusion models for eye-gaze trajectory generation using position and velocity representations
Authors: Laxman Basnet, Alexander Szorkovszky, Pedro G. Lind, Anis Yazidi, Shailendra Bhandari
Organizations: Department of Computer Science, Oslo Metropolitan University, Oslo, Norway · Department of Numerical Analysis and Scientific Computing, Simula Research Laboratory, Oslo, Norway · Department of Technology, Kristiania University of Applied Sciences, Oslo, Norway · Department of Informatics, University of Oslo, Oslo, Norway
Eye-tracking data are expensive to collect, requiring specialized hardware and controlled laboratory conditions, and difficult to share because of privacy constraints. We address this using two complementary denoising diffusion probabilistic models (DDPMs) for unconditional generation of eye-gaze dynamics from visual-search data. Both use an identical FiLM-conditioned one-dimensional U-Net with self-attention (19.35,M parameters), trained on 8,s sliding-window sequences from 28 participants. One model generates raw two-dimensional gaze-position sequences, while the other generates two-component velocity sequences; each uses representation-specific preprocessing, training settings, data partitions, and evaluation protocols. Both are evaluated across three independent training seeds, with aggregated metrics reported as mean,±,SD. The position-space model achieves a mean Jensen-Shannon (JS) divergence of 0.016±0.004 across nine kinematic features, with the highest feature-wise mean below 0.030, fixation duration within 2% of real data, and a Fr'echet Gaze Distance more than an order of magnitude below statistical and Markovian baselines. Under a Train-on-Synthetic-Test-on-Real protocol, synthetic-only training achieves R2=0.66±0.02, or 82.7% of the real-data R2 point estimate. The velocity-space model achieves a mean JS divergence of 0.0065 across velocity components, speed, log-speed, and turning angle, with a maximum of 0.015±0.005. Reconstructed path length is less accurate (0.21±0.02 versus 0.03±0.01 in position space), although the protocols differ. Overall, unconditional diffusion captures local gaze kinematics and short-range temporal and directional structure, while long-range properties such as saccade counts and cumulative path geometry remain targets for future conditioned models.
Figures & tables
Figure 1: Example recording in both model representations. (a) Gaze path for the full 45 s trial. (b–e) One 8 s segment ( L=2,000 samples, fs=250 Hz) from the same recording: (b) position x(τ) , y(τ) , where fixations appear as plateaus and saccades as steps (input to the position-space model). (c, d) Velocity components vx(τ) , vy(τ) (Eq. 3 ), where saccades become sparse high-amplitude spikes (input to the velocity-space model). (e) Scalar speed vx2+vy2 .
Figure 2: Shared denoising architecture. Position or velocity windows X0∈R2×L are noised to Xt , and the 1D U-Net predicts εθ(Xt,t) . Deterministic DDIM sampling ( η=0 ) produces X^0 ; training and sampling settings differ between representations.
Feature
KS
JS
Mean speed
0.149±0.036
0.0249±0.0096
Max speed
0.150±0.034
0.0152±0.0062
Total path length
0.149±0.036
0.0249±0.0096
x -range
0.159±0.020
0.0122±0.0035
y -range
0.115±0.012
0.0067±0.0018
x -std
0.186±0.016
0.0206±0.0030
Table 1: Position-space model. Feature-level agreement with real held-out trajectories, mean ± SD over three training seeds, validation segments per seed, split held fixed. The two metrics are the KS statistic and the JS divergence (lower values indicate better agreement).
Figure 3: Representative gaze trajectories generated by the position-space DDPM (DDIM, 100 steps, η=0 ). Each panel shows the xy path of one generated 8 s sequence in normalized coordinates.
Figure 4: Gaze-trajectory statistics for empirical data, the position-space DDPM, and the kinematic Markov baseline following Lencastre et al. [46] : (a) speed ∣u∣ , (b) autocorrelation of ∣u∣ , (c) movement direction θ , and (d) turning angle ϕ .
Figure 5: Real and generated gaze-velocity comparison for a representative seed. Pooled empirical distributions over all segments and time points for (a) vx , (b) vy , and (c) scalar speed vx2+vy2 . All three seeds yield closely matching distributions; aggregate metrics are reported as mean ± SD over seeds in Table 2 .
Quantity
JS
Δρ1
DPSD
vx
(2.19±0.50)×10−3
0.0278±0.0038
(1.31±0.02)×10−4
vy
(7.31±0.89)×10−3
0.0568±0.0092
(1.52±0.04)×10−4
Speed
(8.34±1.36)×10−3
0.0157±0.0026
(1.82±0.12)×10−4
Log-speed
0.0146±0.0049
0.0530±0.0115
(1.72±0.36)×10−4
Signed turning angle
(3.0±2.0)×10−4
–
–
Path length
0.207±0.020
–
–
Table 2: Velocity-space model: agreement with real held-out data, reported as mean ± SD over three training seeds (fixed 80/20 participant-level split).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Quantity
Position-space
Velocity-space
Data
Split mode
Participant-stratified, 80/20, split
Participant-level, split
Validation fraction
0.2
0.2
Sampling rate
250 Hz
Window length
2,000 samples (8 s)
Window stride
250 samples
non-overlapping
Participants / segments
28 / 4,401
–
Appendix
Table 3: Combined hyperparameter configuration. Rows with a single value are identical for both models; rows with two values differ. Architecture and diffusion settings are fully shared.
Figure 6: Marginal distributions of the nine kinematic features for real (blue) and generated (red) position-space trajectories ( n=1,136 each); numerical values in Table 1 .
Figure 7: Velocity-space temporal diagnostics for a representative seed. (a) Mean speed profile across segments. (b) Average speed autocorrelation. (c) Normalized speed power spectrum.
Feature
Real μ
KS
JS
Mean speed
0.0058
0.731
0.347
Max speed
0.2095
0.460
0.355
x -range
1.4699
0.589
0.225
y -range
1.3502
0.348
0.108
x -std
0.4405
0.339
0.112
y -std
0.3815
0.157
0.049
Appendix
Table 4: Kinematic Markov baseline [ 46 ] : real-data feature means and per-feature agreement with real held-out trajectories. The baseline is deterministic given the training set, so no seed spread applies. Compared over the eight shared non-redundant features, the position-space DDPM reduces mean KS by 71.4% and mean JS by 93.0% . It also improves on the baseline for every individual feature.
Feature
W 1
Mean speed
0.0006±0.0002
Max speed
0.0379±0.0075
Total path length
1.2195±0.3671
x -range
0.1355±0.0254
y -range
0.1071±0.0056
x -std
0.0631±0.0065
Appendix
Table 5: Position-space model: Wasserstein-1 distance per kinematic feature, mean ± SD over three seeds ( n=1,136 per seed). W 1 is reported in each feature’s own units and is therefore not comparable across rows; total path length is 1,999× mean speed by construction, which is why its W 1 is correspondingly larger.
Setting
KS
JS
DDIM steps = 10
0.191
0.0565
DDIM steps = 25
0.211
0.0348
DDIM steps = 50
0.180
0.0243
DDIM steps = 100 (chosen)
0.158
0.0186
DDIM steps = 200
0.151
0.0140
η=0.0
0.135
0.0135
Appendix
Table 6: Position-space sampling ablations for the reference run. The DDIM-step and EMA rows report KS/JS for mean speed. The η rows report the mean over the eight non-redundant features. Lower is better.
Metric
μr
μg
KS
JS
Fixation duration (s)
0.344
0.351
0.053
0.0017
Fixation dispersion
0.0074
0.0108
0.162
0.0442
Fixation count / traj.
21.81
21.41
0.112
0.0155
Saccade amplitude
0.183
0.217
0.104
0.0027
Saccade peak speed
0.0624
0.0618
0.049
0.0012
Saccade duration (s)
0.0167
0.0195
0.111
0.0057
Appendix
Table 7: I-VT fixation and saccade statistics, real vs. generated ( n=1,136 each). μr,μg : real/generated means.
Condition
R2
MAE
TRTR (real → real, 5-fold)
0.80±0.05
1.9219
TSTR (synthetic → real, 3 seeds)
0.67±0.02
2.74±0.10
AUG (real+synthetic → real)
0.79±0.02
2.04±0.09
Appendix
Table 8: Downstream TSTR protocol: MLP fixation-count regressor. TRTR uses real data only and is therefore identical across generative seeds; its reported spread is the five-fold cross-validation SD and serves as a real-data reference rather than a statistical upper bound.
Diffusion models are increasingly used to generate synthetic training data, but precise label control remains difficult when the conditioning signal is low-dimensional and coarse. Text-conditioned images are judged by broad prompt consistency, whereas supervised training requires precise correspondence between each image and its numerical label. This is challenging in eye tracking, where a 4D binocular gaze is expressed through subtle, spatially localized pupil and iris geometry. We introduce GazeDiT, a diffusion model that generates images for a requested 4D gaze through an internally constructed spatial condition that grounds the global gaze label in this local geometry. During training, a frozen SegFormer extracts pupil/iris geometry from diverse real images, allowing the model to learn realistic appearance conditioned on that geometry. At inference, a physical eye renderer samples gaze-consistent geometries by varying anatomy and camera state, enabling diverse synthesis without a source image. GazeDiT achieves substantially lower tail gaze-label error than other diffusion baselines, approaching the error of the same frozen gaze estimator on real images. Its generated data also improves the downstream eye tracker, reducing gaze error on difficult cases from 3.05° to 2.80° in the smallest cohort.
We study the problem of human gaze modeling, which aims to generate the gaze patterns a viewer produces while observing a visual stimulus. Gaze is primarily captured through two modalities: continuous eye-tracking trajectories, which describe fine-grained motion dynamics, and discrete scanpaths, which describe high-level fixation structure. Because gaze varies substantially across viewers and trials, we treat this variability as a defining property rather than noise and model gaze as a stochastic generative process. Existing generative gaze models supervise on only one of these two representations in isolation. We hypothesize that trajectories and scanpaths describe gaze at complementary scales and are jointly informative during training, and test this hypothesis through ST-DiffEye, a joint trajectory-scanpath diffusion framework that couples both modalities by concatenating them as an additional raw input channel, requiring no architectural overhead beyond an input and output channel expansion. We further introduce a principled evaluation framework based on the Continuous Ranked Probability Score (CRPS), which generalizes any existing sequence similarity metric into a proper scoring rule that jointly assesses the accuracy and diversity of generated gaze. Experiments on task-driven visual search, covering both target-present and target-absent scenarios, and on free-viewing benchmarks demonstrate state-of-the-art performance. These results, along with detailed ablations, confirm the benefit of joint modeling and the value of distribution-aware evaluation in capturing the intrinsic variability of human gaze. Project webpage: https://st-diffeye.github.io/
Eye movement biometrics (EMB) is an emerging behavioral modality for user authentication, particularly in virtual- and augmented-reality systems, where gaze dynamics contain distinctive subject-specific features. However, robust EMB systems require diverse, high-quality gaze recordings that are expensive to collect and often unavailable at the scale needed for model development. Generative models can mitigate data scarcity, but existing methods either synthesize generic gaze behavior or personalize signals primarily by identity, without jointly representing the user's task and subjective state. Consequently, generated signals may appear visually realistic while failing to retain the behavioral properties required for biometric applications. To address this limitation, we propose EyeMakeYou, a multi-conditional denoising diffusion framework for subject-specific, high-frequency gaze synthesis. EyeMakeYou generates 5-s, 1000-Hz bivariate gaze-velocity sequences from an identity-removed reference trajectory and conditions the denoising process on an identity embedding, a task embedding, and self-reported ratings of overall difficulty, mental tiredness, and eye tiredness. Its objective combines diffusion noise prediction and identity preservation with multi-resolution spectral, drift-consistency, and event-weighted local-smoothness losses. Experiments on GazeBase show that EyeMakeYou achieves higher median spatial accuracy and greater real--synthetic similarity in the embedding feature space than the existing generative approaches, while retaining selected task-dependent associations between subjective reports and oculomotor features. These findings support conditional diffusion as a practical approach for augmenting gaze datasets for biometric and interactive applications.
Kamrul Hasan, Mehedi Hasan Raju, Oleg V. Komogortsev