Characterizing Scanpath Decoding Pipelines with Perturbation Profiles
Organizations: Department of Bioinformatics, Kish International Campus, University of Tehran (Tehran office), Tehran 1417955391, Iran · Department of Computer Science, School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran 1417614411, Iran · BRAINS, Gisma University of Applied Sciences, Potsdam, Germany
Abstract
When a scanpath descriptor supports a classification, it is rarely clear what the classification rests on. We propose perturbation profiles. Five transformations each target one stated property of every scanpath (fixation order, the coupling of durations to positions, the duration distribution, orientation and handedness, and fine spatial structure) while preserving a listed set of invariants. The classifier is refit, and the change in participant-held-out AUC is the pipeline's sensitivity to that property. Intervals come from a participant bootstrap that refits the classifier; in simulations they were conservative (99.7% coverage; standard errors about 1.8 times the empirical spread). We profiled nine descriptors on thirteen binary problems from three public datasets, with a within-fold AUC and class-balanced folds and a stated rule for which sensitivities to interpret. Of 565 intervals, 95 excluded zero; 80 were robust to the pooled AUC, 20 fold partitions and longer runs (36 of them within +/-0.05 AUC), and 45 also held with boosted trees. Most came from GazeBase (322 participants). Permuting fixation order raised the AUC of a descriptor based on mean saccade amplitude, because permuted amplitude has as its expectation the mean pairwise distance, a measure of spread. A "comprehension" contrast pooled over two programs was a contrast between their layouts: it vanished among trials of the same program, and synthetic gaze drawn from the layout alone reproduced the program contrast beneath it. For native versus non-native readers, spatial jitter at the scale of text lines reduced recurrence's discrimination. A Python package is released.
Figures & tables
| Transf. | Operation | Destroys | Preserves | Not removed |
|---|---|---|---|---|
| order | permute the fixation sequence | serial order | multisets of locations and durations; every order-free property | — |
| duration coupling | permute durations across the trial’s fixations | which fixation carries which duration, and with it any dependence of duration on sequence position (trends, serial autocorrelation) | locations, order, the trial’s duration distribution (mean, spread, total dwell, count) | marginal duration information |
| duration marginal | redraw the trial’s durations from the pooled durations of the other participants | the trial’s duration distribution, the coupling and the duration–sequence dependence | locations, order, count | — |
| rotation | rigid rotation (uniform angle) and random reflection about a centre | orientation, side relative to the stimulus, and handedness (chirality) | all pairwise distances and unsigned angles, order, durations, each fixation’s eccentricity from the centre | eccentricity; any rotationally invariant location contrast (exact) |
| jitter( ) | add isotropic Gaussian noise, SD , to every location | fine spatial structure, increasingly with (shorthand; not a band-pass filter) | order and durations (exact); orientation and structure coarser than approximately, at the distribution level | — |
| Problems | Label assigned to | Unit permuted | Block | Preserved under the null |
|---|---|---|---|---|
| gb_tex_vd1 , gb_vd1_blg , gb_vd1_vd2 , gb_tex_s1s2 , em_program | trial (task, stimulus or session) | trial | participant | each participant’s label counts; participant clustering |
| gb_vd1_halves , em_halves | half of one recording | half | recording (participant session or program) | one first and one second half per recording |
| sb_native , em_expertise , em_correctrect , em_correctveh | participant | participant (all trials together) | all participants | the number of participants per class; all trials of a participant share one label |
| sb_compr | passage (its pages share one outcome) | passage | participant | each participant’s passage-label counts; pages of a passage share one label |
| em_correct | answer to one program | trial | program | the number of correct and incorrect answers within each program |
| Design | Size | Zero-target int. | Excl. 0 | Rate (%) | Coverage | Width |
|---|---|---|---|---|---|---|
| no signal | 60 4 (W) | 120 | 0 | 0.0 0.0 | 1.000 | 0.351 |
| no signal | 112 2 (W) | 120 | 0 | 0.0 0.0 | 1.000 | 0.354 |
| no signal | 95 20 (B) | 120 | 2 | 1.7 1.2 | 0.988 | 0.115 |
| no signal | 65 2 (B) | 120 | 0 | 0.0 0.0 | 0.997 | 0.457 |
| signal elsewhere | 60 4 (W) | 80 | 0 | 0.0 0.0 | 1.000 | 0.167 |
| signal elsewhere | 112 2 (W) | 80 | 0 | 0.0 0.0 | 1.000 | 0.169 |
| Key | Dataset | Contrast | Trials | Part. |
|---|---|---|---|---|
| gb_tex_vd1 | GazeBase | reading vs. video viewing (within) | 1288 | 322 |
| gb_vd1_blg | GazeBase | video viewing vs. ball-catching game (within) | 1287 | 322 |
| gb_vd1_vd2 | GazeBase | video segment 1 vs. 2 (within) | 1288 | 322 |
| gb_vd1_halves | GazeBase | video 1, first vs. second half (within) | 1288 | 322 |
| gb_tex_s1s2 | GazeBase | reading, session 1 vs. 2 (within) | 644 | 322 |
| sb_native | SB-SAT | native vs. non-native English readers, reading pages (between) | 2046 | 95 |
| Problem | Count | Dispersion | Classic | Dur. trend | RQA | RQA | Directed | FAG lattice | FAG AOI | FAG AOI pad. |
|---|---|---|---|---|---|---|---|---|---|---|
| gb_tex_vd1 | 0.969 (.001) | 1.000 (.001) | 0.969 (.001) | 0.957 (.001) | 0.999 (.001) | 1.000 (.001) | 0.977 (.001) | 0.988 (.001) | – | – |
| gb_vd1_blg | 0.690 (.001) | 1.000 (.001) | 0.886 (.001) | 0.752 (.001) | 0.999 (.001) | 1.000 (.001) | 1.000 (.001) | 0.957 (.001) | – | – |
| gb_vd1_vd2 | 0.634 (.001) | 0.707 (.001) | 0.640 (.001) | 0.790 (.001) | 0.677 (.001) | 0.719 (.001) | 0.700 (.001) | 0.664 (.001) | – | – |
| gb_vd1_halves | 0.500 (1.000) | 0.924 (.001) | 0.564 (.001) | 0.590 (.001) | 0.743 (.001) | 0.823 (.001) | 0.805 (.001) | 0.741 (.001) | – | – |
| gb_tex_s1s2 | 0.530 (.013) | 0.519 (.243) | 0.539 (.003) | 0.549 (.010) | 0.576 (.001) | 0.521 (.134) | 0.531 (.090) | 0.591 (.001) | – | – |
| sb_native | 0.624 (.003) | 0.545 (.169) | 0.689 (.001) | 0.654 (.005) | 0.614 (.005) | 0.669 (.002) | 0.614 (.008) | 0.587 (.023) | – | – |
| Problem | Intervals | Excluding zero | Interpreted | Robust | Robust, both learners |
|---|---|---|---|---|---|
| gb_tex_vd1 | 37 | 20 | 20 | 20 | 9 |
| gb_vd1_blg | 37 | 16 | 16 | 15 | 8 |
| gb_vd1_vd2 | 37 | 21 | 21 | 19 | 13 |
| gb_vd1_halves | 37 | 20 | 20 | 20 | 12 |
| gb_tex_s1s2 | 37 | 2 | 2 | 0 | 0 |
| sb_native | 37 | 5 | 4 | 3 | 1 |
| Problem | Descriptor | Transformation | AUC | Sensitivity [95% interval] | Normalized | Trees |
|---|---|---|---|---|---|---|
| gb_tex_vd1 | Dur. trend | duration marginal | 0.957 | +0.473 [+0.442, +0.503] | ||
| gb_vd1_blg | Classic | order | 0.886 | 0.114 [ 0.137, 0.091] | ||
| gb_vd1_blg | Dur. trend | duration marginal | 0.752 | +0.252 [+0.199, +0.304] | ||
| gb_vd1_vd2 | Classic | duration marginal | 0.640 | +0.124 [+0.099, +0.150] | ||
| gb_vd1_vd2 | Dispersion | jitter | 0.707 | +0.116 [+0.085, +0.148] | ||
| gb_vd1_vd2 | Dur. trend | duration coupling | 0.790 | +0.158 [+0.131, +0.185] |
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.969 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 1.000 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.021 [+0.01, +0.03] | +0.000 [ 0.00, +0.00] | 0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] |
| Classic | 0.969 | 0.022 [ 0.03, 0.01] | 0 (ex.) | +0.035 [+0.02, +0.05] | 0 (ex.) | 0.003 [ 0.01, 0.00] | 0.006 [ 0.01, 0.00] | 0.007 [ 0.01, 0.00] | 0.004 [ 0.01, +0.00] |
| Dur. trend | 0.957 | +0.003 [ 0.00, +0.01] | +0.003 [ 0.00, +0.01] | +0.473 [+0.44, +0.50] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.999 | +0.010 [+0.00, +0.02] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.001 [ 0.00, +0.00] | +0.002 [ 0.00, +0.00] | +0.004 [+0.00, +0.01] | +0.015 [+0.01, +0.02] |
| RQA | 1.000 | +0.006 [+0.00, +0.01] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.000 [ 0.00, +0.00] | +0.001 [ 0.00, +0.00] | +0.001 [ 0.00, +0.00] | +0.006 [+0.00, +0.01] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.690 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 1.000 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] | 0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] |
| Classic | 0.886 | 0.114 [ 0.14, 0.09] | 0 (ex.) | +0.033 [+0.02, +0.05] | 0 (ex.) | 0.003 [ 0.00, 0.00] | 0.007 [ 0.01, 0.00] | 0.014 [ 0.02, 0.01] | 0.014 [ 0.02, 0.01] |
| Dur. trend | 0.752 | +0.011 [ 0.00, +0.02] | +0.011 [ 0.00, +0.02] | +0.252 [+0.20, +0.30] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.999 | +0.013 [+0.01, +0.02] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.000 [ 0.00, +0.00] | 0.000 [ 0.00, +0.00] | 0.000 [ 0.00, +0.00] | +0.003 [+0.00, +0.01] |
| RQA | 1.000 | +0.001 [+0.00, +0.00] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.000 [ 0.00, +0.00] | 0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] | +0.000 [ 0.00, +0.00] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.634 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.707 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.098 [+0.07, +0.13] | +0.004 [ 0.00, +0.01] | +0.012 [+0.00, +0.02] | +0.045 [+0.03, +0.06] | +0.116 [+0.08, +0.15] |
| Classic | 0.640 | +0.007 [ 0.00, +0.01] | 0 (ex.) | +0.124 [+0.10, +0.15] | 0 (ex.) | +0.001 [ 0.00, +0.00] | +0.004 [+0.00, +0.01] | +0.005 [ 0.00, +0.01] | +0.005 [ 0.00, +0.01] |
| Dur. trend | 0.790 | +0.157 [+0.13, +0.19] | +0.158 [+0.13, +0.19] | +0.286 [+0.23, +0.34] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.677 | +0.035 [+0.01, +0.06] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.006 [ 0.03, +0.01] | +0.002 [ 0.03, +0.03] | +0.046 [+0.00, +0.09] | +0.108 [+0.07, +0.14] |
| RQA | 0.719 | +0.073 [+0.04, +0.10] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.007 [ 0.01, +0.02] | +0.013 [ 0.01, +0.03] | +0.043 [+0.01, +0.08] | +0.118 [+0.08, +0.16] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.500 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.924 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.249 [+0.22, +0.27] | +0.001 [ 0.00, +0.00] | +0.007 [+0.00, +0.01] | +0.029 [+0.02, +0.04] | +0.130 [+0.11, +0.15] |
| Classic | 0.564 | 0.090 [ 0.11, 0.07] | 0 (ex.) | +0.008 [ 0.01, +0.03] | 0 (ex.) | 0.001 [ 0.00, +0.00] | +0.002 [ 0.00, +0.01] | +0.005 [ 0.01, +0.02] | +0.015 [ 0.00, +0.03] |
| Dur. trend | 0.590 | +0.076 [+0.03, +0.12] | +0.085 [+0.04, +0.13] | +0.082 [+0.04, +0.12] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.743 | +0.046 [+0.02, +0.07] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.012 [ 0.01, +0.03] | +0.025 [ 0.00, +0.05] | +0.079 [+0.05, +0.11] | +0.143 [+0.10, +0.19] |
| RQA | 0.823 | +0.088 [+0.06, +0.11] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.003 [ 0.01, +0.02] | +0.026 [+0.01, +0.05] | +0.088 [+0.05, +0.12] | +0.182 [+0.15, +0.21] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.530 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.519 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.036 [ 0.05, +0.12] | +0.003 [ 0.01, +0.01] | 0.000 [ 0.03, +0.03] | +0.011 [ 0.03, +0.06] | +0.031 [ 0.01, +0.07] |
| Classic | 0.539 | +0.005 [ 0.02, +0.03] | 0 (ex.) | +0.031 [ 0.04, +0.10] | 0 (ex.) | +0.001 [ 0.00, +0.01] | 0.000 [ 0.01, +0.01] | 0.005 [ 0.02, +0.02] | 0.001 [ 0.03, +0.03] |
| Dur. trend | 0.549 | +0.014 [ 0.04, +0.07] | +0.016 [ 0.03, +0.06] | +0.036 [ 0.05, +0.12] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.576 | +0.093 [+0.01, +0.17] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.037 [ 0.05, +0.12] | +0.071 [+0.02, +0.12] | +0.072 [ 0.01, +0.15] | +0.062 [ 0.03, +0.16] |
| RQA | 0.521 | +0.047 [ 0.03, +0.12] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.007 [ 0.05, +0.04] | +0.017 [ 0.06, +0.09] | +0.028 [ 0.06, +0.12] | +0.007 [ 0.08, +0.09] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.624 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.545 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.005 [ 0.06, +0.07] | +0.000 [ 0.01, +0.01] | +0.000 [ 0.02, +0.02] | 0.003 [ 0.04, +0.03] | +0.006 [ 0.06, +0.07] |
| Classic | 0.689 | +0.037 [ 0.08, +0.15] | 0 (ex.) | +0.017 [ 0.07, +0.10] | 0 (ex.) | +0.001 [ 0.00, +0.00] | +0.004 [ 0.00, +0.01] | +0.012 [ 0.01, +0.03] | +0.022 [ 0.04, +0.08] |
| Dur. trend | 0.654 | +0.011 [ 0.04, +0.06] | +0.012 [ 0.03, +0.06] | +0.144 [+0.00, +0.29] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.614 | +0.074 [ 0.01, +0.16] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.012 [ 0.02, +0.04] | +0.029 [ 0.01, +0.07] | +0.079 [+0.01, +0.15] | +0.088 [+0.01, +0.16] |
| RQA | 0.669 | +0.105 [ 0.00, +0.21] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.006 [ 0.03, +0.02] | +0.015 [ 0.02, +0.05] | +0.086 [+0.02, +0.16] | +0.140 [+0.05, +0.23] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.554 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.413 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.047 [ 0.12, +0.03] | 0.004 [ 0.02, +0.01] | 0.007 [ 0.03, +0.01] | 0.035 [ 0.06, 0.01] | 0.041 [ 0.11, +0.03] |
| Classic | 0.587 | +0.010 [ 0.07, +0.09] | 0 (ex.) | +0.026 [ 0.09, +0.14] | 0 (ex.) | +0.000 [ 0.00, +0.00] | +0.001 [ 0.01, +0.01] | +0.000 [ 0.01, +0.01] | +0.003 [ 0.04, +0.05] |
| Dur. trend | 0.575 | 0.006 [ 0.04, +0.03] | 0.007 [ 0.03, +0.02] | +0.070 [ 0.05, +0.19] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.497 | +0.011 [ 0.12, +0.14] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.009 [ 0.06, +0.07] | 0.003 [ 0.10, +0.09] | +0.001 [ 0.08, +0.08] | 0.005 [ 0.11, +0.09] |
| RQA | 0.529 | +0.050 [ 0.04, +0.14] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.017 [ 0.06, +0.03] | 0.014 [ 0.07, +0.04] | +0.006 [ 0.05, +0.07] | +0.032 [ 0.08, +0.15] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.561 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.957 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.139 [+0.08, +0.20] | 0.001 [ 0.01, +0.00] | 0.001 [ 0.01, +0.01] | +0.001 [ 0.01, +0.01] | +0.002 [ 0.01, +0.02] |
| Classic | 0.539 | 0.249 [ 0.34, 0.16] | 0 (ex.) | +0.032 [ 0.08, +0.15] | 0 (ex.) | 0.000 [ 0.02, +0.02] | +0.003 [ 0.03, +0.04] | 0.004 [ 0.06, +0.05] | +0.008 [ 0.10, +0.11] |
| Dur. trend | 0.540 | +0.017 [ 0.13, +0.16] | +0.038 [ 0.12, +0.20] | +0.055 [ 0.07, +0.18] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.580 | +0.036 [ 0.07, +0.14] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.004 [ 0.09, +0.09] | +0.061 [ 0.08, +0.21] | +0.068 [ 0.12, +0.25] | +0.000 [ 0.17, +0.17] |
| RQA | 0.547 | +0.003 [ 0.15, +0.16] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.010 [ 0.10, +0.08] | +0.003 [ 0.13, +0.13] | 0.046 [ 0.22, +0.13] | 0.067 [ 0.26, +0.12] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.500 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.616 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.012 [ 0.05, +0.07] | 0.001 [ 0.01, +0.01] | +0.000 [ 0.01, +0.01] | +0.001 [ 0.02, +0.02] | +0.008 [ 0.02, +0.04] |
| Classic | 0.776 | +0.172 [+0.11, +0.23] | 0 (ex.) | +0.004 [ 0.01, +0.02] | 0 (ex.) | 0.004 [ 0.01, +0.00] | 0.003 [ 0.02, +0.01] | 0.002 [ 0.03, +0.02] | +0.043 [ 0.00, +0.09] |
| Dur. trend | 0.582 | +0.048 [ 0.04, +0.14] | +0.036 [ 0.08, +0.15] | +0.086 [+0.00, +0.17] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.613 | +0.058 [ 0.04, +0.15] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.036 [ 0.09, +0.02] | 0.050 [ 0.11, +0.01] | 0.085 [ 0.15, 0.02] | 0.053 [ 0.11, +0.01] |
| RQA | 0.680 | +0.081 [ 0.00, +0.16] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.006 [ 0.02, +0.01] | 0.018 [ 0.05, +0.01] | 0.052 [ 0.10, 0.01] | 0.043 [ 0.09, +0.01] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.472 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.469 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.028 [ 0.21, +0.15] | +0.006 [ 0.03, +0.04] | +0.004 [ 0.05, +0.06] | +0.007 [ 0.06, +0.08] | 0.004 [ 0.10, +0.09] |
| Classic | 0.438 | 0.032 [ 0.28, +0.21] | 0 (ex.) | 0.023 [ 0.34, +0.29] | 0 (ex.) | 0.002 [ 0.04, +0.03] | 0.008 [ 0.05, +0.03] | 0.043 [ 0.10, +0.01] | 0.031 [ 0.16, +0.09] |
| Dur. trend | 0.541 | +0.046 [ 0.13, +0.23] | +0.070 [ 0.23, +0.37] | +0.031 [ 0.27, +0.33] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.419 | 0.016 [ 0.37, +0.34] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.022 [ 0.21, +0.17] | 0.033 [ 0.11, +0.04] | +0.068 [ 0.13, +0.27] | +0.056 [ 0.21, +0.32] |
| RQA | 0.416 | 0.037 [ 0.37, +0.30] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.017 [ 0.13, +0.09] | +0.016 [ 0.24, +0.28] | +0.028 [ 0.21, +0.27] | +0.090 [ 0.24, +0.42] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.495 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.716 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.096 [+0.01, +0.18] | +0.000 [ 0.01, +0.01] | +0.003 [ 0.01, +0.01] | 0.001 [ 0.01, +0.01] | +0.005 [ 0.03, +0.04] |
| Classic | 0.501 | 0.123 [ 0.27, +0.03] | 0 (ex.) | +0.054 [ 0.07, +0.18] | 0 (ex.) | 0.000 [ 0.02, +0.02] | 0.000 [ 0.03, +0.03] | 0.004 [ 0.06, +0.05] | 0.002 [ 0.07, +0.07] |
| Dur. trend | 0.511 | 0.013 [ 0.17, +0.15] | 0.023 [ 0.17, +0.13] | +0.046 [ 0.09, +0.18] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.571 | +0.022 [ 0.10, +0.14] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.040 [ 0.05, +0.13] | +0.056 [ 0.05, +0.16] | +0.097 [ 0.06, +0.25] | +0.061 [ 0.12, +0.24] |
| RQA | 0.525 | 0.031 [ 0.19, +0.13] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.018 [ 0.05, +0.09] | +0.069 [ 0.04, +0.18] | +0.062 [ 0.08, +0.20] | +0.009 [ 0.18, +0.20] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.589 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.472 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.062 [ 0.25, +0.13] | +0.002 [ 0.02, +0.02] | +0.006 [ 0.03, +0.04] | +0.034 [ 0.02, +0.09] | +0.073 [ 0.02, +0.17] |
| Classic | 0.523 | 0.002 [ 0.19, +0.19] | 0 (ex.) | +0.048 [ 0.19, +0.28] | 0 (ex.) | +0.004 [ 0.03, +0.04] | +0.008 [ 0.05, +0.06] | 0.006 [ 0.08, +0.06] | +0.003 [ 0.08, +0.09] |
| Dur. trend | 0.507 | 0.069 [ 0.23, +0.09] | 0.077 [ 0.24, +0.08] | 0.037 [ 0.29, +0.22] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.451 | 0.054 [ 0.38, +0.27] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.046 [ 0.25, +0.16] | 0.072 [ 0.22, +0.08] | 0.004 [ 0.26, +0.25] | 0.041 [ 0.28, +0.20] |
| RQA | 0.462 | 0.065 [ 0.38, +0.25] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.015 [ 0.15, +0.12] | 0.026 [ 0.18, +0.13] | 0.046 [ 0.24, +0.15] | 0.048 [ 0.29, +0.20] |
| Descriptor | AUC | order | dur. coupl. | dur. marg. | rotation | jit. 0.25 | jit. 0.5 | jit. 1 | jit. 2 |
|---|---|---|---|---|---|---|---|---|---|
| Count | 0.444 | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| Dispersion | 0.476 | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.091 [ 0.20, +0.38] | +0.006 [ 0.02, +0.03] | +0.007 [ 0.04, +0.06] | +0.015 [ 0.04, +0.07] | +0.022 [ 0.10, +0.15] |
| Classic | 0.500 | +0.070 [ 0.15, +0.29] | 0 (ex.) | +0.025 [ 0.20, +0.25] | 0 (ex.) | +0.000 [ 0.04, +0.04] | +0.001 [ 0.06, +0.06] | +0.003 [ 0.05, +0.06] | +0.017 [ 0.10, +0.14] |
| Dur. trend | 0.521 | +0.012 [ 0.22, +0.24] | 0.024 [ 0.26, +0.21] | +0.002 [ 0.30, +0.31] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0 (ex.) |
| RQA | 0.518 | 0.026 [ 0.22, +0.17] | 0 (ex.) | 0 (ex.) | 0 (ex.) | +0.034 [ 0.15, +0.21] | +0.026 [ 0.23, +0.28] | +0.104 [ 0.14, +0.35] | +0.098 [ 0.11, +0.31] |
| RQA | 0.559 | +0.036 [ 0.24, +0.31] | 0 (ex.) | 0 (ex.) | 0 (ex.) | 0.004 [ 0.11, +0.10] | +0.093 [ 0.10, +0.29] | +0.106 [ 0.09, +0.30] | +0.140 [ 0.07, +0.35] |
| Problem | Count | Dispersion | Classic | Dur. trend | RQA | RQA | Directed | FAG lattice | FAG AOI | FAG AOI pad. |
|---|---|---|---|---|---|---|---|---|---|---|
| gb_tex_vd1 | 0.968 (.005) | 1.000 (.005) | 0.968 (.005) | 0.956 (.005) | 0.999 (.005) | 1.000 (.005) | 0.977 (.005) | 0.989 (.005) | – | – |
| gb_vd1_blg | 0.691 (.005) | 1.000 (.005) | 0.888 (.005) | 0.751 (.005) | 0.999 (.005) | 1.000 (.005) | 1.000 (.005) | 0.957 (.005) | – | – |
| gb_vd1_vd2 | 0.633 (.005) | 0.703 (.005) | 0.636 (.005) | 0.791 (.005) | 0.672 (.005) | 0.715 (.005) | 0.700 (.005) | 0.666 (.005) | – | – |
| gb_vd1_halves | 0.500 (1.000) | 0.921 (.005) | 0.561 (.005) | 0.590 (.005) | 0.742 (.005) | 0.822 (.005) | 0.803 (.005) | 0.739 (.005) | – | – |
| gb_tex_s1s2 | 0.516 (.065) | 0.517 (.224) | 0.537 (.005) | 0.545 (.015) | 0.574 (.005) | 0.515 (.159) | 0.531 (.070) | 0.588 (.005) | – | – |
| sb_native | 0.610 (.010) | 0.530 (.164) | 0.673 (.005) | 0.637 (.015) | 0.611 (.005) | 0.659 (.005) | 0.594 (.025) | 0.575 (.050) | – | – |
| Problem | Within-fold AUC | Pooled AUC |
|---|---|---|
| gb_tex_vd1 | 0.499–0.501 | 0.491–0.496 |
| gb_vd1_blg | 0.499–0.502 | 0.494–0.497 |
| gb_vd1_vd2 | 0.499–0.501 | 0.494–0.499 |
| gb_vd1_halves | 0.499–0.501 | 0.496–0.500 |
| gb_tex_s1s2 | 0.499–0.501 | 0.494–0.498 |
| sb_native | 0.497–0.504 | 0.476–0.490 |
| Problem | Descriptor | Transformation | Primary | Pooled AUC | Partitions | Long runs | Trees | Norm. | Norm. (pooled) | Robust |
|---|---|---|---|---|---|---|---|---|---|---|
| gb_tex_vd1 | Classic | duration marginal | +0.035 [+0.021, +0.048] | +0.035 [+0.022, +0.048] | to | to (3/3) | +0.045 [+0.025, +0.065] | |||
| gb_tex_vd1 | Classic | order | 0.022 [ 0.034, 0.010] | 0.022 [ 0.034, 0.009] | to | to (3/3) | 0.019 [ 0.033, 0.005] | |||
| gb_tex_vd1 | Classic | jitter | 0.003 [ 0.005, 0.001] | 0.003 [ 0.005, 0.001] | to | to (3/3) | 0.002 [ 0.009, +0.004] | |||
| gb_tex_vd1 | Classic | jitter | 0.006 [ 0.009, 0.002] | 0.006 [ 0.008, 0.003] | to | to (3/3) | 0.005 [ 0.011, +0.001] | |||
| gb_tex_vd1 | Classic | jitter | 0.007 [ 0.011, 0.003] | 0.007 [ 0.011, 0.003] | to | to (3/3) | 0.006 [ 0.014, +0.003] | |||
| gb_tex_vd1 | Directed | order | 0.014 [ 0.020, 0.008] | 0.014 [ 0.020, 0.008] | to | to (3/3) | 0.004 [ 0.011, +0.002] |
| Design | Size | Contrasts | SE ratio, median (range) | Coverage at 80% | at 90% | at 95% |
|---|---|---|---|---|---|---|
| no signal | 65 2 (B) | 8 | 1.66 (1.35–2.04) | 0.959 | 0.988 | 0.997 |
| no signal | 95 20 (B) | 8 | 1.48 (1.23–2.03) | 0.922 | 0.978 | 0.988 |
| no signal | 112 2 (W) | 8 | 1.80 (1.66–2.02) | 0.963 | 0.994 | 1.000 |
| no signal | 60 4 (W) | 8 | 1.80 (1.48–2.22) | 0.966 | 0.987 | 1.000 |
| signal elsewhere | 65 2 (B) | 5 | 2.40 (1.94–3.10) | 0.985 | 0.990 | 1.000 |
| signal elsewhere | 95 20 (B) | 5 | 1.46 (1.28–1.65) | 0.925 | 0.985 | 1.000 |
| Coverage | Width | Excl. 0 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Design | Size | Descr. | Contrast | direct | corr. | direct | corr. | direct | corr. |
| no signal | 65 2 (B) | rqa | order | 0.985 | 0.990 | 0.367 | 0.434 | 0.015 | 0.010 |
| no signal | 65 2 (B) | rqa | jitter 160 px | 0.995 | 1.000 | 0.419 | 0.505 | 0.005 | 0.000 |
| no signal | 65 2 (B) | rqa | jitter 40 px | 0.985 | 0.990 | 0.417 | 0.498 | 0.015 | 0.010 |
| no signal | 60 4 (W) | classic | order | 0.995 | 0.985 | 0.241 | 0.286 | 0.005 | 0.015 |
| no signal | 60 4 (W) | rqa | order | 0.990 | 0.995 | 0.272 | 0.324 | 0.010 | 0.005 |
| Design | Size | Contrast | Target (SE) | Coverage | Excl. 0 | Width | Obs. AUC |
|---|---|---|---|---|---|---|---|
| order hides spread (classic) | 65 2 (B) | duration marginal | +0.006 (0.009) | 0.975 | 0.025 | 0.397 | 0.520 |
| order hides spread (classic) | 65 2 (B) | order | 0.499 (0.010) | 1.000 | 1.000 | 0.405 | 0.520 |
| order hides spread (classic) | 60 4 (W) | duration marginal | 0.001 (0.005) | 1.000 | 0.000 | 0.296 | 0.509 |
| order hides spread (classic) | 60 4 (W) | order | 0.497 (0.008) | 1.000 | 1.000 | 0.292 | 0.509 |
| no signal (rqa) | 65 2, heterog. (B) | order | 0 (by arg.) | 1.000 | 0.000 | 0.429 | 0.493 |
| no signal (rqa) | 65 2, heterog. (B) | jitter 160 px | +0.001 (0.012) | 1.000 | 0.000 | 0.498 | 0.493 |
| Program contrast | Correctness, both programs | ||||
|---|---|---|---|---|---|
| Descriptor | Real gaze | Synthetic gaze | All pairs | Perm. mean | Same-program pairs [95% interval] |
| Count | – | – | 0.495 | 0.515 | 0.517 [0.317, 0.646] ( ) |
| Dispersion | 0.957 | 1.000 | 0.716 | 0.710 | 0.490 [0.398, 0.581] ( ) |
| Classic | – | – | 0.501 | 0.505 | 0.523 [0.319, 0.625] ( ) |
| Dur. trend | – | – | 0.511 | 0.502 | 0.538 [0.373, 0.640] ( ) |
| RQA | 0.580 | 0.604 | 0.571 | 0.517 | 0.527 [0.403, 0.645] ( ) |
| Fold | Classic | Directed | FAG AOI | |||
|---|---|---|---|---|---|---|
| test | train | test | train | test | train | |
| 1 | 0.686 | 0.532 | 0.400 | 0.593 | 0.229 | 0.588 |
| 2 | 0.167 | 0.582 | 0.033 | 0.616 | 0.300 | 0.533 |
| 3 | 0.143 | 0.595 | 0.381 | 0.589 | 0.095 | 0.598 |
| 4 | 0.929 | 0.497 | 0.964 | 0.485 | 0.107 | 0.523 |
| 5 | 0.429 | 0.551 | 0.257 | 0.594 | 0.200 | 0.595 |
| Transformation | em_program | em_halves | em_expertise | em_correct | em_correctrect | em_correctveh |
|---|---|---|---|---|---|---|
| none (observed) | 0.974 / 0.975 | 0.960 / 0.990 | 0.977 / 0.973 | 0.973 / 0.974 | 0.972 / 0.972 | 0.978 / 0.975 |
| order | 0.974 / 0.975 | 0.960 / 0.990 | 0.977 / 0.973 | 0.973 / 0.974 | 0.972 / 0.972 | 0.978 / 0.975 |
| duration coupling | 0.975 / 0.977 | 0.962 / 0.990 | 0.978 / 0.975 | 0.974 / 0.975 | 0.973 / 0.972 | 0.978 / 0.977 |
| duration marginal | 0.975 / 0.977 | 0.962 / 0.990 | 0.978 / 0.975 | 0.974 / 0.975 | 0.973 / 0.972 | 0.979 / 0.977 |
| rotation | 0.901 / 0.767 | 0.788 / 0.888 | 0.838 / 0.814 | 0.864 / 0.816 | 0.897 / 0.882 | 0.742 / 0.785 |
| jitter 0.25 | 0.970 / 0.970 | 0.956 / 0.983 | 0.971 / 0.968 | 0.968 / 0.970 | 0.968 / 0.968 | 0.971 / 0.970 |
| Problem | Contrast | AOI graph | AOI graph, support-matched | Support-only descriptor |
|---|---|---|---|---|
| em_halves (AUC 0.682; 0.773) | rotation | +0.098 [+0.031, +0.166] | +0.032 [ 0.059, +0.124] (99%) | +0.054 [ 0.001, +0.110] |
| em_halves (AUC 0.682; 0.773) | jitter 0.25 | +0.036 [ 0.029, +0.102] | +0.019 [ 0.048, +0.086] (100%) | +0.048 [+0.018, +0.078] |
| em_halves (AUC 0.682; 0.773) | jitter 0.5 | +0.023 [ 0.025, +0.071] | +0.035 [ 0.029, +0.099] (100%) | +0.112 [+0.065, +0.159] |
| em_halves (AUC 0.682; 0.773) | jitter 1 | +0.045 [ 0.032, +0.123] | +0.057 [ 0.027, +0.141] (89%) | +0.119 [+0.059, +0.178] |
| em_halves (AUC 0.682; 0.773) | jitter 2 | +0.113 [+0.014, +0.212] | +0.121 [+0.035, +0.207] (31%) | +0.123 [+0.054, +0.192] |
| Problem | AUC obs. | AUC, LOO pool | AUC, training-fold pool (SD) | LOO | training-fold |
|---|---|---|---|---|---|
| gb_tex_vd1 | 0.969 | 0.935 | 0.935 (0.000) | +0.035 | +0.034 |
| gb_vd1_blg | 0.886 | 0.852 | 0.852 (0.001) | +0.033 | +0.034 |
| gb_vd1_vd2 | 0.640 | 0.516 | 0.523 (0.009) | +0.124 | +0.117 |
| gb_tex_s1s2 | 0.539 | 0.508 | 0.503 (0.020) | +0.031 | +0.036 |
| gb_vd1_halves | 0.564 | 0.556 | 0.553 (0.004) | +0.008 | +0.011 |
| sb_native | 0.689 | 0.672 | 0.672 (0.002) | +0.017 | +0.017 |
| Problem | Descriptor | Transformation | Observed AUC | Transformed AUC | Contrast | (lower tail) | ||
|---|---|---|---|---|---|---|---|---|
| em_program | Classic | order | 0.539 | 0.095 | 0.788 | 0.001 | 0.001 | |
| gb_vd1_halves | Classic | order | 0.564 | 0.001 | 0.653 | 0.001 | 0.001 | |
| gb_tex_vd1 | Classic | order | 0.969 | 0.001 | 0.991 | 0.001 | 0.142 | |
| gb_vd1_blg | Classic | order | 0.886 | 0.001 | 1.000 | 0.001 | 0.001 | |
| em_halves | RQA | jitter | 0.613 | 0.001 | 0.692 | 0.001 | 0.027 |
| Design | Descriptor | Contrast | Coverage | Excl. 0 at zero target | Width | SE ratio | Coverage at 80% |
|---|---|---|---|---|---|---|---|
| no signal | classic | order | 40/40 | 0/40 | 0.111 | 1.65 | 0.975 |
| no signal | directed | order | 38/40 | 2/40 | 0.139 | 1.39 | 0.950 |
| no signal | directed | jitter 160 px | 39/40 | – | 0.148 | 1.26 | 0.900 |
| no signal | directed | jitter 40 px | 39/40 | – | 0.150 | 1.39 | 0.975 |
| no signal | directed | rotation | 40/40 | – | 0.144 | 1.42 | 0.900 |
| no signal | rqa | order | 40/40 | 0/40 | 0.133 | 1.63 | 0.950 |
| Problem | Descriptor | Transformation | Trees, , | Trees, , |
|---|---|---|---|---|
| gb_tex_vd1 | Dur. trend | duration marginal | +0.371 [+0.318, +0.424] | +0.378 [+0.327, +0.428] |
| gb_vd1_blg | Classic | order | 0.118 [ 0.148, 0.088] | 0.118 [ 0.146, 0.090] |
| gb_vd1_blg | Dur. trend | duration marginal | +0.226 [+0.161, +0.291] | +0.231 [+0.168, +0.293] |
| gb_vd1_vd2 | Classic | duration marginal | +0.130 [+0.073, +0.187] | +0.127 [+0.074, +0.180] |
| gb_vd1_vd2 | Dispersion | jitter | +0.102 [+0.038, +0.166] | +0.100 [+0.036, +0.164] |
| gb_vd1_vd2 | Dur. trend | duration coupling | +0.176 [+0.133, +0.219] | +0.172 [+0.128, +0.215] |
| Trials | Participants | Fixations per trial | |||||
|---|---|---|---|---|---|---|---|
| Problem | Positive class | positive | negative | positive | negative | median (IQR) | range |
| gb_tex_vd1 | VD1 | 644 | 644 | 322 | 322 | 179 (132–225) | 37–300 |
| gb_vd1_blg | BLG | 643 | 644 | 322 | 322 | 121 (96–146) | 33–392 |
| gb_vd1_vd2 | VD2 | 644 | 644 | 322 | 322 | 141 (121–159) | 32–233 |
| gb_vd1_halves | VD1#2 | 644 | 644 | 322 | 322 | 66 (57–75) | 18–113 |
| gb_tex_s1s2 | TEX@S2 | 322 | 322 | 322 | 322 | 225 (206–243) | 97–300 |
Eye Tracking
Eye tracking measures where people look and how their eyes move to study visual attention, behavior, and gaze-controlled systems.
See all 65 papers