Hidden In Plain Gaze: Gaze Representations as Privacy Controls for Utility and Re-identification Risk in XR

Cory Ilo; Brendan-David John; Doug A. Bowman

cs.CVSep 4, 2026

Hidden In Plain Gaze: Gaze Representations as Privacy Controls for Utility and Re-identification Risk in XR

Authors: Cory Ilo, Brendan-David John, Doug A. Bowman

Organizations: Virginia Tech Blacksburg, Virginia, USA

Abstract

Intelligent extended reality (XR) systems increasingly use eye and head tracking to infer user intent, task, and attention, but the same signals can also reveal biometric identity. We study whether gaze data representation choice can serve as a lightweight privacy control at feature extraction, before adding perturbation or formal privacy mechanisms. Using the egocentric HoloAssist dataset, we compare three gaze representations under matched model capacity: raw gaze, spatial attention heatmaps, and engineered eye-movement features. We evaluate each representation on action recognition as task utility and closed-set user re-identification as privacy leakage. Representation choice substantially changes the privacy-utility tradeoff. Engineered features retain roughly 85% of raw gaze's action-recognition accuracy while reducing re-identification by about an order of magnitude, to roughly four times the chance rate across 206 identities. This reduction attenuates rather than eliminates identity leakage, and the differences across representations show that abstraction alone does not guarantee privacy. Engineered features expose interpretable and auditable structure, giving designers a transparent privacy lever that complements mechanisms such as differential privacy.

Figures & tables

Figure 1. The three gaze representations, from the same eye and head stream: (left) raw gaze/head signal, (center) spatial attention heatmap, (right) engineered continuous-feature vector. Abstraction increases left to right, discarding fine-grained temporal structure that carries identity.

Figure 2. Comparison setup. Only the input gaze representation varies; the CNN+BiGRU encoder, its capacity, and the training budget are fixed. The learned representation is read out on two competing objectives—action recognition (utility) and closed-set re-identification (privacy)—after two-phase SupCon pretraining and per-objective fine-tuning.

Figure 3. Recording sessions per identity in the filtered HoloAssist subset. Most identities contribute several sessions, so the stratified-by-identity split holds out segments while keeping every identity represented across train, validation, and test.

Figure 4. Action-recognition accuracy across representations and label granularities (higher is better). The engineered representation tracks raw gaze closely, with the gap narrowing on the coarser verb-level task.

Figure 5. Closed-set re-identification across representations (log scale; lower is more private). The engineered representation reduces top-1 re-identification by roughly an order of magnitude relative to raw gaze while remaining above the

1/206

chance floor.

Figure 6. Action top-1 accuracy by class-frequency stratum. All representations degrade sharply on tail (rare) classes, which accounts for the low balanced-accuracy values on the primary task.

Figure 7. The privacy–utility exchange rate: utility retained vs. raw gaze (action top-1 ratio) against privacy gain vs. raw gaze (re-identification top-1 fold reduction); raw sits at

(1,1)

. The engineered representation occupies the favorable upper-right region.

Privacy-Utility Tradeoff

Privacy-utility tradeoffs study how to limit disclosure of sensitive information while preserving useful data, models, and services.

See all 71 papers

Lensless Gaze Is Not Private by Default: Auditing Identity Leakage Across Disclosure Surfaces
Aug 31, 2026Rahul Vimalkanth, Kaushik MitraPrivacy Auditing Gaze Estimation
Indian Institute of Technology Madras, Chennai, India
MIRAGE: Full-Body Bystander Privacy for Smart Glasses with Consent-Based Restoration
Sep 21, 2026Muhammad Umair, Muhammad Danial Maqbool, Fatima Arshad Cheema +4Privacy-Preserving ML Person Re-Identification
Lahore University of Management Sciences (LUMS) Pakistan · Munster Technological University Ireland
CityGuard: Graph-Aware Private Descriptors for Bias-Resilient Identity Search Across Urban Cameras
Feb 20, 2026Rong Fu, Yibo Meng, Rui Lu +2Person Re-Identification Metric Learning
Independent Researcher

Aug 31, 2026J/K move · Enter open · S save

Lensless Gaze Is Not Private by Default: Auditing Identity Leakage Across Disclosure Surfaces

Rahul Vimalkanth, Kaushik Mitra

Indian Institute of Technology Madras, Chennai, India

Privacy Auditing Gaze Estimation Biometric Identification

Open paper AskPDF arXiv

Lensless near-eye sensing is often described as privacy-friendly because its coded measurements are visually unintelligible. Yet visual unintelligibility reflects human interpretation, not what a learned adversary can recover. We therefore treat identity privacy as a systems property of disclosure surfaces: representations crossing sensing, storage, computation, and output boundaries. We audit a simulated lensless gaze pipeline under a 36-subject known-gallery closed-set identification protocol with a fixed, known PSF; privacy from an unknown or varying optical key is outside our scope. Reported accuracies are empirical attack success rates under matched linear and MLP probes and do not upper-bound stronger adversaries. Simulated lensless measurements yield 96.7% top-1 identification versus 97.7% for matched original eye crops, while an MAE embedding retains 94.3%. Compression alone offers little protection: 8-D PCA and a matched 8-D bottleneck retain 93.2% and 91.8%, whereas separately trained 8-D GSPL bottlenecks yield 77.5% mean recovery across three seeds. A released 128-way gaze token lowers single-frame recovery to 38.1%, while its residual and continuous gaze output expose 62.1% and 72.6%, respectively. Under a source-frame-disjoint tiled protocol, token summaries reach 39.9% at T=25, showing that repeated-output risk depends on representation and aggregation. These rates reflect all subject-correlated information in the evaluated dataset, including acquisition and behavioral cues, rather than isolating intrinsic ocular biometrics. Ordinary least squares residualization against a six-dimensional crop geometry and intensity summary still leaves lensless recovery at 95.1%. Our results show that privacy claims for lensless sensing must be tested at disclosure boundaries rather than inferred from appearance.

Abstract

Privacy-Utility Tradeoff

Explore similar work

Lensless Gaze Is Not Private by Default: Auditing Identity Leakage Across Disclosure Surfaces

MIRAGE: Full-Body Bystander Privacy for Smart Glasses with Consent-Based Restoration

CityGuard: Graph-Aware Private Descriptors for Bias-Resilient Identity Search Across Urban Cameras