AOI-Net: Structural Face AOI-Guided Eye-Gaze Track Representation Learning for Autism Spectrum Disorder Detection

Zhanpei Huang; Binbin Sun; Jialiang Chen; Yiou Wang; Taochen Chen; Yuzhu Ji; Yiqun Zhang; Yiu-Ming Cheung

cs.CVAug 29, 2026

AOI-Net: Structural Face AOI-Guided Eye-Gaze Track Representation Learning for Autism Spectrum Disorder Detection

Authors: Zhanpei Huang, Binbin Sun, Jialiang Chen, Yiou Wang, Taochen Chen, Yuzhu Ji, Yiqun Zhang, Yiu-Ming Cheung

Organizations: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China · Shenzhen Maternity and Child Healthcare Hospital Affiliated to Southern Medical University, Shenzhen, China · Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China

Abstract

Eye-movement tracking has emerged as a promising non-invasive approach to Autism Spectrum Disorder (ASD) screening, with systematic differences in attentional allocation and revisit behaviors observed during socially interactive tasks. Existing computational methods typically characterize eye-movements using discrete gaze trajectories and fixation events, yielding representations dominated by short-range temporal dynamics and limiting models that primarily emphasize long-range dependencies. Meanwhile, gaze behavior is naturally organized across semantically meaningful Areas of Interest (AOIs), whose attention allocation and transitions provide important structural cues, yet their relationships are rarely modeled explicitly. To address these limitations, we propose a structural face AOI-guided Eye-Gaze Track Network (AOI-Net) that jointly models short-term temporal dynamics and AOI-level structural organization. A network gating mechanism adaptively integrates the complementary temporal and structural representations according to their contributions to gaze-behavior characterization. To mitigate the pronounced class imbalance commonly encountered between individuals with ASD and Typically Developing (TD) participants in clinical datasets, class-distribution-aware learning is further employed to facilitate discriminative embedding learning under skewed class distributions. Experiments on a unique and large-scale clinical eye-tracking database comprising eight stimulus subsets and more than 1,300 participants show that AOI-Net consistently outperforms state-of-the-art methods. The proposed framework also enables interpretable gaze-behavior modeling and provides a practical basis for scalable AI-driven ASD screening in real-world healthcare. The code is available at https://github.com/Zhanpei-ai/CIM-AOI-Net/tree/main/Code

Figures & tables

Fig. 1: Schematic Illustration of the Eye-Tracking Stimulus Paradigm and AOI Statistics for ASD Screening. Red dashed lines indicate transitions within social regions, and yellow dashed lines represent shifts from social to non-social regions. (Note: Due to copyright restrictions of the original visual stimuli, the woman’s face shown in this figure is an AI-generated synthetic image used solely for illustrative purposes.)

Fig. 2: Overview of AOI-Net for temporal eye gaze data representation learning and ASD detection.

Method	Metrics	Speaking	Walking1	Walking2	Helicopter	Baby	Tablet	Attention	Sad
Informer	ACC	0.7194 $\pm$ 0.0095	0.7819 $\pm$ 0.0118	0.7826 $\pm$ 0.0131	0.7515 $\pm$ 0.0084	0.7852 $\pm$ 0.0161	0.8109 $\pm$ 0.0109	0.7345 $\pm$ 0.0085	0.7344 $\pm$ 0.0103
[ 19 ] 2020	F1	0.6657 $\pm$ 0.0089	0.7465 $\pm$ 0.0099	0.7452 $\pm$ 0.0107	0.7060 $\pm$ 0.0037	0.7603 $\pm$ 0.0070	0.7751 $\pm$ 0.0103	0.7151 $\pm$ 0.0066	0.7073 $\pm$ 0.0106
	Spe.	0.4941 $\pm$ 0.0194	0.6190 $\pm$ 0.0123	0.5435 $\pm$ 0.0171	0.6190 $\pm$ 0.0195	0.6617 $\pm$ 0.0109	0.6786 $\pm$ 0.0130	0.5366 $\pm$ 0.0159	0.4810 $\pm$ 0.0189
	Sen.	0.8497 $\pm$ 0.0173	0.8138 $\pm$ 0.0125	0.8272 $\pm$ 0.0111	0.8115 $\pm$ 0.0129	0.8298 $\pm$ 0.0120	0.8691 $\pm$ 0.0085	0.8032 $\pm$ 0.0141	0.8362 $\pm$ 0.0175
Crossformer	ACC	0.6631 $\pm$ 0.0150	0.6985 $\pm$ 0.0195	0.6944 $\pm$ 0.0186	0.6824 $\pm$ 0.0400	0.7087 $\pm$ 0.0174	0.7430 $\pm$ 0.0172	0.6457 $\pm$ 0.0057	0.6524 $\pm$ 0.0305
[ 38 ] 2023	F1	0.6275 $\pm$ 0.0068	0.6803 $\pm$ 0.0203	0.6726 $\pm$ 0.0192	0.6607 $\pm$ 0.0322	0.6858 $\pm$ 0.0103	0.7144 $\pm$ 0.0198	0.6185 $\pm$ 0.0030	0.6245 $\pm$ 0.0253

TABLE I: ACC, F1, specificity (Spe.), and sensitivity (Sen.) comparison. Bold / underlined /“-” denote best/second-best/prediction failure.

Dataset	Metric	Variants
Dataset	Metric	AOI-Net	Variant I	Variant II
Speaking	ACC	0.8201	0.7950	0.7950
Speaking	F1	0.8637	0.8571	0.8589
Walking1	ACC	0.8248	0.8076	0.8064
Walking1	F1	0.8739	0.8641	0.8594
Walking2	ACC	0.8140	0.8128	0.8056

TABLE II: Comparison of ablated AOI-Net variants. Bold / underlined denote best/second-best results, respectively.

Fig. 3: ROC curves and AUC of different methods on Speaking and Walking1. The histogram in the figure also serves as the legend for the ROC curves.

Fig. 4: Behavioral graphs for TD (b) and ASD (c) participants on the Speaking dataset. Nodes: E (Eye), N (Nose), M (Mouth), OFA (Other Facial Area), PR (Peripheral Region); node size encodes attention strength (values shown). Edge color (cool-warm) indicates transition strength; self‑loops represent staying in the same region. Edges with weight $<0.05$ are omitted.

Dataset	Metric	KNN-Based Construction Strategy			AOI Edge Construction Strategy
Dataset	Metric	KNN ( $k=2$ )	KNN ( $k=4$ )	KNN ( $k=6$ )	AOI-Net (Dynamic)	AOI-Net (Weighted)	AOI-Net (Full)
Speaking	ACC	0.7482	0.7662	0.7482	0.8034	0.8165	0.8201
Speaking	F1	0.8293	0.8441	0.8293	0.8646	0.8741	0.8637
Walking1	ACC	0.7941	0.7978	0.7904	0.8199	0.8113	0.8248
Walking1	F1	0.8470	0.8533	0.8480	0.8655	0.8620	0.8739
Walking2	ACC	0.7971	0.8007	0.7971	0.8056	0.8007	0.8140

TABLE III: AOI edge strategies: KNN, Dynamic, Weighted, and Full. Bold / underlined denote best/second-best.

Fig. 5: Efficiency comparison of training and inference time.

Fig. 6: Comparison of different numbers of GNN layers across datasets.

Fig. 7: Comparison of different temporal modeling scales across datasets.

Fig. 8: Generalization evaluation under missing stimulus paradigms. After a sample-level random split, the model is trained on samples containing all eight paradigms and evaluated on the test samples with one designated paradigm omitted.

Eye Tracking

Eye tracking measures where people look and how their eyes move to study visual attention, behavior, and gaze-controlled systems.

See all 61 papers

Jul 8, 2026

Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies

Autism spectrum disorder (ASD) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited. This study addresses two complementary challenges in automated detection of autism-related self-stimulatory behaviors from video: (1) identifying the optimal sequence-based neural network architecture and temporal sampling rate, and (2) characterizing data augmentation strategies for training on small behavioral datasets. For the first objective, long short-term memory (LSTM) and gated recurrent unit (GRU) models were trained on pose-derived features from the Self-Stimulatory Behavior Diagnosis (SSBD) dataset at frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames. Both architectures exceeded prior convolutional neural network (CNN) baselines (62-76% accuracy), with peak accuracies of 97.5% (LSTM) and 98.75% (GRU) at a sampling interval of every 15 frames. For the second objective, ten data augmentation strategies were applied to an I3D transfer learning pipeline, with an ablation study quantifying the marginal contribution of each technique. Horizontal flip achieved the highest standalone accuracy (48.78%), while exclusion of upsampling from the augmentation pipeline produced the largest performance degradation, indicating its necessity for complex behavioral video augmentation. A personalized machine learning approach, in which per-subject models were trained and tested on temporally split segments of each video, produced consistent predictions (mean loss 1.84, SD 0.79). These results provide practitioners with concrete guidance on architecture selection, sampling rate, and augmentation strategy for video-based behavioral classification in data-scarce clinical domains.

Raunak Mondal, Peter Washington

School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Information and Computer Sciences, University of Hawaiʻi at Mānoa, Honolulu, HI, USA

Data Augmentation Human Activity Recognition Autism Spectrum Disorder Recurrent Neural Networks Video Action Recognition

Ask PDF

Jun 3, 2026

3D Temporal Analysis for Autism Spectrum Disorder Screening During Attention Tasks

Accurate Autism Spectrum Disorder (ASD) screening for school-age children is crucial to identify cases that may have been missed earlier and to enable timely interventions supporting social, cognitive, and academic development. Current ASD screening relies on subjective assessments and 2D analysis methods that fail to capture spatial displacement patterns characteristic of ASD behaviors. In this study, a novel 3D temporal analysis framework is presented, built on top of DECA (Detailed Expression Capture and Animation), a 3D modeling framework, to extract comprehensive head pose parameters (including translational components

T_x, T_y, T_z

) and facial expressions independent of pose variations. LSTM and GRU-based temporal classifiers were trained on the extracted 3D features from video data collected from 39 participants (19 ASD, 20 TD) aged 7-12 years during Virtual Reality-Continuous Performance Test tasks. The GRU-based models demonstrated superior performance, with 3D head pose features achieving 83.9% accuracy and 3D facial features reaching 81.4% accuracy, outperforming 2D baseline approaches by 10.7% and 7.5%, respectively. Furthermore, multimodal fusion of 3D head pose and facial features with PCA-based dimensionality reduction achieved the highest accuracy of 84.6%, outperforming unimodal approaches. This work establishes a foundation for objective, automated screening tools addressing current diagnostic limitations in ASD identification for school-age populations.

Inam Qadir, Elizabeth B Varghese, Dena Al-Thani +1

College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha, Qatar

Autism Spectrum Disorder

Ask PDF

Sep 25, 2026

UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term developmental outcomes. However, existing ASD recognition methods are often constrained by the scarcity of diagnostic text data, forcing them to rely mainly on visual analysis and limiting their ability to model clinically meaningful semantic reasoning. To address this challenge, we propose UniAR, a unified framework enhanced by multi-granularity prompt learning for robust ASD recognition under heterogeneous data variations. Specifically, UniAR leverages a large multimodal model to generate hierarchical diagnostic descriptions at the word, phrase, and sentence levels, compensating for the lack of paired clinical reports. To align the generated semantics with visual evidence, we further design a Mixture-of-Experts-based Multi-Scale Alignment Module, which dynamically matches vector-quantized visual prototypes with semantic representations at corresponding granularities. Extensive experiments on four benchmarks covering brain MRI and facial expression scenarios show that UniAR consistently outperforms existing state-of-the-art methods, achieving average accuracies of 75.9% on MRI benchmarks and 91.6% on facial benchmarks, while improving average Accuracy on MRI benchmarks by 1.5 percentage points and average Accuracy on facial benchmarks by 1.2 percentage points over baselines. These results demonstrate that UniAR offers a robust and interpretable framework for ASD screening under semantic scarcity.

Lei Xin, Zeheng Wang, Jiayin Zhu +6

Wuhan University Wuhan, Hubei, China · Northeast Normal University Changchun, Jilin, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China +4

Autism Spectrum Disorder Multimodal Learning

Ask PDF

Abstract

Eye Tracking

Explore similar work

Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies

3D Temporal Analysis for Autism Spectrum Disorder Screening During Attention Tasks

UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning