Organizations: Toyota Technological Institute, Japan · Idiap Research Institute, Switzerland · ´Ecole Polytechnique F´ed´erale de Lausanne, Switzerland
This paper proposes an end-to-end shared attention estimation method via group detection. Most previous methods estimate shared attention (SA) without detecting the actual group of people focusing on it, or assume that there is a single SA point in a given image. These issues limit the applicability of SA detection in practice and impact performance. To address them, we propose to simultaneously achieve group detection and shared attention estimation using a two step process: (i) the generation of SA heatmaps relying on individual gaze attention heatmaps and group membership scalars estimated in a group inference; (ii) a refinement of the initial group memberships allowing to account for the initial SA heatmaps, and the final prediction of the SA heatmap. Experiments demonstrate that our method outperforms other methods in group detection and shared attention estimation. Additional analyses validate the effectiveness of the proposed components. Code: https://github.com/chihina/sagd-CVPRW2026.
Figures & tables
Figure 1: Difference between previous and our shared attention estimation methods. (a) Shared attention estimation using simple integration of the individual attention of all people in the scene as post-processing. (b) Direct shared attention estimation without group detection. (c) Our shared attention estimation via group detection, in which shared attention is estimated by integration of individual attention based on detected groups.
Figure 2: Overview of our network. First, the individual attention heatmaps A are estimated for each person. They are then exploited to derive group memberships per each group token M and integrated to infer the shared attention heatmap associated with each group token S . Both the group memberships and shared attention heatmaps are further refined in a second step.
Figure 3: Membership-based shared attention heatmap estimation. Individual attentions heatmaps are integrated by weighting them using the group membership obtained in the group detection step.
Figure 4: Overview of group detection. The dot product between the e -th updated group token and the n -th person token is computed as a group membership coefficient (i.e., Me,n ).
Figure 5: Overview of refined group detection in which spatial argmax is applied to the initial SA heatmaps S for refinement.
GroupAP
θIoU =0.5
θIoU=1.0
Method
θDist=0.05
θDist=0.1
θDist=∞
θDist=0.05
θDist=0.1
θDist=∞
MTGS-PP [ 9 ]
16.4
20.7
37.5
7.1
8.6
11.3
MTGS-Soc. [ 9 ]
5.7
7.8
28.6
2.9
3.6
9.8
Gaze-LLE-PP [ 23 ]
15.6
19.5
26.3
5.7
6.9
8.2
Ours
32.4
41.0
61.7
12.5
15.9
17.7
Table 1: Comparison of previous and our methods on the VideoCoAtt dataset. The best result in each column is colored by red . Regarding the group detection, θIoU=0.5 and θIoU=1.0 are used for lenient and strict criteria. For the shared attention estimation, θDist=0.05 , θDist=0.1 , and θDist=∞ are used for evaluation. θDist=0.05 and θDist=0.1 judge whether estimated shared attention is located within a pre-defined distance threshold from ground-truth shared attentions. θDist=∞ is used to only evaluate group detection while ignoring shared attention estimation errors.
Figure 6: Visual comparison on the VideoCoAtt dataset. People joining the same group are enclosed within a rectangle of the same color. The estimated shared attention point (i.e., the peak of the estimated shared attention heatmap) for each group is visualized as each group’s color point. Due to space limitations, we compare our method with MTGS-PP, having the second-best performance in Table 1 .
GroupAP
θIoU =0.5
θIoU=1.0
Method
θDist=0.05
θDist=0.1
θDist=∞
θDist=0.05
θDist=0.1
θDist=∞
MTGS-PP [ 9 ]
30.3
35.6
66.0
5.1
6.3
9.0
MTGS-Soc. [ 9 ]
7.6
9.9
37.2
1.5
1.9
4.7
Gaze-LLE-PP [ 23 ]
7.5
8.5
12.4
2.9
3.0
3.3
Ours
27.2
33.5
52.0
7.5
10.1
12.0
Table 2: Comparison of previous and our methods on the VideoAttentionTarget dataset. The best result in each column is colored by red .
Figure 7: Visual comparison on the VideoAttentionTarget dataset.
GroupAP
θIoU =0.5
θIoU=1.0
Method
θDist=0.05
θDist=0.1
θDist=∞
θDist=0.05
θDist=0.1
θDist=∞
MTGS-PP [ 9 ]
7.8
13.9
25.4
4.6
5.2
5.6
MTGS-Soc. [ 9 ]
0.8
1.8
9.1
0.7
0.7
1.3
Gaze-LLE-PP [ 23 ]
5.8
8.1
14.1
2.4
2.8
4.4
Ours
9.0
15.6
36.3
2.1
2.1
2.4
Table 3: Comparison of previous and our methods on the ChildPlay dataset. The best result in each column is colored by red .
Figure 8: Failure cases on the ChildPlay dataset.
GroupAP
θIoU =1.0
Method
θDist=0.05
θDist=0.1
θDist=∞
w/o refinement
2.5
3.3
3.6
w/o Lsocial
0.9
1.6
1.9
Ours (Full)
12.5
15.9
17.7
Table 4: Ablation study on the VideoCoAtt dataset.
Figure 9: Effectiveness of refined shared attention estimation (Sec. 3.3 ) on the VideoCoAtt dataset.
Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher's performance while being lightweight enough for practical deployment on robots and consumer devices. The code and model checkpoints are available at our project page.
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
Junhui She, Fei Wang, Kun Li +4
University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center · 2Hefei University of Technology +2
Driver gaze estimation is essential for understanding the driver's situational awareness of surrounding traffic. Existing gaze estimation models use driver facial information to predict the Point-of-Gaze (PoG) or the 3D gaze direction vector. We propose a benchmark dataset, Urban Driving-Face Scene Gaze (UD-FSG), comprising synchronized driver-face and traffic-scene images. The scene images provide cues about surrounding traffic, which can help improve the gaze estimation model, along with the face images. We propose SGAP-Gaze, Scene-Grid Attention based Point-of-Gaze estimation network, trained and tested on our UD-FSG dataset, which explicitly incorporates the scene images into the gaze estimation modelling. The gaze estimation network integrates driver face, eye, iris, and scene contextual information. First, the extracted features from facial modalities are fused to form a gaze intent vector. Then, attention scores are computed over the spatial scene grid using a Transformer-based attention mechanism fusing face and scene image features to obtain the PoG. The proposed SGAP-Gaze model achieves a mean pixel error of 104.73 on the UD-FSG dataset and 63.48 on LBW dataset, achieving a 23.5% reduction in mean pixel error compared to state-of-the-art driver gaze estimation models. The spatial pixel distribution analysis shows that SGAP-Gaze consistently achieves lower mean pixel error than existing methods across all spatial ranges, including the outer regions of the scene, which are rare but critical for understanding driver attention. These results highlight the effectiveness of integrating multi-modal gaze cues with scene-aware attention for a robust driver PoG estimation model in real-world driving environments.
Pavan Kumar Sharma, Pranamesh Chakraborty
Department of Civil Engineering, Indian Institute of Technology Kanpur, Kanpur-208016, India