ChildGaze: A Benchmark Dataset for Collaborative Behavior Understanding in Children
Authors: Sindhuja Penchala, Saketh Reddy Kontham, Prachi Bhattacharjee, S. Nima Mahmoodi, Daniel Fonseca, Sareh Karami, Mehdi Garemani, Sudip Mittal, Shahram Rahimi, Noorbakhsh Amiri Golilarz
Organizations: The University of Alabama, Tuscaloosa, AL, USA · Mississippi State University, Mississippi State, MS, USA
Understanding collaborative behavior in children is important for analyzing social participation, peer interaction, shared attention, and engagement during play and learning activities. Reliable recognition of these cues can support research in child development, educational analysis, and human-centered computer vision. However, estimating where a child is looking does not necessarily reveal whether the child is actively participating in a shared activity. To support this higher-level analysis, we introduce ChildGaze, a child-centered behavioral annotation dataset built on the ChildPlay video collection [1]. ChildGaze introduces two behavioral labels, collaborative and non-collaborative, assigned independently to each child within a frame. The dataset provides face, left-hand, and right-hand bounding boxes for children and adults and organizes the annotations at the row, person, and frame levels. The current release contains 27 annotated video files, 10,641 frames, and 73,268 body-part annotation rows. Annotation reliability was evaluated on 1,187 frames using independent annotations from two annotators. The collaboration labels achieved 93.16% raw agreement and a Cohen's kappa of 0.8631, while bounding-box annotations achieved an overall mean IoU of 0.808. Baseline experiments with pretrained ViT and Swin Transformer models achieved up to 97.44% child-person-level accuracy and 96.80% frame-level accuracy, respectively. These results show that ChildGaze provides a reliable benchmark for studying collaborative behavior in naturalistic child-adult and peer interactions.
Figures & tables
Fig. 1: Representative examples of the frame-level collaboration annotation scheme. Each child receives an individual collaboration label, and the frame label is assigned by majority vote across the children: Yes when the collaborative labels are in the majority, No when the non-collaborative labels are in the majority, and Undetermined when the two counts are equal.
Feature
ChildPlay [ 1 ]
ChildPlay-Hand [ 6 ]
ChildPlay-R [ 14 ]
SCB-Dataset3 [ 12 ]
MMASD [ 10 ]
ChildGaze (Ours)
Multiple children per scene
✓
✓
✗
✓
✗
✓
Face/head bounding boxes
✓
✗
✓
✗
✗
✓
Hand bounding boxes
✗
✓
✗
✗
✗
✓
Action/behavior labels
✗
✓
✗
✓
✓
✓
Person-level annotation
✓
✓
✓
✓
✓
✓
Frame-level annotation
✓
✓
✓
✓
✗
✓
TABLE I: Comparison of ChildGaze with existing child-centered gaze, behavior, and interaction datasets in terms of annotation type, spatial cues, multi-person support, behavioral labeling, annotation granularity, and evaluation methodology.
Label
Annotation Criteria
Collaborative
• Child listens to an adult and participates in the ongoing interaction. • Performs, attempts, or continues a shared task with an adult. • Plays with an object that is part of a shared activity. • Works or plays with another child on the same task or object. • Looks away but remains actively involved in the shared activity.
Non-Collaborative
• Child does not pay attention to the ongoing interaction and does not participate. • Plays or works independently without shared participation. • Attends to an unrelated person, object, or activity. • Is present in the scene but shows no observable participation in the shared interaction.
TABLE II: Collaboration Labeling Criteria Used for Child Annotations in ChildGaze
Fig. 2: Overview of the ChildGaze dataset construction and analysis pipeline. Source ChildPlay [ 1 ] videos are prepared and imported into Labelbox, where separate child and adult annotation classes are defined. Face, left-hand, and right-hand bounding boxes are annotated and propagated across frames, followed by assignment of collaborative or non-collaborative labels to individual children. The resulting annotations are organized at the row, person, and frame levels to preserve both individual and scene-level interaction information. The final annotated dataset supports validation and downstream evaluation of collaborative-behavior recognition using pretrained visual models [ 1 ] .
Property
Yes
No
Total
Dataset Overview
Annotated files
–
–
27
Total frames
–
–
10,641
Total annotation rows
51,155
22,113
73,268
Child Annotations
Child annotation rows
17,654
22,113
39,767
TABLE III: Detailed statistics of the ChildGaze dataset across frame-, person-, and body-part-level annotations.
Annotator B: Yes
Annotator B: No
Annotator A: Yes
644
59
Annotator A: No
38
677
TABLE IV: Inter-annotator agreement for child collaboration labels in the validation subset [ 18 ] .
TABLE VI: Baseline performance and inference efficiency of ViT and Swin Transformer for collaborative-behavior recognition at the frame and child-person levels.