Group-DINOmics: Incorporating People Dynamics into DINO for Self-supervised Group Activity Feature Learning

Ryuki Tezuka; Chihiro Nakatani; Norimichi Ukita

cs.CVApr 6, 2026

Group-DINOmics: Incorporating People Dynamics into DINO for Self-supervised Group Activity Feature Learning

Authors: Ryuki Tezuka, Chihiro Nakatani, Norimichi Ukita

Organizations: Toyota Technological Institute

Abstract

This paper proposes Group Activity Feature (GAF) learning without group activity annotations. Unlike prior work, which uses low-level static local features to learn GAFs, we propose leveraging dynamics-aware and group-aware pretext tasks, along with local and global features provided by DINO, for group-dynamics-aware GAF learning. To adapt DINO and GAF learning to local dynamics and global group features, our pretext tasks use person flow estimation and group-relevant object location estimation, respectively. Person flow estimation is used to represent the local motion of each person, which is an important cue for understanding group activities. In contrast, group-relevant object location estimation encourages GAFs to learn scene context (e.g., spatial relations of people and objects) as global features. Comprehensive experiments on public datasets demonstrate the state-of-the-art performance of our method in group activity retrieval and recognition. Our ablation studies verify the effectiveness of each component in our method. Code: https://github.com/tezuka0001/Group-DINOmics.

Figures & tables

Panel of Figure 1 — Figure 1: Our self-supervised GAF learning augmented by two pretext tasks: (1) person flow estimation for local dynamics embedding into GAFs and (2) group-relevant object localization for global context embedding into GAFs. Compared with previous self-supervised methods [ 20 , 32 ] that utilize only local appearance features, our pretext tasks enhance GAF learning.

Figure 2: Overview of our network. (a) Image feature extractor. Group-relevant objects are inpainted to enhance global feature learning in (c). (b) GAF learning network. Image features are fed into the transformer encoder, MLP, and temporal pooling to obtain a GAF. (c) Pretext tasks for GAF learning. The flow of each person and the locations of the group-relevant objects are estimated from the GAF.

Figure 3: Inpainting to enhance global feature embedding into a GAF by localizing group-relevant objects (i.e., ball in our method). Top: w/o inpainting. Bottom: w/ inpainting (our method).

Figure 4: Pretext tasks: overview of person flow estimation, (a) and (b), and group-relevant object location estimation, (c) and (d). $\bm{L}^{p,t}$ and $\bm{T}^{t}$ are omitted for simplicity.

	VBD		NBA
Method	Hit@1	Hit@3	Hit@1	Hit@3
B1-Compact [ 20 ]	30.3	59.9	14.9	39.5
B2-VGG19 [ 20 ]	35.4	65.0	16.8	39.8
HRN [ 20 ]	31.2	57.6	15.5	37.1
GAFL [ 32 ]	61.1	82.4	24.7	50.4
Ours	82.7	93.0	43.9	72.0

Table 1: Comparison with state-of-the-art self-supervised GAF learning methods [ 20 , 32 ] on VBD and NBA. While all scores of the other methods on VBD come from [ 32 ] , we obtained the scores on NBA with the codes and weights provided by [ 32 ] because NBA is not used in experiments shown in [ 32 ] .

Panel of Figure 5 — Figure 5: Visual comparison of group activity retrieval on VBD. (a) R-set query. (b) R-spike query.

Figure 6: Visual comparison of group activity retrieval on NBA when the query video is a 3p-succ video.

	$\mathcal{L}_{\mathrm{F}}$		$\mathcal{L}_{\mathrm{O}}$		VBD	NBA
	$\mathcal{L}_{\mathrm{F},\bm{G}}$	$\mathcal{L}_{\mathrm{F},\bm{I}}$	$\mathcal{L}_{\mathrm{O},\bm{G}}$	$\mathcal{L}_{\mathrm{O},\bm{I}}$	Hit@1	Hit@1
(a)	-		-		43.0	22.6
(b)	$\mathrm{\checkmark}$		-		75.4	34.1
(c)	-		$\mathrm{\checkmark}$		74.0	40.3
(d)	$\color[rgb]{0,0.7,0}\mathrm{\checkmark}$ ( $w_{\mathrm{F}}$ =1)		$\color[rgb]{0,0.7,0}\mathrm{\checkmark}$ ( $w_{\mathrm{O}}$ =1)		77.9	42.9
(e)	$\color[rgb]{1,0,0}\mathrm{\checkmark}$		$\color[rgb]{0,1,1}\mathrm{\checkmark}$		82.7	43.9

Table 2: Ablation studies on VBD and NBA.

\color[rgb]{0,0.7,0}\mathrm{\checkmark}

in (d) and (i) indicates that

\mathcal{L}_{\mathrm{F}}

and

\mathcal{L}_{\mathrm{O}}

are used simultaneously.

\mathrm{\checkmark}

and

\mathrm{\checkmark}

indicate a loss used in the first and second stages, respectively.

Input image	VBD		NBA
Input image	Hit@1	Hit@3	Hit@1	Hit@3
Original	81.6	92.0	35.9	65.3
Masking	69.3	87.1	26.7	54.1
Inpainting (Ours)	82.7	93.0	43.9	72.0

Table 3: Detailed analysis of inpainting on VBD and NBA.

Dataset		VBD	NBA
Feature extractor	Architecture	Hit@1	Hit@1
Supervised ViT [ 10 ]	ViT-L/16	41.9	29.8
MAE [ 18 ]	ViT-L/16	55.4	40.1
CLIP [ 41 ]	ViT-L/14	59.8	37.9
SigLIP 2 [ 53 ]	ViT-L/16	62.2	38.1
DINOv3 [ 46 ] (Ours)	ViT-L/16	73.1	40.3

Table 4: Different feature extractors are used in our proposed network. Due to image feature extractor constraints, we use

256\times 256

inputs for “SigLIP 2 [ 53 ] ” and

224\times 224

for the others.

Figure 7: Confusion matrices of GAR by nearest neighbor retrieval on VBD and NBA. Each row and column shows the ground-truth and recognized group activity, respectively.

Figure 8: Visual comparison of group activity retrieval on VBD.

Figure 9: Visual comparison of group activity retrieval on NBA.

Method	Extractor	VBD	NBA
DFWSGAR [ 25 ]	ResNet-18	90.5	75.8
SOGAR [ 6 ]	ViT-Base	93.1	83.3
Flaming-Net [ 34 ]	Inception-v3	93.3	79.1
LiGAR [ 4 ]	ResNet-18	74.8	62.7
SAM [ 7 ]	ResNet-18	86.3	54.3
Dual-AI [ 17 ]	Inception-v3	-	58.1

Table 5: Comparison with supervised group activity recognition methods on VBD and NBA. The results of all methods are copied from their papers. The four methods above the middle line take the whole image as input, while the five methods between the middle line and ours use person bounding boxes together with the whole image in both training and inference. For a fair comparison, in all methods, the only manually annotated labels are group activity classes, and only images are used in inference.

Figure 10: Visualization of GAF spaces on VBD. GAFs extracted from the test set are projected to 2D by t-SNE [ 55 ] . The color of each sample indicates its ground-truth class.

Figure 11: Detail of our pretext tasks. Our pretext tasks comprise person-flow estimation and group-relevant object location estimation. Person flow estimation utilizes a pre-trained optical flow estimator. Group-relevant object location estimation utilizes a pre-trained object detector. Since both pretext tasks use only these pre-trained models, training can be conducted in a self-supervised manner with pseudo labels. The black paths are shared by both pretext tasks, while the red paths are used only for person flow estimation, and the yellow paths are used only for group-relevant object location estimation.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

	VBD
Method	Hit@1	Hit@2	Hit@3
HiGCIN [ 57 ]	50.0	66.3	74.5
DIN [ 58 ]	57.0	73.1	81.1
Dual-AI [ 13 ]	64.4	76.5	82.0
GAFL [ 28 ]	84.8	89.6	91.8
Ours	82.7	90.0	93.0

Appendix

Table 6: Comparison with state-of-the-art self-supervised GAF learning methods trained with supervision of person action classes on VBD. All other results are from [ 28 ].

Input image	VBD	NBA
Input image	Hit@1	Hit@1
Original	72.9	30.5
Masking	45.0	18.5
Inpainting (Ours)	74.0	40.3

Appendix

Table 7: Detailed analysis of inpainting on VBD and NBA. Unlike Table 3 of the main paper, results when only

\mathcal{L}_{\mathrm{O}}

is used for training are shown for further analysis.

Inpainting during inference	VBD	NBA
Inpainting during inference	Hit@1	Hit@1
$\mathrm{\checkmark}$	81.5	43.4
- (Ours)	82.7	43.9

Appendix

Table 8: Effects of inpainting during inference in our GAF learning on VBD and NBA.

\mathrm{\checkmark}

indicates that the inpainting is applied during inference.

	Loss		VBD	NBA
Method	$\mathcal{L}_{\mathrm{F}}$	$\mathcal{L}_{\mathrm{O}}$	Hit@1	Hit@1
Ours w/o $\bm{L}^{p,t}$	$\mathrm{\checkmark}$	-	63.4	29.2
Ours	$\mathrm{\checkmark}$	-	75.4	34.1
Ours w/o $\bm{L}^{p,t}$	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	79.0	37.7
Ours	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	82.7	43.9

Appendix

Table 9: Effects of the location-guidance in our GAF learning on VBD and NBA. Results obtained by the two settings (i.e.,

\mathcal{L}_{\mathrm{F}}

is only used or both

\mathcal{L}_{\mathrm{F}}

and

\mathcal{L}_{\mathrm{O}}

are used) are separated by the line.

	$\mathcal{L}_{\mathrm{F}}$		$\mathcal{L}_{\mathrm{O}}$		VBD	NBA
	$\mathcal{L}_{\mathrm{F},\bm{G}}$	$\mathcal{L}_{\mathrm{F},\bm{I}}$	$\mathcal{L}_{\mathrm{O},\bm{G}}$	$\mathcal{L}_{\mathrm{O},\bm{I}}$	Hit@1	Hit@1
(a)	-		-		43.0	22.6
(j)	-	$\mathrm{\checkmark}$	-		71.3	29.5
(k)	-		-	$\mathrm{\checkmark}$	72.5	37.0

Appendix

Table 10: Additionl ablation studies on VBD and NBA.

Method	Backbone	VBD	NBA
Method	Backbone	Hit@1	Hit@1
GAFL [ 28 ]	VGG19	61.1	24.7
GAFL [ 28 ]	DINOv3	38.1	25.3
Ours	DINOv3	82.7	43.9

Appendix

Table 11: Comparison of GAFL with different image feature extractors. We replace VGG19, the original image feature extractor in GAFL, with DINOv3

Method	All Params	Learnable Params	Time[ms]		Hit@1
Method	All Params	Learnable Params	VBD	NBA	VBD	NBA
GAFL [ 28 ]	79M	68M	82.4	86.3	61.1	24.7
Ours (ViT-B)	99M	21M	86.6	102.9	73.8	38.8
Ours (ViT-L)	326M	48M	272.3	327.7	82.7	43.9

Appendix

Table 12: Effects of architecture size for GAF learning on VBD and NBA. Results obtained by GAFL [ 28 ] and our method are separated by a double line. In our method, results obtained by different ViTs in DINOv3 are shown for further discussion.

	Training setting				VBD	NBA
	Unfrozen Blocks	LoRA	Adapter	Learnable Params	Hit@1	Hit@1
(a)	-	-	-	23M	64.6	30.0
(b)	24	-	-	36M	78.2	40.3
(c)	22, 23, 24	-	-	61M	80.9	36.5
(d)	-	$\mathrm{\checkmark}$	-	24M	76.6	42.3
(e)	-	-	1 linear	24M	65.7	34.4

Appendix

Table 13: Comparison of training strategies for DINOv3 in our method. Results obtained by our method are shown in (i). In (a), (b), (c), and (i), we fine-tuned different numbers of transformer blocks in the ViT of DINOv3. In (d), DINOv3 is updated by LoRA [ 31 ] . In (e), (f), (g), and (h), an additional linear layer or transformer blocks are attached to the frozen DINOv3 for fine-tuning.

Input features	VBD	NBA
Input features	Hit@1	Hit@1
Cropped	58.8	28.8
Whole (Ours)	74.0	40.3

Appendix

Table 14: Comparison of input features on VBD and NBA. While “Cropped” utilizes cropped-image features as input, our method utilizes whole-image features for GAF learning.

	Loss		VBD	NBA
Method	$\mathcal{L}_{\mathrm{F}}$	$\mathcal{L}_{\mathrm{O}}$	Hit@1	Hit@1
Whole image flow	$\mathrm{\checkmark}$	-	51.9	23.8
Person flow (Ours)	$\mathrm{\checkmark}$	-	75.4	34.1
Whole image flow	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	76.7	38.2
Person flow (Ours)	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	82.7	43.9

Appendix

Table 15: Effects of whole image flow estimation on VBD and NBA. While the optical flow image obtained from a whole image is estimated as a pretext task in “Whole image flow,” our method proposes to estimate flow values of each person as a pretext task for learning people-relevant features in

\bm{G}

	VBD
Method	Hit@1
GAFL [ 28 ] (YOLOX)	51.5
GAFL [ 28 ] (GT)	61.1
Ours (YOLOX)	81.2
Ours (GT)	82.7

Appendix

Table 16: Effects of errors in bounding boxes detected by YOLOX [ 12 ] on VBD. “GT” indicates that the experiments utilize the ground-truth person bounding boxes. “YOLOX” indicates that the experiments utilize detected tracklets by Deep-EIoU [ 7 ] .

Figure 12: GAR accuracy curve by the KNN classification on VBD and NBA. K changes from 1 to 20 in our experiment.

Figure 13: Performance under different noise levels.

$\mathcal{L}_{\mathrm{F}}$	$\mathcal{L}_{\mathrm{O}}$		NBA
Person Flow	Ball	Goal	Hit@1
-	$\mathrm{\checkmark}$	-	40.3
-	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	40.8
$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	-	43.9
$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	$\mathrm{\checkmark}$	45.5

Appendix

Table 17: Additional pretext goal-detection task.

Method	Extractor	VBD	NBA
Method	Extractor	Merged MCA	MPCA
DFWSGAR [ 21 ]	ResNet-18	94.4	71.2
SOGAR [ 6]	ResNet-18	95.9	73.5
Flaming-Net [ 29]	Inception-v3	95.2	76.0
LiGAR [ 4]	ResNet-18	76.1	57.1
SAM [ 56]	ResNet-18	93.1	51.5

Appendix

Table 18: Comparison with supervised group activity recognition methods on VBD and NBA. The results of all methods are copied from their papers. The four methods above the middle line take the whole image as input, while the four methods between the middle line and ours use person bounding boxes together with the whole image in both training and inference. For a fair comparison, in all methods, the only manually annotated labels are group activity classes, and only images are used in inference.

Figure 14: Visualization of the learned GAF space on VBD. Unlike Fig. 10 of the main paper, we also show actual images corresponding to the GAFs. The orange data points (i.e., “R-spike”) are divided into (a) and (b) based on the spiker’s position. The blue data points (i.e., “R-set”) are divided into (c) and (d) based on the setter’s position as similar to (a) and (b). In (e) and (f), the green data points (i.e., “R-pass”) are split into two clusters that differ in terms of the receiver’s position.

Method	3 frames	6 frames	12 frames	18 frames
DFWSGAR [ 21 ]	65.3/60.2	71.8/65.4	74.4/69.4	73.9/68.6
MP-GCN [ 26 ]	49.4/45.8	63.7/59.3	70.8/66.6	74.4/70.8
Ours	67.4/61.4	73.1/67.7	73.0/68.2	73.2/68.4

Appendix

Table 19: Effect of temporal resolutions on NBA. MCA/MPCA are shown. For fair comparison under equivalent conditions, we select [ 21, 26 ] whose authors provide publicly available code.

Explore similar work

May 21, 2026

H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning

Parametric human models capture global pose but cannot represent the non-rigid surface dynamics of clothing and soft tissue. Generic scene flow estimates dense motion but breaks down on articulated bodies, where pixel-level supervision is also intractable to acquire. We introduce H-Flow, a dense human scene flow that captures both skeletal kinematics and surface deformation. A unified multi-head transformer estimates flow from monocular video, jointly predicting pose and depth as companion outputs. The challenge lies in the lack of supervision. In place of unattainable labels, we anchor the network in the physics of human motion, encoding geometric, structural, and biomechanical priors as cross-modal training objectives. We further introduce DynAct4D, a high-fidelity synthetic benchmark providing dense flow annotations across diverse subjects, garments, and motions. On standard benchmarks, H-Flow outperforms scene-flow and parametric baselines, and generalizes zero-shot to in-the-wild video. Code, models, and the DynAct4D benchmark will be released upon publication

Zhanbo Huang, Xiaoming Liu, Yu Kong

Michigan State University · University of North Carolina, Chapel Hill

Optical Flow

Ask PDF

Apr 11, 2026

Attention-Guided Dual-Stream Learning for Group Engagement Recognition: Fusing Transformer-Encoded Motion Dynamics with Scene Context via Adaptive Gating

Student engagement is crucial for improving learning outcomes in group activities. Highly engaged students perform better both individually and contribute to overall group success. However, most existing automated engagement recognition methods are designed for online classrooms or estimate engagement at the individual level. Addressing this gap, we propose DualEngage, a novel two-stream framework for group-level engagement recognition from in-classroom videos. It models engagement as a joint function of both individual and group-level behaviors. The primary stream models person-level motion dynamics by detecting and tracking students, extracting dense optical flow with the Recurrent All-Pairs Field Transforms network, encoding temporal motion patterns using a transformer encoder, and finally aggregating per-student representations through attention pooling into a unified representation. The secondary stream captures scene-level spatiotemporal information from the full video clip, leveraging a pretrained three-dimensional Residual Network. The two-stream representations are combined via softmax-gated fusion, which dynamically weights each stream's contribution based on the joint context of both features. DualEngage learns a joint representation of individual actions with overarching group dynamics. We evaluate the proposed approach using fivefold cross-validation on the Classroom Group Engagement Dataset developed by Ocean University of China, achieving an average classification accuracy of 0.9621+/-0.0161 with a macro-averaged F1 of 0.9530+/-0.0204. To understand the contribution of each branch, we further conduct an ablation study comparing single-stream variants against the two-stream model. This work is among the first in classroom engagement recognition to adopt a dual-stream design that explicitly leverages motion cues as an estimator.

Saniah Kayenat Chowdhury, Muhammad E. H. Chowdhury

Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh · Department of Electrical Engineering, Qatar University, Doha, 2713, Qatar

AI in Education Temporal Video Understanding Human Activity Recognition

Ask PDF

Mar 23, 2026

EgoGroups: A Benchmark For Detecting Social Groups of People in the Wild

Social group detection, or the identification of humans involved in reciprocal interpersonal interactions (e.g., family members, friends, and customers and merchants), is a crucial component of social intelligence needed for agents transacting in the world. The few existing benchmarks for social group detection are limited by low scene diversity and reliance on third-person camera sources (e.g., surveillance footage). Consequently, these benchmarks generally lack real-world evaluation on how groups form and evolve in diverse cultural contexts and unconstrained settings. To address this gap, we introduce EgoGroups, a first-person view dataset that captures social dynamics in cities around the world. EgoGroups spans 65 countries covering low, medium, and high-crowd settings under four weather/time-of-day conditions. We include dense human annotations for person and social groups, along with rich geographic and scene metadata. Using this dataset, we performed an extensive evaluation of state-of-the-art VLM/LLMs and supervised models on their group detection capabilities. We found several interesting findings, including VLMs and LLMs can outperform supervised baselines in a zero-shot setting, while crowd density and cultural regions clearly influence model performance.

Jeffri Murrugarra-Llerena, Pranav Chitale, Zicheng Liu +4

Stony Brook University · Rice University

Topic pending

Ask PDF