Organizations: School of Social Information Science, University of Hyogo, Hyogo, Japan · Graduate School of Information Science, University of Hyogo, Hyogo, Japan
In animal facilities, continuous surveillance of penguins is essential yet technically challenging due to their homogeneous visual characteristics, rapid and frequent posture changes, and substantial environmental noise such as water reflections. In this study, we propose a framework that enhances both detection and identification performance by integrating appearance and motion features. For detection, we adapted YOLO11 to process consecutive frames to overcome the lack of temporal consistency in single-frame detectors. This approach leverages motion cues to detect targets even when distinct visual features are obscured. Our evaluation shows that fine-tuning the model with two-frame inputs improves [email protected] from 0.922 to 0.933, outperforming the baseline, and successfully recovers individuals that are indistinguishable in static images. For identification, we introduce a tracklet-based contrastive learning approach applied after tracking. Through qualitative visualization, we demonstrate that the method produces coherent feature embeddings, bringing samples from the same individual closer in the feature space, suggesting the potential for mitigating ID switching.
Figures & tables
Figure 1: Architecture of the proposed detection method
Name
Abbr.
Input Content
Sequential
RGB-Seq
{It,…,It−N+1}
Interval
RGB-Int
{It,It−Δ}
Diff-Sequential
Diff-Seq
{It,d(t−1,1),…,d(t−N+1,1)}
Diff-Interval
Diff-Int
{It,d(t−Δ,Δ)}
Table 1: Four types of input formats defined in this study
Table 5: Effect of frame interval Δ in Diff-Int ( N=2 )
Figure 2: Detection results during swimming. Individuals difficult to distinguish in still images (baseline, left) are detected by utilizing video information (proposed method, right).
Figure 3: Detection results in background regions unseen during training. Moving individuals were detected in both methods (top), whereas stationary individuals were not detected (bottom), illustrating the contribution of motion cues.
Figure 4: Visualization of feature embeddings using t-SNE
Figure 5: Grad-CAM visualization examples for IDs 15 and 21