Automated Re-Identification of Holstein-Friesian Cattle in Dense Crowds
Authors: Phoenix Yu, Tilo Burghardt, Andrew W Dowsey, Neill W Campbell
Organizations: School of Computer Science, Merchant Venturers Building, Woodland Road, University of Bristol, Bristol, BS8 1UB, Bristol, United Kingdom · Bristol Veterinary School, University of Bristol, Langford House, Dolberry, Churchill, Bristol, BS40 5DU, Bristol, United Kingdom
Holstein-Friesian detection and re-identification (Re-ID) methods capture individuals well when targets are spatially separate. However, existing approaches, including YOLO-based species detection, break down when cows group closely together. This is particularly prevalent for species which have outline-breaking coat patterns. To boost both effectiveness and transferability in this setting, we propose a new detect-segment-identify pipeline that leverages the Open-Vocabulary Weight-free Localisation and the Segment Anything models as pre-processing stages alongside Re-ID networks. To evaluate our approach, we publish a collection of nine days CCTV data filmed on a working dairy farm. Our methodology overcomes detection breakdown in dense animal groupings, resulting in a 98.93% accuracy. This significantly outperforms current oriented bounding box-driven, as well as SAM species detection baselines with accuracy improvements of 47.52% and 27.13%, respectively. We show that unsupervised contrastive learning can build on this to yield 94.82% Re-ID accuracy on our test data. Our work demonstrates that Re-ID in crowded scenarios is both practical as well as reliable in working farm settings with no manual intervention. Code and dataset are provided for reproducibility.
Figures & tables
Figure 1: Existing Detector Performance on Crowded Cows. Visualisations of performance with YOLO-v11x [ 26 ] (left) and RetinaNet [ 62 ] (right) on detecting cows without fine-tuning: the RetinaNet species detector module from MultiCamCows2024, pretrained on sparsely grouped cows, underperforms in detecting individuals from crowds. The YOLO-v11x model pretrained on the COCO dataset completely fails to detect and localise any cow.
Figure 2: Bounding Box Results from Text-Prompted Detection Models. The inference results of both OWLv2 [ 34 ] and GroundingDINO [ 31 ] on our cow herds with no prior fine-tuning. GroundingDINO, though much faster than OWLv2 inference, fails to identify individuals when using the same text prompt as input.
Figure 3: Instance Segmentation on Crowded Holstein-Friesians. Illustrations of instance segmentations from our pipeline (left) and in failure of segmentation with GroundedSAM [ 43 ] when used without prior fine-tuning (right) . GroundedSAM and our pipeline are, in principle, both capable of segmenting polygon masks based on text inputs. Yet GroundedSAM, segmenting one single region using the same text prompt ‘cow’ as our pipeline, or a varied selection of prompts such as ‘a cow’, ‘individual cow’, or ‘cows’, fails to separate and outline the torso of each cow individually. Other prompts, such as ‘all instances of cows among a group’ or ‘30 individual cows in the densely packed group’, fail to provide any detection for OWLv2, or segmentation for GroundedSAM.
Figure 4: Processing Pipeline Overview. Our framework consists of two main components. For data acquisition (left) of both mask segmentations and ID, we initiate tracking and relocate targets at 1s intervals via OWLv2-enhanced and baseline SAM2 models. For ground truth image set generation, we manually label the bounding boxes and use them as the initial input for tracking. Then, two sections of evaluation were performed. We first apply IoU matching between SAM2 binary masks and ground truth to evaluate their localisation performance (upper-right) . After integrating binary masks with their original frames to get the RGB masks, we applied a UCL module to evaluate Re-ID performance (lower-right) .
Figure 5: Automated Data Acquisition. Our proposed approach for using prompts to generate segmentation masks has two stages: first in (red) , we use a text prompt along with video frames as the inputs to a pre-trained OWLv2 [ 34 ] model to get axis-aligned bounding boxes. Then, in (purple) , the same video frames along with the bounding box outputs are used as inputs to a pre-trained SAM2 [ 27 ] model to yield segmentations.
Figure 6: Unsupervised Contrastive Learning Module Overview. During the training phase, we feed the masks from different individuals within the same time instance into our NTXentLoss function. The learned embeddings are analysed using K-Nearest Neighbours, and K-Means is used for clustering. UMAP is used for visualisations of raw embeddings coloured with the ground-truth labels, and also with K-Means labels for inference.
Figure 7: SAM-based Species Detector Comparison. Visualisations of detections from the standard SAM2 baseline and our OWLv2-SAM2 pipeline. Note both over-segmentations (upper-left and middle small bounding boxes capturing only distinct coloured blobs rather than the entire cow) and missing targets (darker cows on the right) when using the baseline.
Backbone
IoU
TP
Usage
Matching
Accuracy
Rate
Rate
RetinaNet [ 62 ]
0.256
0.00%
1.04%
51.41%
SAM2Baseline [ 42 ]
0.383
20.83%
26.97%
71.80%
OWLv2+SAM2 (ours)
0.450
25.00%
33.80%
98.93%
Table 1: Oriented Bounding Box Localisation. Shown are metrics to evaluate oriented bounding box detection results from MultiCamCows2024(RetinaNet), baseline SAM2 and our pipeline with ground truth data. IoU : Intersection over Union with respect to ground truth versus detected bounding boxes. TP Accuracy : Average IoU for all individuals which have been well-detected (individuals’ average IoU >0.7 over all frames). This metric punishes missing detections more than IoU alone. Usage Rate : Fraction of total detections for which IoU >0.7 . Matching Rate : Fraction of ground-truth bounding boxes that have a corresponding well-detected bounding box. This metric helps explain over- versus under-segmentation.
Backbone
IoU
TP
Usage
Matching
Accuracy
Rate
Rate
SAM2Baseline [ 42 ]
0.821
41.67%
43.80%
46.96%
OWLv2+SAM2 (ours)
0.898
70.83%
75.39%
94.11%
Table 2: Pixel-Level Segmentation Evaluation. Shown are metrics to evaluate baseline SAM2 and our instance segmentations with ground truth data. Here, the same metrics are evaluated on the pixel-level masks, rather than for the oriented bounding boxes of Table 1 .
Metric
Manual
Automated (ours)
kNNAccuracy
99.98±0.01%
97.77±0.65%
ARI
0.974±0.001
0.866±0.007
AMI
0.987±0.004
0.944±0.006
NMI
0.988±0.004
0.945±0.006
HA
97.73±1.35%
87.55±0.55%
Table 3: Unsupervised Contrastive Learning Clustering. The metrics to evaluate clustering performance between manually annotated ground truth masks and the automated mask dataset enhanced with OWLv2-SAM2, with k-fold cross-validation where k=5 . The kNN Accuracy is the accuracy of classification. The Adjusted Random Index (ARI), Adjusted Mutual Information (AMI), Normalised Mutual Information (NMI) and accuracy of the Hungarian Algorithm (HA) represent the metrics for analysing clustering similarities based on ground truth labels and K-Means results.
Target Dataset
Test Accuracy
2024−10−18
98.91%
2024−10−31
99.52%
2024−11−01
99.89%
2024−11−02
93.96%
2024−11−03
91.31%
2024−11−04
88.27%
Table 4: kNN Accuracy on Inference and Retraining Models of Unseen Data. Using a customised k-fold cross-validation with k=9 , the Test Accuracy uses the corresponding target dataset for inference. For each instance, its previous day was used as the validation set, and the remaining seven days were used for training. (e.g. The first row metric is obtained from a model instance trained with data from 2024-10-31 to 2024-11-06, validated on data from 2024-11-07, and tested on the data from 2024-10-18.)
Figure 8: Embeddings for Re-ID over Nine Days. Trained on seven days of data, validated and tested on the other two days, as part of a k-fold cross-validation framework. The visualisations show testing performance, using each day’s inference via the UCL-based Re-ID module. Each day shows not only different numbers of frames, but also different numbers of individuals.
Figure 9: Sample Frames from the Nine Days of the Re-ID Experiment. Images show a sample frame for each day considered for the re-identification experiment. Note that the timestamp for the first image is one hour off due to summer time adjustment. In addition, as cameras are motion-triggered for recording, the exact time of first capture (after 12:00 pm) varies slightly.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Oriented Bounding Box Results Metrics. Additional metrics for oriented bounding boxes using a part of our new dataset. We show results for RetinaNet-based, baseline SAM2 and our OWL2-enhanced SAM2 detection. From a video with a total of 1500 frames, the (upper) row represents the average IoU of the corresponding segmentation per video frame. These are based on the manually labelled ground truths of a total of 24 individuals. The (lower) row demonstrates the ratio of well-detected bounding boxes, a frame-level Usage Rate.
Figure 11: Mask Segmentations Metrics. Additional metrics for masks using a part of our new dataset. We show results for baseline SAM2 and our OWL2-enhanced SAM2 detection. From a video with a total of 1500 frames, the (upper) row represents the average IoU of the corresponding segmentation per video frame. These are based on the manually labelled ground truths of a total of 24 individuals. The (lower) row demonstrates the ratio of well-detected bounding boxes(Usage Rate) per frame.
Figure 12: Embedding Visualisations. Embeddings of our hand-labelled (single day worth of) ground truth versus automatically extracted masks. The (upper) row represents the results from the hand-labelled dataset, while the (lower) from our automated framework. (Left) shows the embedding, for the training set, coloured with the best K-Means proposed labels. (Middle) column is the same embedding but coloured with the ground truth labels. (Right) shows the test set coloured using the ground truths. Ideally, they would all be the same, with every cluster having a single colour.
Figure 13: Visualisations of Single-Day UCL Embeddings. Embeddings showing clustering for each day in turn. The dataset is obtained with our automated OWLv2-SAM2 pipeline. The Figure demonstrates that Re-ID may be possible using only a single day’s data.
Target Dataset
Single-Day kNN Accuracy
2024−10−18
98.96%
2024−10−31
99.92%
2024−11−01
97.57%
2024−11−02
95.04%
2024−11−03
95.30%
2024−11−04
90.03%
Appendix
Table 5: Single-Day UCL Accuracy The kNN accuracies of the single-day UCL pipelines shown from Figure 13 . Each instance trained and tested on data from the same day. 80% of data is used for training and validation, and 20% for testing.
Camera-based animal re-identification (Animal Re-ID) can support wildlife monitoring and precision livestock management in large outdoor environments with limited wireless connectivity. In these settings, inference must run directly on collar tags or low-power edge nodes built around microcontrollers (MCUs), yet most Animal Re-ID models are designed for workstations or servers and are too large for devices with small memory and low-resolution inputs. We propose an on-device framework. First, we characterise the gap between state-of-the-art Animal Re-ID models and MCU-class hardware, showing that straightforward knowledge distillation from large teachers offers limited benefit once memory and input resolution are constrained. Second, guided by this analysis, we design a high-accuracy Animal Re-ID architecture by systematically scaling a CNN-based MobileNetV2 backbone for low-resolution inputs. Third, we evaluate the framework with a real-world dataset and introduce a data-efficient fine-tuning strategy to enable fast adaptation with just three images per animal identity at a new site. Across six public Animal Re-ID datasets, our compact model achieves competitive retrieval accuracy while reducing model size by over two orders of magnitude. On a self-collected cattle dataset, the deployed model performs fully on-device inference with only a small accuracy drop and unchanged Top-1 accuracy relative to its cluster version. We demonstrate that practical, adaptable Animal Re-ID is achievable on MCU-class devices, paving the way for scalable deployment in real field environments.
AnimalCLEF26 addresses discovery-oriented animal re-identification, where systems must both attach query images to known individuals and discover unseen individuals by clustering them correctly. We present a similarity-to-clustering pipeline for this setting across Eurasian lynx, fire salamander, loggerhead sea turtle, and Texas horned lizard images. The method first isolates the target specimen using segmentation and then applies lightweight species-specific preprocessing for lynx, sea turtle, and salamander images to enhance identity-relevant visual cues, while Texas horned lizard images are used after segmentation only. Pairwise similarities are then estimated with WildFusion by calibrating and combining a MiewID global descriptor with two local matching branches, ALIKED + LightGlue and DISK + LightGlue. The resulting query-query similarities are refined and converted into identity clusters using graph-based clustering, while query-database similarities are used to attach confident samples to known identities. We evaluate training-free and fine-tuned MiewID variants, including Dynamic ArcFace and SphereFace2-Focal adaptations, and combine them in the final ensemble. Our selected ensemble substantially improves on the WildFusion baseline, achieving the best public ARI of 0.72124 and a private ARI of 0.70393, while a simpler preprocessing-before-calibration variant achieves the best private ARI of 0.71087. These results indicate that calibrated global-local fusion with species-aware preprocessing choices is effective for open-set wildlife re-identification under challenging field conditions and visual variation. The implementation code is available on GitHub.
Made In Alexandria Artificial Intelligence Team, Alexandria, Egypt · Faculty of Computers and Data Science, Alexandria University, Alexandria, Egypt · Faculty of Computer Science and Engineering, Alamein International University, New Alamein City, 51718, Egypt +3
Fine-grained wildlife re-identification remains a challenging area in research. Current state-of-the-art approaches apply a detection and re-identification pipeline. We propose a one-stage end-to-end detection and re-identification model that performs identity searching within the latent space. We adopt DINOv2 for robust spatial geometry and MegaDescriptor for wildlife re-identification. We enhance latent queries with prompt re-identification features. A detection decoder queries the scene latent space to establish object boundaries around the target identity. Preliminary findings reflect a competitive mean average precision score of 30.584% compared to the state-of-the-art two stage approach of 44.89%. Qualitative results depict effective bounding and identification of animal identities.
Mufhumudzi Muthivhi, Jiahao Huo, Terence van Zyl +1
University of Johannesburg, South Africa · Linköping University, Sweden