cs.CVFeb 17, 2026
SaveAutomated Re-Identification of Holstein-Friesian Cattle in Dense Crowds
Organizations: School of Computer Science, Merchant Venturers Building, Woodland Road, University of Bristol, Bristol, BS8 1UB, Bristol, United Kingdom · Bristol Veterinary School, University of Bristol, Langford House, Dolberry, Churchill, Bristol, BS40 5DU, Bristol, United Kingdom
Abstract
Holstein-Friesian detection and re-identification (Re-ID) methods capture individuals well when targets are spatially separate. However, existing approaches, including YOLO-based species detection, break down when cows group closely together. This is particularly prevalent for species which have outline-breaking coat patterns. To boost both effectiveness and transferability in this setting, we propose a new detect-segment-identify pipeline that leverages the Open-Vocabulary Weight-free Localisation and the Segment Anything models as pre-processing stages alongside Re-ID networks. To evaluate our approach, we publish a collection of nine days CCTV data filmed on a working dairy farm. Our methodology overcomes detection breakdown in dense animal groupings, resulting in a 98.93% accuracy. This significantly outperforms current oriented bounding box-driven, as well as SAM species detection baselines with accuracy improvements of 47.52% and 27.13%, respectively. We show that unsupervised contrastive learning can build on this to yield 94.82% Re-ID accuracy on our test data. Our work demonstrates that Re-ID in crowded scenarios is both practical as well as reliable in working farm settings with no manual intervention. Code and dataset are provided for reproducibility.
Figures & tables
Figure 1: Existing Detector Performance on Crowded Cows. Visualisations of performance with YOLO-v11x [ 26 ] (left) and RetinaNet [ 62 ] (right) on detecting cows without fine-tuning: the RetinaNet species detector module from MultiCamCows2024, pretrained on sparsely grouped cows, underperforms in detecting individuals from crowds. The YOLO-v11x model pretrained on the COCO dataset completely fails to detect and localise any cow.
Figure 2: Bounding Box Results from Text-Prompted Detection Models. The inference results of both OWLv2 [ 34 ] and GroundingDINO [ 31 ] on our cow herds with no prior fine-tuning. GroundingDINO, though much faster than OWLv2 inference, fails to identify individuals when using the same text prompt as input.
Figure 3: Instance Segmentation on Crowded Holstein-Friesians. Illustrations of instance segmentations from our pipeline (left) and in failure of segmentation with GroundedSAM [ 43 ] when used without prior fine-tuning (right) . GroundedSAM and our pipeline are, in principle, both capable of segmenting polygon masks based on text inputs. Yet GroundedSAM, segmenting one single region using the same text prompt ‘cow’ as our pipeline, or a varied selection of prompts such as ‘a cow’, ‘individual cow’, or ‘cows’, fails to separate and outline the torso of each cow individually. Other prompts, such as ‘all instances of cows among a group’ or ‘30 individual cows in the densely packed group’, fail to provide any detection for OWLv2, or segmentation for GroundedSAM.
Figure 4: Processing Pipeline Overview. Our framework consists of two main components. For data acquisition (left) of both mask segmentations and ID, we initiate tracking and relocate targets at intervals via OWLv2-enhanced and baseline SAM2 models. For ground truth image set generation, we manually label the bounding boxes and use them as the initial input for tracking. Then, two sections of evaluation were performed. We first apply IoU matching between SAM2 binary masks and ground truth to evaluate their localisation performance (upper-right) . After integrating binary masks with their original frames to get the RGB masks, we applied a UCL module to evaluate Re-ID performance (lower-right) .
Figure 5: Automated Data Acquisition. Our proposed approach for using prompts to generate segmentation masks has two stages: first in (red) , we use a text prompt along with video frames as the inputs to a pre-trained OWLv2 [ 34 ] model to get axis-aligned bounding boxes. Then, in (purple) , the same video frames along with the bounding box outputs are used as inputs to a pre-trained SAM2 [ 27 ] model to yield segmentations.
Figure 6: Unsupervised Contrastive Learning Module Overview. During the training phase, we feed the masks from different individuals within the same time instance into our NTXentLoss function. The learned embeddings are analysed using K-Nearest Neighbours, and K-Means is used for clustering. UMAP is used for visualisations of raw embeddings coloured with the ground-truth labels, and also with K-Means labels for inference.
Figure 7: SAM-based Species Detector Comparison. Visualisations of detections from the standard SAM2 baseline and our OWLv2-SAM2 pipeline. Note both over-segmentations (upper-left and middle small bounding boxes capturing only distinct coloured blobs rather than the entire cow) and missing targets (darker cows on the right) when using the baseline.
| Backbone | IoU | TP | Usage | Matching |
|---|---|---|---|---|
| Accuracy | Rate | Rate | ||
| [ 62 ] | ||||
| [ 42 ] | ||||
| (ours) | 0.450 | 25.00% | 33.80% | 98.93% |
Table 1: Oriented Bounding Box Localisation. Shown are metrics to evaluate oriented bounding box detection results from MultiCamCows2024(RetinaNet), baseline SAM2 and our pipeline with ground truth data. IoU : Intersection over Union with respect to ground truth versus detected bounding boxes. TP Accuracy : Average IoU for all individuals which have been well-detected (individuals’ average IoU over all frames). This metric punishes missing detections more than IoU alone. Usage Rate : Fraction of total detections for which IoU . Matching Rate : Fraction of ground-truth bounding boxes that have a corresponding well-detected bounding box. This metric helps explain over- versus under-segmentation.
| Backbone | IoU | TP | Usage | Matching |
|---|---|---|---|---|
| Accuracy | Rate | Rate | ||
| [ 42 ] | ||||
| (ours) | 0.898 | 70.83% | 75.39% | 94.11% |
Table 2: Pixel-Level Segmentation Evaluation. Shown are metrics to evaluate baseline SAM2 and our instance segmentations with ground truth data. Here, the same metrics are evaluated on the pixel-level masks, rather than for the oriented bounding boxes of Table 1 .
| Metric | Manual | Automated (ours) |
|---|---|---|
Table 3: Unsupervised Contrastive Learning Clustering. The metrics to evaluate clustering performance between manually annotated ground truth masks and the automated mask dataset enhanced with OWLv2-SAM2, with k-fold cross-validation where . The kNN Accuracy is the accuracy of classification. The Adjusted Random Index (ARI), Adjusted Mutual Information (AMI), Normalised Mutual Information (NMI) and accuracy of the Hungarian Algorithm (HA) represent the metrics for analysing clustering similarities based on ground truth labels and K-Means results.
| Target Dataset | Test Accuracy |
|---|---|
Table 4: kNN Accuracy on Inference and Retraining Models of Unseen Data. Using a customised k-fold cross-validation with , the Test Accuracy uses the corresponding target dataset for inference. For each instance, its previous day was used as the validation set, and the remaining seven days were used for training. (e.g. The first row metric is obtained from a model instance trained with data from 2024-10-31 to 2024-11-06, validated on data from 2024-11-07, and tested on the data from 2024-10-18.)
Figure 8: Embeddings for Re-ID over Nine Days. Trained on seven days of data, validated and tested on the other two days, as part of a k-fold cross-validation framework. The visualisations show testing performance, using each day’s inference via the UCL-based Re-ID module. Each day shows not only different numbers of frames, but also different numbers of individuals.
Figure 9: Sample Frames from the Nine Days of the Re-ID Experiment. Images show a sample frame for each day considered for the re-identification experiment. Note that the timestamp for the first image is one hour off due to summer time adjustment. In addition, as cameras are motion-triggered for recording, the exact time of first capture (after 12:00 pm) varies slightly.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Oriented Bounding Box Results Metrics. Additional metrics for oriented bounding boxes using a part of our new dataset. We show results for RetinaNet-based, baseline SAM2 and our OWL2-enhanced SAM2 detection. From a video with a total of frames, the (upper) row represents the average IoU of the corresponding segmentation per video frame. These are based on the manually labelled ground truths of a total of individuals. The (lower) row demonstrates the ratio of well-detected bounding boxes, a frame-level Usage Rate.
Figure 11: Mask Segmentations Metrics. Additional metrics for masks using a part of our new dataset. We show results for baseline SAM2 and our OWL2-enhanced SAM2 detection. From a video with a total of frames, the (upper) row represents the average IoU of the corresponding segmentation per video frame. These are based on the manually labelled ground truths of a total of individuals. The (lower) row demonstrates the ratio of well-detected bounding boxes(Usage Rate) per frame.
Figure 12: Embedding Visualisations. Embeddings of our hand-labelled (single day worth of) ground truth versus automatically extracted masks. The (upper) row represents the results from the hand-labelled dataset, while the (lower) from our automated framework. (Left) shows the embedding, for the training set, coloured with the best K-Means proposed labels. (Middle) column is the same embedding but coloured with the ground truth labels. (Right) shows the test set coloured using the ground truths. Ideally, they would all be the same, with every cluster having a single colour.
Figure 13: Visualisations of Single-Day UCL Embeddings. Embeddings showing clustering for each day in turn. The dataset is obtained with our automated OWLv2-SAM2 pipeline. The Figure demonstrates that Re-ID may be possible using only a single day’s data.
| Target Dataset | Single-Day kNN Accuracy |
|---|---|
Table 5: Single-Day UCL Accuracy The kNN accuracies of the single-day UCL pipelines shown from Figure 13 . Each instance trained and tested on data from the same day. of data is used for training and validation, and for testing.