CowCare: A Unified Visual Framework for Viewpoint-Robust Cattle Tracking and Spatiotemporal Heat-Stress Analysis
Authors: Xin Liu, Weiran Li, Minghao Zhu, Mina Han, Zhenbo Li
Organizations: College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China · National Innovation Center for Digital Fishery, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China · Key Laboratory of Smart Farming Technologies for Aquatic Animal and Livestock, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China · Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China
Reliable individual monitoring is essential for dairy cattle welfare and early heat stress detection in Precision Livestock Farming (PLF). However, translating raw multi-camera videos into automated diagnostics remains challenging due to disconnected tracking and recognition pipelines and complex physical environments. Dense mutual occlusions and cross-camera viewpoint variations fragment identities, while environmental noise, including camera vibrations, and limited agricultural data obscure subtle heat stress cues such as rapid breathing and flank undulations. To bridge low-level visual features and high-level behavioral diagnostics, we propose CowCare, a unified framework combining viewpoint-robust tracking with spatiotemporal heat stress recognition. Its tracking module, CowTrack, maintains cross-camera identity consistency through a View-Decoupled Aggregation Network (VDA-Net). Instead of relying on static global representations, VDA-Net aggregates localized textures under heavy occlusion and uses a Viewpoint Decoupling Transformer (VDT) to suppress transient viewpoint variations. CowTrack further incorporates a causal online self-supervision mechanism for continuous adaptation to dynamic farm scenes. The resulting tracklets are subsequently processed by TSDF-Net, a fine-grained video understanding network. Its Temporal Stabilization and Aggregation Module (TSAM) suppresses high-frequency jitter, while a Diffusion Semantic Prior (DSP) uses frozen pre-trained diffusion semantics to enhance subtle behavioral signals and mitigate overfitting. Extensive evaluations on two real-world farm benchmarks demonstrate the framework's effectiveness: CowTrack achieves 83.48% HOTA and 87.72% IDF1 for global multi-object tracking on MVCow, and TSDF-Net achieves 87.72% Top-1 accuracy on HSCow. These results establish CowCare as a robust end-to-end framework for automated livestock health monitoring.
Figures & tables
Figure 1: Overview of the visual challenges and the comprehensive performance of our proposed methods. (a) Critical challenges in cattle tracking and heat-stress analysis, including appearance confusion under dense occlusion, identity fragmentation across non-overlapping views, and domain gaps in dynamic scenes. (b) Performance comparison of our CowTrack and VDA-Net against state-of-the-art MOT and ReID methods. (c) Performance comparison of our TSDF-Net against existing spatiotemporal video understanding models for fine-grained heat-stress recognition.
Figure 2: Equipment deployment and the actual farming environment for dataset collection.
Figure 3: Representative samples from the MVCow dataset illustrating herd occlusion, severe individual occlusion, and cross-view appearance distortion.
Figure 4: Multi-view images of the same individual cow, highlighting the visual variations across different camera angles.
Figure 5: The annotation workflow for the HSCow heat stress dataset, including multi-object tracking annotation, tracklet clipping, and expert evaluation.
Dataset
Farm Location
Area
Time
Videos
Purpose
MVCow
Yanqing, Beijing
Intersection of milking channel and parlor
Apr 2025
12 Clips
Train and evaluate object detection, ReID, and MOT models
HSCow
Shandong
Barn cooling zone
Jun-Sep 2025
6 Clips
Train and evaluate heat stress behavior recognition models
Table 1: Data collection information for the constructed benchmarks.
Table 2: Task-specific data split configurations for the MVCow dataset.
Category
Number of Clips
Avg. Duration (s)
Normal
748
15.17
Heat Stress
383
13.78
Total
1131
14.70
Table 3: Statistical overview of the HSCow behavior dataset.
Figure 6: The overall architecture of the CowCare framework.
Figure 7: The overall architecture of the CowTrack framework.
Figure 8: Architecture and core components of the View-Decoupled Aggregation Network (VDA-Net). Top: The overall VDA-Net pipeline, illustrating the backbone integrated with Outlook attention layers and the Viewpoint Decoupling Transformer (VDT). Bottom left: The structure of the Outlook block, utilized for localized texture aggregation under severe occlusions. Bottom right: Detailed structure of the VDT block, showcasing the subtractive mechanism for isolating view-invariant identity embeddings.
Figure 9: Flowchart of the causal online self-supervised fine-tuning process for dynamic scene adaptation.
Figure 10: The overall architecture of TSDF-Net, highlighting the Temporal Stabilization and Aggregation Module (TSAM) and the Diffusion Semantic Prior (DSP).
Figure 11: Schematic of the Temporal Stabilization and Aggregation Module (TSAM) and the Diffusion Semantic Prior (DSP). TSAM performs temporal smoothing through 1D depthwise convolution and quality-aware gating, whereas DSP injects generalized motion semantics through a frozen diffusion encoder.
Method
Rank-1 ↑
Rank-5 ↑
mAP ↑
CLIP-ReID ( Li et al., 2023 )
97.36
99.84
82.45
DGNet ( Zheng et al., 2019 )
92.89
97.81
81.79
HOReID ( Wang et al., 2021 )
61.28
62.89
62.67
OSNet ( Zhou et al., 2019 )
98.90
99.90
99.60
PCB ( Sun et al., 2018 )
61.79
63.04
63.07
PersonViT ( Hu et al., 2025 )
94.90
99.10
93.20
Table 4: Cross-camera ReID performance on MVCow (%). Best and second-best results are shown in bold and underlined, respectively. ↑ indicates higher values are better.
Method
HOTA ↑
MOTA ↑
IDF1 ↑
BoostTrack ( Stanojević and Todorović, 2024 )
77.49
82.76
81.40
ByteTrack ( Zhang et al., 2022b )
75.18
79.47
82.09
BoT-SORT ( Aharon et al., 2022 )
68.87
79.04
72.11
DeepSORT ( Wojke et al., 2017 )
75.07
80.03
81.30
Deep OC-SORT ( Maggiolino et al., 2023 )
75.29
80.00
81.58
Depth-OC-SORT ( Zhu et al., 2026 )
74.99
75.77
80.13
Table 5: Global multi-object tracking performance on MVCow (%). Best and second-best results are shown in bold and underlined, respectively. ↑ indicates higher values are better.
Model
Top-1 ↑
Precision ↑
Recall ↑
F1 ↑
ActionDiff ( Guimarães et al., 2026 )
76.32
78.30
76.08
75.78
SlowOnly-R50 ( Feichtenhofer et al., 2019 )
78.07
82.36
78.39
77.44
SlowFast ( Feichtenhofer et al., 2019 )
70.18
81.11
70.69
67.62
TSN ( Wang et al., 2016 )
78.07
78.51
77.96
77.93
TSM ( Lin et al., 2019 )
76.32
76.97
76.45
76.23
TimeSformer ( Bertasius et al., 2021 )
70.18
70.55
70.04
69.94
Table 6: Heat stress behavior recognition performance on the HSCow dataset (%). The best results are shown in bold, while the second-best results are underlined. ↑ indicates that higher values are better.
Configuration
Rank-1 ↑
Rank-5 ↑
Rank-10 ↑
mAP ↑
Baseline
96.03
98.72
99.05
96.69
+ Outlook
97.50
99.29
99.33
97.54
vs. previous
↑1.47pp
↑0.57pp
↑0.28pp
↑0.85pp
+ VDT
99.83
99.98
99.98
99.67
vs. previous
↑2.33pp
↑0.69pp
↑0.65pp
↑2.13pp
Table 7: Ablation of VDA-Net components on cross-camera ReID (%). Changes are absolute percentage-point differences from the preceding row.
Configuration
HOTA ↑
MOTA ↑
IDF1 ↑
Baseline
77.49
82.76
81.40
+ VDA-Net (Static)
81.67
89.50
85.55
vs. previous
↑4.18pp
↑6.74pp
↑4.15pp
+ Online Adaptation
83.48
89.50
87.72
vs. previous
↑1.81pp
→0.00pp
↑2.17pp
Table 8: Ablation of CowTrack components on global tracking (%). Changes are absolute percentage-point differences from the preceding row.
Configuration
Top-1 ↑
Precision ↑
Recall ↑
F1 ↑
Baseline
82.46
83.06
82.57
82.41
+ TSAM
83.30
87.33
83.62
82.95
vs. previous
↑0.84pp
↑4.27pp
↑1.05pp
↑0.54pp
+ DSP
87.72
89.35
87.90
87.62
vs. previous
↑4.42pp
↑2.02pp
↑4.28pp
↑4.67pp
Table 9: Ablation studies of TSDF-Net components (%). Changes are absolute percentage-point differences from the preceding row.
Setup
Rank-1 ↑
Rank-5 ↑
Rank-10 ↑
mAP ↑
Single-CAM
98.30
99.20
99.73
88.93
Multi-CAM
99.83
99.98
99.98
99.67
Gain (pp)
↑1.53
↑0.78
↑0.25
↑10.74
Table 10: Comparison of cross-camera ReID performance between single-camera and multi-camera setups (%).
Figure 12: Heatmap visualization of TSDF-Net performance sensitivity to temporal sampling parameters (input frame count T and frame interval Δt ) in terms of Top-1 accuracy (left) and Heat Stress F1 score (right).
Input Frames T
Frame Interval Δt
Top-1 ↑
Precision ↑
Recall ↑
F1 ↑
Heat Stress F1 ↑
Heat Stress Recall ↑
4
1
88.60
88.68
88.64
88.60
88.70
91.07
4
8
89.47
89.63
89.53
89.47
89.66
92.86
6
1
86.84
88.17
87.01
86.76
87.80
96.43
6
8
87.72
88.83
87.87
87.66
88.52
96.43
8
1
86.84
88.74
87.04
86.72
88.00
98.21
8
8
87.72
89.35
87.90
87.62
88.71
98.21
Table 11: Sensitivity analysis of selected representative temporal sampling configurations, parameterized by the input frame count T and frame interval Δt , on TSDF-Net performance (%). The complete results for all 25 configurations are shown in Fig. 12 . Precision, Recall, and F1 are macro-averaged across both classes. Heat Stress F1 and Heat Stress Recall treat the heat stress category as the positive class. For T=10 , all interval results are identical, hence only one representative row is listed.
Figure 13: Qualitative comparison of tracking performance under severe occlusion. The baseline method (top) suffers from an ID switch (from ID 43 to ID 23), whereas our CowTrack (bottom) successfully maintains the correct identity (ID 31).
Figure 14: t-SNE feature space visualization of hard ReID samples in the MVCow dataset. Compared to the baseline (left), our VDA-Net (right) yields tighter intra-class clustering and larger inter-class margins.
Figure 15: Comparison of feature space distributions using t-SNE. (a) The Video-FocalNet baseline; (b) Our TSDF-Net, which demonstrates clearer decision boundaries between normal and heat-stressed samples.
Figure 16: Cross-camera retrieval rankings produced by the baseline and VDA-Net in challenging cases involving occlusion and viewpoint changes.
Figure 17: Grad-CAM visualization of a heat stress video sequence constrained by the cow body ROI. The heatmaps demonstrate that our network correctly focuses on physiological indicators such as the flank undulations.
Figure 18: Attention heatmap comparison. Compared to the baseline (left), TSDF-Net (right) effectively concentrates on critical physiological areas (e.g., flank undulations) while avoiding background distractions.
Figure 19: Temporal sequence comparison between frame quality scores generated by TSAM (red) and optical flow magnitude (blue), illustrating the model’s ability to adaptively down-weight high-jitter frames.
Figure 20: Continuous heat stress prediction probability curve generated by the CowCare framework in a simulated real-world deployment queue.
Promity, Warsaw, Poland · Institute of Animal Science, Warsaw University of Life Sciences, Warsaw, Poland · Faculty of Electronics and Information Technology, Warsaw University of Technology, Warsaw, Poland
Bristol Veterinary School, University of Bristol, Langford House, Dolberry, Churchill, Bristol, BS40 5DU, United Kingdom · School of Computer Science, University of Bristol, Merchant Venturers Building, Woodland Road, Bristol, BS8 1UB, United Kingdom
Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh · Department of Data Science and Artificial Intelligence, Monash University, Melbourne 3800, Australia +5