CowCare: A Unified Visual Framework for Viewpoint-Robust Cattle Tracking and Spatiotemporal Heat-Stress Analysis

Xin Liu; Weiran Li; Minghao Zhu; Mina Han; Zhenbo Li

cs.CVOct 10, 2026

CowCare: A Unified Visual Framework for Viewpoint-Robust Cattle Tracking and Spatiotemporal Heat-Stress Analysis

Authors: Xin Liu, Weiran Li, Minghao Zhu, Mina Han, Zhenbo Li

Organizations: College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China · National Innovation Center for Digital Fishery, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China · Key Laboratory of Smart Farming Technologies for Aquatic Animal and Livestock, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China · Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China

Abstract

Reliable individual monitoring is essential for dairy cattle welfare and early heat stress detection in Precision Livestock Farming (PLF). However, translating raw multi-camera videos into automated diagnostics remains challenging due to disconnected tracking and recognition pipelines and complex physical environments. Dense mutual occlusions and cross-camera viewpoint variations fragment identities, while environmental noise, including camera vibrations, and limited agricultural data obscure subtle heat stress cues such as rapid breathing and flank undulations. To bridge low-level visual features and high-level behavioral diagnostics, we propose CowCare, a unified framework combining viewpoint-robust tracking with spatiotemporal heat stress recognition. Its tracking module, CowTrack, maintains cross-camera identity consistency through a View-Decoupled Aggregation Network (VDA-Net). Instead of relying on static global representations, VDA-Net aggregates localized textures under heavy occlusion and uses a Viewpoint Decoupling Transformer (VDT) to suppress transient viewpoint variations. CowTrack further incorporates a causal online self-supervision mechanism for continuous adaptation to dynamic farm scenes. The resulting tracklets are subsequently processed by TSDF-Net, a fine-grained video understanding network. Its Temporal Stabilization and Aggregation Module (TSAM) suppresses high-frequency jitter, while a Diffusion Semantic Prior (DSP) uses frozen pre-trained diffusion semantics to enhance subtle behavioral signals and mitigate overfitting. Extensive evaluations on two real-world farm benchmarks demonstrate the framework's effectiveness: CowTrack achieves 83.48% HOTA and 87.72% IDF1 for global multi-object tracking on MVCow, and TSDF-Net achieves 87.72% Top-1 accuracy on HSCow. These results establish CowCare as a robust end-to-end framework for automated livestock health monitoring.

Figures & tables

Figure 1: Overview of the visual challenges and the comprehensive performance of our proposed methods. (a) Critical challenges in cattle tracking and heat-stress analysis, including appearance confusion under dense occlusion, identity fragmentation across non-overlapping views, and domain gaps in dynamic scenes. (b) Performance comparison of our CowTrack and VDA-Net against state-of-the-art MOT and ReID methods. (c) Performance comparison of our TSDF-Net against existing spatiotemporal video understanding models for fine-grained heat-stress recognition.

Figure 2: Equipment deployment and the actual farming environment for dataset collection.

Figure 3: Representative samples from the MVCow dataset illustrating herd occlusion, severe individual occlusion, and cross-view appearance distortion.

Figure 4: Multi-view images of the same individual cow, highlighting the visual variations across different camera angles.

Figure 5: The annotation workflow for the HSCow heat stress dataset, including multi-object tracking annotation, tracklet clipping, and expert evaluation.

Dataset	Farm Location	Area	Time	Videos	Purpose
MVCow	Yanqing, Beijing	Intersection of milking channel and parlor	Apr 2025	12 Clips	Train and evaluate object detection, ReID, and MOT models
HSCow	Shandong	Barn cooling zone	Jun-Sep 2025	6 Clips	Train and evaluate heat stress behavior recognition models

Table 1: Data collection information for the constructed benchmarks.

Task	Data Format	Split Method	Training Set	Testing Set
Object Detection	YOLO	Frame	7,128 Frames	3,056 Frames
Multi-Object Tracking	MOT17	Video Clip	—	12 Clips
Re-Identification	Market-1501	Identity	115 IDs / 54,230 Images	Query: 33 IDs / 103 Images Gallery: 33 IDs / 24,223 Images

Table 2: Task-specific data split configurations for the MVCow dataset.

Category	Number of Clips	Avg. Duration (s)
Normal	748	15.17
Heat Stress	383	13.78
Total	1131	14.70

Table 3: Statistical overview of the HSCow behavior dataset.

Figure 6: The overall architecture of the CowCare framework.

Figure 7: The overall architecture of the CowTrack framework.

Figure 8: Architecture and core components of the View-Decoupled Aggregation Network (VDA-Net). Top: The overall VDA-Net pipeline, illustrating the backbone integrated with Outlook attention layers and the Viewpoint Decoupling Transformer (VDT). Bottom left: The structure of the Outlook block, utilized for localized texture aggregation under severe occlusions. Bottom right: Detailed structure of the VDT block, showcasing the subtractive mechanism for isolating view-invariant identity embeddings.

Figure 9: Flowchart of the causal online self-supervised fine-tuning process for dynamic scene adaptation.

Figure 10: The overall architecture of TSDF-Net, highlighting the Temporal Stabilization and Aggregation Module (TSAM) and the Diffusion Semantic Prior (DSP).

Figure 11: Schematic of the Temporal Stabilization and Aggregation Module (TSAM) and the Diffusion Semantic Prior (DSP). TSAM performs temporal smoothing through 1D depthwise convolution and quality-aware gating, whereas DSP injects generalized motion semantics through a frozen diffusion encoder.

Method	Rank-1 $\uparrow$	Rank-5 $\uparrow$	mAP $\uparrow$
CLIP-ReID ( Li et al., 2023 )	97.36	99.84	82.45
DGNet ( Zheng et al., 2019 )	92.89	97.81	81.79
HOReID ( Wang et al., 2021 )	61.28	62.89	62.67
OSNet ( Zhou et al., 2019 )	98.90	99.90	99.60
PCB ( Sun et al., 2018 )	61.79	63.04	63.07
PersonViT ( Hu et al., 2025 )	94.90	99.10	93.20

Table 4: Cross-camera ReID performance on MVCow (%). Best and second-best results are shown in bold and underlined, respectively.

\uparrow

indicates higher values are better.

Method	HOTA $\uparrow$	MOTA $\uparrow$	IDF1 $\uparrow$
BoostTrack ( Stanojević and Todorović, 2024 )	77.49	82.76	81.40
ByteTrack ( Zhang et al., 2022b )	75.18	79.47	82.09
BoT-SORT ( Aharon et al., 2022 )	68.87	79.04	72.11
DeepSORT ( Wojke et al., 2017 )	75.07	80.03	81.30
Deep OC-SORT ( Maggiolino et al., 2023 )	75.29	80.00	81.58
Depth-OC-SORT ( Zhu et al., 2026 )	74.99	75.77	80.13

Table 5: Global multi-object tracking performance on MVCow (%). Best and second-best results are shown in bold and underlined, respectively.

\uparrow

indicates higher values are better.

Model	Top-1 $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
ActionDiff ( Guimarães et al., 2026 )	76.32	78.30	76.08	75.78
SlowOnly-R50 ( Feichtenhofer et al., 2019 )	78.07	82.36	78.39	77.44
SlowFast ( Feichtenhofer et al., 2019 )	70.18	81.11	70.69	67.62
TSN ( Wang et al., 2016 )	78.07	78.51	77.96	77.93
TSM ( Lin et al., 2019 )	76.32	76.97	76.45	76.23
TimeSformer ( Bertasius et al., 2021 )	70.18	70.55	70.04	69.94

Table 6: Heat stress behavior recognition performance on the HSCow dataset (%). The best results are shown in bold, while the second-best results are underlined.

\uparrow

indicates that higher values are better.

Configuration	Rank-1 $\uparrow$	Rank-5 $\uparrow$	Rank-10 $\uparrow$	mAP $\uparrow$
Baseline	96.03	98.72	99.05	96.69
$+$ Outlook	97.50	99.29	99.33	97.54
vs. previous	$\uparrow\,1.47\,\mathrm{pp}$	$\uparrow\,0.57\,\mathrm{pp}$	$\uparrow\,0.28\,\mathrm{pp}$	$\uparrow\,0.85\,\mathrm{pp}$
$+$ VDT	99.83	99.98	99.98	99.67
vs. previous	$\uparrow\,2.33\,\mathrm{pp}$	$\uparrow\,0.69\,\mathrm{pp}$	$\uparrow\,0.65\,\mathrm{pp}$	$\uparrow\,2.13\,\mathrm{pp}$

Table 7: Ablation of VDA-Net components on cross-camera ReID (%). Changes are absolute percentage-point differences from the preceding row.

Configuration	HOTA $\uparrow$	MOTA $\uparrow$	IDF1 $\uparrow$
Baseline	77.49	82.76	81.40
$+$ VDA-Net (Static)	81.67	89.50	85.55
vs. previous	$\uparrow\,4.18\,\mathrm{pp}$	$\uparrow\,6.74\,\mathrm{pp}$	$\uparrow\,4.15\,\mathrm{pp}$
$+$ Online Adaptation	83.48	89.50	87.72
vs. previous	$\uparrow\,1.81\,\mathrm{pp}$	$\rightarrow\,0.00\,\mathrm{pp}$	$\uparrow\,2.17\,\mathrm{pp}$

Table 8: Ablation of CowTrack components on global tracking (%). Changes are absolute percentage-point differences from the preceding row.

Configuration	Top-1 $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
Baseline	82.46	83.06	82.57	82.41
$+$ TSAM	83.30	87.33	83.62	82.95
vs. previous	$\uparrow\,0.84\,\mathrm{pp}$	$\uparrow\,4.27\,\mathrm{pp}$	$\uparrow\,1.05\,\mathrm{pp}$	$\uparrow\,0.54\,\mathrm{pp}$
$+$ DSP	87.72	89.35	87.90	87.62
vs. previous	$\uparrow\,4.42\,\mathrm{pp}$	$\uparrow\,2.02\,\mathrm{pp}$	$\uparrow\,4.28\,\mathrm{pp}$	$\uparrow\,4.67\,\mathrm{pp}$

Table 9: Ablation studies of TSDF-Net components (%). Changes are absolute percentage-point differences from the preceding row.

Setup	Rank-1 $\uparrow$	Rank-5 $\uparrow$	Rank-10 $\uparrow$	mAP $\uparrow$
Single-CAM	98.30	99.20	99.73	88.93
Multi-CAM	99.83	99.98	99.98	99.67
Gain (pp)	$\uparrow\,1.53$	$\uparrow\,0.78$	$\uparrow\,0.25$	$\uparrow\,10.74$

Table 10: Comparison of cross-camera ReID performance between single-camera and multi-camera setups (%).

Figure 12: Heatmap visualization of TSDF-Net performance sensitivity to temporal sampling parameters (input frame count $T$ and frame interval $\Delta t$ ) in terms of Top-1 accuracy (left) and Heat Stress F1 score (right).

Input Frames $T$	Frame Interval $\Delta t$	Top-1 $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$	Heat Stress F1 $\uparrow$	Heat Stress Recall $\uparrow$
4	1	88.60	88.68	88.64	88.60	88.70	91.07
4	8	89.47	89.63	89.53	89.47	89.66	92.86
6	1	86.84	88.17	87.01	86.76	87.80	96.43
6	8	87.72	88.83	87.87	87.66	88.52	96.43
8	1	86.84	88.74	87.04	86.72	88.00	98.21
8	8	87.72	89.35	87.90	87.62	88.71	98.21

Table 11: Sensitivity analysis of selected representative temporal sampling configurations, parameterized by the input frame count

T

and frame interval

\Delta t

, on TSDF-Net performance (%). The complete results for all 25 configurations are shown in Fig. 12 . Precision, Recall, and F1 are macro-averaged across both classes. Heat Stress F1 and Heat Stress Recall treat the heat stress category as the positive class. For

T=10

, all interval results are identical, hence only one representative row is listed.

Figure 13: Qualitative comparison of tracking performance under severe occlusion. The baseline method (top) suffers from an ID switch (from ID 43 to ID 23), whereas our CowTrack (bottom) successfully maintains the correct identity (ID 31).

Figure 14: t-SNE feature space visualization of hard ReID samples in the MVCow dataset. Compared to the baseline (left), our VDA-Net (right) yields tighter intra-class clustering and larger inter-class margins.

Figure 15: Comparison of feature space distributions using t-SNE. (a) The Video-FocalNet baseline; (b) Our TSDF-Net, which demonstrates clearer decision boundaries between normal and heat-stressed samples.

Figure 16: Cross-camera retrieval rankings produced by the baseline and VDA-Net in challenging cases involving occlusion and viewpoint changes.

Figure 17: Grad-CAM visualization of a heat stress video sequence constrained by the cow body ROI. The heatmaps demonstrate that our network correctly focuses on physiological indicators such as the flank undulations.

Figure 18: Attention heatmap comparison. Compared to the baseline (left), TSDF-Net (right) effectively concentrates on critical physiological areas (e.g., flank undulations) while avoiding background distractions.

Figure 19: Temporal sequence comparison between frame quality scores generated by TSAM (red) and optical flow magnitude (blue), illustrating the model’s ability to adaptively down-weight high-jitter frames.

Figure 20: Continuous heat stress prediction probability curve generated by the CowCare framework in a simulated real-world deployment queue.

Precision Livestock Farming

See all 12 papers

Oct 9, 2026

Deep Learning Approach for RealTime Contraction Detection in Laboring Dairy Cattle

Currently, the animal production sector is undergoing rapid development of techniques that allow automatic monitoring of animal behaviour and health. This paper presents a computer vision-based approach for the automatic recognition of labor contractions in cows. During the course of the study, videos depicting 100 cow births were collected. From the obtained videos, 8500 cow contraction actions and about 25 000 non-contraction sequences were extracted which constituted the dataset. Leveraging image processing techniques and deep learning algorithms like Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), our methodology extracts relevant features from video frames to detect and classify contractions. Our study demonstrates a high accuracy rate of over 95% in detecting labor contractions. This automated system has the potential to enhance productivity and animal welfare in the dairy industry by providing real-time monitoring and facilitating timely intervention during difficult calvings. The video demonstration of the system can be found at the following URL: https://www.youtube.com/watch?v=8fTSG5D7dZ0

Sebastian Puchala, Dominik Gronkiewicz, Marcin Gołębiewski +2

Promity, Warsaw, Poland · Institute of Animal Science, Warsaw University of Life Sciences, Warsaw, Poland · Faculty of Electronics and Information Technology, Warsaw University of Technology, Warsaw, Poland

Agricultural Image Analysis Video Action Recognition

Ask PDF

Oct 10, 2025

Cattle-CLIP: A Multimodal Framework for Dairy Cattle Behaviour Recognition from Video

Monitoring cattle behaviour provides valuable insights into animal health, productivity and welfare. However, robust behaviour recognition in real-world farm environments remains challenging because well-annotated livestock video datasets are scarce and agricultural surveillance footage differs substantially from large-scale pretraining data. To address these challenges, we propose Cattle-CLIP, a domain-adaptive vision-language framework based on Contrastive Language-Image Pretraining (CLIP), which reformulates cattle behaviour recognition as a cross-modal semantic alignment task. Cattle-CLIP incorporates a temporal integration module that extends image-level contrastive pretraining to video-based behaviour understanding and promotes consistent semantic alignment across time. To reduce domain discrepancy, customised data augmentation and behaviour-oriented prompts are employed. We also construct CattleBehaviours6, a curated dataset containing 1,905 annotated clips from six indoor behaviours: feeding, drinking, standing-self-grooming, standing-ruminating, lying-self-grooming and lying-ruminating. The dataset provides standardised ethogram definitions for model training, evaluation and future livestock behaviour research. Cattle-CLIP was evaluated under supervised and few-shot learning settings, with particular emphasis on data-constrained behaviour recognition. In supervised experiments, the model achieved 96.1% accuracy across six behaviours and showed high precision and recall for feeding, drinking and standing-ruminating. It also maintained promising few-shot performance with limited labelled data, demonstrating the potential of multimodal learning for cattle behaviour recognition in data-constrained farm environments.

Huimin Liu, Jing Gao, Daria Baran +3

Bristol Veterinary School, University of Bristol, Langford House, Dolberry, Churchill, Bristol, BS40 5DU, United Kingdom · School of Computer Science, University of Bristol, Merchant Venturers Building, Woodland Road, Bristol, BS8 1UB, United Kingdom

Animal Behavior Analysis Video-Language Models Contrastive Learning for VLMs

Ask PDF

Apr 5, 2026

A Physics-Informed, Behavior-Aware Digital Twin for Robust Multimodal Forecasting of Core Body Temperature in Precision Livestock Farming

Precision livestock farming requires accurate and timely heat stress prediction to ensure animal welfare and optimize farm management. This study presents a physics-informed digital twin (DT) framework combined with an uncertainty-aware, expert-weighted stacked ensemble for multimodal forecasting of Core Body Temperature (CBT) in dairy cattle. Using the high-frequency, heterogeneous MmCows dataset, the DT integrates an ordinary differential equation (ODE)-based thermoregulation model that simulates metabolic heat production and dissipation, a Gaussian process for capturing cow-specific deviations, a Kalman filter for aligning predictions with real-time sensor data, and a behavioral Markov chain that models activity-state transitions under varying environmental conditions. The DT outputs key physiological indicators, such as predicted CBT, heat stress probability, and behavioral state distributions are fused with raw sensor data and enriched through multi-scale temporal analysis and cross-modal feature engineering to form a comprehensive feature set. The predictive methodology is designed in a three-stage stacked ensemble, where stage 1 trains modality-specific LightGBM 'expert' models on distinct feature groups, stage 2 collects their predictions as meta-features, and at stage 3 Optuna-tuned LightGBM meta-model yields the final CBT forecast. Predictive uncertainty is quantified via bootstrapping and validated using Prediction Interval Coverage Probability (PICP). Ablation analysis confirms that incorporating DT-derived features and multimodal fusion substantially enhances performance. The proposed framework achieves a cross-validated R2 of 0.783, F1 score of 84.25% and PICP of 92.38% for 2-hour ahead forecasting, providing a robust, uncertainty-aware, and physically principled system for early heat stress detection and precision livestock management.

Riasad Alvi, Mohaimenul Azam Khan Raiaan, Sadia Sultana Chowa +6

Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh · Department of Data Science and Artificial Intelligence, Monash University, Melbourne 3800, Australia +5

Precision Livestock Farming Time Series Forecasting Digital Twins Uncertainty-Aware Multimodal Fusion Physics-Informed ML

Ask PDF

Abstract

Precision Livestock Farming

Explore similar work

Deep Learning Approach for RealTime Contraction Detection in Laboring Dairy Cattle

Cattle-CLIP: A Multimodal Framework for Dairy Cattle Behaviour Recognition from Video

A Physics-Informed, Behavior-Aware Digital Twin for Robust Multimodal Forecasting of Core Body Temperature in Precision Livestock Farming