BeeWhere: Segmenting Bumble Bee Colonies to Quantify Behavioral Effects
Authors: Roberta Hunt, August Easton-Calabria, James Crall
Organizations: University of Copenhagen, 2100 Copenhagen, Denmark · Lund University, 223 62 Lund, Sweden · University of Wisconsin-Madison, Madison, WI 53706, USA
Social bees are important pollinators that support biodiversity and crop pollination globally and serve as important model systems for collective behavior, but scalable measurement of individual- and colony-level behavior remains difficult in dense, occluded nest environments. Existing monitoring workflows use fiducial tags (e.g., ArUco) to preserve individual identity, yet tag-based tracking can fail when markers are obscured and provide limited information about body extent, spatial context, and untagged individuals. We present BeeWhere, an AI-assisted annotation and analysis workflow that combines ArUco detections with deep-learnt instance segmentations to quantify bumble bee behavior from high-resolution colony images and videos. Using bumble bee (Bombus impatiens) microcolonies as a test case, we annotate 483 frames containing 8,443 bee instances. We additionally annotate pollen balls, nest structures, and chamber boundaries, and train YOLO instance segmentation models for downstream behavioral analysis. Instance segmentations enable quantification of important behavioral metrics based on body contours, including nearest-neighbor distance, proximity to nest structures, spatial occupancy within the nest, and detection counts over time. We apply the BeeWhere models to tag-based tracking in an exploratory validation study assessing the behavioral impacts of neonicotinoid pesticide exposure. BeeWhere increased detection rates compared to tag-based tracking, particularly when bees were partially obscured or under challenging imaging conditions, and also captured treatment-associated changes in bee spatial organization not captured using tag-based tracking alone. These results suggest that instance segmentation can complement fiducial-marker tracking by recovering behaviorally meaningful signals under challenging colony conditions.
Figures & tables
Figure 1: Overview of BeeWhere pipeline. (a) Bumblebox microcolonies of Bombus impatiens are imaged under control and imidacloprid treatment conditions. (b) An AI-assisted annotation workflow produces training data for bees, nest structures, pollen balls, and chambers using SAM2 mask proposals and iterative YOLO retraining. (c) Trained models generate instance segmentations that can be linked across time with tracking and ArUco identity information. (d) Segmentation and tracking outputs are converted into behavioral proxies to compare treatments.
Class
No. Videos
No. Frames
No. Bbox
No. Seg
Train
Val
Train
Val
Train
Val
Train
Val
Bee
151
73
321
162
5,237
3,206
4,252
692
Pollen Ball
69
12
-
-
256
38
256
38
Nest Structure
89
28
-
-
290
115
286
115
Chamber
55
17
-
-
98
33
97
33
Table 1: Training and Validation dataset statistics. Note that some frames have all class annotations and others do not. Pollen Balls, Nest Structures and Chambers are annotated per video since they are mostly static over the course of the 10s video.
Figure 2: Number of YOLO detections per frame for each trial, shown with a rolling average. Control R1 showed an increase in population over the experiment.
Figure 3: Distributions of estimated median distance to nearest bee per frame using (a) ArUco markers and (b) instance segmentations. n_frames is the number of frames with detections included in the calculation. It should be noted that the R1 trial does not have any ArUco detections as the videos were too blurry to identify any, but bees were detected with the instance segmentation model. The mismatch in frame numbers between the two approaches for control is due to skipped frames which encountered errors during segmentation inference.
Model
Val AP0.5
Val AP0.5:0.95
No. Epochs
YOLO-BBox-Bee
0.9532
0.75067
323
YOLO-BBox-Bee + SAM2 1 1 1 These authors contributed equally to this work.
0.9343
0.4428
–
YOLO-BBox-Bee + YOLO-Bee-Fine
0.9099
0.6466
490
YOLO-Seg-Bee
0.97739
0.75688
240
YOLO-Seg-PollenBall
0.93417
0.7432
221
YOLO-Seg-Nest
0.79717
0.55338
360
Table 2: Average precision (AP) results per model, and number of epochs run until early stopping. The results of the bounding box and fine grained bee model are shown for completeness but used only during annotation. The segmentation (‘Seg’) models are used for inference and further results are based on these models.
Figure 4: Spatial bee detections for each trial from ArUco and YOLO-Seg-Bee. Average chamber, nest structure, and pollen ball contours from 100 random frames are shown. Note that chambers moved slightly over time. This can bee seen in the ’Avg’ column. Round 1’s imidacloprid treatment is missing ArUco detections completely because the images were blurred. We can see from the ’Last’ column that often nest structures and pollen balls are not entirely captured, partly due to pollen ball movement.
Prediction Source
Tracking Method
MOTA (%)
IDF1 (%)
ID Switches
Ground Truth
ByteTrack
89.7
89.7
322
SimpleIOU
93.7
93.7
196
Centroid - 200
99.0
99.0
34
YOLO-BBox-Bee
ByteTrack
84.7
88.3
233
SimpleIOU
87.9
91.5
138
Centroid - 200
90.9
91.5
44
Table 3: Tracking performance for segmentation and bounding box models measured using Multiple Object Tracking Accuracy (MOTA) and Identity F1 Score (IDF1). Best results are bolded.
Method
Detections
ArUco Identified Detections
FPs
FNs
ArUco
915
915
-
-
Yolo-Seg-Bee
2378
0
-
-
ArUco + Yolo-Seg-Bee
2378
1110
4
32
Ground truth
2441
1142
-
-
Table 4: Comparison of the number of bees detected using the Segmentation model only; the identified ArUco tags only; and their combination on the validation tracking dataset. FPs (False Positives) are wrong identifications due to ID switches. FNs (False Negatives) are identifications we failed to pick up, but could have with perfect tracking and identification. Results generated using Centroid - 200 tracking algorithm.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Overview of the staged annotation pipeline. Arrows going both ways represent an iterative process of train-predict-correct-repeat. Icons next to each step represent which classes were included in the process. Since the bees are the hardest to predict and also have small structures (antennae and legs) which are important for social behavior, and therefore the most time was spent annotating them, therefore they have extra steps (training Yolo-Bee-Fine) to reduce the human effort involved in the annotation process.
No. Videos
No. Frames
Bounding Boxes
Segmentations
Data Source
Year
Train
Val
Train
Val
Train
Val
A2019
2019
33
4
54
4
54
4
B2021
2021
42
7
57
10
36
4
C2022
2022
12
5
32
4
24
1
D2024
2024
37
9
111
14
108
5
Appendix
Table 5: Object Detection: Data sources for training and validation data. Each row is a different experiment run on a bumblebee microcolony. Note that the extents of the segmentations also count as bounding boxes, hence the number of bounding boxes always includes the segmentations.
No. Videos
No. Frames
Bounding Boxes
Segmentations
Data Source
Year
Val
Val
Val
A2019
2019
2
22
22
B2021
2021
1
10
0
C2022
2022
0
0
0
D2024
2024
1
11
11
Appendix
Table 6: Object Tracking: Data sources for training and validation data.
Figure 6: Example frames from training and validation data sources showing a variety of chamber and lighting arrangements.
Figure 7: Distributions of estimated distance to the nest across treatments, predicted using segmentations
The CVPPA@ECCV 2026 BuzzSpot Challenge asks us to detect bees, bumblebees, hoverflies, and moths in 1920x1080 field keyframes. Its annotations carry 2 difficulties: the median box occupies 0.16% of a frame, and bees account for 80% of the labels. To cope with the small boxes, we compare 10 recorded detector configurations on held-out keyframes; plain Co-DINO with a Swin-L backbone has the highest mAP in this comparison, so we select it. Training then addresses the bee dominance in 2 ways: fine-tuning on a crop-mosaic pool in which the combined annotation share of the 3 rare classes rises from 19.9% to 55.1%, and a class-weighted simplex equiangular tight frame (ETF) loss that pulls the projected states of matched decoder queries toward fixed class directions. The full schedule spans 12+3+2 epochs. Without inference-time ensembling or test-time augmentation, we rank first on FinalTest at 0.5062 mAP@[.5:.95].
This work presents an automatic bee entrance monitoring system based on YOLO11 transfer learning and the ByteTrack tracking algorithm. The study investigates the influence of data augmentation, backbone freezing, and tracker parameter optimization on the detection and counting of small, fast-moving bees. The detector with progressive backbone unfreezing strategy achieved about 97.0% precision and 98.7% mAP50, while providing more stable convergence than full fine-tuning. Experiments also showed that light augmentation outperformed heavy augmentation. For tracking, ByteTrack parameters were optimized to improve trajectory continuity under low-confidence detections. On an independent 25 FPS side-view video, the optimized YOLO11-ByteTrack system correctly counted 43 of 47 incoming bees (91.5%) and 7 of 30 outgoing bees (23.3%). Error analysis showed that most counting errors were caused by missed detections due to rapid bee motion and motion blur, while tracking failures became less frequent after parameter optimization. Overall, the results indicate that moderate augmentation, progressive backbone unfreezing, and ByteTrack tuning improve the reliability of automatic bee entrance monitoring under realistic recording conditions.
Thi Thu Thao Nguyen, Johannes Reschke
IoT Engineering Savonia University of Applied Sciences Finland · Electrical and Information Engineering Ostbayerische Technische Hochschule Regensburg Germany
Detecting pollinators in field video is challenging: targets are small, visually similar, and observed against cluttered vegetation under blur and occlusion. We present a systematic empirical study of small-pollinator detection under a practical single-GPU compute budget. Using the BuzzSpot challenge dataset, we compare YOLO and RF-DETR models across input resolutions and evaluate sliced inference, class-gated fusion, size-routed ensembling, and post-hoc temporal processing. RF-DETR Large at 1344-pixel resolution achieved our best hidden-test result, reaching 0.405 mAP50:95 and outperforming the 1120-pixel model (0.379) and the best single-model YOLO26m baseline (0.366). The strongest gains came from adopting RF-DETR and increasing its input resolution, indicating that detector choice and input resolution were more effective levers than added inference-time complexity; the resolution gain was strongest for small objects and the rarer bumblebee and moth classes. Sliced-inference fusion, size-routed ensembling, and warm-started 1536-pixel continuation did not surpass this result, while post-hoc temporal processing did not improve the leaked diagnostic evaluation. Error analysis identified bee-hoverfly discrimination as the clearest remaining bottleneck: neighboring frames rarely supplied correctly classified hoverfly evidence for post-hoc correction. These findings motivate learned feature-level temporal aggregation before the final classification decision.
Onur Onal, Chen Chen
Iowa State University · Iowa State University, Ames, IA, USA · Institute of AI, University of Central Florida +1