Combining Object Detection with Geometry-Aware Clustering to Distinguish Overlapping Plants in UAV Imagery
Authors: Ik Jae Lee, Hieu D. Nguyen, Mahbubur Meenar, Carlos Morrison Martinez, Cameron Connelly
Organizations: Department of Mathematics, Rowan University · Department of Geography, Planning, and Sustainability, Rowan University · Department of Computer Science, Rowan University
Reliable plant-level information from unmanned aerial vehicle (UAV) imagery is important for automated crop monitoring. However, in dense crop canopies, adjacent plants frequently overlap and are detected as a single object, reducing the reliability of plant-level measurements. This study presents a geometry-aware post-detection framework for resolving overlapping plant instances using standard RGB UAV imagery. The framework combines object detection with geometric clustering of plant components. Leaves or branches detected within each bush-level region are represented using two complementary geometric features: component centroids and radial intersection points (RIPs) derived from detected plant structures. K-means and Gaussian mixture models determine whether a detected region contains a single plant or two overlapping plants. Density filtering suppresses spurious radial intersections, and a post-pipeline ensemble combines spatial and directional geometric information. The framework was evaluated using UAV imagery of eggplant and tomato crops under field conditions. Centroid-based clustering achieved an F1-score of 0.89 for eggplant, while the combined centroid-RIP approach achieved the best tomato performance, with an accuracy of 0.80, precision of 1.00, and F1-score of 0.75 using K-means. Density filtering substantially improved RIP-based clustering for tomato. The proposed approach provides a lightweight, modular engineering solution that can be integrated with existing RGB UAV monitoring pipelines without additional depth sensors, pixel-level segmentation, three-dimensional reconstruction, or retraining of the primary bush detector. The results demonstrate that geometric reasoning applied to existing detector outputs can complement deep-learning-based object detection and improve plant-level interpretation in dense agricultural canopies.
Figures & tables
Figure 1: Two adjacent tomato plants mis-detected as a single plant (red bounding box) due to canopy overlap.
Figure 2: Schematic illustration of approximately radial leaf or branch structure in eggplant and tomato plants. Illustrative images were generated using ChatGPT (OpenAI, March 2026) and reviewed by the authors for accuracy.
Vegetable
Number of Images
Number of Bush Annotations
Eggplant
10
111
Tomato
20
138
Table 1: Bush datasets used for training the object detection model.
Figure 3: Examples of bush-level bounding-box annotations for eggplant and tomato.
Vegetable
Number of Images
Number of Annotations
Eggplant
14
5336 (leaf)
Tomato
10
457 (branch)
Table 2: Leaf/branch datasets used for fine-scale component detection.
Figure 4: Examples of fine-grained plant-part annotations (leaf/branch).
Crop
Class
Tuning set
Validation set
Eggplant
1-in-1
30
10
2-in-1
30
10
Tomato
1-in-1
15
10
2-in-1
15
10
Table 3: Bush-level dataset split used for tuning and evaluation of the clustering pipeline.
Vegetable
Precision
Recall
mAP50
Eggplant
0.93728
0.93411
0.96104
Tomato
0.92753
0.8125
0.85404
Table 4: Bush model accuracy results.
Vegetable
Precision
Recall
mAP50
Eggplant (Leaf)
0.91406
0.60297
0.72498
Tomato (Branch)
0.69936
0.75
0.73945
Table 5: Leaf/branch model accuracy results.
Figure 5: Representative 2-in-1 misdetection cases from the bush detector. Each ROI contains two plants but is initially detected as a single bush box, motivating bush-level clustering for post-processing.
Figure 6: Centroid-based representation for a single bush ROI (yellow), showing leaf detections inside the ROI (blue boxes) and their computed centroids (red dots).
Figure 7: Bounding box misdetection of two overlapping plants.
Figure 8: Radial intersection points (RIP).
Figure 9: Leaf centroid versus radial intersection point (RIP) clustering using K-means.
Figure 10: RIP density thresholding.
Figure 11: Block diagram of the base bush-level clustering pipeline. The same steps are applied independently to each bush ROI using either centroid-based or RIP-based feature points, followed by clustering with K-means or GMM and silhouette-based classification into 1-in-1 or 2-in-1 cases.
Figure 12: Example of the RIP-based processing pipeline on real drone imagery. (a) Radial line construction from detected leaf boxes. (b) Generation of raw intersection points. (c) Filtering of intersection points based on ROI membership. (d) Clustering of retained points into two groups. Cluster centroids are indicated by × symbols.
Figure 13: Example of minimum-density filtering applied to intersection points. Raw intersection points are first restricted to the ROI and then aggregated into a density map, low-density cells are removed, and the remaining points are retained for downstream clustering.
Metric
K-means
GMM
Accuracy
0.90000
0.90000
Precision (2-in-1)
1.00000
1.00000
Recall (2-in-1)
0.80000
0.80000
F1-score (2-in-1)
0.88889
0.88889
Table 6: Eggplant centroid-based clustering results on the validation set without density filtering.
Figure 14: Representative eggplant centroid-based clustering examples using K-means on ground-truth 2-in-1 ROIs. The left panel shows a correctly resolved case, while the right panel shows a missed 2-in-1 case that is incorrectly classified as 1-in-1 because its silhouette score falls below the decision threshold. Cluster centroids are indicated by × .
Figure 15: Eggplant centroid-based borderline failure case. The left panel shows the actual centroid-based pipeline output for a bush ROI labeled as 2-in-1 but predicted as 1-in-1. The right panel shows the same ROI under a forced two-cluster fit ( k=2 ), included for diagnostic comparison. Cluster centroids are marked by × .
Figure 16: Representative eggplant true-negative example for centroid-based clustering using K-means. The ground-truth label is 1-in-1, and the centroid-based pipeline correctly predicts 1-in-1. Silhouette score: 0.361. Cluster centroid is indicated by × .
Metric
K-means
GMM
Accuracy
0.75000
0.75000
Precision (2-in-1)
1.00000
1.00000
Recall (2-in-1)
0.50000
0.50000
F1-score (2-in-1)
0.66667
0.66667
Table 7: Tomato centroid-based clustering results on the validation set without density filtering.
Figure 17: Representative tomato centroid-based clustering examples using K-means on ground-truth 2-in-1 ROIs. The left panel shows a correctly resolved case, while the right panel shows a missed 2-in-1 case that is incorrectly classified as 1-in-1 because its silhouette score falls below the decision threshold. Cluster centroids are indicated by × .
Metric
K-means
GMM
Accuracy
0.80000
0.75000
Precision (2-in-1)
0.87500
0.85714
Recall (2-in-1)
0.70000
0.60000
F1-score (2-in-1)
0.77778
0.70588
Table 8: Eggplant intersection-based clustering results on the validation set using radial intersection points (RIP) without density thresholding.
Figure 18: Representative eggplant RIP-based clustering examples without density filtering using K-means on ground-truth 2-in-1 ROIs. The left panel shows a correctly resolved case, while the right panel shows a missed 2-in-1 case that is incorrectly classified as 1-in-1 because its silhouette score falls below the decision threshold. Cluster centroids are indicated by × .
Metric
K-means
GMM
Accuracy
0.70000
0.65000
Precision (2-in-1)
0.66667
0.66667
Recall (2-in-1)
0.80000
0.60000
F1-score (2-in-1)
0.72727
0.63158
Table 9: Eggplant intersection-based clustering results on the validation set using RIP with density thresholding.
Figure 20: Representative eggplant RIP-based examples obtained with minimum-density filtering and K-means for ground-truth 2-in-1 ROIs. In each row, the left panel shows the raw intersection points and the right panel shows the final clustering result. The top row shows a correctly resolved case, while the bottom row shows a missed 2-in-1 case. Cluster centroids are indicated by × .
Figure 21: Representative eggplant false-positive RIP result after minimum-density filtering using K-means. The left panel shows the raw intersection points, and the right panel shows the final filtered clustering output. Although the ROI is labeled as 1-in-1, the filtered RIP pipeline predicts 2-in-1. Cluster centroids are indicated by × .
Metric
K-means
GMM
Accuracy
0.55000
0.55000
Precision (2-in-1)
0.66667
1.00000
Recall (2-in-1)
0.20000
0.10000
F1-score (2-in-1)
0.30769
0.18182
Table 10: Tomato intersection-based clustering results on the validation set using radial intersection points (RIP) without density thresholding.
Metric
K-means
GMM
Accuracy
0.70000
0.35000
Precision (2-in-1)
0.70000
0.38462
Recall (2-in-1)
0.70000
0.50000
F1-score (2-in-1)
0.70000
0.43478
Table 11: Tomato intersection-based clustering results on the validation set using RIP with density thresholding.
Crop
Grid
K-means F1
GMM F1
25 × 25
0.78
0.70
Eggplant
30 × 30
0.72
0.57
35 × 35
0.70
0.625
20 × 20
0.625
0.67
Tomato
25 × 25
0.78
0.57
30 × 30
0.70
0.43
Table 12: Sensitivity of RIP-based clustering performance to grid resolution in the minimum-density filtering stage.
Metric
K-means
GMM
Accuracy
0.80000
0.80000
Precision (2-in-1)
0.80000
0.80000
Recall (2-in-1)
0.80000
0.80000
F1-score (2-in-1)
0.80000
0.80000
Table 13: Eggplant combined centroid-based and intersection-based clustering results on the validation set with density filtering.
Metric
K-means
GMM
Accuracy
0.80000
0.40000
Precision (2-in-1)
1.00000
0.40000
Recall (2-in-1)
0.60000
0.40000
F1-score (2-in-1)
0.75000
0.40000
Table 14: Tomato combined centroid-based and intersection-based clustering results on the validation set with density filtering.
Multimodal 3D object detection based on LiDAR and cameras has demonstrated excellent performance in ground-vehicle scenarios, but has not been explored for Unmanned Aerial Vehicle (UAV) platforms. In UAV top-down scenes, frequent groundobject occlusion dominated by tree canopies causes spatially varying and modality-dependent information degradation. Existing multimodal fusion frameworks neither explicitly model such ground-object occlusion nor embed occlusion awareness into the detection pipeline, limiting their performance in occluded UAV scenes. To address these challenges, we propose CAMF-Det, a closure-aware multimodal fusion framework for LiDAR-camera 3D object detection on UAV platforms, which derives dual-modal occlusion intensity through physics-inspired modeling and embeds them as priors throughout the detection pipeline. First, a dual-modal closure modeling module explicitly constructs occlusion intensity ground truth for both modalities offline via a Beer-Lambert-inspired formulation and building-mask correction. Second, using these ground-truth maps as supervision, a dual-modal prediction network converts the offline modeling results into online occlusion intensity predictions under single-frame inference. Third, both ground-truth and predicted occlusion intensity are injected into data augmentation, feature encoding, multimodal fusion, and detection head, enabling adaptive detection under spatially varying and modality-dependent information degradation. Experiments on two self-built UAV-based multimodal datasets, SI3D-DI and SI3D-DII, demonstrate that CAMF-Det achieves the best performance across all difficulty levels, with hard-level mAPBEV improvements of 9.43% and 4.88% over the best competing methods, respectively. These results confirm the effectiveness of explicit occlusion prior modeling and exploitation for robust multimodal 3D detection in UAV scenes.
Yanze Jiang, Yanfeng Gu, Xian Li
School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China
Accurate 3D crop monitoring underpins data-driven precision agriculture by enabling field-scale analysis of plant structure, growth dynamics, and management response. Modern 3D reconstruction methods perform strongly on generic benchmarks, but rendered appearance may not translate into metrically and agronomically useful geometry in crop fields. We introduce UAV3DCrop, a public benchmark of repeated multi-angle unmanned aerial vehicle (UAV) crop surveys. It contains 88,830 RGB images at 5280×3956 pixels, with a ground sampling distance of 3.6-5.8 mm, from 91 scenes spanning corn, soybean, wheat, and oat. Track A evaluates seven scene-optimized methods -- Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) variants -- on held-out views, photogrammetry-referenced depth, and canopy-height recovery. Track B tests four pretrained feed-forward models on zero-shot camera-pose and geometry estimation. The scene-optimized methods rank differently across the three targets: Splatfacto-big leads appearance, whereas Scaffold-GS leads depth and is statistically tied with Splatfacto for canopy height. Among feed-forward models, MapAnything leads on seven of the eight metrics, while the remaining models vary more across crops and fail severely on absolute scale in a way that alignment conceals. Repeated acquisitions reveal further sensitivities that differ by output type and by model, associated with position within the acquisition sequence and with tie-point multiplicity. Current 3D reconstruction methods are therefore not yet interchangeable for agronomic use: no single method wins on appearance, geometry, and canopy height at once, and only one of four feed-forward models recovers usable metric scale. The dataset is publicly available at https://link-dev.github.io/UAV3DCrop/
Junxiong Zhou, Xuechen Li, Chonghao Qiu +11
University of Minnesota, Twin Cities, USA · University of Pittsburgh, USA · Max Planck Institute for Biogeochemistry, Germany +3
Reliable detection of humans beneath forest canopy remains a difficult remote-sensing challenge due to sparse, structured, and viewpoint-dependent occlusion. This paper presents a multimodal proof-of-concept pipeline that integrates three complementary approaches: (i) experimental evaluation of LiDAR returns through vegetation to assess the feasibility of active sensing, (ii) visible--thermal image fusion using a multi-scale transform and sparse-representation framework to enhance human saliency, and (iii) synthetic-aperture image formation via Airborne Optical Sectioning (AOS) to suppress canopy clutter. A YOLOv5 detector is fine-tuned on the Teledyne FLIR thermal dataset and evaluated on thermal and fused imagery. Results show that the tested terrestrial LiDAR configuration provides limited penetration for object-level detection, while visible--thermal fusion improves target visibility in low-contrast scenes and AOS enhances ground-plane detection in synthetic forest imagery. The fine-tuned YOLOv5 achieves a mean average precision of ∼0.83 on the top three FLIR classes. These findings establish an initial baseline for UAV-deployable search-and-rescue and surveillance systems operating in forested environments, and motivate future work on dedicated forest datasets and real-time multimodal integration.
Nitik Jain, Mangal Kothari
Robotics & AI, Johns Hopkins University, USA. This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur. · This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur. · Senior Principal Flight Control Engineer, ADASI, EDGE Group, Abu Dhabi, UAE. This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur.