Saliency-Depth Conditioning for Zero-Shot Segmentation of Communication-Tower Components in Cluttered UAV Imagery
Authors: Ali Lesani, Chul Min Yeum, Su-Min Kang
Organizations: Computer Vision for Smart Structures (CViSS) Lab, Waterloo, Canada · University of Waterloo, Waterloo, Canada · School of Architecture, Soongsil University, Seoul, South Korea
Fine-grained segmentation of communication-tower components in UAV imagery is essential for automated inspection, yet task-specific models are hard to develop due to limited instance-level annotations. Zero-shot segmentation models offer a promising alternative, but in cluttered scenes, visually similar background structures interfere with component localization, causing missed instances and false positives. We propose a model-agnostic saliency-depth foreground-conditioning strategy combining appearance-based saliency with monocular relative depth to construct a coarse tower prior and suppress irrelevant content. We integrate this module with Grounded-SAM and SAM 3, yielding SD-Grounded-SAM and SD-SAM 3. SD-Grounded-SAM further applies geometric and depth-aware box refinement before mask generation, while SD-SAM 3 relies on SAM 3's internal setup. On TOW-300, a dataset of 340 communication-tower UAV images, our strategy improves both baselines: SD-SAM 3 achieves the strongest instance-segmentation performance, while SD-Grounded-SAM produces fewer false positives. Ablations confirm complementary gains from saliency, depth, and box refinement, improving robustness in cluttered scenes.
Figures & tables
Figure 1: Overview of the proposed saliency–depth conditioning framework. Given an input UAV image, saliency estimation and monocular depth prediction are combined to construct a refined foreground mask and a conditioned image with reduced background interference. The conditioned image is then processed by two zero-shot segmentation frameworks. In SD-Grounded-SAM, Grounding DINO generates candidate boxes that are refined using tower-relative geometry, pairwise containment, and depth cues before being passed to SAM. In SD-SAM 3, the conditioned image and text prompt are provided directly to SAM 3, which performs localization and segmentation internally.
Figure 2: Representative instance-level annotations from TOW-300. All component instances are shown using the same overlay color (pink) for improved visibility and visual clarity.
Figure 3: Representative challenges in TOW-300. (a) Background structures remain visible through the sparse tower lattice and exhibit colors and appearances similar to tower-mounted components. (b) A small background structure resembles the communication tower, creating ambiguity for coarse text prompts such as antenna . (c) The tower occupies only a small portion of the full high-resolution image and is surrounded by substantial background clutter. (d) A near top-down view exposes rooftop objects through the sparse tower structure, some of which visually resemble mounted communication equipment.
Table 1: Instance segmentation performance on the TOW-300 test set. Mean matched IoU is computed over true-positive instance matches at an IoU threshold of 0.5. Best results are shown in bold, and second-best results are underlined.
Method
IoU ( ↑ )
Dice ( ↑ )
Grounded-SAM
0.2876
0.4467
SAM 3
0.3630
0.5326
SD-Grounded-SAM (Ours)
0.6281
0.7716
SD-SAM 3 (Ours)
0.4376
0.6088
Table 2: Semantic segmentation results obtained by merging all predicted instance masks into a binary foreground mask. Best results are shown in bold, and second-best results are underlined.
Figure 4: Qualitative comparison of Grounded-SAM, SAM 3, and their saliency–depth-conditioned variants. Each row presents a representative UAV image, its ground-truth annotation, and the corresponding predictions.
Figure 5: Representative failure cases. In the first and second rows, SD-SAM 3 detects several valid components but also produces an oversized tower-level mask, whereas SD-Grounded-SAM suppresses this prediction through box refinement. The third and fourth rows show near top-down views in which foreground conditioning retains visually similar background regions, leading to localization errors.
Table 3: Ablation study of saliency–depth conditioning and box refinement across the Grounded-SAM and SAM 3 frameworks. Best and second-best results are identified separately within each framework and are shown in bold and underlined, respectively.