Multimodal Urban Tree Detection from Satellite and Street-Level Imagery via Annotation-Efficient Deep Learning Strategies
Authors: In Seon Kim, Ali Moghimi
Organizations: Department of Computer Science, University of California, Davis, , Davis, 95616, CA, USA · Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, 95616, USA
Beyond the immediate biophysical benefits, urban trees play a foundational role in environmental sustainability and disaster mitigation. Precise mapping of urban trees is essential for environmental monitoring, post-disaster assessment, and strengthening policy. However, the transition from traditional, labor-intensive field surveys to scalable automated systems remains limited by high annotation costs and poor generalization across diverse urban scenarios. This study introduces a multimodal framework that integrates high-resolution satellite imagery with ground-level Google Street View to enable scalable and detailed urban tree detection under limited-annotation conditions. The framework first leverages satellite imagery to localize tree candidates and then retrieves targeted ground-level views for detailed detection, significantly reducing inefficient street-level sampling. To address the annotation bottleneck, domain adaptation is used to transfer knowledge from an existing annotated dataset to a new region of interest. To further minimize human effort, we evaluated three learning strategies: semi-supervised learning, active learning, and a hybrid approach combining both, using a transformer-based detection model. The hybrid strategy achieved the best performance with an F1-score of 0.90, representing a 12% improvement over the baseline model. In contrast, semi-supervised learning exhibited progressive performance degradation due to confirmation bias in pseudo-labeling, while active learning steadily improved results through targeted human intervention to label uncertain or incorrect predictions. Error analysis further showed that active and hybrid strategies reduced both false positives and false negatives. Our findings highlight the importance of a multimodal approach and guided annotation for scalable, annotation-efficient urban tree mapping to strengthen sustainable city planning.
Figures & tables
Platform
Source/Target domain
Train Images
Validation Images
Test Images
Total Images
Satellite Imagery
Open-source Satellite (source)
154
44
0
198
Google Satellite
176
100
500
776
In total
330
144
500
974
Ground-Level Imagery
Chinese urban tree dataset (source)
3,509
440
440
4,389
GSV Images
254
1,780
1,712
3,746
In total
3,763
2,220
2,152
8,135
Table 1: Dataset partitioning for satellite and ground-level imagery across source and target domains used for model training, validation, and testing.
Figure 1: Illustration of the bearing angle calculation. The bearing angle θ from the panorama vehicle position (pe,pn) to the tree target (te,tn) is computed in UTM coordinates, and passed as the bearing parameter to the Street View Static API to center the camera on the target tree.
Model
Precision
Recall
Best F1
mAP50
YOLOv11
0.85
0.67
0.75
0.79
RF-DETR
0.86
0.75
0.80
0.85
Table 2: Model performance comparison for ground-level urban tree detection.
Figure 2: Flowcharts for the three annotation-efficient learning strategies. (A) Flowchart for the semi-supervised pipeline. The model is trained on the labeled train/validation dataset and deployed on an unlabeled pool. Detections with confidence above 0.8 are automatically accepted as pseudo-labels and merged with the original dataset for retraining, while lower-confidence predictions are placed back in the unlabeled pool. (B) Active learning and hybrid learning strategies. In active learning, samples with prediction confidence below 0.5 are selected for human annotation. In the hybrid approach, predictions with confidence above 0.8 are automatically accepted as pseudo-labels (semi-supervised learning), while samples with confidence below 0.5 are assigned for human annotation (active learning) before merging and deduplication.
Figure 3: Precision, Recall, and F1-Score curve of the satellite model canopy detection. The best F1-score of 0.78 is achieved at a confidence threshold of 0.41. Lower thresholds increase recall at the cost of lower precision, while higher thresholds improve precision but reduce recall.
Model
Precision
Recall
F1-Score
Satellite Model
0.81
0.75
0.78
Semi-Supervised (Avg)
0.85
0.70
0.77
Active Learning (Avg)
0.89
0.84
0.86
Hybrid Learning (Avg)
0.90
0.84
0.87
Table 3: Performance comparison of learning strategies on ground-level urban tree detection.
Figure 4: Performance comparison of three learning strategies over 10 rounds: Semi-Supervised learning (SS), Active Learning (AL), and hybrid AL + Semi-Supervised learning. (A) Precision trajectories across training rounds. (B) Recall trajectories across training rounds. (C) F1 score trajectories with peak performance markers (stars) for each method.
Learning
Initial TP
Final TP
Initial FN
Final FN
Initial FP
Final FP
Strategy
(Round 1)
(%)
(Round 1)
(%)
(Round 1)
(%)
SS
2,836 (out of 3,791)
2,538 ( − 10.5%)
955
1,253 ( + 31.2%)
469
369 ( − 21.3%)
AL
3,295 ( + 16.2%)
496 ( − 48.1%)
316 ( − 32.6%)
Hybrid
3,327 ( + 17.3%)
464 ( − 51.4%)
303 ( − 35.4%)
Table 4: Quantitative analysis of detection counts across learning strategies using true positives (TP), false positives (FP), and false negatives (FN). Percentages shown in parentheses indicate relative change from the initial round.
Figure 5: Error analysis and performance metric dynamics across the three learning strategies over iterative training rounds, including true positives (A), false positives (B), and false negatives (C).
Figure 6: Error analysis of the tree detection model. (A) Multiple overlapping bounding boxes for a single tree instance due to the complex branching structure and occlusion. (B) Model detecting trees accurately, but inaccurate human annotation of ambiguous cases leads to false positives. (C) An urban scene with a modern building featuring a large reflective glass window; a nearby tree is correctly detected, while the model additionally identified the tree’s reflection in the building’s windows as a separate tree instance, resulting in false positive detections driven by specular reflectance.
Figure 7: Distribution of the number of pseudo-labeled and human-annotated samples added to the training dataset per round for different learning strategies. (A) Semi-supervised vs. hybrid learning: a decline in pseudo-label generation for the semi-supervised approach. (B) Active learning vs. hybrid learning: active learning requires substantially higher human annotation effort, while the hybrid approach achieves comparable dataset growth with fewer human annotations.