Multimodal Urban Tree Detection from Satellite and Street-Level Imagery via Annotation-Efficient Deep Learning Strategies
Authors: In Seon Kim, Ali Moghimi
Organizations: Department of Computer Science, University of California, Davis, , Davis, 95616, CA, USA · Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, 95616, USA
Beyond the immediate biophysical benefits, urban trees play a foundational role in environmental sustainability and disaster mitigation. Precise mapping of urban trees is essential for environmental monitoring, post-disaster assessment, and strengthening policy. However, the transition from traditional, labor-intensive field surveys to scalable automated systems remains limited by high annotation costs and poor generalization across diverse urban scenarios. This study introduces a multimodal framework that integrates high-resolution satellite imagery with ground-level Google Street View to enable scalable and detailed urban tree detection under limited-annotation conditions. The framework first leverages satellite imagery to localize tree candidates and then retrieves targeted ground-level views for detailed detection, significantly reducing inefficient street-level sampling. To address the annotation bottleneck, domain adaptation is used to transfer knowledge from an existing annotated dataset to a new region of interest. To further minimize human effort, we evaluated three learning strategies: semi-supervised learning, active learning, and a hybrid approach combining both, using a transformer-based detection model. The hybrid strategy achieved the best performance with an F1-score of 0.90, representing a 12% improvement over the baseline model. In contrast, semi-supervised learning exhibited progressive performance degradation due to confirmation bias in pseudo-labeling, while active learning steadily improved results through targeted human intervention to label uncertain or incorrect predictions. Error analysis further showed that active and hybrid strategies reduced both false positives and false negatives. Our findings highlight the importance of a multimodal approach and guided annotation for scalable, annotation-efficient urban tree mapping to strengthen sustainable city planning.
Figures & tables
Platform
Source/Target domain
Train Images
Validation Images
Test Images
Total Images
Satellite Imagery
Open-source Satellite (source)
154
44
0
198
Google Satellite
176
100
500
776
In total
330
144
500
974
Ground-Level Imagery
Chinese urban tree dataset (source)
3,509
440
440
4,389
GSV Images
254
1,780
1,712
3,746
In total
3,763
2,220
2,152
8,135
Table 1: Dataset partitioning for satellite and ground-level imagery across source and target domains used for model training, validation, and testing.
Figure 1: Illustration of the bearing angle calculation. The bearing angle θ from the panorama vehicle position (pe,pn) to the tree target (te,tn) is computed in UTM coordinates, and passed as the bearing parameter to the Street View Static API to center the camera on the target tree.
Model
Precision
Recall
Best F1
mAP50
YOLOv11
0.85
0.67
0.75
0.79
RF-DETR
0.86
0.75
0.80
0.85
Table 2: Model performance comparison for ground-level urban tree detection.
Figure 2: Flowcharts for the three annotation-efficient learning strategies. (A) Flowchart for the semi-supervised pipeline. The model is trained on the labeled train/validation dataset and deployed on an unlabeled pool. Detections with confidence above 0.8 are automatically accepted as pseudo-labels and merged with the original dataset for retraining, while lower-confidence predictions are placed back in the unlabeled pool. (B) Active learning and hybrid learning strategies. In active learning, samples with prediction confidence below 0.5 are selected for human annotation. In the hybrid approach, predictions with confidence above 0.8 are automatically accepted as pseudo-labels (semi-supervised learning), while samples with confidence below 0.5 are assigned for human annotation (active learning) before merging and deduplication.
Figure 3: Precision, Recall, and F1-Score curve of the satellite model canopy detection. The best F1-score of 0.78 is achieved at a confidence threshold of 0.41. Lower thresholds increase recall at the cost of lower precision, while higher thresholds improve precision but reduce recall.
Model
Precision
Recall
F1-Score
Satellite Model
0.81
0.75
0.78
Semi-Supervised (Avg)
0.85
0.70
0.77
Active Learning (Avg)
0.89
0.84
0.86
Hybrid Learning (Avg)
0.90
0.84
0.87
Table 3: Performance comparison of learning strategies on ground-level urban tree detection.
Figure 4: Performance comparison of three learning strategies over 10 rounds: Semi-Supervised learning (SS), Active Learning (AL), and hybrid AL + Semi-Supervised learning. (A) Precision trajectories across training rounds. (B) Recall trajectories across training rounds. (C) F1 score trajectories with peak performance markers (stars) for each method.
Learning
Initial TP
Final TP
Initial FN
Final FN
Initial FP
Final FP
Strategy
(Round 1)
(%)
(Round 1)
(%)
(Round 1)
(%)
SS
2,836 (out of 3,791)
2,538 ( − 10.5%)
955
1,253 ( + 31.2%)
469
369 ( − 21.3%)
AL
3,295 ( + 16.2%)
496 ( − 48.1%)
316 ( − 32.6%)
Hybrid
3,327 ( + 17.3%)
464 ( − 51.4%)
303 ( − 35.4%)
Table 4: Quantitative analysis of detection counts across learning strategies using true positives (TP), false positives (FP), and false negatives (FN). Percentages shown in parentheses indicate relative change from the initial round.
Figure 5: Error analysis and performance metric dynamics across the three learning strategies over iterative training rounds, including true positives (A), false positives (B), and false negatives (C).
Figure 6: Error analysis of the tree detection model. (A) Multiple overlapping bounding boxes for a single tree instance due to the complex branching structure and occlusion. (B) Model detecting trees accurately, but inaccurate human annotation of ambiguous cases leads to false positives. (C) An urban scene with a modern building featuring a large reflective glass window; a nearby tree is correctly detected, while the model additionally identified the tree’s reflection in the building’s windows as a separate tree instance, resulting in false positive detections driven by specular reflectance.
Figure 7: Distribution of the number of pseudo-labeled and human-annotated samples added to the training dataset per round for different learning strategies. (A) Semi-supervised vs. hybrid learning: a decline in pseudo-label generation for the semi-supervised approach. (B) Active learning vs. hybrid learning: active learning requires substantially higher human annotation effort, while the hybrid approach achieves comparable dataset growth with fewer human annotations.
Counting individual trees is a fundamental task for environmental monitoring, yet remains largely unexplored with satellite imagery. At these resolutions, isolated trees may still be identifiable, but crown boundaries become ambiguous in dense forests, making the notion of an individual tree inherently ill-defined. Moreover, large-scale manual annotations of individual trees are prohibitively expensive. While scalable supervision can be derived from airborne LiDAR, the resulting annotations are noisy and difficult to exploit effectively. We address these challenges by formulating tree counting as a spatial density matching problem supervised through Unbalanced Optimal Transport. This formulation naturally accommodates both precise localization of isolate trees and robust density estimation in dense forests. We further introduce a self-correction mechanism that leverages transport residuals to progressively refine noisy supervision during training. We evaluate our approach on TinyTrees, a new benchmark spanning three continents and three satellite sensors, comprising over 216 million tree annotations (including 639k manually verified instances) across 25890 km2. Our method consistently outperforms detection-based, regression-based, and transport-based distribution-matching baselines, demonstrating the effectiveness of unbalanced transport and reliability-aware supervision for large-scale tree counting from satellite imagery. Code, data and models are available at https://github.com/dgominski/treematch.
Dimitri Gominski, Maurice Mugabowindekwe, Qiue Xu +6
University of Copenhagen · University of Rwanda · University of Chinese Academy of Sciences +3
The fine grained classification of street trees is a crucial task for urban planning, streetscape management, and the assessment of urban ecosystem services. However, progress in this field has been hindered by the lack of large scale, geographically diverse, and publicly available benchmark datasets specifically designed for street trees. To address this critical gap, we introduce StreetTree, the world's first large scale benchmark dataset dedicated to fine grained street tree classification. The dataset contains over 12 million images covering more than 8,300 common street tree species, collected from urban streetscapes across 133 countries spanning five continents, and supplemented with expert verified observational data. StreetTree poses challenges for pretrained vision models under complex urban environments including high inter species visual similarity, long tailed natural distributions, significant intra class variations caused by seasonal changes, and diverse imaging conditions such as lighting, occlusions from buildings, and varying camera angles. In addition, we provide a hierarchical taxonomy (order, family, genus, and species) to support research in hierarchical classification and representation learning. Through extensive experiments with various vision models, we establish solid baselines and reveal the limitations of existing methods in handling such real world complexities. We believe that StreetTree will serve as a key resource for driving new advancements at the intersection of computer vision and urban science.
Reliable agricultural data is essential for food security, land-use planning, and economic resilience, yet in Palestine, such data remains difficult to collect at scale because of fragmented landscapes, limited field access, and restrictions on aerial monitoring. This paper presents ResAF-Net, a satellite-based tree detection framework designed for large-scale agricultural monitoring in resource-constrained settings. The proposed architecture combines a ResNet-50 encoder, Atrous Spatial Pyramid Pooling (ASPP), a feature-fusion stage, a multi-head self-attention refinement module, and an anchor-free FCOS detection head to improve tree localization in dense and heterogeneous scenes. Trained on the MillionTrees benchmark, the model achieved 82% Recall, 63.03% [email protected], and 35.47% [email protected]:0.95 on the validation split, indicating strong sensitivity to tree presence while maintaining competitive localization quality. Beyond benchmark evaluation, we implemented the model within a web-based GIS application integrated with Palestinian cadastral data from GeoMolg, enabling tree analysis at scene, parcel, and community levels. This deployment demonstrates the practical feasibility of AI-assisted agricultural inventorying in Palestine. It provides a foundation for data-driven monitoring, reporting, and future species-level analysis of Mediterranean tree crops.