Organizations: School of Geography and Information Engineering, China University of Geosciences (Wuhan), Wuhan 430074, China · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China · Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China · Department of Building Environment and Energy Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China · School of Urban Planning and Design, Peking University, Shenzhen, 518055, China
High-fidelity photovoltaic (PV) module mapping requires complete instance recovery and accurate delineation. Vision foundation models offer zero-shot segmentation, but converting incomplete and inconsistent predictions across views into a complete, geometrically accurate map of individual modules remains challenging. We propose the Multi-View PV Module Mapper (MV-PVMapper), a training-free framework coupling vision foundation models with unmanned aerial vehicle (UAV) photogrammetry. First, Segment Anything Model 3 (SAM 3) generates candidates from original images using text prompts. Their contours are then transferred into object space through per-module slope correction to limit distortion from surface-model noise, and verified against representative module dimensions to guide cross-view instance selection. Finally, accepted instances return to all covering images as box prompts for SAM~3, supplementing text prompts with spatial cues for iterative segmentation, verification and fusion. On five-direction surveys of three operational rooftop PV plants at approximately 1 cm nadir ground sampling distance, two feedback iterations recover 7,710 of 7,711 reference modules with no false positives at an intersection-over-union (IoU) matching threshold of 0.5. Macro-averaged area precision and recall reach 99.00 % and 98.52 %, respectively. Averaged across sites, the aggregated Jaccard index reaches 97.55 %, compared with 60.73 % and 41.24 % for orthomosaic segmentation and multi-view pixel-voting baselines, respectively; average precision at the stricter IoU threshold of 0.95 reaches 95.67 %. Ablation studies show that module-size constraints are critical for instance separation, while slope correction and feedback to all covering views improve boundary accuracy. These results are achieved without task-specific image annotation or segmentation-model weight updates.
Figures & tables
Site 1
Site 2
Site 3
Reference modules
1 772
2 305
3 634
Module size (m)
2.22×1.09
2.17×1.10
2.20×1.09
Mounting levels
1
1
2
Mounting type
Roof-flush
Racked
Roof-flush
Survey date
2025-10
2025-11
2024-08
Relative flight height (m)
35.0
38.0
41.8
Table 1: Study sites and UAV acquisition. Module size is the representative length × width of the modules at each site, taken in this study as the mean length × width of the reference modules.
Figure 1: DOMs (top) and oblique views of the reconstructed 3D models (bottom) of Sites 1–3.
Figure 2: Workflow of MV-PVMapper. Panels (a)–(d) correspond to Sections 4.1 – 4.4 . The snowflake marks the frozen weights of the segmentor, the promptable segmentation model described in Section 4.1 . Cv is the camera pose of image v , Mvk is the mask of its k -th candidate, and L and W are the representative module length and width.
Figure 3: Object-space contours of a row of five modules, transferred from the image-space masks in (a) by (b) intersecting the viewing ray of each contour vertex with the DSM, (c) a two-dimensional image-to-object affine transformation estimated for each candidate from DSM point correspondences and (d) per-module slope correction.
Area-level
Instance-level
Method
IoU area
F1
P area
R area
RQ
SQ
PQ
AJI
AP95
DOM-IterSeg
60.73
72.58
99.80
60.77
73.93
96.97
71.66
60.73
60.51
MV-PixelVote
86.11
92.19
96.79
89.11
49.40
96.03
47.64
41.24
27.52
MV-PVMapper
97.55
98.76
99.00
98.52
99.99
97.56
97.56
97.55
95.67
Table 2: Module-mapping accuracy of MV-PVMapper and the two baselines at t=2 . Values (%) are macro-averages over the three sites. Higher values are better; bold marks the highest value in each column.
Figure 4: Per-site comparison of the three methods at t=2 . (a) Metric profiles; IoU, F1, P and R are the area-level metrics of Table 2 , and the dashed circle marks 50%. (b) Reference-module outcomes and prediction-side false positives. Reference and prediction bars are normalised by their respective populations. Numbers below the reference bars give matched, merged, split and missed counts; numbers below the prediction bars give unmatched predictions (FP).
Figure 5: Site-wide module maps at t=2 , overlaid on the DOM. Columns show Sites 1–3 and rows show the reference modules, DOM-IterSeg, MV-PixelVote and MV-PVMapper.
Figure 6: Detail views of the module maps in Figure 5 . Each site is shown by two 5 m-scale crops; rows show the reference modules, DOM-IterSeg, MV-PixelVote and MV-PVMapper.
Area-level
Instance-level
Projection
IoU area
F1
P area
R area
RQ
SQ
PQ
AJI
AP95
Collinearity-based projection
89.94
94.52
96.78
92.56
96.85
94.45
91.54
87.70
46.56
Affine-based projection
97.24
98.60
98.82
98.38
99.98
97.28
97.26
97.24
93.46
Slope correction projection (ours)
97.55
98.76
99.00
98.52
99.99
97.56
97.56
97.55
95.67
Table 3: Ablation of contour projection. All configurations use the full geometry prior and feedback to all covering views and are evaluated at t=2 . Values (%) are macro-averages over the three sites; bold marks the highest value in each column.
Figure 7: Per-site results of the contour-projection ablation at t=2 . (a) RQ and SQ; (b) AP95 ; (c) missed reference modules (Missed/GT) and false positives among the predictions (FP/Pred).
Figure 8: Object-space contours obtained with the three projection methods (rows) in two windows per site (columns); TP, FP and FN are shown at the pixel level.
Figure 9: Module-level 3D geometry at Sites 1–3 (columns). (a) Recovered module planes above the DSM, coloured by tilt on a separate scale for each site; the yellow dashed line marks the transect. (b) Elevation of the DSM and the module planes along the transect.
Area-level
Instance-level
Geometry prior
IoU area
F1
P area
R area
RQ
SQ
PQ
AJI
AP95
No geometry prior
72.99
84.38
74.05
98.09
5.16
91.91
4.84
19.63
3.80
+ rect
95.34
97.60
98.18
97.04
84.03
97.09
81.64
83.69
87.54
+ area
97.20
98.58
98.63
98.53
99.95
97.29
97.24
97.19
90.81
+ rect + area (ours)
97.55
98.76
99.00
98.52
99.99
97.56
97.56
97.55
95.67
Table 4: Ablation of the module-geometry prior. + rect and + area each use only the corresponding term. All configurations use slope correction and feedback to all covering views and are evaluated at t=2 . Values (%) are macro-averages over the three sites; bold marks the highest value in each column.
Figure 10: Per-site results of the module-geometry prior ablation at t=2 . (a) Quantity and coverage: number of predictions relative to the number of reference modules (#Pred/#GT) and area F1 . (b) Instance quality: RQ and SQ (top), AJI and AP95 (bottom). (c) Partition errors: merged and split reference modules as percentages of the reference count.
Figure 11: Module maps of the four geometry-prior configurations (rows) in two windows per site and one window over a non-PV area (columns). Predictions are coloured by outcome as shown in the legend.
Area-level
Instance-level
Feedback routing
IoU area
F1
P area
R area
RQ
SQ
PQ
AJI
AP95
No feedback ( t=0 )
83.19
89.52
98.83
83.99
91.05
96.67
88.04
83.18
71.96
Image-space self re-prompting
95.36
97.62
98.74
96.53
99.35
96.53
95.90
95.32
79.88
Source-view routing
94.80
97.30
98.84
95.86
98.77
97.05
95.86
94.80
86.77
Covering-view routing (ours)
97.55
98.76
99.00
98.52
99.99
97.56
97.56
97.55
95.67
Table 5: Ablation of feedback routing. No feedback is evaluated at t=0 and the other configurations at t=2 . All configurations use slope correction and the full geometry prior. Values (%) are macro-averages over the three sites; bold marks the highest value in each column.
Figure 12: Per-site results of the feedback-routing ablation. No feedback is evaluated at t=0 and the other routes at t=2 . (a) SQ and AP95 ; (b) missed reference modules (Missed/GT) and false positives among the predictions (FP/Pred).
Figure 13: Comparison of the four feedback routes (rows) in two windows per site (columns); TP, FP and FN are shown at the pixel level.
Area-level
Instance-level
Iteration
IoU area
F1
P area
R area
RQ
SQ
PQ
AJI
AP95
t=0
83.19
89.52
98.83
83.99
91.05
96.67
88.04
83.18
71.96
t=1
96.33
98.12
99.01
97.26
99.42
97.43
96.86
96.33
92.44
t=2
97.55
98.76
99.00
98.52
99.99
97.56
97.56
97.55
95.67
Table 6: Module-mapping accuracy at t=0 , 1 and 2 using all five viewing directions. The t=2 row corresponds to MV-PVMapper in Table 2 . Values (%) are macro-averages over the three sites; bold marks the highest value in each column.
Figure 14: Per-site accuracy of MV-PVMapper at t=0 , 1 and 2 . Each panel shows one metric for the three sites; Missed/GT is the percentage of reference modules that are missed. The table under each panel gives the change in percentage points made by the first ( 0→1 ) and second ( 1→2 ) iteration. In the Missed/GT panel, the lines of Sites 1 and 2 coincide at zero.
Figure 15: Module maps at t=0 , 1 and 2 , overlaid on the DOM of each site. Each site is shown by two crops; TP, FP and FN are shown at the pixel level.
Figure 16: Per-site accuracy of the nested viewpoint subsets at t=0 , 1 and 2 . The horizontal axis gives the number of viewpoints: the nadir images alone, followed by one to four oblique directions added in a fixed order. The number at the end of each line is its change in percentage points from one to five viewpoints. Missed/GT is the percentage of reference modules that are missed.
Component-level information supports rooftop photovoltaic (PV) inspection, but recognizing individual components across complex scenes remains challenging. We present PVIS-10K, a dataset of 10,065 drone images with component-level annotations covering 63,632 physical PV modules across 22 rooftop installations. The high-resolution red-green-blue (RGB) images were acquired by unmanned aerial vehicles (UAVs, also known as drones) in Ningbo, Shaoxing and Shenzhen, China. Multi-angle oblique observations capture installation-height and scale variation, dense arrangements, occlusion, shadows and reflections. Three-dimensional delineation, reprojection and image-by-image review produced 3,516,675 image-level polygon annotations, stored in Common Objects in Context (COCO) format. Station-disjoint partitions support evaluation on unseen stations within and across cities. Four supervised models and text-prompted zero-shot Segment Anything Model 3 (SAM 3) provide full-resolution benchmarks of instance matching and foreground coverage. PVIS-10K supports component localization, delineation and image-based counting for drone-based rooftop PV inspection research.
Penglei Zhang, Wan Zhang, Ruizhe Deng +8
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, P.R. China · Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, P.R. China · Department of Building Environment and Energy Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China +2
Spatio-temporal PV data are essential for understanding adoption processes in off-grid regions, yet such data remain largely unavailable. Automated segmentation of remote sensing (RS) imagery offers a promising solution; yet, residential PV systems remain challenging targets because of their small size and sparse distribution, resulting in severe target-background imbalance. Vision-language foundation models (FMs) provide a data-efficient paradigm through prompt-based semantic and spatial guidance, but the relative contribution of different prompt types remains unclear. We systematically evaluate SAM3 for small-scale PV segmentation in RS imagery by comparing textual, geometric, and hybrid prompting, under varying supervision levels, training strategies, spatial resolutions, and imaging conditions. Multi-temporal aerial imagery from a large off-grid rural region serves as a study site, with findings validated across three additional datasets. Prompting strategy emerged as the dominant factor governing model behavior. Textual prompting consistently produced the lowest performance and showed the greatest sensitivity to supervision and imaging conditions. In contrast, spatial guidance substantially improved both segmentation accuracy and robustness. Hybrid prompting achieved the highest accuracy and stability, indicating that semantic and spatial guidance provide complementary information. Most performance gains were achieved with only a few hundred annotated samples, demonstrating strong data efficiency. Transfer learning had limited overall impact, with only modest improvements observed for textual prompting under limited supervision. Overall, our findings establish prompting strategy as a key determinant of SAM3 adaptation, robustness, and generalization, highlighting the potential of promptable FMs for scalable PV mapping in data-constrained off-grid regions.
Roni Blushtein-Livnon, Tal Svoray, Osher Rafaeli +4
Department of Environmental, Geoinformatics and Urban Planning Sciences, Ben-Gurion University of the Negev, Israel · Department of Psychology, Ben-Gurion University of the Negev, Israel · Department of Geography, The Hebrew University of Jerusalem, Isreal +1
Electroluminescence (EL) imaging is widely used to detect defects in photovoltaic (PV) modules, and machine learning methods have been applied to enable large-scale analysis of EL images. However, existing methods cannot assign multiple labels to the same pixel, limiting their ability to capture overlapping degradation features. We present a multi-channel U-Net architecture for pixel-level multi-label segmentation of EL images. The model outputs independent probability maps for cracks, busbars, dark areas, and non-cell regions, enabling accurate co-classification of interacting features such as cracks crossing busbars. The model achieved an accuracy of 98% and has been shown to generalize to unseen datasets. This framework offers a scalable, extensible tool for automated PV module inspection, improving defect quantification and lifetime prediction in large-scale PV systems.
Ojas Sanghi, Norman Jost, Benjamin G. Pierce +3
Sandia National Laboratories · Sandia National Laboratories, 1515 Eubank Blvd SE, Albuquerque, New Mexico, 87123, USA · Case Western Reserve University +3