High-fidelity training-free photovoltaic module mapping from multi-view UAV imagery
Organizations: School of Geography and Information Engineering, China University of Geosciences (Wuhan), Wuhan 430074, China · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China · Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China · Department of Building Environment and Energy Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China ·
Abstract
High-fidelity photovoltaic (PV) module mapping requires complete instance recovery and accurate delineation. Vision foundation models offer zero-shot segmentation, but converting incomplete and inconsistent predictions across views into a complete, geometrically accurate map of individual modules remains challenging. We propose the Multi-View PV Module Mapper (MV-PVMapper), a training-free framework coupling vision foundation models with unmanned aerial vehicle (UAV) photogrammetry. First, Segment Anything Model 3 (SAM 3) generates candidates from original images using text prompts. Their contours are then transferred into object space through per-module slope correction to limit distortion from surface-model noise, and verified against representative module dimensions to guide cross-view instance selection. Finally, accepted instances return to all covering images as box prompts for SAM~3, supplementing text prompts with spatial cues for iterative segmentation, verification and fusion. On five-direction surveys of three operational rooftop PV plants at approximately 1 cm nadir ground sampling distance, two feedback iterations recover 7,710 of 7,711 reference modules with no false positives at an intersection-over-union (IoU) matching threshold of 0.5. Macro-averaged area precision and recall reach 99.00 % and 98.52 %, respectively. Averaged across sites, the aggregated Jaccard index reaches 97.55 %, compared with 60.73 % and 41.24 % for orthomosaic segmentation and multi-view pixel-voting baselines, respectively; average precision at the stricter IoU threshold of 0.95 reaches 95.67 %. Ablation studies show that module-size constraints are critical for instance separation, while slope correction and feedback to all covering views improve boundary accuracy. These results are achieved without task-specific image annotation or segmentation-model weight updates.
Figures & tables
| Site 1 | Site 2 | Site 3 | |
| Reference modules | 1 772 | 2 305 | 3 634 |
| Module size (m) | |||
| Mounting levels | 1 | 1 | 2 |
| Mounting type | Roof-flush | Racked | Roof-flush |
| Survey date | 2025-10 | 2025-11 | 2024-08 |
| Relative flight height (m) | 35.0 | 38.0 | 41.8 |
| Area-level | Instance-level | ||||||||
| Method | IoU area | P area | R area | RQ | SQ | PQ | AJI | ||
| DOM-IterSeg | 60.73 | 72.58 | 99.80 | 60.77 | 73.93 | 96.97 | 71.66 | 60.73 | 60.51 |
| MV-PixelVote | 86.11 | 92.19 | 96.79 | 89.11 | 49.40 | 96.03 | 47.64 | 41.24 | 27.52 |
| MV-PVMapper | 97.55 | 98.76 | 99.00 | 98.52 | 99.99 | 97.56 | 97.56 | 97.55 | 95.67 |
| Area-level | Instance-level | ||||||||
| Projection | IoU area | P area | R area | RQ | SQ | PQ | AJI | ||
| Collinearity-based projection | 89.94 | 94.52 | 96.78 | 92.56 | 96.85 | 94.45 | 91.54 | 87.70 | 46.56 |
| Affine-based projection | 97.24 | 98.60 | 98.82 | 98.38 | 99.98 | 97.28 | 97.26 | 97.24 | 93.46 |
| Slope correction projection (ours) | 97.55 | 98.76 | 99.00 | 98.52 | 99.99 | 97.56 | 97.56 | 97.55 | 95.67 |
| Area-level | Instance-level | ||||||||
| Geometry prior | IoU area | P area | R area | RQ | SQ | PQ | AJI | ||
| No geometry prior | 72.99 | 84.38 | 74.05 | 98.09 | 5.16 | 91.91 | 4.84 | 19.63 | 3.80 |
| + rect | 95.34 | 97.60 | 98.18 | 97.04 | 84.03 | 97.09 | 81.64 | 83.69 | 87.54 |
| + area | 97.20 | 98.58 | 98.63 | 98.53 | 99.95 | 97.29 | 97.24 | 97.19 | 90.81 |
| + rect + area (ours) | 97.55 | 98.76 | 99.00 | 98.52 | 99.99 | 97.56 | 97.56 | 97.55 | 95.67 |
| Area-level | Instance-level | ||||||||
| Feedback routing | IoU area | P area | R area | RQ | SQ | PQ | AJI | ||
| No feedback ( ) | 83.19 | 89.52 | 98.83 | 83.99 | 91.05 | 96.67 | 88.04 | 83.18 | 71.96 |
| Image-space self re-prompting | 95.36 | 97.62 | 98.74 | 96.53 | 99.35 | 96.53 | 95.90 | 95.32 | 79.88 |
| Source-view routing | 94.80 | 97.30 | 98.84 | 95.86 | 98.77 | 97.05 | 95.86 | 94.80 | 86.77 |
| Covering-view routing (ours) | 97.55 | 98.76 | 99.00 | 98.52 | 99.99 | 97.56 | 97.56 | 97.55 | 95.67 |
| Area-level | Instance-level | ||||||||
| Iteration | IoU area | P area | R area | RQ | SQ | PQ | AJI | ||
| 83.19 | 89.52 | 98.83 | 83.99 | 91.05 | 96.67 | 88.04 | 83.18 | 71.96 | |
| 96.33 | 98.12 | 99.01 | 97.26 | 99.42 | 97.43 | 96.86 | 96.33 | 92.44 | |
| 97.55 | 98.76 | 99.00 | 98.52 | 99.99 | 97.56 | 97.56 | 97.55 | 95.67 | |
Remote Sensing Image Segmentation
Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.
See all 232 papers