PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery
Organizations: The University of Texas at Austin · Amazon · University of Maryland · Emory University
Abstract
Vector polygon generation converts visual inputs, e.g., remote sensing (RS) images, into vectorized polygonal geometries, supporting applications such as autonomous driving, vector map construction, and remote sensing. Early pipelines predict raster masks and post-process them into polygons, which prevents end-to-end optimization and may miss small objects or introduce inaccurate vertices. Recent methods directly generate vector polygons, but most focus on simple exterior contours, while they either cannot represent complex polygons with holes or fail to preserve their topology. In this paper, we propose PolyTopoBench, a unified evaluation framework for vector polygon generation from RS images with explicit emphasis on complex polygons. PolyTopoBench evaluates both exterior and interior rings, and benchmarks 11 representative methods, including segmentation-based polygonization pipelines, vision foundation model baselines, and specialized vector polygon generators, on two RS-image datasets covering buildings, roads, vegetation, and unvegetated regions. Experiments show that existing methods often recover simple exterior boundaries but degrade substantially on polygons with holes or multiple rings. These results reveal complex polygon generation as an unresolved challenge and motivate topology-aware benchmarks and model designs. Code and data are available at https://github.com/seai-lab/PolyTopoBench.
Figures & tables
| Dataset | Class | Instances | Simple | Complex | Holes | Holes / complex | Avg. verts |
|---|---|---|---|---|---|---|---|
| Inria | Building | 254,133 | 248,621 | 5,512 | 19,854 | 3.60 | 10.27 |
| Deventer | Road | 4,533 | 3,467 | 1,066 | 9,992 | 9.37 | 62.57 |
| Deventer | Vegetation | 23,517 | 22,658 | 859 | 1,477 | 1.72 | 11.98 |
| Deventer | Unvegetated | 16,552 | 13,610 | 2,942 | 10,749 | 3.65 | 21.53 |
| Method | Type and Description | Ring |
| Seg: Segmentation-then-polygonization | ||
| U-Net + Poly. [ 38 , 55 ] | Predict a semantic mask and convert its contours into polygons with rule-based simplification. | Imp. |
| Mask R-CNN + Poly. [ 11 ] | Predict instance masks and convert each mask contour into a simplified polygon. | Imp. |
| FM: Foundation-model-assisted polygonization | ||
| SAM2 + Poly. [ 33 ] | Generate masks with SAM2 prompts or priors and polygonize the resulting mask contours. | Imp. |
| Rep.: Learned representation-to-vector |
| Region | Vector-boundary quality | Ring structure | |||||||||||
| Method | Task | AP50 | BIoU | POLIS | MTA | Hole-F1 | Topo-EM | ||||||
| O | E | H | O | E | H | O | E | H | |||||
| U-Net + Poly. | Building | 0.670 | 0.294 | 0.308 | 0.234 | 2.62 | 2.99 | 2.79 | 47.52 | 47.02 | 48.23 | 0.514 | 0.531 |
| Road | 0.520 | 0.328 | 0.311 | 0.337 | 5.16 | 8.44 | 2.66 | 52.66 | 52.58 | 46.93 | 0.466 | 0.144 | |
| Veg. | 0.382 | 0.340 | 0.313 | 0.208 | 4.41 | 6.12 | 5.17 | 50.52 | 49.48 | 48.83 | 0.022 | 0.222 | |
| Unveg. | 0.247 | 0.222 | 0.206 | 0.253 | 5.39 | 8.24 | 3.15 | 56.70 | 55.43 | 50.68 | 0.428 | 0.088 | |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | AP50 | E-BIoU | Hole-F1 | Topo-EM | POLIS ( / / ) |
|---|---|---|---|---|---|
| U-Net + Poly. | 0.520 | 0.311 | 0.466 | 0.144 | 5.16 / 5.13 / 5.12 |
| HiSup | 0.389 | 0.257 | 0.303 | 0.137 | 6.49 / 6.51 / 6.50 |
| ACPV-Net | 0.361 | 0.281 | 0.394 | 0.137 | 6.35 / 6.31 / 6.29 |
| RoIPoly | 0.002 | 0.221 | 0.192 | 0.009 | 9.27 / 9.32 / 9.32 |
| Setting | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|
| Train from the released WHU-Mix checkpoint | 0.190 | 0.588 | 0.000 | 0.004 |
| Train from scratch | 0.631 | 0.312 | 0.015 | 0.065 |
| Setting | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|
| Corner threshold | 0.034 | 0.558 | 0.000 | 0.000 |
| Corner threshold | 0.034 | 0.558 | 0.000 | 0.000 |
| Corner threshold | 0.000 | – | 0.000 | 0.000 |
| Proposal source | Setting | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|---|
| Ground truth boxes (upper bound) | Evaluate the RoIPoly polygon head with ground truth object boxes. | 0.158 | 0.680 | 0.310 | 0.082 |
| Sparse R-CNN boxes | Train a separate Sparse R-CNN detector on Inria and use its predicted boxes as RoIPoly proposals. | 0.001 | 0.135 | 0.002 | 0.001 |
| Variant | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|
| input with affine rotation, maximum vertices | 0.000 | 0.592 | 0.000 | 0.000 |
| input without affine rotation, maximum vertices | 0.000 | 0.608 | 0.000 | 0.000 |
| input without affine rotation, maximum vertices | 0.102 | 0.183 | 0.047 | 0.084 |
| Variant | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|
| Zero-shot using pretrained checkpoint | 0.004 | 0.600 | 0.000 | 0.001 |
| Fine-tuning | 0.012 | 0.544 | 0.000 | 0.000 |
| Train from scratch | 0.016 | 0.180 | 0.002 | 0.021 |
| Prompting variant | Prompt construction | AP50 | O-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|---|
| Ground truth box prompt (upper bound) | Use each ground truth instance box as the SAM2 box prompt. | 0.824 | 0.266 | 0.000 | 0.050 |
| Mask R-CNN box prompt | Use Mask R-CNN predicted boxes as SAM2 box prompts. | 0.431 | 0.242 | 0.017 | 0.377 |
| U-Net mask + box | Use a U-Net predicted mask together with its bounding box. | 0.506 | 0.251 | 0.000 | 0.029 |
| Iterative mask refinement | Run SAM2 with U-Net box and mask prompts, then feed the predicted mask back for a second refinement pass. | 0.473 | 0.242 | 0.000 | 0.027 |
| U-Net box only | Use the bounding box of each U-Net predicted instance mask. | 0.402 | 0.243 | 0.000 | 0.024 |
| U-Net mask only | Use the U-Net predicted mask as the only SAM2 prompt. | 0.035 | 0.203 | 0.000 | 0.001 |
| Method | Type | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|---|
| U-Net + Poly. | Seg. | 0.455 [0.437, 0.478] | 0.294 [0.288, 0.299] | 0.357 [0.335, 0.379] | 0.246 [0.235, 0.257] |
| Mask R-CNN + Poly. | Seg. | 0.343 [0.329, 0.361] | 0.260 [0.252, 0.268] | 0.112 [0.100, 0.127] | 0.243 [0.233, 0.254] |
| SAM2 + Poly. | FM | 0.134 [0.128, 0.145] | 0.274 [0.264, 0.285] | 0.011 [0.008, 0.013] | 0.127 [0.121, 0.133] |
| HiSup | Rep. | 0.315 [0.298, 0.338] | 0.256 [0.250, 0.262] | 0.286 [0.263, 0.309] | 0.187 [0.177, 0.198] |
| ACPV-Net | Rep. | 0.323 [0.306, 0.341] | 0.263 [0.256, 0.270] | 0.298 [0.277, 0.318] | 0.189 [0.180, 0.199] |
| FFL | Rep. | 0.207 [0.194, 0.221] | 0.217 [0.207, 0.229] | 0.000 [0.000, 0.000] | 0.096 [0.092, 0.100] |
| Hole-bearing patches in the training stream | AP50 | E-BIoU | Hole-F1 | Topo-EM |
|---|---|---|---|---|
| Down-sampled ( ) | 0.678 | 0.304 | 0.455 | 0.529 |
| Natural ( ) | 0.670 | 0.308 | 0.514 | 0.531 |
| Up-sampled ( ) | 0.679 | 0.304 | 0.559 | 0.533 |
Remote Sensing Benchmarks
Remote sensing benchmarks measure how well models interpret Earth-observation data across sensors, regions, time, tasks, and operating conditions.
See all 194 papers