PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery

Zeping Liu; Ni Lao; Weiwei Sun; Gil Wolff; Yiqun Xie; Liang Zhao; Junfeng Jiao; Gengchen Mai

cs.CVSep 26, 2026

PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery

Authors: Zeping Liu, Ni Lao, Weiwei Sun, Gil Wolff, Yiqun Xie, Liang Zhao, Junfeng Jiao, Gengchen Mai

Organizations: The University of Texas at Austin · Amazon · University of Maryland · Emory University

Abstract

Vector polygon generation converts visual inputs, e.g., remote sensing (RS) images, into vectorized polygonal geometries, supporting applications such as autonomous driving, vector map construction, and remote sensing. Early pipelines predict raster masks and post-process them into polygons, which prevents end-to-end optimization and may miss small objects or introduce inaccurate vertices. Recent methods directly generate vector polygons, but most focus on simple exterior contours, while they either cannot represent complex polygons with holes or fail to preserve their topology. In this paper, we propose PolyTopoBench, a unified evaluation framework for vector polygon generation from RS images with explicit emphasis on complex polygons. PolyTopoBench evaluates both exterior and interior rings, and benchmarks 11 representative methods, including segmentation-based polygonization pipelines, vision foundation model baselines, and specialized vector polygon generators, on two RS-image datasets covering buildings, roads, vegetation, and unvegetated regions. Experiments show that existing methods often recover simple exterior boundaries but degrade substantially on polygons with holes or multiple rings. These results reveal complex polygon generation as an unresolved challenge and motivate topology-aware benchmarks and model designs. Code and data are available at https://github.com/seai-lab/PolyTopoBench.

Figures & tables

Figure 1: Overview of PolyTopoBench. PolyTopoBench evaluates complex vector polygon generation from remote sensing imagery. It consists of diverse remote sensing imagery datasets with simple and complex polygon annotations on different geographic features, including buildings, roads, unvegetated regions, and vegetated regions . It supports eleven diverse polygon generation baselines. A unified evaluator is developed, which jointly considers polygonal region agreement, vector-boundary quality, and ring-structure correctness .

Dataset	Class	Instances	Simple	Complex	Holes	Holes / complex	Avg. verts
Inria	Building	254,133	248,621	5,512	19,854	3.60	10.27
Deventer	Road	4,533	3,467	1,066	9,992	9.37	62.57
Deventer	Vegetation	23,517	22,658	859	1,477	1.72	11.98
Deventer	Unvegetated	16,552	13,610	2,942	10,749	3.65	21.53

Table 1: Dataset complexity statistics for the four single-class PolyTopoBench tasks. Instances denotes the number of total polygon annotations in each dataset/class. Simple and Complex denote instances with

H(G)=0

and

H(G)\geq 1

. Holes counts all interior rings in complex instances. Holes / complex is the average number of holes per complex instance. Avg. verts is the average number of vertices per instance, including exterior and interior rings and excluding the repeated closing point.

Method	Type and Description	Ring
	Seg: Segmentation-then-polygonization
U-Net + Poly. [ 38 , 55 ]	Predict a semantic mask and convert its contours into polygons with rule-based simplification.	Imp.
Mask R-CNN + Poly. [ 11 ]	Predict instance masks and convert each mask contour into a simplified polygon.	Imp.
	FM: Foundation-model-assisted polygonization
SAM2 + Poly. [ 33 ]	Generate masks with SAM2 prompts or priors and polygonize the resulting mask contours.	Imp.
	Rep.: Learned representation-to-vector

Table 2: Comparing polygon generation baselines. Ring mode is explicit (denoted as Exp. ) when holes are natively modeled and implicit (denoted as Imp. ) otherwise.

		Region	Vector-boundary quality									Ring structure
Method	Task	AP50	BIoU			POLIS			MTA			Hole-F1	Topo-EM
		$\uparrow$	O $\uparrow$	E $\uparrow$	H $\uparrow$	O $\downarrow$	E $\downarrow$	H $\downarrow$	O $\downarrow$	E $\downarrow$	H $\downarrow$	$\uparrow$	$\uparrow$
U-Net + Poly.	Building	0.670	0.294	0.308	0.234	2.62	2.99	2.79	47.52	47.02	48.23	0.514	0.531
	Road	0.520	0.328	0.311	0.337	5.16	8.44	2.66	52.66	52.58	46.93	0.466	0.144
	Veg.	0.382	0.340	0.313	0.208	4.41	6.12	5.17	50.52	49.48	48.83	0.022	0.222
	Unveg.	0.247	0.222	0.206	0.253	5.39	8.24	3.15	56.70	55.43	50.68	0.428	0.088

Table 3: Main results across the four single-class polygon generation tasks. Building uses the Inria dataset [ 29 ] ; Road , Veg. (vegetation), and Unveg. (unvegetated) use the Deventer dataset [ 17 ] . O, E, and H denote the Overall, Ext., and Hole ring-aware scopes. AP50, BIoU, Hole-F1, and Topo-EM are higher-is-better; POLIS and MTA are lower-is-better. “–” marks scores that are undefined because no hole match is accepted on that task. Best value per task and per column is bold .

Figure 4: Qualitative vector prediction results. Rows from top to bottom show building, road, vegetation, and unvegetated examples. Columns show five methods, one per type, which are U-Net + Poly. ( Seg. ), SAM2 + Poly. ( FM ), ACPV-Net [ 17 ] ( Rep. , Exp. ), HiSup [ 45 ] ( Rep. , Imp. ), and RoIPoly [ 16 ] ( Direct , Imp. )). Each patch is annotated with its ground-truth or predicted hole count. Ground-truth exterior rings are black, interior is white, predicted exterior rings use per-method colors, holes are dashed yellow. Vertices are marked with small circles. See Appendix A.4 for more.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Metric sensitivity to the minimum hole area on the Inria Building dataset. We vary the minimum area used to keep ground-truth and predicted interior rings while fixing the ring matching threshold to $\gamma=0.1$ . Hole-F1 and H-BIoU are more sensitive to this filtering because they directly evaluate interior rings, while AP50 and Topo-EM remain relatively stable. This shows that the proposed hole-aware metrics are not dominated by a single choice of minimum hole size.

Figure 6: Metric sensitivity to the ring matching threshold $\gamma$ . Scores are macro-averaged over the four tasks and grouped by method type. Increasing $\gamma$ makes ring matching stricter, which lowers both Hole-F1 and Topo-EM, but the leading methods remain largely stable across thresholds.

Method	AP50	E-BIoU	Hole-F1	Topo-EM	POLIS ( $1\times$ / $2\times$ / $4\times$ )
U-Net + Poly.	0.520	0.311	0.466	0.144	5.16 / 5.13 / 5.12
HiSup	0.389	0.257	0.303	0.137	6.49 / 6.51 / 6.50
ACPV-Net	0.361	0.281	0.394	0.137	6.35 / 6.31 / 6.29
RoIPoly	0.002	0.221	0.192	0.009	9.27 / 9.32 / 9.32

Appendix

Table 4: Metric robustness to vertex density on Deventer Road. Collinear midpoints are inserted into every predicted ring to obtain

2\times

and

4\times

more vertices without changing the shape. AP50, E-BIoU, Hole-F1, and Topo-EM are identical for all three densities; POLIS (Overall scope) is reported for

1\times

2\times

4\times

vertices.

Figure 7: Additional qualitative vector predictions. Rows show building, road, vegetation, and unvegetated examples. Columns show the ground-truth vector and five representative methods: Mask R-CNN + Poly. ( Seg. ), SAM2 + Poly. ( FM ), GCP ( Rep. ), FFL ( Rep. ), and PolyWorld ( Direct ). The first column uses black exterior rings for ground truth. Prediction columns show only the predicted vector output: colored solid lines are exterior rings, dashed yellow lines are interior rings, and small circles are vertices. Patch labels report the ground-truth or predicted hole count.

Figure 8: Additional qualitative vector predictions with segmentation and representation-to-vector baselines. Rows show the four object categories in the same order as Figure 7 . Columns show the ground-truth vector, U-Net + Poly. and Mask R-CNN + Poly. ( Seg. ), HiSup, HoliTracer, and ACPV-Net ( Rep. ). Prediction columns contain no ground-truth overlay, so the figure directly shows each method’s predicted exterior ring, interior rings, and vertices.

Figure 9: Additional qualitative vector predictions comparing all four method types. Columns show the ground-truth vector, U-Net + Poly. ( Seg. ), SAM2 + Poly. ( FM ), ACPV-Net and GCP ( Rep. ), and RoIPoly ( Direct ). Across the four object categories, exterior geometry can remain visually plausible while the number and placement of interior rings differ strongly from the ground truth.

Figure 10: Additional qualitative vector predictions emphasizing representation-to-vector methods. Columns show the ground-truth vector, Mask R-CNN + Poly. ( Seg. ), SAM2 + Poly. ( FM ), FFL, HoliTracer, and GCP ( Rep. ). The examples illustrate common topology errors: missing holes, extra holes from fragmented evidence, and exterior rings that trace nearby structures rather than the target instance.

Figure 11: Additional qualitative vector predictions including direct vector decoders. Columns show the ground-truth vector, U-Net + Poly. ( Seg. ), ACPV-Net and HiSup ( Rep. ), Pix2Poly and RoIPoly ( Direct ). Direct decoders often produce simple or distorted exterior rings and miss many interior rings, while representation-to-vector methods recover more local structure but still fail to match the complete ring topology.

Setting	AP50	E-BIoU	Hole-F1	Topo-EM
Train from the released WHU-Mix checkpoint	0.190	0.588	0.000	0.004
Train from scratch	0.631	0.312	0.015	0.065

Appendix

Table 5: GCP setting ablation on Inria Building. Since GCP releases its checkpoint on WHU-Mix [ 28 ] , we ablate the transfer learning setting and the training from scratch setting.

Setting	AP50	E-BIoU
Corner threshold $0.00$	0.034	0.558
Corner threshold $0.10$	0.034	0.558
Corner threshold $0.50$	0.000	–

Appendix

Table 6: HoliTracer setting ablation on the Inria Building dataset. We sweep the corner confidence threshold used by the vector tracing stage.

Proposal source	Setting	AP50	E-BIoU	Hole-F1	Topo-EM
Ground truth boxes (upper bound)	Evaluate the RoIPoly polygon head with ground truth object boxes.	0.158	0.680	0.310	0.082
Sparse R-CNN boxes	Train a separate Sparse R-CNN detector on Inria and use its predicted boxes as RoIPoly proposals.	0.001	0.135	0.002	0.001

Appendix

Table 7: RoIPoly setting ablation on the Inria Building dataset. We compare the polygon head under ground truth boxes with a setting where a separately trained Sparse R-CNN [ 40 ] detector provides the bounding boxes.

Variant	AP50	E-BIoU	Hole-F1	Topo-EM
$512$ input with affine rotation, $192$ maximum vertices	0.000	0.592	0.000	0.000
$512$ input without affine rotation, $192$ maximum vertices	0.000	0.608	0.000	0.000
$512$ input without affine rotation, $224$ maximum vertices	0.102	0.183	0.047	0.084

Appendix

Table 8: Pix2Poly setting ablation on the Inria Building dataset. Pix2Poly predicts vertex tokens with an image-to-sequence Transformer and recovers connectivity with an optimal matching network. We mainly ablate the maximum vertex-sequance length and image data augmentation strategy.

Variant	AP50	E-BIoU	Hole-F1	Topo-EM
Zero-shot using pretrained checkpoint	0.004	0.600	0.000	0.001
Fine-tuning	0.012	0.544	0.000	0.000
Train from scratch	0.016	0.180	0.002	0.021

Appendix

Table 9: PolyWorld setting ablation on the Inria Building dataset. PolyWorld detects vertex peaks, extracts visual descriptors, and predicts vertex connectivity with a GNN and Sinkhorn-style optimal matching. The variants test whether better vertex supervision or longer fine-tuning fixes topology errors.

Prompting variant	Prompt construction	AP50	O-BIoU	Hole-F1	Topo-EM
Ground truth box prompt (upper bound)	Use each ground truth instance box as the SAM2 box prompt.	0.824	0.266	0.000	0.050
Mask R-CNN box prompt	Use Mask R-CNN predicted boxes as SAM2 box prompts.	0.431	0.242	0.017	0.377
U-Net mask + box	Use a U-Net predicted mask together with its bounding box.	0.506	0.251	0.000	0.029
Iterative mask refinement	Run SAM2 with U-Net box and mask prompts, then feed the predicted mask back for a second refinement pass.	0.473	0.242	0.000	0.027
U-Net box only	Use the bounding box of each U-Net predicted instance mask.	0.402	0.243	0.000	0.024
U-Net mask only	Use the U-Net predicted mask as the only SAM2 prompt.	0.035	0.203	0.000	0.001

Appendix

Table 10: SAM2-assisted polygonization setting ablation on the Inria Building dataset. Each variant changes only the SAM2 prompt source: ground truth boxes, Mask R-CNN predicted boxes, U-Net predicted boxes, U-Net masks with boxes, iterative SAM2 mask refinement, U-Net mask-only prompts, or U-Net box-only prompts. All SAM2 masks are converted to polygons using the same contour extraction and simplification procedure.

Figure 12: Qualitative comparison of all $11$ methods on the same patches. Rows show building, road, vegetation, and unvegetated examples; columns show the ground truth followed by every benchmark method. All methods share one color scheme: black lines are ground-truth exterior rings, red lines are predicted exterior rings, dashed lines are interior rings, and white dots are vertices. Each panel reports its ground-truth or predicted hole count, and “no pred” means the method produced no output overlapping the patch. Seg. and Rep. methods such as U-Net + Poly., HiSup, and ACPV-Net generally recover exterior boundaries and some interior rings, but the predicted hole counts and locations often differ from the ground truth. SAM2 + Poly. tends to hallucinate holes (e.g., $44$ predicted holes on the road example with $5$ ground-truth holes). The direct methods frequently produce fragmented or incomplete instances and almost no interior rings, consistent with their low AP50.

Method	Type	AP50	E-BIoU	Hole-F1	Topo-EM
U-Net + Poly.	Seg.	0.455 [0.437, 0.478]	0.294 [0.288, 0.299]	0.357 [0.335, 0.379]	0.246 [0.235, 0.257]
Mask R-CNN + Poly.	Seg.	0.343 [0.329, 0.361]	0.260 [0.252, 0.268]	0.112 [0.100, 0.127]	0.243 [0.233, 0.254]
SAM2 + Poly.	FM	0.134 [0.128, 0.145]	0.274 [0.264, 0.285]	0.011 [0.008, 0.013]	0.127 [0.121, 0.133]
HiSup	Rep.	0.315 [0.298, 0.338]	0.256 [0.250, 0.262]	0.286 [0.263, 0.309]	0.187 [0.177, 0.198]
ACPV-Net	Rep.	0.323 [0.306, 0.341]	0.263 [0.256, 0.270]	0.298 [0.277, 0.318]	0.189 [0.180, 0.199]
FFL	Rep.	0.207 [0.194, 0.221]	0.217 [0.207, 0.229]	0.000 [0.000, 0.000]	0.096 [0.092, 0.100]

Appendix

Table 11: Macro-average benchmark scores with 95% image-level bootstrap confidence intervals over 500 resamples.

Hole-bearing patches in the training stream	AP50	E-BIoU	Hole-F1	Topo-EM
Down-sampled ( $4.3\%$ )	0.678	0.304	0.455	0.529
Natural ( $15.2\%$ )	0.670	0.308	0.514	0.531
Up-sampled ( $41.8\%$ )	0.679	0.304	0.559	0.533

Appendix

Table 12: Effect of hole-bearing patch sampling on U-Net + Poly. (Inria Building). Only the sampling probability of patches containing at least one complex polygon is changed; data, gradient steps, and seed are fixed. The natural distribution corresponds to the main benchmark setting.

Remote Sensing Benchmarks

Remote sensing benchmarks measure how well models interpret Earth-observation data across sensors, regions, time, tasks, and operating conditions.

See all 194 papers

ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery
Mar 17, 2026Weiqin Jiao, Hao Cheng, George Vosselman +1
Faculty of Geo-Information Science and Earth Observation (ITC) University of Twente, The Netherlands
Vector Map as Language: Toward Unified Remote Sensing Vector Mapping
Jun 9, 2026Yinglong Yan, Yunkai Yang, Haoyi Wang +7Remote Sensing Image Understanding Vision-Language Models
school of Artificial Intelligence and Robotics, Hunan University, Changsha 410082, China · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong · Department of Geomatics Engineering, Changsha University of Science and Technology, Changsha 410114, China +1
Topologically Consistent Agricultural Parcel Vectorization with Semantic-Guided Diffusion and Topology-Aware Polygonization
Sep 7, 2026Weiqin Jiao, Xiaolong Zuo, Claudio PerselloPrecision Agriculture Diffusion Models
Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, 7522 NH, The Netherlands · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430072, China

Mar 17, 2026J/K move · Enter open · S save

ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery

Weiqin Jiao, Hao Cheng, George Vosselman, Claudio Persello

Faculty of Geo-Information Science and Earth Observation (ITC) University of Twente, The Netherlands

Open paper AskPDF arXiv

We tackle the problem of generating a complete vector map representation from aerial imagery in a single run: producing polygons for all land-cover classes with shared boundaries and without gaps or overlaps. Existing polygonization methods are typically class-specific; extending them to multiple classes via per-class runs commonly leads to topological inconsistencies, such as duplicated edges, gaps, and overlaps. We formalize this new task as All-Class Polygonal Vectorization (ACPV) and release the first public benchmark, Deventer-512, with standardized metrics jointly evaluating semantic fidelity, geometric accuracy, vertex efficiency, per-class topological fidelity and global topological consistency. To realize ACPV, we propose ACPV-Net, a unified framework introducing a novel Semantically Supervised Conditioning (SSC) mechanism coupling semantic perception with geometric primitive generation, along with a topological reconstruction that enforces shared-edge consistency by design. While enforcing such strict topological constraints, ACPV-Net surpasses all class-specific baselines in polygon quality across classes on Deventer-512. It also applies to single-class polygonal vectorization without any architectural modification, achieving the best-reported results on WHU-Building. Data, code, and models will be released at: https://github.com/HeinzJiao/ACPV-Net.

Abstract

Remote Sensing Benchmarks

Explore similar work

ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery

Vector Map as Language: Toward Unified Remote Sensing Vector Mapping

Topologically Consistent Agricultural Parcel Vectorization with Semantic-Guided Diffusion and Topology-Aware Polygonization