Generalizable Multiscale Segmentation of Heterogeneous Map Collections
Authors: Remi Petitpierre
Organizations: EPFL, Swiss Federal Institute of Technology in Lausanne. Digital Humanities Laboratory (DHLAB), School of Computer and Communication Sciences (IC). CM 1 468, Station 10, 1015 Lausanne, Switzerland. · EPFL. Laboratory of Urban Sociology (LASUR), School of Architecture, Civil and Environmental Engineering (ENAC).
Historical map collections are highly diverse in style, scale, and geographic focus, often consisting of many single-sheet documents. Yet most work in map recognition focuses on specialist models tailored to homogeneous map series. In contrast, this article aims to develop generalizable semantic segmentation models and ontology. First, we introduce Semap, a new open benchmark dataset comprising 1,439 manually annotated patches designed to reflect the variety of historical map documents. Second, we present a segmentation framework that combines procedural data synthesis with multiscale integration to improve robustness and transferability. This framework achieves state-of-the-art performance on both the HCMSSD and Semap datasets, showing that a diversity-driven approach to map recognition is not only viable but also beneficial. The results indicate that segmentation performance remains largely stable across map collections, scales, geographic regions, and publication contexts. By proposing benchmark datasets and methods for the generic segmentation of historical maps, this work opens the way to integrating the long tail of cartographic archives to historical geographic studies.
Figures & tables
Figure 1: Manually annotated training samples. Real map crops (top) with corresponding labels (bottom) . The label color code follows the legend of Figure 2 .
Figure 2: Semantic masks retrieved from MapTiler API, displayed as a function of the zoom level.
Figure 3: Synthetically generated training samples. Synthetic map images (top) and corresponding labels (bottom) . The label color code follows the legend of Figure 2 .
background
boundary
built
non-built
water
road network
Semap
36.6%
3.8%
11.7%
31.8%
7.9%
8.2%
Synthetic data
7.9%
4.3%
2.8%
72.9%
9.8%
2.3%
Table 1: Relative class–area distribution in the Semap and synthetic training datasets, expressed as a percentage of total pixel area.
Test set (base)
Test set (no multiscale)
Test set (no synth. pretrain)
class
IoU
Recall
Precision
IoU
Recall
Precision
IoU
Recall
Precision
background
76.8
78.2
90.5
78.7
81.0
87.9
86.3
89.4
91.1
boundary
40.7
59.0
56.2
41.9
55.9
60.6
39.7
58.2
55.4
built*
79.8
84.4
92.4
70.8
75.5
89.4
74.6
84.1
87.7
non-built*
81.8
86.0
87.2
79.3
84.4
85.1
78.4
83.0
87.0
water*
72.2
74.4
81.2
68.4
70.6
73.6
67.0
68.6
80.3
Table 2: Performance of semantic segmentation on Semap dataset ( ntest=144 ). IoU denotes the Intersection over Union. The mean is computed across the four geographic classes (*).
Reference
Model (training)
mIoU
mR
mP
F1
(Polák, 2024)
UNet–Transformer
28.1
43.9
45.2
44.6
(Petitpierre et al., 2021)
UNet–ResNet101
54.3
77.3
66.5
71.9
(Arzoumanidis et al., 2025)
SCGCN–ResNet101
63.5
82.7
74.2
74.2
(Jan, 2022)
HRNet–OCRNet
66.2
–
–
–
Ours
Mask2Former–Swin-L (”few-shots”)
71.5
77.9
89.1
83.5
Ours
Mask2Former–Swin-L (transfer)
76.0
83.8
88.2
86.0
Table 3: Performance benchmark on HCMSSD–Paris dataset (validation set). mIoU denotes the average intersection over union of the four geographic classes (built, non-built, water, road network). mR = mean recall, mP = mean precision, PRm = (mP+mR)/2. Note that Polák Polák (2024) aggregates both HCMSS–Paris and HCMSSD–World datasets. The multiscale strategy is not employed here.
Reference
Model (training)
mIoU
mR
mP
F1
(Polák, 2024)
UNet–Transformer
28.1
43.9
45.2
44.6
(Petitpierre et al., 2021)
UNet–ResNet101
45.2
64.9
62.6
63.8
Ours
Mask2Former–Swin-L (”few-shots”)
74.4
81.7
88.6
85.2
Ours
Mask2Former–Swin-L (transfer)
76.3
88.4
92.6
90.5
Ours
Mask2Former–Swin-L (retrained)
74.2
86.8
84.1
85.4
Table 4: Performance benchmark on HCMSSD–World dataset (validation set). mIoU denotes the average intersection over union of the four geographic classes (built, non-built, water, road network). mR = mean recall, mP = mean precision, PRm = (mP+mR)/2. Note that Polák Polák (2024) aggregates both HCMSS–Paris and HCMSSD–World datasets. The multiscale strategy is not employed here.
Set/treatment
mIoU
mR
mP
PRm
Acc
Test set (base)
74.2
79.4
85.4
82.4
80.2
– multiscale
70.0
75.5
81.5
78.5
79.6
– synth. pretrain.
69.1
75.4
84.2
79.8
81.5
– finetuning
69.1
73.3
83.7
78.5
81.4
Validation set
75.5
80.5
87.8
84.1
84.5
– finetuning
70.9
76.3
86.7
81.5
82.7
Table 5: Ablation study and comparison of the results on test, validation, and training sets ( ntest=144 , nval=289 , ntrain=1006 ). Removing one feature at a time, hereafter specified as (–) treatment. mIoU = mean Intersection over Union, mR = mean recall, mP = mean precision, Acc = accuracy. Means are computed over the four geographic classes (built, non-built, water, road network), while Acc is computed as the overall pixel accuracy. PRm=(mP+mR)/2
Figure 4: Confusion matrix based on Semap test set predictions. The matrix is computed as the average over all test samples (micro-average), normalized per class, (a) w.r.t. predictions or (b) w.r.t to the ground-truth. The diagonal values correspond to (a) precision, and (b) recall.
Figure 5: Standardized effect of map-metadata variables on mIoU. Vertical bars indicate multivariate linear ordinary least-squares regression coefficients, with 95% CI. The dependent variable, mIoU, denotes the per-patch (n=801) average intersection over union standardized within each dataset partition. No systematic bias is detected, apart from a slight positive effect for maps covering locations in Indonesia or Turkey. Overall, the regression explains little variance ( R2=0.04 ), suggesting stable segmentation performance across evaluated metadata classes.
Figure 6: Results of inference on the test set for a random subset of 8 samples.
Figure 7: (top) Map of railway trade volumes, 1883. Emile Cheysson. Mouvement sur les chemins de fer du monde, 1833. Ministère des Travaux Publics, Paris. 49 x 60 cm. David Rumsey Collection, 12516.014. URL: davidrumsey.com/luna/servlet/detail/RUMSEY8130924090079149; (bottom) Result of the semantic segmentation. The recognition of coastlines is impacted, although not entirely hindered by the presence of overlapping circles. Dark blue areas tend to be mistaken for water.
Figure 8: (top) Ethnographic and religious map, 1883. Carl Diercke, Eduard Gaebler. Völkerkarte der Erde. Völker- und Religionskarte von Europa, 1883. Published by George Westermann, Braunschweig. 32 x 31 cm. David Rumsey Collection, 12198.011. URL: davidrumsey.com/luna/servlet/detail/RUMSEY8131188990081579, (bottom) Result of the semantic segmentation. The distinction of land masses from seas seems affected by the use of a color code to represent thematic categories.
Figure 9: (top) New map of America, part of the Atlas of Gerard Mercator and Jodocus Hondius, 1633. Hendrik Hondius. America noviter delineata, 1633. Published in Amsterdam. Copperplate, hand colored. 38 x 50 cm. David Rumsey Coll., 10621.195. URL: davidrumsey.com/luna/servlet/detail/RUMSEY8134503990107477. (bottom) Result of the semantic segmentation. Landmasses are overall well segmented, whereas the recognition of seas seems impacted by the figuration of ship icons.
Figure 10: (top) View of Thomaston, Connecticut, 1879. O.H. Bailey. View of Thomaston, Conn., 1879. O.H. Bailey & Co, Boston. Hand colored. 43 x 63 cm. Library of Congress, G3784.T43A3 1879.B3. URL: hdl.loc.gov/loc.gmd/g3784t.pm000974. (bottom) Result of the semantic segmentation. Dwellings are well recognized despite the strong iconographic character of the representation.
Figure 11: (top) Fire insurance plan of New Egypt, Ocean, New Jersey, 1900. New Egypt N.J. 2, 1900. Sanborn Map Company, New York. 64 x 54 cm. Princeton library, HMC04. Rotated 90°. URL: catalog.princeton.edu/catalog/9955740603506421. (bottom) Result of the semantic segmentation. Building segmentation is accurate, whereas a lack of spatial context impairs the recognition of the road network and non-built surfaces.
Table 7: Performance assessment strategies. Variant formulae for computing the mIoU. For mR, the term ∪i,k is replaced by Yi,k . For mP, it is replaced by Y^i,k .
Set
Performance assessment strategy
mIoU
mR
mP
PRm
Semap
Sample-normalized macro average (base)
74.2
79.4
85.4
82.4
Micro average
74.4
80.3
80.3
80.3
Macro average
67.0
80.3
80.3
80.3
Per class macro average
57.4
79.4
67.8
73.6
HCMSSDParis
Sample-normalized macro average
79.3
84.2
87.7
85.9
Micro average
84.6
90.6
90.6
90.6
Appendix
Table 8: Comparison of performance assessment strategies on Semap, HCMSSD–Paris, and –World datasets ( retrained models). Based on the formulae provided in Tab. 7 . PRm = (mP+mR)/2.
Figure 12: (top) Map of the Old World, 1655. Pierre Duval. Orbis Vetus, 1655. Published in Paris. Copperplate, hand colored. 38.5 x 54 cm. BnF, GE D-17771. URL: gallica.bnf.fr/ark:/12148/btv1b84953694. (bottom) Result of the semantic segmentation. Seas and continents are correctly segmented despite their unusual shape.
Figure 13: (top) Map of Ile-de-France region, 18th century. Nicolas de Fer, P. Starckman. Les environs de Paris […], 18th century. Danet, Paris. Copperplate, hand colored. 48.5 x 67.5 cm. BnF, GE DD-2987 (786B). URL: gallica.bnf.fr/ark:/12148/btv1b530532084. (bottom) Result of the semantic segmentation. Riverways are well segmented; City icons are correctly classified as built.
Figure 14: (top) Jambudipa in the topographic map of Java at the scale of 1:20,000, 1907. Java Res Preanger Regentschappen, Blad G.XIII, 1907. Dutch Topographic Bureau, Batavia. Photoengraving. Leiden university library, DG 14,67. Rotated 90°. URL: hdl.handle.net/1887.1/item:815459. (bottom) Result of the semantic segmentation. Urban areas are well segmented in spite of their unusual morphology; the recognition of boundaries seems impaired by the dense representation of relief.
Figure 15: (top) New York City Atlas, 1879. George W. Bromley, Edward Robinson, August H. Mueller. New York City, Ward 15 & Part of Ward 9, 1879. Printed by F. Bourquin, Philadelphia. Published by G.W. Bromley & Co, and E. Robinson, New York. Lithography, hand colored. 46 x 67 cm. David Rumsey Collection, 2597.010. URL: davidrumsey.com/luna/servlet/detail/RUMSEY81306281150150. (bottom) Result of the semantic segmentation. The street grid and land plots tend to be well segmented, although disruptions in the urban form, or the limited context in directly adjacent blank areas generates confusion.
Class
Feature
Category
Zoom min
Zoom max
Query filter
built
landuse_residential
landuse
10
11
(class IN (’residential’,’suburb’,’neighborhood’))
nblt1
landcover_grass
landcover
(class IS ’grass’)
nblt2
landcover_wood
landcover
(class IS ’wood’)
nblt1
landcover_sand
landcover
(class IS ’sand’)
nblt1
landcover_glacier
landcover
(subclass IN (’glacier’,’ice_shelf’))
water
water
water
NOT (intermittent) AND (brunnel IS NOT ’tunnel’)
Appendix
Table 9: List of vector objects queried from MapTiler Planet API. The feature and category fields correspond to MapTiler’s data structure. The filter field is adapted for legibility. nblt = non-built (1 base, 2 forest), roadn = road network (1 base, 2 railway), bnd = boundary.
Symbols and icons in archival maps represent an expansive source of evidence for historical geography. In this paper, we present a two-stage framework for (1) detecting cartographic signs in historical maps and (2) generating interpretable sign embeddings that capture both semantics and stylistic variations. We also introduce CaSiDD, an open benchmark dataset spanning multiple sign categories and diverse cartographic styles, designed for generic sign detection. Using YOLOv10-X, we demonstrate the practicality of automated sign detection on historical maps. We then present strategies for creating contrasted, informative embeddings of cartographic signs. We approach embedding design systematically, maximizing expressiveness while suppressing digitization or background material confounders. Our final method, based on an adapted RADIOv4 backbone, effectively encodes both sign form and semantics, while producing interpretable representations. This work lays a foundation for scalable evidence extraction for the comparative study of map signs within cartographic culture and geohistorical research.
Remi Petitpierre
EPFL, Swiss Federal Institute of Technology in Lausanne. Digital Humanities Laboratory (DHLAB), School of Computer and Communication Sciences (IC). CM 1 468, Station 10, 1015 Lausanne, Switzerland. · EPFL. Laboratory of Urban Sociology (LASUR), School of Architecture, Civil and Environmental Engineering (ENAC).
Prior to modern Earth observation technologies, historical maps provide a unique record of long-term urban transformation and offer a lens on the evolving identity of cities. However, extracting consistent and fine-grained change information from historical map series remains challenging due to spatial misalignment, cartographic variation, and degrading document quality, limiting most analyses to small-scale or qualitative approaches. We propose a deep-learning-based framework for fine-grained urban change analysis from large collections of historical maps, built on a modular design that integrates spatial map alignment, multi-temporal object extraction, and change estimation. This framework shifts the analysis of historical maps from ad hoc visual comparison toward systematic, quantitative characterization of urban change. Experiments demonstrate the robust performance of the proposed alignment and object extraction methods. Applied to Paris between 1868 and 1937, the framework reveals the spatial and temporal heterogeneity in urban transformation, suggesting its potential relevance for research in the social sciences and humanities. The modular design of our framework provides flexibility for future adaptation to different cartographic contexts and downstream applications.
Sidi Wu, Yizi Chen, Maurizio Gribaudi +4
Institute of Cartography and Geoinformation, ETH Zurich, Switzerland · CRH, École des hautes études en sciences sociales (EHESS), France · Photogrammetry and Remote Sensing, ETH Zurich, Switzerland +1
In remote sensing, open-vocabulary segmentation is particularly important for flexible and scalable land-cover understanding across diverse regions, as it improves generalization to novel categories without requiring exhaustive pixel-level annotations. Despite its great promise, progress in this area remains largely constrained by existing datasets with limited category coverage and overly coarse taxonomies, and therefore struggles to support fine-grained semantic understanding required in real-world applications such as urban functional zoning, carbon accounting, and post-disaster assessment. To address these challenges, we present OpenEarth-OVS44, a multimodal benchmark with 290,587 expert-verified segments from 2,832 RGB images and 1,341 strictly registered RGB-SAR pairs at sub-meter spatial resolution, covering 40 cities across France, Japan, and the United States under a unified 44-class taxonomy organized into 9 base categories. Three evaluation tracks are provided: an RGB-only fine-grained benchmark, a multimodal track with diverse synthetic cloudy conditions for weather-robust evaluation, and a held-out cross-region track. Extensive experiments on representative open-vocabulary segmentation models demonstrate that training on OpenEarth-OVS44 substantially improves fine-grained segmentation performance over the strongest training-free baseline, with improvements of up to 7.1 mIoU points on unseen categories and 27.6 mIoU points on seen categories. The dataset and code will be released.
Yimin Wei, Hongruixuan Chen, Hao Zheng +5
The University of Tokyo · RIKEN AIP · East China Normal University +1