Generalizable Multiscale Segmentation of Heterogeneous Map Collections

Remi Petitpierre

cs.CVMar 5, 2026

Generalizable Multiscale Segmentation of Heterogeneous Map Collections

Authors: Remi Petitpierre

Organizations: EPFL, Swiss Federal Institute of Technology in Lausanne. Digital Humanities Laboratory (DHLAB), School of Computer and Communication Sciences (IC). CM 1 468, Station 10, 1015 Lausanne, Switzerland. · EPFL. Laboratory of Urban Sociology (LASUR), School of Architecture, Civil and Environmental Engineering (ENAC).

Abstract

Historical map collections are highly diverse in style, scale, and geographic focus, often consisting of many single-sheet documents. Yet most work in map recognition focuses on specialist models tailored to homogeneous map series. In contrast, this article aims to develop generalizable semantic segmentation models and ontology. First, we introduce Semap, a new open benchmark dataset comprising 1,439 manually annotated patches designed to reflect the variety of historical map documents. Second, we present a segmentation framework that combines procedural data synthesis with multiscale integration to improve robustness and transferability. This framework achieves state-of-the-art performance on both the HCMSSD and Semap datasets, showing that a diversity-driven approach to map recognition is not only viable but also beneficial. The results indicate that segmentation performance remains largely stable across map collections, scales, geographic regions, and publication contexts. By proposing benchmark datasets and methods for the generic segmentation of historical maps, this work opens the way to integrating the long tail of cartographic archives to historical geographic studies.

Figures & tables

Figure 1: Manually annotated training samples. Real map crops (top) with corresponding labels (bottom) . The label color code follows the legend of Figure 2 .

Figure 2: Semantic masks retrieved from MapTiler API, displayed as a function of the zoom level.

Figure 3: Synthetically generated training samples. Synthetic map images (top) and corresponding labels (bottom) . The label color code follows the legend of Figure 2 .

	background	boundary	built	non-built	water	road network
Semap	36.6%	3.8%	11.7%	31.8%	7.9%	8.2%
Synthetic data	7.9%	4.3%	2.8%	72.9%	9.8%	2.3%

Table 1: Relative class–area distribution in the Semap and synthetic training datasets, expressed as a percentage of total pixel area.

	Test set (base)			Test set (no multiscale)			Test set (no synth. pretrain)
class	IoU	Recall	Precision	IoU	Recall	Precision	IoU	Recall	Precision
background	76.8	78.2	90.5	78.7	81.0	87.9	86.3	89.4	91.1
boundary	40.7	59.0	56.2	41.9	55.9	60.6	39.7	58.2	55.4
built*	79.8	84.4	92.4	70.8	75.5	89.4	74.6	84.1	87.7
non-built*	81.8	86.0	87.2	79.3	84.4	85.1	78.4	83.0	87.0
water*	72.2	74.4	81.2	68.4	70.6	73.6	67.0	68.6	80.3

Table 2: Performance of semantic segmentation on Semap dataset (

n_{test}=144

). IoU denotes the Intersection over Union. The mean is computed across the four geographic classes (*).

Reference	Model (training)	mIoU	mR	mP	F1
(Polák, 2024)	UNet–Transformer	28.1	43.9	45.2	44.6
(Petitpierre et al., 2021)	UNet–ResNet101	54.3	77.3	66.5	71.9
(Arzoumanidis et al., 2025)	SCGCN–ResNet101	63.5	82.7	74.2	74.2
(Jan, 2022)	HRNet–OCRNet	66.2	–	–	–
Ours	Mask2Former–Swin-L (”few-shots”)	71.5	77.9	89.1	83.5
Ours	Mask2Former–Swin-L (transfer)	76.0	83.8	88.2	86.0

Table 3: Performance benchmark on HCMSSD–Paris dataset (validation set). mIoU denotes the average intersection over union of the four geographic classes (built, non-built, water, road network). mR = mean recall, mP = mean precision, PRm = (mP+mR)/2. Note that Polák Polák (2024) aggregates both HCMSS–Paris and HCMSSD–World datasets. The multiscale strategy is not employed here.

Reference	Model (training)	mIoU	mR	mP	F1
(Polák, 2024)	UNet–Transformer	28.1	43.9	45.2	44.6
(Petitpierre et al., 2021)	UNet–ResNet101	45.2	64.9	62.6	63.8
Ours	Mask2Former–Swin-L (”few-shots”)	74.4	81.7	88.6	85.2
Ours	Mask2Former–Swin-L (transfer)	76.3	88.4	92.6	90.5
Ours	Mask2Former–Swin-L (retrained)	74.2	86.8	84.1	85.4

Table 4: Performance benchmark on HCMSSD–World dataset (validation set). mIoU denotes the average intersection over union of the four geographic classes (built, non-built, water, road network). mR = mean recall, mP = mean precision, PRm = (mP+mR)/2. Note that Polák Polák (2024) aggregates both HCMSS–Paris and HCMSSD–World datasets. The multiscale strategy is not employed here.

Set/treatment	mIoU	mR	mP	PRm	Acc
Test set (base)	74.2	79.4	85.4	82.4	80.2
– multiscale	70.0	75.5	81.5	78.5	79.6
– synth. pretrain.	69.1	75.4	84.2	79.8	81.5
– finetuning	69.1	73.3	83.7	78.5	81.4
Validation set	75.5	80.5	87.8	84.1	84.5
– finetuning	70.9	76.3	86.7	81.5	82.7

Table 5: Ablation study and comparison of the results on test, validation, and training sets (

n_{test}=144

n_{val}=289

n_{train}=1006

). Removing one feature at a time, hereafter specified as (–) treatment. mIoU = mean Intersection over Union, mR = mean recall, mP = mean precision, Acc = accuracy. Means are computed over the four geographic classes (built, non-built, water, road network), while Acc is computed as the overall pixel accuracy.

PRm=(mP+mR)/2

Figure 4: Confusion matrix based on Semap test set predictions. The matrix is computed as the average over all test samples (micro-average), normalized per class, (a) w.r.t. predictions or (b) w.r.t to the ground-truth. The diagonal values correspond to (a) precision, and (b) recall.

Figure 5: Standardized effect of map-metadata variables on mIoU. Vertical bars indicate multivariate linear ordinary least-squares regression coefficients, with 95% CI. The dependent variable, mIoU, denotes the per-patch (n=801) average intersection over union standardized within each dataset partition. No systematic bias is detected, apart from a slight positive effect for maps covering locations in Indonesia or Turkey. Overall, the regression explains little variance (

R^{2}=0.04

), suggesting stable segmentation performance across evaluated metadata classes.

Figure 6: Results of inference on the test set for a random subset of 8 samples.

Panel of Figure 7 — Figure 7: (top) Map of railway trade volumes, 1883. Emile Cheysson. Mouvement sur les chemins de fer du monde, 1833. Ministère des Travaux Publics, Paris. 49 x 60 cm. David Rumsey Collection, 12516.014. URL: davidrumsey.com/luna/servlet/detail/RUMSEY81~~309240~~90079149; (bottom) Result of the semantic segmentation. The recognition of coastlines is impacted, although not entirely hindered by the presence of overlapping circles. Dark blue areas tend to be mistaken for water.

Panel of Figure 8 — Figure 8: (top) Ethnographic and religious map, 1883. Carl Diercke, Eduard Gaebler. Völkerkarte der Erde. Völker- und Religionskarte von Europa, 1883. Published by George Westermann, Braunschweig. 32 x 31 cm. David Rumsey Collection, 12198.011. URL: davidrumsey.com/luna/servlet/detail/RUMSEY81~~311889~~90081579, (bottom) Result of the semantic segmentation. The distinction of land masses from seas seems affected by the use of a color code to represent thematic categories.

Panel of Figure 9 — Figure 9: (top) New map of America, part of the Atlas of Gerard Mercator and Jodocus Hondius, 1633. Hendrik Hondius. America noviter delineata, 1633. Published in Amsterdam. Copperplate, hand colored. 38 x 50 cm. David Rumsey Coll., 10621.195. URL: davidrumsey.com/luna/servlet/detail/RUMSEY81~~345039~~90107477. (bottom) Result of the semantic segmentation. Landmasses are overall well segmented, whereas the recognition of seas seems impacted by the figuration of ship icons.

Panel of Figure 10 — Figure 10: (top) View of Thomaston, Connecticut, 1879. O.H. Bailey. View of Thomaston, Conn., 1879. O.H. Bailey & Co, Boston. Hand colored. 43 x 63 cm. Library of Congress, G3784.T43A3 1879.B3. URL: hdl.loc.gov/loc.gmd/g3784t.pm000974. (bottom) Result of the semantic segmentation. Dwellings are well recognized despite the strong iconographic character of the representation.

Panel of Figure 11 — Figure 11: (top) Fire insurance plan of New Egypt, Ocean, New Jersey, 1900. New Egypt N.J. 2, 1900. Sanborn Map Company, New York. 64 x 54 cm. Princeton library, HMC04. Rotated 90°. URL: catalog.princeton.edu/catalog/9955740603506421. (bottom) Result of the semantic segmentation. Building segmentation is accurate, whereas a lack of spatial context impairs the recognition of the road network and non-built surfaces.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Institution(s)	Part of	Landscape	Scale	Period	# samples
Vaud Cantonal Archives	Lausanne cadastre	urban, periurban	very high	1722–1882	42
Neuchatel Cadastre Office	Neuchatel cadastre	urban, periurban	very high	1886–1888	10
Geneva Cadastre Office	Geneva cadastre	urban, periurban	very high	1845–1856	8
French Dept. Archives & others	Napoleonic cadastre	urban, rural	very high	1808–1888	18
BHVP	Paris Atlas	urban	very high	1898–1926	3
BHVP	HCMSSD – Paris	urban	very high	1785–1950	70

Appendix

Table 6: Composition of Semap dataset.

Sample-normalized macro average		$mIoU=\frac{1}{K}\sum_{k=3}^{6}\frac{\sum_{i=1}^{N}\sum_{\forall p}\cap_{i,k}Y_{i,k}}{\sum_{i=1}^{N}\sum_{\forall p}\cup_{i,k}Y_{i,k}}$
Micro average		$mIoU=\frac{1}{N}\sum_{i=1}^{N}\frac{\sum_{k=3}^{6}\sum_{\forall p}\cap_{i,k}Y_{i,k}}{\sum_{k=3}^{6}\sum_{\forall p}\cup_{i,k}Y_{i,k}}$
Macro average		$mIoU=\frac{\sum_{k=3}^{6}\sum_{i=1}^{N}\sum_{\forall p}\cap_{i,k}}{\sum_{k=3}^{6}\sum_{i=1}^{N}\sum_{\forall p}\cup_{i,k}}$
Per class macro average		$mIoU=\frac{1}{K}\sum_{k=3}^{6}\frac{\sum_{i=1}^{N}\sum_{\forall p}\cap_{i,k}}{\sum_{i=1}^{N}\sum_{\forall p}\cup_{i,k}}$

Appendix

Table 7: Performance assessment strategies. Variant formulae for computing the mIoU. For mR, the term

\cup_{i,k}

is replaced by

Y_{i,k}

. For mP, it is replaced by

\hat{Y}_{i,k}

Set	Performance assessment strategy	mIoU	mR	mP	PRm
Semap	Sample-normalized macro average (base)	74.2	79.4	85.4	82.4
	Micro average	74.4	80.3	80.3	80.3
	Macro average	67.0	80.3	80.3	80.3
	Per class macro average	57.4	79.4	67.8	73.6
$\text{HCMSSD}_{\text{{{Paris}}}}$	Sample-normalized macro average	79.3	84.2	87.7	85.9
	Micro average	84.6	90.6	90.6	90.6

Appendix

Table 8: Comparison of performance assessment strategies on Semap, HCMSSD–Paris, and –World datasets ( retrained models). Based on the formulae provided in Tab. 7 . PRm = (mP+mR)/2.

Panel of Figure 12 — Figure 12: (top) Map of the Old World, 1655. Pierre Duval. Orbis Vetus, 1655. Published in Paris. Copperplate, hand colored. 38.5 x 54 cm. BnF, GE D-17771. URL: gallica.bnf.fr/ark:/12148/btv1b84953694. (bottom) Result of the semantic segmentation. Seas and continents are correctly segmented despite their unusual shape.

Panel of Figure 13 — Figure 13: (top) Map of Ile-de-France region, 18th century. Nicolas de Fer, P. Starckman. Les environs de Paris […], 18th century. Danet, Paris. Copperplate, hand colored. 48.5 x 67.5 cm. BnF, GE DD-2987 (786B). URL: gallica.bnf.fr/ark:/12148/btv1b530532084. (bottom) Result of the semantic segmentation. Riverways are well segmented; City icons are correctly classified as built.

Panel of Figure 14 — Figure 14: (top) Jambudipa in the topographic map of Java at the scale of 1:20,000, 1907. Java Res Preanger Regentschappen, Blad G.XIII, 1907. Dutch Topographic Bureau, Batavia. Photoengraving. Leiden university library, DG 14,67. Rotated 90°. URL: hdl.handle.net/1887.1/item:815459. (bottom) Result of the semantic segmentation. Urban areas are well segmented in spite of their unusual morphology; the recognition of boundaries seems impaired by the dense representation of relief.

Panel of Figure 15 — Figure 15: (top) New York City Atlas, 1879. George W. Bromley, Edward Robinson, August H. Mueller. New York City, Ward 15 & Part of Ward 9, 1879. Printed by F. Bourquin, Philadelphia. Published by G.W. Bromley & Co, and E. Robinson, New York. Lithography, hand colored. 46 x 67 cm. David Rumsey Collection, 2597.010. URL: davidrumsey.com/luna/servlet/detail/RUMSEY81~~30628~~1150150. (bottom) Result of the semantic segmentation. The street grid and land plots tend to be well segmented, although disruptions in the urban form, or the limited context in directly adjacent blank areas generates confusion.

Class	Feature	Category	Zoom min	Zoom max	Query filter
built	landuse_residential	landuse	10	11	(class IN (’residential’,’suburb’,’neighborhood’))
nblt1	landcover_grass	landcover			(class IS ’grass’)
nblt2	landcover_wood	landcover			(class IS ’wood’)
nblt1	landcover_sand	landcover			(class IS ’sand’)
nblt1	landcover_glacier	landcover			(subclass IN (’glacier’,’ice_shelf’))
water	water	water			NOT (intermittent) AND (brunnel IS NOT ’tunnel’)

Appendix

Table 9: List of vector objects queried from MapTiler Planet API. The feature and category fields correspond to MapTiler’s data structure. The filter field is adapted for legibility. nblt = non-built (1 base, 2 forest), roadn = road network (1 base, 2 railway), bnd = boundary.

Semantic Segmentation

Semantic segmentation assigns a class label to every image pixel, enabling detailed scene understanding across domains and sensors.

See all 414 papers

Automated Detection and Encoding of Cartographic Signs in Historical Maps
Oct 9, 2026Remi PetitpierreHistorical Document Processing Geospatial Representation Learning
EPFL, Swiss Federal Institute of Technology in Lausanne. Digital Humanities Laboratory (DHLAB), School of Computer and Communication Sciences (IC). CM 1 468, Station 10, 1015 Lausanne, Switzerland. · EPFL. Laboratory of Urban Sociology (LASUR), School of Architecture, Civil and Environmental Engineering (ENAC).
Deep learning enables urban change profiling through alignment of historical maps
Feb 2, 2026Sidi Wu, Yizi Chen, Maurizio Gribaudi +4
Institute of Cartography and Geoinformation, ETH Zurich, Switzerland · CRH, École des hautes études en sciences sociales (EHESS), France · Photogrammetry and Remote Sensing, ETH Zurich, Switzerland +1
OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation
Oct 21, 2026Yimin Wei, Hongruixuan Chen, Hao Zheng +5Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation
The University of Tokyo · RIKEN AIP · East China Normal University +1

Oct 9, 2026J/K move · Enter open · S save

Automated Detection and Encoding of Cartographic Signs in Historical Maps

Remi Petitpierre

EPFL, Swiss Federal Institute of Technology in Lausanne. Digital Humanities Laboratory (DHLAB), School of Computer and Communication Sciences (IC). CM 1 468, Station 10, 1015 Lausanne, Switzerland. · EPFL. Laboratory of Urban Sociology (LASUR), School of Architecture, Civil and Environmental Engineering (ENAC).

Historical Document Processing Geospatial Representation Learning

Open paper AskPDF arXiv

Symbols and icons in archival maps represent an expansive source of evidence for historical geography. In this paper, we present a two-stage framework for (1) detecting cartographic signs in historical maps and (2) generating interpretable sign embeddings that capture both semantics and stylistic variations. We also introduce CaSiDD, an open benchmark dataset spanning multiple sign categories and diverse cartographic styles, designed for generic sign detection. Using YOLOv10-X, we demonstrate the practicality of automated sign detection on historical maps. We then present strategies for creating contrasted, informative embeddings of cartographic signs. We approach embedding design systematically, maximizing expressiveness while suppressing digitization or background material confounders. Our final method, based on an adapted RADIOv4 backbone, effectively encodes both sign form and semantics, while producing interpretable representations. This work lays a foundation for scalable evidence extraction for the comparative study of map signs within cartographic culture and geohistorical research.

Abstract

Semantic Segmentation

Explore similar work

Automated Detection and Encoding of Cartographic Signs in Historical Maps

Deep learning enables urban change profiling through alignment of historical maps

OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation