Although semantic 3D city models are internationally available and becoming increasingly detailed, the incorporation of material information remains largely untapped. However, a structured representation of materials and their physical properties could substantially broaden the application spectrum and analytical capabilities for urban digital twins. At the same time, the growing number of repeated mobile laser scans of cities and their street spaces yields a wealth of observations influenced by the material characteristics of the corresponding surfaces. To leverage this information, we propose radiometric fingerprints of object surfaces by grouping LiDAR observations reflected from the same semantic object under varying distances, incident angles, environmental conditions, sensors, and scanning campaigns. Our study demonstrates how 312.4 million individual beams acquired across four campaigns using five LiDAR sensors on the Audi Autonomous Driving Dataset (A2D2) vehicle can be automatically associated with 6368 individual objects of the semantic 3D city model. The model comprises a comprehensive and semantic representation of four inner-city streets at Level of Detail (LOD) 3 with centimeter-level accuracy. It is based on the CityGML 3.0 standard and enables fine-grained sub-differentiation of objects. The extracted radiometric fingerprints for object surfaces reveal recurring intra-class patterns that indicate class-dominant materials. The semantic model, the method implementations, and the developed geodatabase solution 3DSensorDB are released under: https://github.com/tum-gis/sensordb
Figures & tables
Figure 1: Deriving radiometric fingerprints by grouping electromagnetic radiation responses of object surfaces measured by multiple LiDAR sensors across varying distances, angles, and environmental conditions to infer surface-specific properties and material composition information. 0 0 0 Delorean by Luis Adrian licensed under CC-BY-4.0 .
Figure 2: The workflow of this study.
Figure 3: Controlled intensity measurements were conducted with the Velodyne VLP-16 LiDAR sensor and a Spectralon target featuring four calibrated diffuse reflectance strips of approximately 20 , 90 , 4.3 , and 53% for varying distances and angles of incidence.
Figure 4: Semantic model of the road spaces in the city of Ingolstadt in Germany according to CityGML 3.0.
Figure 5: Vehicle of the Audi Autonomous Driving Dataset (A2D2) equipped with 6 cameras and 5 LiDAR sensors ( Geyer et al., 2020 ) .
Campaign 1
Campaigns 2–4
C2
C3
C4
Start time
13:06:26
12:16:47
12:27:22
12:34:11
Duration
90s
98s
107s
118s
Date
14 January 2020
18 November 2020
Astronomical season
early winter
late autumn
Weather
sunny, few clouds
sunny, few clouds
Table 1: Environmental conditions during the four sensor data acquisition campaigns. Parameter values marked with † are derived from historical meteorological records ( Time and Date AS, 2025 ) .
Figure 6: UML diagram of the 3DSensorDB database schema in green alongside a selected set of CityGML classes depicting the association between individual LiDAR beams and their thematic surface from which they were reflected. Due to the mapping to database tables, the 3DSensorDB classes use PostgreSQL data types. Classes of the CityGML 3.0 space concept are colored blue and two classes of thematic modules are shown in yellow as examples.
Figure 7: Raycasting-based association approach between LiDAR beam and object surface that is implemented on the 3DSensorDB leveraging the 3DCityDB.
Figure 8: Intensity values measured on the four Spectralon strips as a function of range R with the zenith angle θ<3∘ , where a point denotes the mean intensity and the shaded region indicates the standard deviation.
Figure 9: Individual intensity values measured as a function of the zenith angle θ at a 2-meter distance from the Spectralon rotation center with approximated cosine functions.
Figure 10: Bird’s-eye view of the reference point clouds and the point clouds of each SLAM submap from the acquisition campaigns after coregistration. Variations in submap coverage across campaigns necessitate combining the sensor data using each platform’s georeferenced pose trajectory, which is also required for associating the sensor data with the semantic road space model.
Campaign 1
Campaign 2
Campaign 3
Campaign 4
ET
1 year 4 months
1 year 9 months
1 year 9 months
1 year 9 months
∑
91.1M points
109.6M points
104.5M points
113.2M points
SM
#
f
RMSE
#
f
RMSE
#
f
RMSE
#
f
RMSE
1
7.1M
96.4%
0.18m
7.3M
96.2%
0.21m
6.3M
99.3%
0.16m
6.1M
95.9%
0.22m
2
8.2M
99.3%
0.11m
9.5M
98.9%
0.14m
6.8M
95.9%
0.21m
7.4M
98.3%
0.13m
3
7.8M
99.1%
0.15m
9.5M
99.0%
0.15m
8.0M
99.2%
0.12m
10.8M
94.7%
0.19m
Table 2: Fitness and RMSE scores of the point cloud submaps of each campaign after coregistration with the reference point cloud. ET = Elapsed time since reference surveying campaign, SM = submap of graph-based SLAM.
Figure 11: Distribution of the 15816 thematic objects in the LOD 3 road space model broken down by CityGML 3.0 classes (top diagram). The model’s rich semantic information supports fine subdifferentiations (bottom diagrams) and therefore enables flexible grouping of objects and their associated sensor data.
r≤15meter
r≤30meter
all ranges
associated
all
associated
all
associated
all
Camp. 1
79.6%
58.3M
73.2M
75.5%
68.8M
91.1M
74.6%
71.2M
95.5M
Camp. 2
75.6%
65.9M
87.2M
70.8%
77.6M
109.6M
69.8%
80.3M
115.0M
Camp. 3
76.9%
66.8M
86.8M
72.2%
75.5M
104.5M
71.3%
77.5M
108.8M
Camp. 4
75.1%
72.4M
96.4M
71.9%
81.4M
113.2M
70.9%
83.4M
117.5M
Σ/μ
76.8%
263.5M
343.6M
72.6%
303.2M
418.4M
71.7%
312.4M
436.8M
Table 3: Distribution of associated points broken down by campaign and range threshold. Points within a 15-meter range are associated more frequently than those beyond this threshold.
Figure 12: Association distribution of LiDAR points according to the CityGML 3.0 classes (top) and distribution of total object observations (bottom). Our method automatically associated 312.4 million points and identified 87345 object observations, each counted individually per sensor and campaign.
Figure 16
Figure 14: Selection of walls A and B for point-level analyses, as they were comprehensively captured by the sensors at varying ranges and angles. Additional cropping ensures that only points reflected from the same material are included.
Figure 15: t-SNE analysis at a perplexity of 200 for 10000 points reflected by the rear-left sensor on wall A. While the zenith angle strongly influences the measured intensity, no such effect is observed for the azimuth angle.
Figure 16: t-SNE analysis for the wall paint (left) and wall base plinth (right) of wall B at perplexity 100 revealing that the measured intensities are influenced by the sensor, range, and campaign.
Figure 20
1. SolitaryVegetationObject
0.0
3.8
13.2
9.2
10.8
114.9
124.3
2. SolitaryVegetationObject
0.0
9.7
12.7
14.1
118.3
127.7
3. DoorSurface
0.0
22.2
23.8
127.8
137.3
4. WallSurface
0.0
4.6
106.5
115.7
5. WallSurface
0.0
104.2
113.6
6. CityFurniture
0.0
13.9
Table 4: RMSE-based distance distQ3 quantifying differences between the radiometric fingerprints of only those objects in Figure 17 that were fully covered by the sensor throughout all zenith-angle intervals [θj,θj+1) .
CityGML class
count
AuxiliaryTrafficArea
6
7.5
33.0
91.9
33.3
34.1
22.4
14.7
40.0
31.2
BuildingInstallation
84
31.4
77.7
32.5
25.7
25.7
35.2
34.4
31.1
CityFurniture
193
78.5
77.4
73.9
78.8
93.5
75.5
79.1
DoorSurface
37
33.3
27.4
27.0
36.8
34.3
32.3
OuterFloorSurface
1
16.5
33.1
26.2
25.0
SolitaryVegetationObject
39
15.6
25.3
29.4
25.0
Table 5: Mean distance dQ3 between the radiometric fingerprints of objects grouped by CityGML class indicating class-specific material composition patterns.
function of object
count
bench
5
25
28
28
40
53
20
70
49
50
96
123
119
70
22
controller box
4
27
37
35
60
31
58
50
35
83
106
102
62
22
fence
3
34
47
51
24
80
51
58
105
132
129
76
31
other
11
47
66
42
62
58
41
82
102
99
68
34
pole
4
79
47
89
65
75
103
122
120
82
54
railing
13
12
78
48
57
103
131
127
73
22
Table 6: Mean distance dQ3 between radiometric fingerprints of CityFurniture objects grouped by function demonstrating function-specific material patterns.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Specification
Channels
16
Field of View: Vertical
30\SIUnitSymbolDegree ( −15\SIUnitSymbolDegree to 15\SIUnitSymbolDegree )
Measurement range
100m
Field of View: Horizontal
360\SIUnitSymbolDegree
Rotation Rate
520Hz ( 10Hz used for experiments)
Range accuracy
up to ± 3cm (typical)
Appendix
Table 7: Specifications of the VLP-16 LiDAR sensor by Velodyne Lidar, Inc. (2022) .
Figure 25Figure 26
Parameter
Value
Line segment length ℓ
1m
Spherocylinder radius ρ
5cm
Max. object surface associations per point
1
Appendix
Table 8: Parameter values utilized for the beam-surface association process.
Figure 28
Parameter
Value
Range rk
[0m,15m)
Zenith angle θk
{[0∘,20∘),[20∘,40∘),[40∘,60∘),[60∘,90∘)}
Azimuth angle ϕk
[0,2π)
Campaignk
campaign 1
Sensork
rear-left sensor
Objectk
selected objects highlighted in Figure 17
Appendix
Table 9: Filters applied according to subsection 4.6 for extracting radiometric fingerprints, as discussed in Section 4.6 and illustrated in Figure 17 and Table 4 .
Parameter
Value
Range rk
[0m,15m)
Zenith angle θk
{[0∘,20∘),[20∘,40∘),[40∘,60∘),[60∘,90∘)}
Azimuth angle ϕk
[0,2π)
Campaignk
campaign 1
Sensork
front-center sensor
Objectk
–
Appendix
Table 10: Filters applied according to subsection 4.6 for extracting radiometric fingerprints, as discussed in Section 4.7 and listed in Table 5 and 6 .
The paradigm of digital twin cities is shifting from coarse visual mapping toward more precise and actionable digitization of urban assets. However, existing datasets predominantly focus on coarse visual perception, lacking the strict multi-modal alignment and attribute and status diagnosis required for automated infrastructure maintenance. To bridge this gap, we introduce WHU-Infra3D, a large-scale, multi-modal benchmark dataset dedicated to roadside infrastructure inventory. Covering 53.8 km across three cities, WHU-Infra3D uniquely integrates panoramic imagery and LiDAR point clouds with rigorous 2D-3D instance association and cross-frame tracking. Comprising over 175k multi-view 2D bounding boxes alongside thousands of 3D infrastructure instances, the dataset provides over 181k detailed attribute and status annotations (e.g., rust, occlusion) to empower operational health assessment. We establish comprehensive baselines across five core tasks: 2D detection, 2D cross-view matching, 3D geo-identification, 3D point cloud segmentation, and attribute recognition. Extensive evaluations expose significant cross-city domain gaps and inherent vulnerabilities of current models on long-tailed defective statuses, establishing WHU-Infra3D as an essential testbed for advancing scalable, AI-driven urban infrastructure inventory and lifecycle management. The WHU-Infra3D dataset is available at https://github.com/WHU-USI3DV/WHU-Infra3D.
Chong Liu, Luxuan Fu, Xuyu Feng +2
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan 430079, China
We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42% (dense) and 89.67% (sparse).
Hojung Jung, Yuki Oto, Oscar M. Mozos +2
Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan · Polytechnic University of Cartagena (UPCT), Spain · Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan
Semantic segmentation of 3D LiDAR point clouds is important in urban remote sensing for understanding real-world street environments. This task, by projecting LiDAR point clouds and 3D semantic labels as sparse maps, can be reformulated as a 2D problem. However, the intrinsic sparsity of the projected LiDAR and label maps can result in sparse and inaccurate intermediate 2D semantic predictions, which in return limits the final 3D accuracy. To address this issue, we enhance this task by shaping dense and accurate 2D predictions. Specifically, we develop a multi-modal segmentation model, MM2D3D. By leveraging camera images as auxiliary data, we introduce cross-modal guided filtering to overcome label map sparsity by constraining intermediate 2D semantic predictions with dense semantic relations derived from the camera images; and we introduce dynamic cross pseudo supervision to overcome LiDAR map sparsity by encouraging the 2D predictions to emulate the dense distribution of the semantic predictions from the camera images. Experiments show that our techniques enable our model to achieve intermediate 2D semantic predictions with dense distribution and higher accuracy, which effectively enhances the final 3D accuracy. Comparisons with previous methods demonstrate our superior performance in both 2D and 3D spaces.
Xiaoyu Dong, Tiankui Xian, Wanshui Gan +1
The University of Tokyo, Tokyo, Japan · RIKEN AIP, Tokyo, Japan