Image-Guided Pavement Defect Recognition in GPR Data with novel 3D Deep Learning Architecture
Authors: Yuandong Pan, Linjun Lu, Mudan Wang, Florian Noichl, Fan Xue, Brian Sheil, Lavindra de Silva, André Borrmann, Ioannis Brilakis
Organizations: Postdoctoral scholar, School of Engineering, Stanford University, 473 Via Ortega, Stanford · Research fellow, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Assistant research professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Postdoctoral Scholar, School of Engineering and Design, Technical University of Munich, Arcisstrasse 21, Munich · Associate professor, Department of Real Estate and Construction, University of Hong Kong, Pok Fu Lam Rd, Hong Kong · Associate professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Research professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Professor, School of Engineering and Design, Technical University of Munich, Arcisstrasse 21, Munich · Professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge
Ground Penetrating Radar (GPR) is a widely adopted non-destructive sensing technology for subsurface inspection in civil and transportation engineering. Despite its potential for pavement condition assessment, the large-scale application of GPR in automated inspection has two key challenges: the scarcity of annotated real-world datasets and the lack of deep learning models designed for the unique characteristics of 3-Dimensional (3D) GPR data. This study addresses these limitations by firstly introducing a cost-effective data preparation pipeline that integrates orthomosaic Red Green Blue (RGB) imagery with 3D GPR scans to generate annotated 3D GPR datasets. The proposed method uses the aligned segments of RGB and GPR data, using pavement surface images as a reference to transfer labels of surface-visible defects to corresponding GPR segments, enabling efficient large-scale annotation in a real-world dataset collected on a highway section under operation. In addition to the dataset contribution, we propose a specialised 3D Convolutional Neural Network (CNN) architecture incorporating residual connections, mixed convolutional kernel sizes, and both depthwise and channelwise attention mechanisms to enhance feature representation and defect classification. The model is evaluated on binary classification tasks for detecting patch and crack defects in pavement structures. Experimental results demonstrate that the proposed network outperforms baseline architectures across multiple evaluation metrics. Ablation studies further confirm the effectiveness of the designed architectural components. This work contributes a scalable and practical method for real-world dataset generation, along with a novel deep learning framework.
Figures & tables
Figure 1: Equipment used in CAMHighways Dataset [ 7 ]
Figure 2: Orthomosaics built from the pavement images and defect masks
Figure 3: Illustration of spatial misalignment between orthomosaics and GPR data for the same road section
Figure 4: Exemplar raw GPR data collected from two lanes of road section
Figure 5: Extraction of overlapping area from example orthomosaics and GPR data
Figure 6: Extracted orthomosaics and GPR data layers for one lane
Figure 7: Overlapping area of orthomosaics and GPR coverage and corresponding fitted centerline
Figure 8: Composite dataset of one lane divided into segments using one moving plane orthogonal to lane direction
dp,i=∣(xp−xi)⋅nyi−(yp−yi)⋅nxi∣
Algorithm 1 Assign GPR Points to Nearest Segment
Figure 9: Visualisation of prepared data segments
Figure 10: Proposed network architecture for GPR segment classification tasks
Figure 11: RGB-GPR segments with annotations
Class
Training Set
Test Set
Total
Patch
652
220
872
Crack
633
140
773
No defect
800
300
1100
Total
2085
660
2745
Table 1: Sample number of different classes in prepared dataset
Figure 12: Training curves for proposed network
Defect
Model
Accuracy
Precision
Recall
F1
Patch
Resnet18
59.6
58.9
59.0
58.9
Resnet18 (pretrain)
59.3
59.4
59.4
58.8
3D Conv
67.6
66.9
65.8
65.9
Proposed network
69.0
68.2
68.0
68.1
Crack
Resnet18
54.9
52.4
52.7
51.8
Resnet18 (pretrain)
54.0
51.4
51.6
50.7
Table 2: Prediction performance of various networks on test set (%)
Defect type
Class
Precision
Recall
F1
Patch
Normal
72.5
75.7
74.1
Patch
64.7
60.9
62.8
Average
68.6
68.3
68.4
Crack
Normal
78.5
77.7
78.1
Crack
53.2
54.3
53.7
Average
65.8
66.0
65.9
Table 3: Classwise comparison for patch and crack classification (%)
Figure 13: Confusion matrix for patch and crack classification tasks
Model
Accuracy
Precision
Recall
F1
3DCNN
67.6
66.9
65.8
65.9
Residual block
66.8
66.0
65.9
66.0
Depth attention
68.3
67.4
67.2
67.3
Channel attention
67.5
66.9
66.7
66.6
Mixed kernel size
66.7
65.9
65.8
65.8
Proposed network
69.0
68.2
68.0
68.1
Table 4: Ablation study of individual techniques on patch classification (%)
Layer Number
Accuracy
Precision
Recall
F1
5
67.5
66.8
65.9
65.9
10
68.1
67.3
67.0
67.1
15
68.5
67.7
67.5
67.5
20
69.0
68.2
68.0
68.1
Table 5: Results of evaluating different number of input layers on proposed model for patch classification (%)
Layer Number
Accuracy
Precision
Recall
F1
5
66.1
60.6
55.5
53.4
10
70.6
65.8
64.1
64.4
15
70.0
65.5
65.5
65.4
20
69.0
68.2
68.0
68.1
Table 6: Results of evaluating different number of input layers on proposed model for crack classification (%)
Three-dimensional convolutional neural networks (3D CNNs) learn spatial and spatiotemporal features from volumetric images, video, and other structured data.
Post-Doctoral Research Fellow, Federal Highway Administration, Turner-Fairbank Highway Research Center, McLean, VA 22101 · Nondestructive Evaluation Program and Laboratory Manager, Federal Highway Administration, Turner-Fairbank Highway Research Center, McLean, VA 22101
AI Research Group, Department of Civil Engineering, Faculty of Engineering, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand · Center for Disaster Management Informatics Research, Ehime University, Matsuyama, Japan · Canaan Geo Research Co., Ltd., Matsuyama, Ehime, Japan
Dept. of Electrical Engineering & Computer Science Sir Syed CASE Institute of Technology Islamabad, Pakistan · Dept. of Computer & Information Sciences University of Strathclyde Glasgow, UK