Image-Guided Pavement Defect Recognition in GPR Data with novel 3D Deep Learning Architecture
Authors: Yuandong Pan, Linjun Lu, Mudan Wang, Florian Noichl, Fan Xue, Brian Sheil, Lavindra de Silva, André Borrmann, Ioannis Brilakis
Organizations: Postdoctoral scholar, School of Engineering, Stanford University, 473 Via Ortega, Stanford · Research fellow, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Assistant research professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Postdoctoral Scholar, School of Engineering and Design, Technical University of Munich, Arcisstrasse 21, Munich · Associate professor, Department of Real Estate and Construction, University of Hong Kong, Pok Fu Lam Rd, Hong Kong · Associate professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Research professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge · Professor, School of Engineering and Design, Technical University of Munich, Arcisstrasse 21, Munich · Professor, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge
Ground Penetrating Radar (GPR) is a widely adopted non-destructive sensing technology for subsurface inspection in civil and transportation engineering. Despite its potential for pavement condition assessment, the large-scale application of GPR in automated inspection has two key challenges: the scarcity of annotated real-world datasets and the lack of deep learning models designed for the unique characteristics of 3-Dimensional (3D) GPR data. This study addresses these limitations by firstly introducing a cost-effective data preparation pipeline that integrates orthomosaic Red Green Blue (RGB) imagery with 3D GPR scans to generate annotated 3D GPR datasets. The proposed method uses the aligned segments of RGB and GPR data, using pavement surface images as a reference to transfer labels of surface-visible defects to corresponding GPR segments, enabling efficient large-scale annotation in a real-world dataset collected on a highway section under operation. In addition to the dataset contribution, we propose a specialised 3D Convolutional Neural Network (CNN) architecture incorporating residual connections, mixed convolutional kernel sizes, and both depthwise and channelwise attention mechanisms to enhance feature representation and defect classification. The model is evaluated on binary classification tasks for detecting patch and crack defects in pavement structures. Experimental results demonstrate that the proposed network outperforms baseline architectures across multiple evaluation metrics. Ablation studies further confirm the effectiveness of the designed architectural components. This work contributes a scalable and practical method for real-world dataset generation, along with a novel deep learning framework.
Figures & tables
Figure 1: Equipment used in CAMHighways Dataset [ 7 ]
Figure 2: Orthomosaics built from the pavement images and defect masks
Figure 3: Illustration of spatial misalignment between orthomosaics and GPR data for the same road section
Figure 4: Exemplar raw GPR data collected from two lanes of road section
Figure 5: Extraction of overlapping area from example orthomosaics and GPR data
Figure 6: Extracted orthomosaics and GPR data layers for one lane
Figure 7: Overlapping area of orthomosaics and GPR coverage and corresponding fitted centerline
Figure 8: Composite dataset of one lane divided into segments using one moving plane orthogonal to lane direction
dp,i=∣(xp−xi)⋅nyi−(yp−yi)⋅nxi∣
Algorithm 1 Assign GPR Points to Nearest Segment
Figure 9: Visualisation of prepared data segments
Figure 10: Proposed network architecture for GPR segment classification tasks
Figure 11: RGB-GPR segments with annotations
Class
Training Set
Test Set
Total
Patch
652
220
872
Crack
633
140
773
No defect
800
300
1100
Total
2085
660
2745
Table 1: Sample number of different classes in prepared dataset
Figure 12: Training curves for proposed network
Defect
Model
Accuracy
Precision
Recall
F1
Patch
Resnet18
59.6
58.9
59.0
58.9
Resnet18 (pretrain)
59.3
59.4
59.4
58.8
3D Conv
67.6
66.9
65.8
65.9
Proposed network
69.0
68.2
68.0
68.1
Crack
Resnet18
54.9
52.4
52.7
51.8
Resnet18 (pretrain)
54.0
51.4
51.6
50.7
Table 2: Prediction performance of various networks on test set (%)
Defect type
Class
Precision
Recall
F1
Patch
Normal
72.5
75.7
74.1
Patch
64.7
60.9
62.8
Average
68.6
68.3
68.4
Crack
Normal
78.5
77.7
78.1
Crack
53.2
54.3
53.7
Average
65.8
66.0
65.9
Table 3: Classwise comparison for patch and crack classification (%)
Figure 13: Confusion matrix for patch and crack classification tasks
Model
Accuracy
Precision
Recall
F1
3DCNN
67.6
66.9
65.8
65.9
Residual block
66.8
66.0
65.9
66.0
Depth attention
68.3
67.4
67.2
67.3
Channel attention
67.5
66.9
66.7
66.6
Mixed kernel size
66.7
65.9
65.8
65.8
Proposed network
69.0
68.2
68.0
68.1
Table 4: Ablation study of individual techniques on patch classification (%)
Layer Number
Accuracy
Precision
Recall
F1
5
67.5
66.8
65.9
65.9
10
68.1
67.3
67.0
67.1
15
68.5
67.7
67.5
67.5
20
69.0
68.2
68.0
68.1
Table 5: Results of evaluating different number of input layers on proposed model for patch classification (%)
Layer Number
Accuracy
Precision
Recall
F1
5
66.1
60.6
55.5
53.4
10
70.6
65.8
64.1
64.4
15
70.0
65.5
65.5
65.4
20
69.0
68.2
68.0
68.1
Table 6: Results of evaluating different number of input layers on proposed model for crack classification (%)
Three-dimensional convolutional neural networks (3D CNNs) learn spatial and spatiotemporal features from volumetric images, video, and other structured data.
The research explores the pioneering integration of Physics-Informed Neural Networks (PINNs) into the domain of Ground-Penetrating Radar (GPR) data prediction. This research presents a detailed development framework for a specialized PINN model, proficient at interpreting and forecasting GPR data, much like how medical imaging models predict tumor behavior. By harnessing the synergy between deep learning algorithms and the physical laws governing subsurface structures or in medical terms, human tissues the model effectively embeds the physics of electromagnetic wave propagation into its architecture. This ensures that predictions not only align with fundamental physical principles but also mirror the precision needed in medical diagnostics for detecting and monitoring tumors. The suggested deep learning structure comprises three components: a CNN, a spatial feature channel attention (SFCA) mechanism, and ConvLSTM, along with temporal feature frame attention (TFFA) modules. The attention mechanism computes channel attention and temporal attention weights using self-adaptation, thereby fine tuning the visual and temporal feature responses to extract the most pertinent and significant visual and temporal features. By integrating physics directly into the neural network, our model has shown enhanced accuracy in forecasting GPR data. This improvement is vital for conducting effective assessments of bridge deck conditions and other evaluations related to civil infrastructure. The use of Physics Informed Neural Networks (PINNs) has demonstrated the potential to transform the field of Non-Destructive Evaluation (NDE) by enhancing the precision of infrastructure deterioration predictions. Moreover, it offers a deeper insight into the fundamental mechanisms of deterioration, viewed through the prism of physics-based models.
Mehrdad Shafiei Dizaji, Hoda Azari
Post-Doctoral Research Fellow, Federal Highway Administration, Turner-Fairbank Highway Research Center, McLean, VA 22101 · Nondestructive Evaluation Program and Laboratory Manager, Federal Highway Administration, Turner-Fairbank Highway Research Center, McLean, VA 22101
Automated detection of subsurface cavities from Ground Penetrating Radar (GPR) is most difficult in soft, high-water-content ground, where conductive, water-saturated soil attenuates the signal and degrades cavity reflections, yet this is also the condition under which cavities most readily form. This paper proposes TriView-YOLO, a multi-view YOLOv12 detector for road cavity screening in such ground. Three co-registered views (longitudinal B-scan, horizontal C-scan, and cross-section B-scan) form a 9-channel input fused by a TripleInputConv layer that replaces the YOLOv12 stem; the rest of the network is unchanged, and bounding boxes are required on the longitudinal view only. Training used 1,600 expert-verified field samples, principally metropolitan road surveys of Bangkok, Thailand, acquired with a vehicle-mounted multichannel three-dimensional GPR mobile mapping system, with surveys over the firmer subgrades of Japan added to training and validation only. The test set comes exclusively from the Bangkok surveys, over soft marine clay with 80-140% water content and a water table at 1-2 m depth, a ground condition for which no dedicated deep learning cavity-detection evaluation has been reported. On this unaugmented, field-only test set, split randomly within surveys, the proposed model attains mAP50 of 0.558 +/- 0.028 over three seeds at 23.6 GFLOPs and 3.1 ms per image. Ablations show that removing the auxiliary views lowers mAP50 and recall, whereas public and synthetic training images, DINOv3 features, larger model scale, and COCO pretraining bring no gain.
AI Research Group, Department of Civil Engineering, Faculty of Engineering, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand · Center for Disaster Management Informatics Research, Ehime University, Matsuyama, Japan · Canaan Geo Research Co., Ltd., Matsuyama, Ehime, Japan
Clandestine tunneling beneath oil and gas pipelines enables fuel theft, smuggling, and sabotage, yet conventional monitoring detects damage only after a pipeline has been compromised. Ground-penetrating radar (GPR) can image such tunnels non-invasively, but manual radargram interpretation does not scale to continuous corridor surveillance, and supervised detectors require tunnel examples that are scarce in practice. We present a fully unsupervised detection pipeline trained exclusively on normal subsurface radargrams collected at a purpose-built field site containing three buried tunnels at 1.5-3 m depth. A denoising convolutional autoencoder learns the structure of anomaly-free ground; at inference, tunnels are flagged by reconstruction error. Our central contribution is a depth-restricted top-k anomaly score, which pools the highest reconstruction errors only within the depth band where tunnels can physically occur. This physically motivated rule raises AUC from 0.986 to 0.994 and cuts missed detections from 74 to 17 of 634 tunnel windows, relative to whole-image scoring, without any retraining or labels. We further show that the optimal top-k fraction interacts with the depth restriction - 1% pooling is best on full images, 5% once scoring is depth-restricted - and that spatial voting across overlapping survey windows helps weak per-image detectors but offers no benefit once the scoring rule is strong. The final system attains AUC 0.994, F1 0.975, recall 0.973, and precision 0.976 on 1,600 field test windows spanning 55 survey lines, at a 1.6% false-alarm rate, using no tunnel labels for training, scoring, or threshold calibration.
Muhammad Junaid, Shoab A. Khan, Nisar Ahmed
Dept. of Electrical Engineering & Computer Science Sir Syed CASE Institute of Technology Islamabad, Pakistan · Dept. of Computer & Information Sciences University of Strathclyde Glasgow, UK