Balancing Real and Synthetic Data for CNN-based Masonry Crack Detection
Authors: Mattia Forlesi, Alfonso Esposito, Ivan Zyrianoff, Alessandro Marzani, Marco Di Felice
Abstract
Cracks are a critical indicator of building health, and early stage identification is fundamental to prevent harmful damages. Advances in deep learning (DL), particularly convolutional neural networks (CNNs), have enabled scalable solutions for automated crack detection. However, CNN performance strongly depends on the availability of large and diverse datasets, which is particularly challenging for complex surfaces such as masonry. Collecting sufficient real data is time-consuming, while publicly available datasets may not be adequate. To address this limitation, we explored generating synthetic crack data, which complements real data and improves training effectiveness. The real dataset consists of masonry crack images collected from buildings in Bologna and surrounding areas. In contrast, the synthetic dataset was generated using a crack overlay tool that adds cracks to background images in a controlled orientation and placement. The real dataset was used to train several DL architectures, to identify the best-performing model (InceptionV4) employed for experiments with generated data. Six training scenarios were tested in InceptionV4 by varying the ratio of real and synthetic data, with evaluation performed on a test set composed of real images using the F1-score and mean Intersection over Union (mIoU) metrics. Results show that training on synthetic data plus a modest addition of 20% real data achieves results comparable to training on real data only. Moreover, the 20/80 scenario (synthetic/real) achieved an 76% F1-score and 80% mean IoU, outperforming the real-only case. As can be seen, the method demonstrates the potential of synthetic data to reduce collection efforts while enhancing crack detection accuracy.
Reliable crack detection and segmentation are vital for structural health monitoring, yet the scarcity of well-annotated data constitutes a major challenge. To address this limitation, we propose a novel context-aware generative framework designed to synthesize realistic crack growth patterns for data augmentation. Unlike existing methods that primarily manipulate textures or background content, CrackForward explicitly models crack morphology by combining directional crack elongation with learned thickening and branching. Our framework integrates two key innovations: (i) a contextually guided crack expansion module, which uses local directional cues and adaptive random walk to simulate realistic propagation paths; and (ii) a two-stage U-Net-style generator that learns to reproduce spatially varying crack characteristics such as thickness, branching, and growth. Experimental results show that the generated samples preserve target-stage saturation and thickness characteristics and improve the performance of several crack segmentation architectures. These results indicate that structure-aware synthetic crack generation can provide more informative training data than conventional augmentation alone.
Vision-based crack inspection depends on segmentation networks whose reliability depends on the quantity, diversity and label quality of their training data. Pixel-level annotations are costly, and crack images of specific structures are scarce. Generative augmentation can supply additional data, but existing methods address isolated steps. They reuse annotated masks, offer limited control over crack geometry, and adopt the conditioning mask as the label without checking it. This paper presents an end-to-end pipeline that produces labelled crack data without manual annotation and assesses the reliability of these data and of the detectors trained on them. Procedurally sampled Bézier skeletons with guaranteed geometric properties are converted into crack masks by a generative adversarial network (GAN). A dual-ControlNet Stable Diffusion model renders the masks as crack images, either on text-described surfaces or on user-provided backgrounds. An ensemble of segmentation networks trained on real images combines its agreement with the inherited label and its internal disagreement into a pixel-wise label confidence. This confidence weights the training loss instead of removing samples with a threshold. The trained detectors are evaluated with image-space probability of detection (POD) and calibration analyses. On CRACK500 and CrackTree200, the pipeline improves five segmentation networks over conventional, diffusion-based and flow-matching-based augmentation, and on CRACK500 confidence weighting yields a higher accuracy than threshold filtering at every tested threshold. On CRACK500, the crack width that U-Net detects with 90% probability at 95% confidence decreases from 8.0 to 4.3 pixels, and the expected calibration error decreases from 14.2% to 9.6%.
While automated defect detection such as the detection of surface scratched is an important aspect in industrial quality control, the scarcity of annotated defect data make this task challenging. This paper presents a procedural rendering pipeline that generates large-scale annotated synthetic training data using BlenderProc, with configurable material appearance, camera modes, and domain randomization, producing automatic COCO-format annotations. To show the potential of our approach, we evaluate four training strategies, namely synthetic-only, real-only, mixed, and fine-tuning from synthetic weights, across two objects with different material properties and three lightweight edge-deployable detectors, YOLOX, YOLO26, and LW-DETR. Our evaluation show that fine-tuning from synthetic weights consistently outperforms real-only training, and that mixed training effectively recovers performance under scarce real-data conditions, with findings validated across both convolutional and transformer-based architectures. The proposed approach enables scalable defect detection without the burden of large real annotated datasets, making it practical for on-device industrial inspection. The pipeline scripts, 3D model, and both synthetic and real annotated scratch datasets for a glossy toy Ferrari car will be made available through the project website upon acceptance.
Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist +3