cs.CVMar 14, 2026

Efficient Semi-Automated Material Microstructure Analysis Using Deep Learning: A Case Study in Additive Manufacturing

Authors: Sanjeev S. Navaratna, Nikhil Thawari, Gunashekhar Mari, Amritha V P, Murugaiyan Amirthalingam, Rohit Batra

Organizations: Department of Metallurgical and Materials Engineering, Indian Institute of Technology Madras, Chennai 600036, India · Center for Atomistic Modelling and Materials Design, IIT Madras, Chennai 600036, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Image segmentation is fundamental to microstructural analysis for defect identification and structure-property correlation, yet remains challenging due to pronounced heterogeneity in materials images arising from varied processing and testing conditions. Conventional image processing techniques often fail to capture such complex features rendering them ineffective for large-scale analysis. Even deep learning approaches struggle to generalize across heterogeneous datasets due to scarcity of high-quality labeled data. Consequently, segmentation workflows often rely on manual expert-driven annotations which are labor intensive and difficult to scale. Using an additive manufacturing (AM) dataset as a case study, we present a semi-automated active learning based segmentation pipeline that integrates a U-Net based convolutional neural network with an interactive user annotation and correction interface and a representative core-set image selection strategy. The active learning workflow iteratively updates the model by incorporating user corrected segmentations into the training pool while the core-set strategy identifies representative images for annotation. Three subset selection strategies, manual selection, uncertainty driven sampling and proposed maximin Latin hypercube sampling from embeddings (SMILE) method were evaluated over six refinement rounds. The SMILE strategy consistently outperformed other approaches, improving the macro F1 score from 0.74 to 0.93 while reducing manual annotation time by about 65 percent. The segmented defect regions were further analyzed using a coupled classification model to categorize defects based on microstructural characteristics and map them to corresponding AM process parameters. The proposed framework reduces labeling effort while maintaining scalability and robustness and is broadly applicable to image based analysis across diverse materials systems.

Figures & tables

Figure 1: Semi-automated pipeline for microstructure image analysis. The workflow consists of two stages: defect detection and defect classification. In the defect detection stage, an active learning framework combined with a subset selection strategy and a user-friendly annotation interface is employed to segment regions of microstructural images containing defects. The subset selection strategy ensures the use of diverse and representative images, active learning enables iterative improvement of the detection model, and the annotation tool allows users to efficiently correct model-predicted labels rather than generating ground-truth annotations from scratch. Subsequently, in the defect classification stage, the obtained segmented defect regions are mapped to corresponding etched microstructural images, containing critical microstructural features such as melt pool boundaries and grain morphology. This combined information is used to train a defect classification model, and the resulting defect classification information is mapped to associated manufacturing process parameters.
Figure 2: Defect detection model performance. (a) Iterative improvement in the macro F1 score of the defect detection model with increasing active learning rounds for different subset selection strategies. Results obtained using the baseline Otsu thresholding method are also shown for comparison. Markers denote the mean performance on the test set, and error bars indicate ±1σ\pm 1\sigma standard deviation. The dashed line and shaded region represent the mean macro F1 score and ±1σ\pm 1\sigma standard deviation of the Otsu method, respectively. (b) Example segmentation results produced by the final best-performing SMILES-based model on test images.
Figure 3: Diversity and coverage of subset selection strategies and Representative Grad-CAM visualization using the final SMILE-trained model. (a) Isomap projection of the full training and the independent test set, highlighting image samples selected using manual, deep ensemble, and SMILE-based approaches across all active learning iterations. Regions that are either underrepresented or overrepresented by different selection strategies are also highlighted. (b) t-SNE projection of the image samples selected using the SMILE approach. While selection of points across various clusters promotes coverage of the embedding space, the LHS + maximin criteria enforces intra-cluster diversity. (c) The left panel shows the original microstructural image, while the right panel displays the Grad-CAM heatmap overlaid on the image. High activation regions are localized around defect areas, confirming that the network focuses on physically meaningful microstructural features rather than background intensity variations.
Image No.Manual (min.)Model (min.)
A1A2A3A1A2A3
14535301277
2302525201310
32515151595
4302015854
510108112
Table 1: Annotation time savings . Comparison of manual and model-assisted annotation time (in minutes) across three annotators (A1–A3) for five test images. The model-assisted approach reduced the annotation time by an average of 65%.
Figure 4: Defect classification model performance and defect statistics measured across different AM process parameters. (a) Confusion matrix summarizing performance of the defect classification model on the test set. (b) Representative test patches comparing ground-truth and predicted labels, illustrating both correct and incorrect classifications. Defect statistics measured across different laser power and scan speed combinations for (c) Inconel 625 and (d) CoCrMo alloy AM systems. Each cell represents one processing condition and contains two bar plots, one capturing relative fraction of lack of fusion (blue) and porosity (orange), with the white colored number indicating the porosity fraction, and second bar denoting % defect area fraction (red color) with the average number of defects reported in brackets. The results highlight clear variations in defect type and severity across processing conditions, demonstrating the strong influence of process parameters on defect formation.
More in this topic

Interactive Image Segmentation

See all 11 papers

Explore similar work

CardsList
Jun 18, 2026

Speeding up the annotation process in semantic segmentation industrial applications

Current machine learning models commonly require large and well-annotated datasets. However, the annotation process often becomes a bottleneck, with increased complexity leading to higher chances of human errors. Within this context, our goal in this paper is to leverage unsupervised algorithms to improve data annotation efficiency for complex semantic segmentation problems in industrial materials science. Previous research has quantified labeling time and others explored unsupervised methods. However, to the best of our knowledge, this is the first study to quantify how much unsupervised algorithms accelerate the labeling process. We aim to validate the extent to which this laborious process can be accelerated, focusing on semantic segmentation tasks that involve annotating each pixel of high-resolution images, such as the microstructure characterization challenge in materials science. Specifically, we demonstrate that by using unsupervised computer vision algorithms, the time required for the labeling process can be reduced from 170 hours to 37 hours, achieving an approximate reduction of 78%. The dataset we work with includes large images of dimensions 1280x959 and 960x703, which further increases the complexity of the annotation task. Despite these challenges, we create and share the largest public steel microstructure segmentation dataset to date, available under MIT License with permanent DOI, contributing a fully annotated, high-resolution dataset to the field. Additionally, this is the first work to compare the labeling time from scratch (a common approach in previous studies) to the labeling time when using these unsupervised algorithms as a pre-annotation step. Furthermore, we provide a Deep Learning model trained on this dataset, validated by field experts, and deployed in an industrial setting, serving as an initial benchmark for this public dataset.

Marta Fernandez-Moreno, Margarita Guerrero, Rosalia Rementeria +2

Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence, DaSCI, University of Granada, 18071, Granada, Spain · Ingénica STS, Av. Pablo Iglesias 38, Bj, 33205, Gijón, Spain · ArcelorMittal Global R&D, SLab—Steel Labs, Calle Marineros 4, 33490, Avilés, Spain +1

Human-in-the-Loop AnnotationSemantic SegmentationUnsupervised Image Segmentation
AskPDF
Aug 22, 2025

EM3M: An Electron Micrograph Dataset for Microstructural Segmentation and Generation

Quantitative microstructural characterization is fundamental to materials science, and electron micrographs (EMs) provide indispensable high-resolution insights. However, progress in deep learning-based analysis of EMs has been hampered by the scarcity of large-scale, expert-annotated public datasets. To address this issue, we introduce EM3M, a large-scale and multimodal dataset for instance-level understanding of EMs. EM3M comprises 5,091 high-quality EMs, approximately 3 million instance segmentation annotations, and image-level textual descriptions with disentangled attributes. The dataset is constructed through a rigorous multi-stage curation and validation pipeline, with comprehensive statistical analyses to ensure reliability and reproducibility. Building upon these curated image-text pairs, we further provide a text-to-image diffusion model that serves as a controllable data augmentation engine, demonstrating that synthetic augmentation consistently improves downstream segmentation performance. To establish a systematic benchmark, we evaluate representative instance segmentation methods on EM3M. Our results reveal that conventional detection-based and query-based methods struggle with the extreme instance densities and textural complexities inherent in EMs. We additionally provide an optimized flow-based baseline to facilitate fair comparison and future research. EM3M {Dataset: https://huggingface.co/datasets/UniParser/EM3M}, the generative engine {Generation: https://huggingface.co/UniParser/EM3M-Gen}, and an online demo {Segmentation demo: https://www.bohrium.com/apps/uni-aims} are publicly available to support future research in automated materials analysis.

Nan Wang, Zhiyi Xia, Yiming Li +7

DP Technology, Beijing, China

Synthetic Data AugmentationInstance SegmentationMicroscopy Image AnalysisElectron Microscopy
AskPDF
Jul 27, 2026

Unifying Active Learning and Semi-Supervised Learning for Medical Image Segmentation

In practical settings, medical image segmentation models are often developed with limited annotated data rather than fully labeled datasets. Training frequently begins in ultra-low labeled regimes where only a small number of volumes are annotated. In such scenarios, practitioners must simultaneously decide which cases to annotate and how to best use the remaining unlabeled data. Although active learning (AL) and semi-supervised learning (SSL) both target annotation scarcity, they are typically designed and optimized independently, resulting in objective mismatch and unstable training during early-stage "cold start" conditions. We propose RegAL, a unified active semi-supervised framework governed by a shared topology-aware Pareto optimization that couples sample acquisition with unlabeled data utilization. RegAL evaluates images along three complementary axes, voxel-wise uncertainty, feature diversity, and a novel topological consistency metric, to select anatomically informative edge cases for annotation. On the other hand, the same criteria are used to identify geometrically stable atlas candidates for diffeomorphic registration-guided augmentation to train a self-supervised Mean Teacher segmentation network. Across BraTS 2021, dHCP, and ProstateX, RegAL remains stable with few labeled volumes and consistently outperforms state-of-the-art AL, SSL, and active semi-supervised baselines across Dice and boundary-distance (ASD, HD95) metrics under extreme annotation scarcity.

Bahram Jafrasteh, Cheng Wan, Heejong Kim +2

Department of Radiology, Weill Cornell Medicine, New York, NY, USA · Department of Electrical and Computer Engineering, Cornell University, Ithaca, NY, USA

Active LearningSemi-Supervised Medical Image SegmentationMedical Image Segmentation
AskPDF