ScrewSense: Mapping the Invisible - Anomaly Detection and Error Classification in Screwdriving Processes
Authors: Tobias Lettner, Tatyana Ivanovska, Anwar Al Assadi, Christian Bergler
Organizations: Department of Electrical Engineering, Media and Computer Science, Ostbayerische Technische Hochschule Amberg-Weiden, Amberg, Germany · Fraunhofer Institute for Manufacturing Engineering and Automation, Stuttgart, Germany
The growing volume of process data generated by screwdriving systems, combined with advancements in computing power, data storage, and algorithms, is facilitating the application of artificial intelligence in this field. Accurately detecting and classifying screwdriving errors is crucial for ensuring product quality, avoiding costly rework in manufacturing, and supporting sustainability by facilitating the maintenance, reuse, and recycling of screw-connected products. In this context, machine learning approaches offer a promising alternative to traditional monitoring techniques, such as window-based methods or envelope curves. This study examines the use of deep learning techniques for anomaly detection and error classification in screwdriving processes. The data corpus consists of multivariate time series capturing process parameters, such as torque and rotation angle. For anomaly detection, the embedding space of a vector quantized variational autoencoder was analyzed. In terms of error classification, four different model architectures were evaluated. The results demonstrate the effectiveness of anomaly detection and confirm that deep learning models can distinguish between multiple error categories beyond traditional binary classification. This extends the capabilities of conventional monitoring approaches, achieving accuracies of up to 100%. Additionally, real-world data material was examined in order to further demonstrate model robustness. Overall the results indicate strong generalization capability and highlight the practical potential of the developed models for use in industrial screwdriving processes, eliminating the need for expert knowledge in error identification and significantly reducing manual effort and analysis time.
Figures & tables
Class
assm_a
dism_a
reasm_a
Class
dism_b
dism_c
OK
58
59
59
OK
–
50
BURSTING_TUBE
48
48
46
REFERENCE
485
–
TIGHT_SCREW
–
50
50
REFERENCE_LARGE
372
–
CROSS_THREADING
–
–
47
INCORRECT_DRIVE
480
101
DAMAGED_SCREW_DRIVE
–
51
–
TIGHT_SCREW
46
50
SCREW_EXTRACTION
51
53
–
DAMAGED_HEAD_LARGE
381
–
Table 1: Class distribution of all datasets used in this study, showing the available classes and the corresponding number of samples per dataset. The dataset identifiers follow the naming convention assm (assembly), dism (dismantling), and reasm (reassembly), with suffixes a , b , and c denoting different system configurations.
Fig. 1 : Example of image conversion results. From left to right: original input, density, GAF, and spectrogram representations of the torque signal. Each image has a size of 224×224 pixel. The model uses three-channel inputs, with torque, rotation angle, and time as features. Density images use a 224×224 bin size; GAF is generated with the summation method; spectrogram parameters are nfft = 512, hop_length = 32, and a Hamming window.
Fig. 2 : Embedding space of the dism_a dataset. Each point represents a screwdriving sample described by torque, rotation angle, and time, with colors indicating class labels. Multiple anomalies occurred and are highlighted as A1 - A5.
Fig. 3 : Embedding space of the reasm_a dataset. Each point represents a screwdriving sample described by torque, rotation angle, and time, with colors indicating class labels. Multiple anomalies occurred. As an example, two OK samples are highlighted within the cluster dominated by CROSS_THREADING instances.
Fig. 4 : Curve overlay of the highlighted anomaly samples from Figure 3 , compared to the normal class samples. The anomaly samples, displayed with higher opacity, exhibit typical cross-threading behavior, characterized by a higher friction torque during the initial rotations.
Model
Dataset performance - BAC
Mean
assm_a
dism_a
reasm_a
dism_b
dism_c
PTSM
1.0000
0.9157
0.7015
0.9912
0.9602
0.9137
CTN
1.0000
0.9425
0.5362
1.0000
0.9701
0.8898
CTR
0.9889
0.9257
0.6735
0.9867
0.9625
0.9074
ViT-Dens.
0.9944
0.9595
0.7391
0.9944
0.9700
0.9315
ViT-Spec.
0.9944
0.9276
0.6926
0.9532
0.9800
0.9096
Table 2: Model classification performance in terms of BAC across datasets, with the right column showing the averaged mean performance across all datasets. The best score for each dataset is highlighted in bold. ViT-GAF achieves the highest mean performance.
Model: ViT-GAF
Original dataset
Adapted dataset
Dataset
BAC
F1-Score
BAC
F1-Score
assm_a
0.9944
0.9937
1.0000
1.0000
dism_a
0.9757
0.9741
0.9757
0.9741
reasm_a
0.7812
0.7661
0.9234
0.9237
dism_b
0.9934
0.9982
0.9989
0.9989
dism_c
0.9800
0.9799
0.9900
0.9922
Table 3: Error classification results of the ViT-GAF model on adapted compared to original datasets, showing improvements for all datasets except dism_a , which required no adaptation. Better value is highlighted in bold.
Fig. 5 : Confusion matrix of the classification result and a subregion of the embedding space of the adapted reasm_a dataset. Misclassifications mainly occur between OK and PLASTIFICATION_THREAD class due to their similarity.
Fig. 6 : Embedding space of the classified lawn mower data with highlighted anomalies. Sample colors indicate the predicted class labels.
Fig. 7 : Comparison of classified samples from lawn mower data. The upper subfigure shows the curve overlay of samples classified as REFERENCE , and the lower subfigure shows the curve overlay of samples classified as DAMAGED_HEAD_LARGE .
Automatic identification of screw types is important for industrial automation, robotics, and inventory management. However, publicly available datasets for screw classification are scarce, particularly for controlled single-object scenarios commonly encountered in automated sorting systems. In this work, we introduce SortScrews, a dataset for casewise visual classification of screws. The dataset contains 560 RGB images at 512×512 resolution covering six screw types and a background class. Images are captured using a standardized acquisition setup and include mild variations in lighting and camera perspective across four capture settings. To facilitate reproducible research and dataset expansion, we also provide a reusable data collection script that allows users to easily construct similar datasets for custom hardware components using inexpensive camera setups. We establish baseline results using transfer learning with EfficientNet-B0 and ResNet-18 classifiers pretrained on ImageNet. In addition, we conduct a well-explored failure analysis. Despite the limited dataset size, these lightweight models achieve strong classification accuracy, demonstrating that controlled acquisition conditions enable effective learning even with relatively small datasets. The dataset, collection pipeline, and baseline training code are publicly available at https://github.com/ATATC/SortScrews.
Tianhao Fu, Bingxuan Yang, Juncheng Guo +2
University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · Project Neura, Toronto, ON, Canada +2
In the context of Industry 4.0, predictive maintenance is essential to minimize unplanned downtime and ensure operational safety. In machining processes, drill jamming is a critical failure that compromises both workpiece quality and tool integrity. This work proposes an acoustic anomaly detection system using ultrasonic sensors to identify pre-failure patterns in Computer Numerical Control (CNC) machining. We release a public dataset of time- and frequency-domain acoustic features from 17 drills, and benchmark semi-supervised autoencoder-based normality modelling against supervised XGBoost classification using Leave-One-Drill-Out validation. The results demonstrate that the Autoencoder is highly effective for industrial safety, achieving a Recall of 1.00 and a consolidated F1-score of 0.81. Furthermore, the threshold-independent analysis shows that the Autoencoder better separates normal and pre-failure acoustic patterns, achieving a consolidated AUC of 0.8294, compared with 0.7559 for the supervised model under inter-tool covariate shift. In contrast, the supervised XGBoost achieved an average F1-score of 0.55 and failed to detect failures in some of the evaluated drills. The findings highlight that the ultrasonic spectrum provides a reliable window for identifying incipient mechanical instabilities, and the AE model offers a robust safety envelope by anticipating all jamming events across the tested drill bits.
Maria Beatriz Silva, Lucas Edson Silva de Araújo, Sergio F. Chevtchenko +3
Federal Rural University of Pernambuco, Recife, Brazil · ICNS, Western Sydney University, Penrith, Australia
This work presents a one-class anomaly detection framework for vibration-based condition monitoring in machining processes using a convolutional neural network autoencoder. Three-axis vibration signals acquired during industrial broaching are transformed into time--frequency spectrograms and combined into a compact three-channel representation, enabling convolutional feature learning on multivariate time-series data. The autoencoder is trained exclusively on normal process data and detects anomalous strokes based on reconstruction errors. Labeled validation data are used for threshold calibration and an independent test set is reserved for final performance evaluation. To improve sensitivity to short-lived process instabilities such as chatter, a localized residual-based metric termed Segmented Vertical Error (SVE) is introduced. Experimental results on approximately 10{,}000 broaching strokes show a clear separation between normal and anomalous process conditions under strong class imbalance. The processing pipeline is additionally evaluated on an external public turning dataset as a proof of concept, indicating preliminary applicability beyond the original broaching setup. The results suggest that spectrogram-based autoencoder models can provide a practical and scalable basis for one-class anomaly detection in industrial vibration monitoring.