When Can We Trust Deep Neural Networks? Towards Reliable Industrial Deployment with an Interpretability Guide

Deep Neural Networks have become the dominant solution for Autonomous Driving perception, but their opacity conflicts with emerging Trustworthy AI guidelines and complicates safety assurance, debugging, and human oversight. While theoretical frameworks for safe and Explainable AI (XAI) exist, concrete implementations of Trustworthy AI for 3D scene understanding remain scarce. We address this gap by proposing a Trustworthy AI perception module that is remarkably robust, integrates faithful explainability, and calibrated uncertainty estimates. Building on a transformer-based detector, we derive explanation from the attention mechanism at inference time and validate their faithfulness using perturbation-based consistency tests. We further integrate an uncertainty estimation and calibration module, and apply robustness-enhancing training methods. Experiments show faithful saliency behavior, improved robustness, and well-calibrated uncertainty estimates. Finally, we deploy these Trustworthy AI elements in a prototype vehicle and provide an XAI Interface that visualizes documentation artifacts, model uncertainty state, and saliency maps, demonstrating the feasibility of trustworthy perception monitoring in real time. Supplementary materials are available at https://tillbeemelmanns.github.io/trustworthy_ai/ .

This paper presents RobustDefect-LLM, an industrial surface-defect inspection framework integrating deep-learning classification, operator-facing visual evidence, confidence-aware decision support, controlled AI-assisted reporting, traceable storage, and mobile interaction in a unified quality-control workflow. Here, robustness-aware denotes explicit evaluation under controlled image degradation and confidence-aware review routing, not an intrinsic robustness guarantee. Four transfer-learning-based convolutional neural networks, ResNet50, EfficientNet-B0, DenseNet121, and MobileNetV3-Large, were evaluated on 1,799 images from the six-class NEU-DET dataset using fixed training, validation, and held-out in-domain test partitions. MobileNetV3-Large achieved the highest numerical test accuracy (99.26%) and macro F1-score (0.9926), with a bootstrap 95% accuracy CI of 0.9815-1.0000. An exact paired McNemar test found no significant difference from DenseNet121 (p = 1.000). The selected model averaged 0.060 s per CPU forward pass (16.66 FPS). Under combined synthetic degradation, accuracy fell to 87.78% at mild intensity and below 40% at stronger intensities, revealing sensitivity to severe image-quality deterioration. Grad-CAM supplied visual evidence, while predictions with confidence below 0.90 or a top-2 margin below 0.10 were routed to HUMAN REVIEW. This conservative policy provided 12.22% automatic coverage and 100% observed selective accuracy among 33 eligible cases (95% CI: 89.43%-100.00%), while routing both observed classification errors to review. Under nominal controlled conditions, all 100 generated reports passed deterministic consistency checks, with a mean latency of 1.66 s. Results support the feasibility of the integrated workflow while emphasizing the need for calibration, repeated evaluation, and real-world industrial validation.

Automated anomaly detection methods often report strong performance on curated academic benchmarks, but their behavior under real-world industrial conditions is less clear. In this work, we evaluate 19 unsupervised anomaly detection models on the BowTie dataset, a challenging manufacturing dataset with reflective surfaces, subtle defects, and profile-specific variation. In contrast to benchmark results, we observe that model performance is less stable than typically reported on standard benchmarks such as MVTec AD, highly sensitive to preprocessing, and inconsistent across conditions, with no single approach emerging as uniformly robust; a consensus audit further indicates that nominal-data quality affects deployment. Motivated by these findings, we developed and initially deployed a unified human-in-the-loop framework for manufactured-part inspection that combines image annotation, AI-assisted defect detection, and an integrated validation engine, replacing a prior manual visual inspection and documentation workflow. The system supports heatmap-guided defect review, SAM-refined candidate regions for inspector acceptance, rejection, or boundary adjustment, mask evaluation where annotations exist, and review history for inspector consistency and onboarding. Together, the results highlight the gap between benchmark performance and deployment reality, and provide a practical framework for addressing it.