Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection
Authors: Radia Daci, Vito Renò, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante
Organizations: CNR-ISASI, Institute of Applied Sciences and Intelligent Systems / National Research Council of Italy, via Monteroni snc, Lecce, 73100, Italy · CNR-STIIMA, Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, Via G. Amendola 122 D/O, Bari, 70126, Italy · Institute d’Electronique de Microelectronique et de Nanotechnologie (IEMN) / Universite Polytechnique Hauts de France; Universite de Lille; CNRS, UMR 8520, City, 59313, France
Multimodal industrial anomaly detection benefits from integrating RGB appearance with 3D surface geometry, yet existing unsupervised approaches commonly rely on memory banks, teacher-student architectures, or fragile fusion schemes, limiting robustness under noisy depth, weak texture, or missing modalities. This paper introduces CMDR-IAD, a lightweight and modality-flexible unsupervised framework for reliable anomaly detection in 2D+3D multimodal as well as single-modality (2D-only or 3D-only) settings. CMDR-IAD combines bidirectional 2D↔3D cross-modal mapping to model appearance-geometry consistency with dual-branch reconstruction that independently captures normal texture and geometric structure. A two-part fusion strategy integrates these cues: a reliability-gated mapping anomaly highlights spatially consistent texture-geometry discrepancies, while a confidence-weighted reconstruction anomaly adaptively balances appearance and geometric deviations, yielding stable and precise anomaly localization even in depth-sparse or low-texture regions. On the MVTec 3D-AD benchmark, CMDR-IAD achieves state-of-the-art performance while operating without memory banks, reaching 97.3% image-level AUROC (I-AUROC), 99.6% pixel-level AUROC (P-AUROC), and 97.6% AUPRO. On a real-world polyurethane cutting dataset, the 3D-only variant attains 92.6% I-AUROC and 92.5% P-AUROC, demonstrating strong effectiveness under practical industrial conditions. These results highlight the framework's robustness, modality flexibility, and the effectiveness of the proposed fusion strategies for industrial visual inspection. Our source code is available at https://github.com/ECGAI-Research/CMDR-IAD/
Figures & tables
Figure 1: Performance, speed and memory occupancy of Multimodal Anomaly Detection methods. The chart reports defect segmentation performance (AUPRO@30%) vs inference speed.
Figure 2: Overview of the proposed CMDR–IAD framework. The method integrates RGB images and point clouds through multimodal feature extraction, cross-modal mapping, and dual-branch 2D–3D reconstruction. Cross-modal discrepancy maps and reconstruction errors are fused into the final anomaly map Ψ , enabling robust detection of both appearance- and geometry-related defects.
Figure 3: Overview of the 2D Reconstruction Branch. Projected features are enhanced using a sparse-attention block and an MLP refinement module with residual connections, then decoded through ConvTranspose2D layers to generate the reconstructed 2D feature map.
Figure 4: Overview of the 3D Reconstruction Branch. After projection and ConvTranspose1D upsampling, a gating block refines the features through sequential 1D convolutions and a sigmoid mask, which is multiplied with a residual pathway to produce the final 3D reconstruction.
Method
Bagel
Cable Gland
Carrot
Cookie
Dowel
Foam
Peach
Potato
Rope
Tire
Mean
2D
PatchCore (2022) [ 32 ]
87.6
88.0
79.1
68.2
91.2
70.1
69.5
61.8
84.1
70.2
77.0
BTF (2023) [ 15 ]
87.6
88.0
79.1
68.2
91.2
70.1
69.5
61.8
84.1
70.2
77.0
BTM (2024) [ 21 ]
90.9
89.5
83.8
74.5
97.5
71.4
79.0
60.5
93.0
75.9
81.6
AST (2023) [ 33 ]
94.7
92.8
85.1
82.5
98.1
95.1
89.5
61.3
99.2
82.1
88.0
M3DM (2023) [ 38 ]
94.4
91.8
89.6
74.9
95.9
76.7
91.9
64.8
93.8
76.7
85.0
Table 1: I-AUROC (%) for anomaly detection on all classes of the MVTec 3D-AD dataset. The best results are highlighted in bold , while the second-best are underlined . Our approach delivers competitive performance in single-modality (2D and 3D) settings and achieves state-of-the-art results in the multimodal (2D+3D) configuration.
Method
Bagel
Cable Gland
Carrot
Cookie
Dowel
Foam
Peach
Potato
Rope
Tire
Mean
2D
PatchCore (2022) [ 32 ]
90.1
94.9
92.8
87.7
89.2
56.3
90.4
93.2
90.8
90.6
87.6
BTF (2023) [ 15 ]
90.1
94.9
92.8
87.7
89.2
56.3
90.4
93.2
90.8
90.6
87.6
BTM (2024) [ 21 ]
90.1
95.8
94.2
90.5
95.1
61.5
90.6
93.8
92.7
91.6
89.6
AST (2023) [ 33 ]
83.4
87.4
76.4
44.7
87.2
56.9
84.5
51.8
86.3
58.8
71.7
M3DM (2023) [ 38 ]
95.2
97.2
97.3
89.1
93.2
84.3
97.0
95.6
96.8
96.6
94.2
Table 2: AUPRO@30% for anomaly detection on all classes of the MVTec 3D-AD dataset. The best results are highlighted in bold , while the second-best are underlined . Our approach delivers competitive performance in single-modality (2D and 3D) settings and achieves state-of-the-art localization accuracy in the multimodal (2D+3D) configuration.
Method
Bagel
CableGland
Carrot
Cookie
Dowel
Foam
Peach
Potato
Rope
Tire
Mean
BTF (2023) [ 15 ]
42.8
36.5
45.2
43.1
37.0
24.4
42.7
47.0
29.8
34.5
38.3
AST (2023) [ 33 ]
38.8
32.2
47.0
41.1
32.8
27.5
47.4
48.7
36.0
47.4
39.8
M3DM (2023) [ 38 ]
41.4
39.5
44.7
31.8
42.2
33.5
44.4
35.1
41.6
39.8
39.4
CFM(2024) [ 10 ]
45.9
43.1
48.5
46.9
39.4
41.3
46.8
48.7
46.4
47.6
45.5
CMDR–IAD (Ours)
47.2
43.9
48.6
46.4
42.2
43.5
48.6
48.5
47.4
48.5
46.5
Table 3: AUPRO@1% on the MVTec 3D-AD dataset for multimodal (2D+3D) anomaly detection. Best results are shown in bold and second-best results are underlined .
Method
Ref
Frame Rate (FPS)
Memory (MB)
I-AUROC
P-AUROC
AUPRO@30%
AUPRO@1%
CFM [ 10 ]
CVPR 2024
3.331
1957.77
95.4
99.3
97.1
45.5
3D-ADNAS [ 25 ]
AAAI 2025
-
-
95.1
-
-
-
MTSJM [ 24 ]
EAAI 2025
-
-
95.7
-
97.2
-
CMDIAD [ 35 ]
Information Fusion 2025
-
-
93.8
99.2
96.2
-
DAK-Net [ 43 ]
Journal of Manufacturing Processes 2026
-
-
93.6
-
-
-
CMDR–IAD (Ours)
-
2.710
2797.65
97.3
99.6
97.6
46.5
Table 4: Inference speed, memory footprint, and anomaly detection performance on MVTec 3D-AD. Frame Rate in FPS and Memory in MB.
Figure 5: Qualitative anomaly localization results on the MVTec 3D-AD dataset using 2D-only features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (2D), M3DM (2D), MTSJM (2D), and the proposed CMDR-IAD (2D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.
Figure 6: Qualitative anomaly localization results on the MVTec 3D-AD dataset using 3D-only point cloud features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (3D), M3DM (3D), MTSJM (3D), and the proposed CMDR–IAD (3D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.
Figure 7: Qualitative anomaly localization results on the MVTec 3D-AD dataset using multimodal (2D+3D) features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (2D+3D), M3DM (2D+3D), MTSJM (2D+3D), and the proposed CMDR–IAD (2D+3D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.
Metric
Frame Rate (FPS)
Memory (MB)
I-AUROC
P-AUROC
CMDR–IAD (Ours)
24.63
465.68
92.6
92.5
Table 5: 3D-only performance of CMDR–IAD on the Polyurethane Cutting dataset. We report image-level AUROC (I-AUROC), point-level AUROC (P-AUROC), inference speed measured in frames per second (FPS), and GPU memory footprint.
Figure 8: Qualitative anomaly localization results on the polyurethane cuts dataset using 3D-only point cloud data. For each sample, the ground truth mask and the corresponding predicted anomaly map are shown. Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.
Configuration
I-AUROC
P-AUROC
AUPRO@30%
AUPRO@1%
Dual Reconstruction only
95.0
98.6
95.2
41.0
Cross-Modal Mapping only
95.4
99.4
97.4
46.0
CMDR–IAD (Ours)
97.3
99.6
97.6
46.5
Table 6: Ablation study on the MVTec 3D-AD dataset evaluating the effect of dual-branch reconstruction and cross-modal mapping. Best results are in bold .
Fusion Variant
I-AUROC
P-AUROC
AUPRO@30%
AUPRO@1%
Case 1: Gated Mapping Fusion (Eq. 1 )
96.7
99.5
97.4
46.3
Case 2: Pure Multiplicative (Eq. 2 )
97.1
99.5
97.6
46.4
Case 3: Soft Adaptive (Eq. 3 )
96.7
99.2
96.8
45.0
Case 4: Softmax + Gated Rec (Eq. 4 )
96.6
99.5
97.4
46.1
Case 5: Dual Gated Fusion (Eq. 5 )
97.0
99.6
97.7
46.5
Case 6: Uniform Avg. (Eq. 6 )
95.0
98.7
95.7
42.5
Table 7: Ablation study of multimodal anomaly scoring variants ΨC1 – ΨC6 (Eqs. 1 – 6 ) compared to the full CMDR–IAD model (Eq. 7 ) on the MVTec 3D–AD dataset.
Detector
Sample Size
Level
I-AUROC
P-AUROC
LOF
4096
low
80.1
94.6
high
80.2
85.6
9216
low
99.5
80.3
high
94.4
79.6
ISO
4096
low
87.5
87.4
high
86.3
84.6
Table 8: Ablation study of outlier detectors for polyurethane preprocessing, comparing LOF, Isolation Forest (ISO), and a hybrid strategy across contamination levels and chunk sizes. The configuration used in the final CMDR–AD pipeline is highlighted.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Bagel
CableGland
Carrot
Cookie
Dowel
Foam
Peach
Potato
Rope
Tire
Mean
2D
PatchCore (2022) [ 32 ]
98.3
98.4
98.0
97.4
97.2
84.9
97.6
98.3
98.7
97.7
96.7
M3DM (2023) [ 38 ]
99.2
99.0
99.4
97.7
98.3
95.5
99.4
99.0
99.5
99.4
98.7
CMDIAD (2025) [ 35 ]
99.2
99.3
99.4
97.7
98.3
95.6
99.3
99.0
99.5
99.4
98.7
CMDR–IAD (Ours)
98.6
98.8
94.8
99.3
99.4
94.0
97.2
99.6
99.6
99.2
98.1
3D
Appendix
Table 9: P-AUROC (%) for anomaly detection on all classes of the MVTec 3D-AD dataset. Best results are shown in bold and second-best results are underlined. Our method achieves the best performance in the 3D and 2D+3D settings, and the second-best performance in the 2D setting.
Case
Metric
Bagel
CableGland
Carrot
Cookie
Dowel
Foam
Peach
Potato
Rope
Tire
Mean
Case 1 (Eq. 1 )
I-AUROC
99.5
91.7
98.6
99.7
98.8
92.1
99.4
91.7
98.8
96.5
96.7
P-AUROC
99.7
99.3
99.8
98.0
99.0
99.5
99.9
99.8
99.8
99.8
99.5
AUPRO@30
97.9
97.3
98.2
95.4
96.0
97.0
98.2
98.3
97.9
98.2
97.4
AUPRO@1
46.9
44.2
48.5
46.5
40.8
43.2
48.6
48.8
47.2
48.4
46.3
Case 2 (Eq. 2 )
I-AUROC
99.7
87.2
98.6
99.7
98.8
93.8
99.6
96.0
99.0
98.7
97.1
P-AUROC
99.7
98.8
99.8
98.6
99.2
99.7
99.9
99.9
99.9
99.8
99.5
Appendix
Table 10: Per-class ablation study for all multimodal fusion variants. Columns correspond to the 10 MVTec 3D-AD classes, while rows report four evaluation metrics for each ablation case defined in Eqs. ( 1 )–( 6 ) and the full model (Eq. 7 ). The rightmost column reports the mean performance across all classes.
Figure 9: Failure case examples on the MVTec 3D-AD dataset. From left to right: RGB image, point cloud, ground truth, and anomaly maps produced by 2D-only, 3D-only, and multimodal (2D+3D) configurations.
Figure 10: Examples of false positive and false negative predictions on the Polyurethane Cutting dataset. Normal samples may exhibit elevated anomaly responses due to unseen appearance or geometric patterns, while subtle defect regions may be weakly detected or missed in the predicted anomaly maps.
Industrial anomaly detection based on RGB-3D multimodal data has emerged as a mainstream paradigm for intelligent quality inspection. However, existing unsupervised methods suffer from two critical limitations: ambiguous cross-modal alignment caused by the lack of high-level semantic guidance and insufficient geometric modeling for RGB-to-3D feature mapping. To address these issues, we propose a unified multimodal industrial anomaly detection framework guided by text semantics. The framework consists of two core modules: a Geometry-Aware Cross-Modal Mapper to preserve geometric structure during modality conversion, and an Object-Conditioned Textual Feature Adaptor to align multimodal features with semantic priors. Furthermore, we establish a unified learning paradigm for multimodal industrial anomaly detection, which breaks the one-model-one-class constraint and enables accurate anomaly detection across diverse classes using a single model. Extensive experiments on the MVTec 3D-AD and Eyecandies datasets demonstrate that our method achieves state-of-the-art performance in classification and localization under unsupervised settings.
Zewen Li, Shuo Ye, Zitong Yu +2
School of Computer Science & Software Engineering, Shenzhen University, China, 518060 · School of Computing and Information Technology, Great Bay University, Dongguan, 523000,China
We present ModMap, a natively multiview and multimodal framework for 3D anomaly detection and segmentation. Unlike existing methods that process views independently, our method draws inspiration from the crossmodal feature mapping paradigm to learn to map features across both modalities and views, while explicitly modelling view-dependent relationships through feature-wise modulation. We introduce a cross-view training strategy that leverages all possible view combinations, enabling effective anomaly scoring through multiview ensembling and aggregation. To process high-resolution 3D data, we train and publicly release a foundational depth encoder tailored to industrial datasets. Experiments on SiM3D, a recent benchmark that introduces the first multiview and multimodal setup for 3D anomaly detection and segmentation, demonstrate that ModMap attains state-of-the-art performance by surpassing previous methods by wide margins.
Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti +1
CVLab, University of Bologna · Ca’ Foscari University of Venice
The purpose of multimodal industrial anomaly detection is to detect complex geometric shape defects such as subtle surface deformations and irregular contours that are difficult to detect in 2D-based methods. However, current multimodal industrial anomaly detection lacks the effective use of crucial geometric information like surface normal vectors and 3D shape topology, resulting in low detection accuracy. In this paper, we propose a novel Geometric Prior-based Anomaly Detection network (GPAD). Firstly, we propose a point cloud expert model to perform fine-grained geometric feature extraction, employing differential normal vector computation to enhance the geometric details of the extracted features and generate geometric prior. Secondly, we propose a two-stage fusion strategy to efficiently leverage the complementarity of multimodal data as well as the geometric prior inherent in 3D points. We further propose attention fusion and anomaly regions segmentation based on geometric prior, which enhance the model's ability to perceive geometric defects. Extensive experiments show that our multimodal industrial anomaly detection model outperforms the State-of-the-art (SOTA) methods in detection accuracy on both MVTec-3D AD and Eyecandies datasets.
Min Li, Jinghui He, Gang Li +3
Faculty of Data Science, City University of Macau, Macau, China. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.