Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection

Radia Daci; Vito Renò; Cosimo Patruno; Angelo Cardellicchio; Abdelmalik Taleb-Ahmed; Marco Leo; Cosimo Distante

cs.CVMar 4, 2026

Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection

Authors: Radia Daci, Vito Renò, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante

Organizations: CNR-ISASI, Institute of Applied Sciences and Intelligent Systems / National Research Council of Italy, via Monteroni snc, Lecce, 73100, Italy · CNR-STIIMA, Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, Via G. Amendola 122 D/O, Bari, 70126, Italy · Institute d’Electronique de Microelectronique et de Nanotechnologie (IEMN) / Universite Polytechnique Hauts de France; Universite de Lille; CNRS, UMR 8520, City, 59313, France

Abstract

Multimodal industrial anomaly detection benefits from integrating RGB appearance with 3D surface geometry, yet existing unsupervised approaches commonly rely on memory banks, teacher-student architectures, or fragile fusion schemes, limiting robustness under noisy depth, weak texture, or missing modalities. This paper introduces CMDR-IAD, a lightweight and modality-flexible unsupervised framework for reliable anomaly detection in 2D+3D multimodal as well as single-modality (2D-only or 3D-only) settings. CMDR-IAD combines bidirectional 2D $\leftrightarrow$ 3D cross-modal mapping to model appearance-geometry consistency with dual-branch reconstruction that independently captures normal texture and geometric structure. A two-part fusion strategy integrates these cues: a reliability-gated mapping anomaly highlights spatially consistent texture-geometry discrepancies, while a confidence-weighted reconstruction anomaly adaptively balances appearance and geometric deviations, yielding stable and precise anomaly localization even in depth-sparse or low-texture regions. On the MVTec 3D-AD benchmark, CMDR-IAD achieves state-of-the-art performance while operating without memory banks, reaching 97.3% image-level AUROC (I-AUROC), 99.6% pixel-level AUROC (P-AUROC), and 97.6% AUPRO. On a real-world polyurethane cutting dataset, the 3D-only variant attains 92.6% I-AUROC and 92.5% P-AUROC, demonstrating strong effectiveness under practical industrial conditions. These results highlight the framework's robustness, modality flexibility, and the effectiveness of the proposed fusion strategies for industrial visual inspection. Our source code is available at https://github.com/ECGAI-Research/CMDR-IAD/

Figures & tables

Figure 1: Performance, speed and memory occupancy of Multimodal Anomaly Detection methods. The chart reports defect segmentation performance (AUPRO@30%) vs inference speed.

Figure 2: Overview of the proposed CMDR–IAD framework. The method integrates RGB images and point clouds through multimodal feature extraction, cross-modal mapping, and dual-branch 2D–3D reconstruction. Cross-modal discrepancy maps and reconstruction errors are fused into the final anomaly map $\Psi$ , enabling robust detection of both appearance- and geometry-related defects.

Figure 3: Overview of the 2D Reconstruction Branch. Projected features are enhanced using a sparse-attention block and an MLP refinement module with residual connections, then decoded through ConvTranspose2D layers to generate the reconstructed 2D feature map.

Figure 4: Overview of the 3D Reconstruction Branch. After projection and ConvTranspose1D upsampling, a gating block refines the features through sequential 1D convolutions and a sigmoid mask, which is multiplied with a residual pathway to produce the final 3D reconstruction.

2D
Method	Bagel	Cable Gland	Carrot	Cookie	Dowel	Foam	Peach	Potato	Rope	Tire	Mean
PatchCore (2022) [ 32 ]	87.6	88.0	79.1	68.2	91.2	70.1	69.5	61.8	84.1	70.2	77.0
BTF (2023) [ 15 ]	87.6	88.0	79.1	68.2	91.2	70.1	69.5	61.8	84.1	70.2	77.0
BTM (2024) [ 21 ]	90.9	89.5	83.8	74.5	97.5	71.4	79.0	60.5	93.0	75.9	81.6
AST (2023) [ 33 ]	94.7	92.8	85.1	82.5	98.1	95.1	89.5	61.3	99.2	82.1	88.0
M3DM (2023) [ 38 ]	94.4	91.8	89.6	74.9	95.9	76.7	91.9	64.8	93.8	76.7	85.0

Table 1: I-AUROC (%) for anomaly detection on all classes of the MVTec 3D-AD dataset. The best results are highlighted in bold , while the second-best are underlined . Our approach delivers competitive performance in single-modality (2D and 3D) settings and achieves state-of-the-art results in the multimodal (2D+3D) configuration.

2D
Method	Bagel	Cable Gland	Carrot	Cookie	Dowel	Foam	Peach	Potato	Rope	Tire	Mean
PatchCore (2022) [ 32 ]	90.1	94.9	92.8	87.7	89.2	56.3	90.4	93.2	90.8	90.6	87.6
BTF (2023) [ 15 ]	90.1	94.9	92.8	87.7	89.2	56.3	90.4	93.2	90.8	90.6	87.6
BTM (2024) [ 21 ]	90.1	95.8	94.2	90.5	95.1	61.5	90.6	93.8	92.7	91.6	89.6
AST (2023) [ 33 ]	83.4	87.4	76.4	44.7	87.2	56.9	84.5	51.8	86.3	58.8	71.7
M3DM (2023) [ 38 ]	95.2	97.2	97.3	89.1	93.2	84.3	97.0	95.6	96.8	96.6	94.2

Table 2: AUPRO@30% for anomaly detection on all classes of the MVTec 3D-AD dataset. The best results are highlighted in bold , while the second-best are underlined . Our approach delivers competitive performance in single-modality (2D and 3D) settings and achieves state-of-the-art localization accuracy in the multimodal (2D+3D) configuration.

Method	Bagel	CableGland	Carrot	Cookie	Dowel	Foam	Peach	Potato	Rope	Tire	Mean
BTF (2023) [ 15 ]	42.8	36.5	45.2	43.1	37.0	24.4	42.7	47.0	29.8	34.5	38.3
AST (2023) [ 33 ]	38.8	32.2	47.0	41.1	32.8	27.5	47.4	48.7	36.0	47.4	39.8
M3DM (2023) [ 38 ]	41.4	39.5	44.7	31.8	42.2	33.5	44.4	35.1	41.6	39.8	39.4
CFM(2024) [ 10 ]	45.9	43.1	48.5	46.9	39.4	41.3	46.8	48.7	46.4	47.6	45.5
CMDR–IAD (Ours)	47.2	43.9	48.6	46.4	42.2	43.5	48.6	48.5	47.4	48.5	46.5

Table 3: AUPRO@1% on the MVTec 3D-AD dataset for multimodal (2D+3D) anomaly detection. Best results are shown in bold and second-best results are underlined .

Method	Ref	Frame Rate (FPS)	Memory (MB)	I-AUROC	P-AUROC	AUPRO@30%	AUPRO@1%
CFM [ 10 ]	CVPR 2024	3.331	1957.77	95.4	99.3	97.1	45.5
3D-ADNAS [ 25 ]	AAAI 2025	-	-	95.1	-	-	-
MTSJM [ 24 ]	EAAI 2025	-	-	95.7	-	97.2	-
CMDIAD [ 35 ]	Information Fusion 2025	-	-	93.8	99.2	96.2	-
DAK-Net [ 43 ]	Journal of Manufacturing Processes 2026	-	-	93.6	-	-	-
CMDR–IAD (Ours)	-	2.710	2797.65	97.3	99.6	97.6	46.5

Table 4: Inference speed, memory footprint, and anomaly detection performance on MVTec 3D-AD. Frame Rate in FPS and Memory in MB.

Figure 5: Qualitative anomaly localization results on the MVTec 3D-AD dataset using 2D-only features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (2D), M3DM (2D), MTSJM (2D), and the proposed CMDR-IAD (2D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.

Figure 6: Qualitative anomaly localization results on the MVTec 3D-AD dataset using 3D-only point cloud features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (3D), M3DM (3D), MTSJM (3D), and the proposed CMDR–IAD (3D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.

Figure 7: Qualitative anomaly localization results on the MVTec 3D-AD dataset using multimodal (2D+3D) features. From top to bottom, the rows show the RGB image, point cloud, ground truth, AST (2D+3D), M3DM (2D+3D), MTSJM (2D+3D), and the proposed CMDR–IAD (2D+3D). Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.

Metric	Frame Rate (FPS)	Memory (MB)	I-AUROC	P-AUROC
CMDR–IAD (Ours)	24.63	465.68	92.6	92.5

Table 5: 3D-only performance of CMDR–IAD on the Polyurethane Cutting dataset. We report image-level AUROC (I-AUROC), point-level AUROC (P-AUROC), inference speed measured in frames per second (FPS), and GPU memory footprint.

Figure 8: Qualitative anomaly localization results on the polyurethane cuts dataset using 3D-only point cloud data. For each sample, the ground truth mask and the corresponding predicted anomaly map are shown. Warmer colors indicate higher anomaly scores, while cooler colors denote normal regions.

Configuration	I-AUROC	P-AUROC	AUPRO@30%	AUPRO@1%
Dual Reconstruction only	95.0	98.6	95.2	41.0
Cross-Modal Mapping only	95.4	99.4	97.4	46.0
CMDR–IAD (Ours)	97.3	99.6	97.6	46.5

Table 6: Ablation study on the MVTec 3D-AD dataset evaluating the effect of dual-branch reconstruction and cross-modal mapping. Best results are in bold .

Fusion Variant	I-AUROC	P-AUROC	AUPRO@30%	AUPRO@1%
Case 1: Gated Mapping Fusion (Eq. 1 )	96.7	99.5	97.4	46.3
Case 2: Pure Multiplicative (Eq. 2 )	97.1	99.5	97.6	46.4
Case 3: Soft Adaptive (Eq. 3 )	96.7	99.2	96.8	45.0
Case 4: Softmax + Gated Rec (Eq. 4 )	96.6	99.5	97.4	46.1
Case 5: Dual Gated Fusion (Eq. 5 )	97.0	99.6	97.7	46.5
Case 6: Uniform Avg. (Eq. 6 )	95.0	98.7	95.7	42.5

Table 7: Ablation study of multimodal anomaly scoring variants

\Psi_{C1}

–

\Psi_{C6}

(Eqs. 1 – 6 ) compared to the full CMDR–IAD model (Eq. 7 ) on the MVTec 3D–AD dataset.

Detector	Sample Size	Level	I-AUROC	P-AUROC
LOF	4096	low	80.1	94.6
	4096	high	80.2	85.6
	9216	low	99.5	80.3
	9216	high	94.4	79.6
ISO	4096	low	87.5	87.4
	4096	high	86.3	84.6

Table 8: Ablation study of outlier detectors for polyurethane preprocessing, comparing LOF, Isolation Forest (ISO), and a hybrid strategy across contamination levels and chunk sizes. The configuration used in the final CMDR–AD pipeline is highlighted.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Bagel	CableGland	Carrot	Cookie	Dowel	Foam	Peach	Potato	Rope	Tire	Mean
2D
PatchCore (2022) [ 32 ]	98.3	98.4	98.0	97.4	97.2	84.9	97.6	98.3	98.7	97.7	96.7
M3DM (2023) [ 38 ]	99.2	99.0	99.4	97.7	98.3	95.5	99.4	99.0	99.5	99.4	98.7
CMDIAD (2025) [ 35 ]	99.2	99.3	99.4	97.7	98.3	95.6	99.3	99.0	99.5	99.4	98.7
CMDR–IAD (Ours)	98.6	98.8	94.8	99.3	99.4	94.0	97.2	99.6	99.6	99.2	98.1
3D

Appendix

Table 9: P-AUROC (%) for anomaly detection on all classes of the MVTec 3D-AD dataset. Best results are shown in bold and second-best results are underlined. Our method achieves the best performance in the 3D and 2D+3D settings, and the second-best performance in the 2D setting.

Case	Metric	Bagel	CableGland	Carrot	Cookie	Dowel	Foam	Peach	Potato	Rope	Tire	Mean
Case 1 (Eq. 1 )	I-AUROC	99.5	91.7	98.6	99.7	98.8	92.1	99.4	91.7	98.8	96.5	96.7
	P-AUROC	99.7	99.3	99.8	98.0	99.0	99.5	99.9	99.8	99.8	99.8	99.5
	AUPRO@30	97.9	97.3	98.2	95.4	96.0	97.0	98.2	98.3	97.9	98.2	97.4
	AUPRO@1	46.9	44.2	48.5	46.5	40.8	43.2	48.6	48.8	47.2	48.4	46.3
Case 2 (Eq. 2 )	I-AUROC	99.7	87.2	98.6	99.7	98.8	93.8	99.6	96.0	99.0	98.7	97.1
	P-AUROC	99.7	98.8	99.8	98.6	99.2	99.7	99.9	99.9	99.9	99.8	99.5

Appendix

Table 10: Per-class ablation study for all multimodal fusion variants. Columns correspond to the 10 MVTec 3D-AD classes, while rows report four evaluation metrics for each ablation case defined in Eqs. ( 1 )–( 6 ) and the full model (Eq. 7 ). The rightmost column reports the mean performance across all classes.

Figure 9: Failure case examples on the MVTec 3D-AD dataset. From left to right: RGB image, point cloud, ground truth, and anomaly maps produced by 2D-only, 3D-only, and multimodal (2D+3D) configurations.

Figure 10: Examples of false positive and false negative predictions on the Polyurethane Cutting dataset. Normal samples may exhibit elevated anomaly responses due to unseen appearance or geometric patterns, while subtle defect regions may be weakly detected or missed in the predicted anomaly maps.

Explore similar work

Text-Guided Multimodal Unified Industrial Anomaly Detection
Apr 24, 2026Zewen Li, Shuo Ye, Zitong Yu +2Unified Multimodal Models Industrial Anomaly Detection
School of Computer Science & Software Engineering, Shenzhen University, China, 518060 · School of Computing and Information Technology, Great Bay University, Dongguan, 523000,China
Modulate-and-Map: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection
Apr 2, 2026Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti +1Multimodal Anomaly Detection Multi-View Learning
CVLab, University of Bologna · Ca’ Foscari University of Venice
Multimodal Industrial Anomaly Detection via Geometric Prior
Mar 24, 2026Min Li, Jinghui He, Gang Li +3Industrial Anomaly Detection Anomaly Localization
Faculty of Data Science, City University of Macau, Macau, China. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.

Apr 24, 2026J/K move · Enter open · S save

Text-Guided Multimodal Unified Industrial Anomaly Detection

Zewen Li, Shuo Ye, Zitong Yu, Weicheng Xie+1

School of Computer Science & Software Engineering, Shenzhen University, China, 518060 · School of Computing and Information Technology, Great Bay University, Dongguan, 523000,China

Unified Multimodal Models Industrial Anomaly Detection Unsupervised Anomaly Detection Multimodal Anomaly Detection Multimodal Fusion Anomaly Detection

Open paper AskPDF arXiv

Industrial anomaly detection based on RGB-3D multimodal data has emerged as a mainstream paradigm for intelligent quality inspection. However, existing unsupervised methods suffer from two critical limitations: ambiguous cross-modal alignment caused by the lack of high-level semantic guidance and insufficient geometric modeling for RGB-to-3D feature mapping. To address these issues, we propose a unified multimodal industrial anomaly detection framework guided by text semantics. The framework consists of two core modules: a Geometry-Aware Cross-Modal Mapper to preserve geometric structure during modality conversion, and an Object-Conditioned Textual Feature Adaptor to align multimodal features with semantic priors. Furthermore, we establish a unified learning paradigm for multimodal industrial anomaly detection, which breaks the one-model-one-class constraint and enables accurate anomaly detection across diverse classes using a single model. Extensive experiments on the MVTec 3D-AD and Eyecandies datasets demonstrate that our method achieves state-of-the-art performance in classification and localization under unsupervised settings.

Abstract

Explore similar work

Text-Guided Multimodal Unified Industrial Anomaly Detection

Modulate-and-Map: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection

Multimodal Industrial Anomaly Detection via Geometric Prior