DCM-SAM: Defect-Conditioned Mixture of LoRA Experts for NPU-Deployed AM Defect Segmentation

Md Mushfiqur Rahaman; Md Mahedi Hasan; Imtiaz Ahmed; Srinjoy Das

cs.CVSep 30, 2026

DCM-SAM: Defect-Conditioned Mixture of LoRA Experts for NPU-Deployed AM Defect Segmentation

Authors: Md Mushfiqur Rahaman, Md Mahedi Hasan, Imtiaz Ahmed, Srinjoy Das

Organizations: Department of Industrial and Management Systems Engineering West Virginia University, Morgantown, WV · School of Mathematical and Data Sciences West Virginia University, Morgantown, WV · Lane Department of Computer Science & Electrical Engineering West Virginia University, Morgantown, WV

Abstract

Metal additive manufacturing parts are inspected by X-ray computed tomography, where labelled data is scarce, the pores and inclusions that matter span a few pixels, and inspection must happen at the machine. We present DCM-SAM, a defect-conditioned adaptive mixture of LoRA experts: one frozen Segment Anything backbone carries a separate Conv-LoRA expert bank and mask decoder per defect class, each trained in its own pass, without prompts, on synthetic slices alone, updating only 4.4% of the parameters. On benchmarks that XCT-SAM reports, DCM-SAM improves on every baseline for both classes from a ViT-B backbone against their ViT-H, and reaches 64.2% pore IoU on real NIST scans having seen no real images during training. Deployment then exposes what adaptation work rarely measures: on a Qualcomm Hexagon NPU, ViT-H and ViT-L compile yet cannot allocate at 1024x1024 image resolution, since activations rather than weights exceed the device ceiling, and quantizing weights does not help. ViT-B alone runs, but the adapted encoder then fails to allocate where the stock one succeeds, until a numerically identical rewrite of the attention lets the complete DCM-SAM run in FP16 at 1024x1024, with no operator falling back to the CPU, masks within 0.01% of pixels of the FP32 reference. Code: https://github.com/MushfiqShovon/DCM-SAM.

Figures & tables

Figure 1: DCM-SAM. (a) Conv-LoRA with a mixture of experts is injected into every block of one frozen SAM ViT-B, giving each defect class its own encoder path and decoder ( $M_{p}=2$ experts for pores, $M_{i}=8$ for inclusions); the heads share the frozen weights but no trainable parameter, and the encoder runs once per head. (b) Within a block, a top- $k$ gate routes the low-rank bottleneck to one of $M$ convolutional experts, each at its own spatial scale.

$(M_{\text{pore}},M_{\text{incl}})$	Role	Synth. pore	Synth. incl.	NIST pore (OOD)
$(1,1)$	corner	32.8	49.6	64.4
$(1,8)$	pore sweep	29.4	47.7	65.1
$(2,2)$	composed optimum	35.0	46.3	64.3
$(2,8)$ (selected)	pore sweep	36.1	48.9	64.2
$(4,8)$	pore sweep	34.6	44.9	65.8
$(8,1)$	incl. sweep	36.5	47.5	65.3

Table 1: ViT-B configurations: mean IoU (%) over the five synthetic splits, and pore IoU on real NIST XCT. Bold marks the column maximum.

Method	Pores (synthetic)	Inclusions (synthetic)	Pores (NIST, OOD)
SAM Kirillov et al. (2023) (zero-shot)	14.3	29.5	49.4
UNet++ Zhou et al. (2020)	14.0	10.2	13.4
MedSAM Ma et al. (2024)	16.3	32.5	33.8
SAM-Med2D Cheng et al. (2023)	18.7	36.0	57.8
Conv-LoRA-SAM Zhong et al. (2024)	27.7	24.1	54.6
XCT-SAM Hasan et al. (2026)	32.5	36.4	58.5

Table 2: Comparison against the baselines reported by XCT-SAM Hasan et al. (2026) (mean IoU, %). DCM-SAM uses ViT-B with

(2,8)

; all baselines use ViT-H.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	ViT-H, $1024^{2}$	ViT-H, $512^{2}$ (raw / rescaled)	ViT-B $(2,8)$ , $1024^{2}$
Pore IoU, image-averaged	37.3	31.7 / 30.8	36.1
Pore IoU, defect-only	35.8	27.9 / 27.6	32.7
Inclusion IoU, image-averaged	52.5	51.8 / 46.6	48.9
Inclusion IoU, defect-only	15.5	9.3 / 20.7	11.8
Real NIST pore IoU	–	–	64.2
Runs on the IQ-9075 with adapters	no	–	yes, verified

Appendix

Table 3: Lowering the resolution against lowering the backbone. The same ViT-H configuration trained and evaluated at

1024^{2}

and at

512^{2}

(“rescaled” applies blob-size filters scaled to

512^{2}

), and the deployed ViT-B

(2,8)

model at

1024^{2}

. IoU in %, mean over the five synthetic splits; “defect-only” averages over defect-bearing images only.

Direct runs: $(1,1)$ $36.3$ / $53.6$ ; $(2,1)$ (selected) $34.6$ / $53.3$
	Pore sweep ( $M_{\text{incl}}{=}8$ )		Incl. sweep ( $M_{\text{pore}}{=}8$ )
$M$	Pore	Incl.	Incl.	Pore
1	35.5	53.8	54.3	34.5
2	36.3	53.2	53.3	36.6
4	36.2	54.3	54.4	32.6
8	36.1	53.5	53.5	36.1

Appendix

Table 4: ViT-H sweeps and corner runs. Each sweep row reports the swept head’s IoU with the fixed companion head’s IoU from the same run; mean IoU (%) over the five synthetic splits.

Component	ViT-H	Share	ViT-B	Share
Frozen SAM backbone (encoder, prompt encoder)	637.0 M	frozen	89.7 M	frozen
Conv-LoRA $A,B$	327,680	0.051%	73,728	0.079%
Convolutional experts	9,728	0.002%	3,648	0.004%
MoE gating ( $W_{g},W_{n}$ )	1,024	$<$ 0.001%	384	$<$ 0.001%
Mask decoder (incl. IoU head)	4.06 M	0.633%	4.06 M	4.33%
Trainable per head	4.40 M	0.69%	4.13 M	4.41%

Appendix

Table 5: Trainable parameters per defect head, counted from the instantiated model (

r=2

M=8

). A checkpoint contains exactly these. The mask decoder is the same module on both backbones.

Graph	Conv-LoRA	Attention	On device	Cosine
Stock ViT-B	no	SDPA	runs	0.99994
Stock ViT-B	no	eager	runs	0.99994
DCM-SAM pore head, as trained	yes	eager	fails, needs 4.07 GB	n/a
DCM-SAM pore head, rewritten	yes	SDPA	runs	0.99988
DCM-SAM inclusion head, rewritten	yes	SDPA	runs	0.99995
Static one-hot routing vs. dynamic top- $1$ (PyTorch, eight slices): $13$ of $8.4$ M mask pixels differ

Appendix

Table 6: Attention-formulation diagnostic on the IQ-9075 at

1024^{2}

, FP16. Cosine is against the corresponding PyTorch FP32 output on a real XCT slice.

Artifact	Experts	NPU ops	Inference (ms)	Peak mem. (MB)	Load (ms)
Pore encoder	$M=2$	1012	1980.8	11.8	825
Pore decoder	n/a	205	3.8	9.6	160
Inclusion encoder	$M=8$	1444	8134.3	13.9	868
Inclusion decoder	n/a	205	3.8	9.6	167

Appendix

Table 7: Measured cost of the four graphs on the IQ-9075, FP16. The model is deployed at

1024^{2}

(§ 3.4 ); the per-operator profiler attaches at

512^{2}

, so the encoder rows are timed there, from an export of the same weights. At

1024^{2}

the decoders measure

15.4

and

15.3

ms. Peak memory excludes the loaded binary.

Split	IoU (%)	Dice (%)	Tol-F1 (%)	Precision (%)	Recall (%)
NIST test2	50.0	66.5	85.1	74.2	100.0
NIST test3	64.2	78.0	94.1	99.2	90.0
NIST test4	54.5	70.4	89.0	80.6	99.9
NIST test5	66.6	79.9	96.2	98.3	94.2
NIST test6 ∗	85.4	92.0	96.8	93.8	100.0
Mean	64.2	77.4	92.2	89.2	96.8

Appendix

Table 8: Per-split NIST results for the selected

(2,8)

ViT-B model. Precision and recall at the

5

px tolerance used for Tol-F1. ∗ test6 uses inversion and non-zero valid-region masking, calibrated for this specimen’s highly porous, inverted-intensity appearance.

Explore similar work

XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentation
Jul 15, 2026Md Mahedi Hasan, Md Mushfiqur Rahaman, Alan Pachkovskiy +3Defect Detection Domain Adaptation
Lane Department of Computer Science & Electrical Engineering West Virginia University, Morgantown, WV · Department of Industrial and Management Engineering West Virginia University, Morgantown, WV · School of Mathematical and Data Sciences West Virginia University, Morgantown, WV
NL-MambaXCT: Self-Supervised Nested-Learning Mamba for Nomex Honeycomb X-ray CT Defect Classification
May 25, 2026Ghaleb Aldoboni, Lobna Nassar, Fakhri Karray +1Defect Detection Cone-Beam Computed Tomography
Department of Computer Science and Engineering, American University of Ras Al Khaimah (AURAK), Ras Al Khaimah, United Arab Emirates · Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates · Centre for Pattern Analysis and Machine Intelligence, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada +1
Interpretable Computer Vision for Defect Detection in X-ray Tomography of Aerospace SiC/SiC Composites
May 19, 2026Antonio Peña Corredor, Julien Lesseur, Romain Nunez +2Defect Detection Cone-Beam Computed Tomography
SES · 1IRT Saint Exup´ery, Toulouse, France · 2Safran Ceramics, Le Haillan, France +1

Jul 15, 2026, cs.CVJ/K move · Enter open · S save

XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentation

Md Mahedi Hasan, Md Mushfiqur Rahaman, Alan Pachkovskiy, Imtiaz Ahmed+2

Lane Department of Computer Science & Electrical Engineering West Virginia University, Morgantown, WV · Department of Industrial and Management Engineering West Virginia University, Morgantown, WV · School of Mathematical and Data Sciences West Virginia University, Morgantown, WV

Defect Detection Domain Adaptation Inductive Bias

Open paper AskPDF arXiv

Defect segmentation in additive manufacturing (AM) X-ray computed tomography (XCT) images remains challenging due to severe class imbalance and large distribution shifts across scan conditions. Although recent foundation models such as the Segment Anything Model (SAM) provide strong general-purpose segmentation priors, their natural-image pre-training transfers poorly to the AM XCT domain, where defects appear as subtle non-semantic microstructural anomalies. Moreover, adapting SAM to the AM domain is further limited by the large domain gap and scarcity of labeled real XCT data. We present XCT-SAM, a sequential parameter-efficient adaptation framework for AM XCT defect segmentation. Instead of adapting SAM directly from natural images to XCT data, we first fine-tune Conv-LoRA adapters on an alloy-microstructure dataset and subsequently transfer the adapted model to XCT images, progressively bridging the domain gap. Using Conv-LoRA adapters with rank r=2, the framework injects convolutional spatial inductive bias into SAM's backbone while training approximately 4.15M parameters and keeping over 99% of the model frozen. We evaluate XCT-SAM on out-of-distribution CycleGAN-XCT benchmarks and real-world NIST XCT scans. Across both settings, XCT-SAM consistently outperforms zero-shot SAM and other domain-adapted SAM baselines, achieving the best overall IoU and Dice scores. These results demonstrate the effectiveness of intermediate domain adaptation with parameter-efficient adapters for industrial XCT defect segmentation. The source code is publicly available at https://github.com/Mahedi-61/XCT-SAM.git

Abstract

Explore similar work

XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentation

NL-MambaXCT: Self-Supervised Nested-Learning Mamba for Nomex Honeycomb X-ray CT Defect Classification

Interpretable Computer Vision for Defect Detection in X-ray Tomography of Aerospace SiC/SiC Composites