CoRe-SAM3: Conditional Semantic--Visual Reconciliation for SAM3 Crack Segmentation
Authors: Shipeng Liu, Liang Zhao, Dengfeng Chen
Organizations: School of Mechanical and Electrical Engineering, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China · School of Artificial Intelligence and Robotics, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China · School of Building Equipment Science and Engineering, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China
Crack segmentation requires a model to recognize target semantics while accurately recovering thin, low-contrast, and topologically continuous local structures. Although SAM3 provides strong open-concept segmentation, its direct application to the crack domain still misses weak cracks, activates crack-like background regions, and produces local boundary errors. We first diagnose the functional differences between the internal prompt-conditioned semantic representation and native visual representation of SAM3 on five crack datasets. The results show that the semantic representation already carries most task information for crack prediction, whereas the utility of the visual representation depends on the current semantic state. Directly combining the two representations does not yield consistent gains. Based on this finding, we propose Conditional Semantic--Visual Reconciliation, termed CoRe. CoRe retains semantic prediction as the primary decision path, applies lightweight semantic calibration to adjust the target-domain decision mapping, and uses spatially aligned native visual evidence to generate a zero-initialized, bounded, and regularized conditional residual that selectively corrects existing predictions. Across five domains, CoRe-SAM3 improves the average Crack IoU from 62.34% to 70.47% and clDice from 81.98% to 89.24%, while introducing only 18.914 K trainable parameters. Prediction-transition analysis further shows that CoRe corrects an average of 34.38% of native errors, with a damage rate of only 0.23% on pixels correctly classified by native SAM3. These results demonstrate that constrained prediction correction based on the functional differences between internal representations provides an effective and parameter-efficient target-domain adaptation strategy for vision foundation models with strong task-specific semantic priors.
Figures & tables
Figure 1: Qualitative visualization of CoRe-SAM3 response correction. From left to right, the columns show the input image, native SAM3 semantic map, native mask, signed CoRe adjustment map, and final CoRe-SAM3 mask. In the color-coded mask overlays, green denotes true-positive crack pixels, blue denotes false negatives, and red denotes false positives. The native semantic map represents the continuous prompt-conditioned crack response. In the signed adjustment map, red indicates positive correction, blue indicates negative correction, and near-white regions indicate negligible changes. From top to bottom, the examples illustrate recovery of weak crack evidence, suppression of unsupported crack-like responses, and preservation of reliable predictions. The inset reports a 34.88% correction rate on native errors and a 0.23% damage rate on pixels correctly classified by native SAM3.
Figure 2: Representation diagnostics across five crack datasets: CamCrack789, CrackMap, Crack500, DeepCrack, and TUT. (a) Label probing compares the average precision (AP) of semantic ( S ) and visual ( V ) representations for predicting crack labels. (b) Error probing evaluates error prediction using semantic cues (Error-S), visual cues (Error-V), and their joint representation (Error-S+V). (c) Joint gain over Error-S reports ΔAP=AP(Error-S+V)−AP(Error-S) , where positive and negative values indicate improvement and degradation, respectively. Light-blue diagonal, light-orange dotted, and light-green cross-hatched bars denote semantic or Error-S, visual or Error-V, and joint S+V or Error-S+V, respectively. Panels (a) and (b) use a truncated AP axis with a lower bound of 0.6 for readability.
Figure 3: Overview of CoRe-SAM3. CoRe adapts SAM3 for crack segmentation by preserving the prompt-conditioned semantic prediction as the primary decision path and exploiting native visual evidence for conditional residual correction. The frozen SAM3 backbone produces semantic features Fs and native visual features Fv . Semantic calibration first adjusts the target-domain decision mapping, while the conditional reconciliation branch predicts a bounded residual ΔZrec from joint semantic–visual evidence. The final prediction conservatively updates the calibrated semantic belief with this residual. Boundary-aware weighted BCE loss, Dice loss, and residual regularization jointly balance effective correction against the magnitude of prediction changes.
Dataset
Train
Validation
Test
Image resolution
CamCrack789
553
79
157
640×480
Crack500
2357
336
675
(360∼648)×(360∼640)
CrackMap
84
12
24
256×256
DeepCrack
368
53
106
(384∼544)×(384∼544)
TUT
987
139
282
640×640
Table 1: Dataset statistics and original image resolutions for the five crack-segmentation benchmarks. The numbers of images in the training, validation, and test subsets follow the dataset splits used in all experiments.
Method
Trainable Params
CamCrack789
Crack500
CrackMap
DeepCrack
TUT
Avg.
Crack specialist
SCSegamba [ 11 ]
2.80 M
66.65
58.43
62.20
81.22
70.23
67.75
MixerCSeg [ 12 ]
2.54 M
69.20
59.37
63.76
83.24
64.62
68.04
RIFT-T [ 13 ]
0.47 M
72.20
61.42
64.09
83.74
68.78
70.04
SAM3-based adaptation
HAT-SAM3 [ 45 ]
85.74 M
62.74
57.44
48.95
76.75
64.15
62.01
Table 2: Overall segmentation performance across five crack datasets. The table reports Crack IoU (%) for CoRe-SAM3, native SAM3, SAM3-based adaptation methods, and dedicated crack-segmentation networks. Results of dedicated crack-segmentation networks serve as task-level performance references because their architectures, pretraining schemes, and input resolutions differ from those of SAM3-based methods.
Method
Crack IoU ↑
mIoU ↑
F1 ↑
Precision ↑
Recall ↑
PAR
clDice ↑
SAM3
62.34
80.33
73.14
74.70
74.16
0.939
81.98
CoRe-SAM3 (ours)
70.47
84.53
81.58
79.94
85.60
1.124
89.24
Δ
+8.13
+4.20
+8.44
+5.24
+11.44
–
+7.26
Table 3: Multi-metric comparison between native SAM3 and CoRe-SAM3 across five crack datasets. Values are macro-averaged over the five datasets. Improvements are absolute differences between CoRe-SAM3 and native SAM3. PAR denotes the ratio of predicted foreground area to ground-truth foreground area.
Semantic calibration
Semantic residual
Visual evidence
CamCrack789
CrackMap
Crack IoU
PAR
clDice
Crack IoU
PAR
clDice
70.48
1.129
93.50
55.61
0.843
77.97
✓
72.98
1.043
93.77
59.16
1.247
84.96
✓
✓
73.53
1.052
93.74
63.30
1.257
85.14
✓
✓
✓
74.71
1.034
94.69
65.78
1.223
89.09
Table 4: Incremental evaluation of semantic calibration and semantic–visual reconciliation. Starting from native SAM3, semantic calibration, semantic residual correction, and native visual evidence are progressively introduced to examine their respective contributions on CamCrack789 and CrackMap.
Method
Semantic prior retained
Visual evidence
Prediction form
Crack IoU
F1
clDice
SAM3
✓
Native prediction
63.05
74.95
85.74
Semantic calibration
✓
Calibrated prediction
66.07
81.60
89.37
Direct S+V fusion
✓
Full prediction
66.04
81.16
89.33
S-only residual
✓
Residual correction
67.96
81.88
88.77
S+V residual
✓
✓
Residual correction
68.55
81.53
90.52
CoRe-SAM3 (ours)
✓
✓
Bounded residual correction
70.24
82.10
91.89
Table 5: Comparison of strategies for incorporating semantic and visual representations. Direct semantic–visual fusion predicts a new segmentation output from the two representations, whereas residual variants preserve the calibrated semantic prediction and use additional representations for prediction correction. Results are averaged over CamCrack789 and CrackMap.
Variant
Zero initialization
Bounded residual
Lres
Crack IoU
CR ↓
DR ↓
Unconstrained residual
68.21
30.64%
0.33%
+ Zero initialization
✓
68.55
30.11%
0.29%
+ Bounded residual
✓
✓
69.16
29.40%
0.19%
CoRe-SAM3
✓
✓
✓
70.24
29.16%
0.18%
Table 6: Effect of conservative constraints on residual reconciliation. Zero initialization, bounded residual prediction, and residual regularization are progressively introduced. CR and DR denote the correction rate on originally misclassified pixels and the damage rate on originally correctly classified pixels, respectively. Results are averaged over CamCrack789 and CrackMap.
Loss configuration
Crack IoU
Precision
Recall
PAR
clDice
BCE + Dice
67.63
75.36
88.50
1.243
90.28
+ Ring weighting
69.16
79.09
85.32
1.150
91.39
+ Residual regularization
70.24
79.59
86.74
1.129
91.89
Table 7: Effect of crack-aware optimization objectives. The comparison progressively introduces crack-boundary ring weighting and residual regularization on top of standard BCE and Dice losses to evaluate their effects on foreground recovery, suppression of false activations, and structural preservation.
Figure 4: Pixel-level prediction transitions from native SAM3 to CoRe-SAM3 across five crack datasets. (a) Log-scale paired counts of pixels misclassified by native SAM3 and those corrected by CoRe-SAM3, together with the corresponding correction rate (CR). (b) Log-scale paired counts of pixels correctly classified by native SAM3 and those damaged by CoRe-SAM3, together with the corresponding damage rate (DR). (c) Class-conditional transition rates for false-negative-to-true-positive (FN → TP), false-positive-to-true-negative (FP → TN), true-positive-to-false-negative (TP → FN), and true-negative-to-false-positive (TN → FP) changes.
Region / Statistic
CamCrack789
CrackMap
Crack500
DeepCrack
TUT
FP R−
98.72%
94.45%
88.64%
97.78%
96.54%
FN R+
68.95%
90.85%
91.49%
74.84%
87.46%
Table 8: Directional statistics of prediction adjustments in false-positive and false-negative regions. R− denotes the proportion of false-positive pixels receiving negative logit adjustments, while R+ denotes the proportion of false-negative pixels receiving positive logit adjustments. FP and FN regions are determined from native SAM3 predictions.
Figure 5: Directional analysis of prediction adjustments induced by CoRe-SAM3. The total prediction adjustment is defined as A=Zfinal−Zsam and incorporates both semantic calibration and conditional reconciliation. (a) Mean ± standard deviation and median signed adjustments for true-positive (TP), true-negative (TN), false-positive (FP), and false-negative (FN) regions under the native SAM3 prediction. (b) Dataset-wise proportions of positive and negative adjustments, denoted by R+ and R− , for the same four regions.
Figure 6: One-factor-at-a-time sensitivity of adapted test mIoU to the residual scale s , Dice loss weight λd , and residual regularization weight λr on CamCrack789 and CrackMap. Dashed vertical lines indicate the default settings, while all remaining hyperparameters are fixed at their default values. The broken y-axis omits the interval 0.83–0.86 for improved visibility.
Figure 7: One-factor-at-a-time sensitivity of adapted test mIoU to the morphological radius r and outside-ring weight ω on CamCrack789 and CrackMap. Dashed vertical lines indicate the default settings, with all other hyperparameters fixed. The broken y-axis omits the interval 0.83–0.86 for improved visibility.
Figure 8: Qualitative examples of recovering weak and incomplete crack responses. Compared with native SAM3, CoRe-SAM3 restores crack regions with weak semantic responses and improves the continuity of thin or locally interrupted structures while largely preserving the original crack topology.
Figure 9: Qualitative examples of suppressing crack-like false-positive responses. CoRe-SAM3 conditionally reconciles the existing semantic prediction with native visual evidence, reducing erroneous activations caused by joints, texture boundaries, shadows, and other crack-like background structures.
Figure 10: Qualitative examples of preserving reliable native SAM3 predictions during target-domain adaptation. For regions already segmented correctly by native SAM3, CoRe-SAM3 applies localized response and boundary adjustments while preserving the overall prediction structure, illustrating the conservative behavior of the proposed reconciliation strategy.
Method
Trainable Params
Params
FLOPs
Latency
Peak GPU Memory
SAM3
0
840.510 M
3.672 T
451.2 ms
3889.7 MiB
CoRe-SAM3
18.914 K
840.529 M
3.678 T
539.1 ms
3889.7 MiB
Table 9: End-to-end computational efficiency of SAM3 and CoRe-SAM3. Latency and peak GPU memory are measured during complete model inference without cached SAM3 features. FLOPs are estimated over the complete forward path. Trainable parameters include only the parameters updated during task adaptation.
Crack segmentation is essential for infrastructure inspection and structural health assessment, but existing high-performance methods typically require task-specific pixel-level annotations and training. Text-promptable vision foundation models enable zero-shot deployment, yet their final mask proposals are poorly suited to thin, fragmented, and low-contrast cracks, whose evidence may be suppressed, truncated, or over-expanded during mask generation. We find that language-conditioned semantic responses within the SAM3 decoder preserve more continuous and complete crack evidence than its final masks. Based on this observation, we propose Semantic-Edge Response Decoding (SERD), which interprets internal responses as a dense crack-likelihood field, calibrates them with a lightweight edge prior, and generates crack masks using a unified global threshold, without annotation or fine-tuning. Experiments on six public datasets show that SERD consistently improves over native SAM3 and outperforms the compared zero-shot and open-vocabulary segmentation methods, achieving an average Crack IoU of 61.14%, 4.63 points higher than SAM3. Further analyses show that most gains arise from directly decoding internal semantic responses, while edge calibration improves structural recovery and false-positive control without increasing end-to-end inference overhead. These results suggest that, for thin and non-compact targets, internal continuous responses can provide a more transferable interface than the final masks of foundation models. Code is available at: https://github.com/xauat-liushipeng/SERD
Reliable crack assessment requires not only accurate pixel-level masks but also connected crack geometry and confidence estimates that remain stable under domain shift. However, existing segmentation models can achieve high overlap scores while fragmenting cracks, missing fine branches, and providing no calibrated uncertainty. To address this gap, this paper proposes CrackGeoFM, a multi-task framework that combines a frozen visual foundation backbone with crack-specific adaptation for mask prediction, skeleton reconstruction, and uncertainty estimation. The framework integrates a Frequency-Guided Crack Enhancement Module (FCEM) to enhance high-frequency crack cues, a Crack-Domain Feature Adaptation Module (CFAM) to adapt frozen backbone features to crack-domain patterns, and a Structure-Aware Multi-Task Decoder (SMTD) to jointly decode masks, skeletons, and uncertainty. Across 20 crack datasets, CrackGeoFM achieves state-of-the-art segmentation, improved topology preservation, calibrated uncertainty, and effective few-shot adaptation with only five labeled images. These results support reliable, generalizable, and engineering-oriented crack analysis for infrastructure assessment.
Recent crack segmentation methods often follow generic semantic segmentation designs, using stronger backbones, hybrid CNN-Transformer-Mamba encoders, and auxiliary enhancement branches. Although effective, this raises whether stronger generic feature mixing is the most suitable direction for crack segmentation. We instead formulate crack segmentation as sparse structural recovery. Cracks have limited category-level semantics but strong morphological regularities, being thin, sparse, anisotropic, locally fragmented, and easily confused with textures or shadows. Thus, the key bottleneck lies in preserving weak structural evidence, recovering directional continuity, and suppressing background coupling. We propose RIFT, a compact family of morphology-aligned crack segmentation models. Rather than compressing a complex generic architecture, RIFT is simple by design, preserving local evidence, aggregating cooperative directional continuity, and restoring crack structures through lightweight multi-scale fusion. Experiments on four public benchmarks show that RIFT achieves the best or tied-best results across the 16 main metrics against reproduced representative baselines. RIFT-B gives the strongest overall accuracy, while RIFT-T provides the best deployment efficiency with only 0.47M parameters and high inference speed. Topology-aware evaluation, ablations, transfer experiments, and visualizations further verify that task-aligned simplicity can match or surpass complex hybrid architectures when its inductive bias fits crack morphology. Code: https://github.com/xauat-liushipeng/RIFT