Weakly supervised multimodal segmentation of acoustic borehole images with depth-aware cross-attention
Authors: Jose Luis Lima de Jesus Silva
Organizations: Federal University of Bahia, Institute of Geosciences, Department of Geophysics, Salvador, BA 40170-115, Brazil · Grupo de Estudos e Aplicac¸ ˜ao de Inteligˆencia Artificial em Geof´ısica (GAIA), Federal University of Bahia, Salvador, BA 40170-115, Brazil
Acoustic borehole images provide high-resolution borehole-wall structure, but large-scale interpretation remains difficult because dense expert annotations are rarely available and subsurface information is intrinsically multimodal. The challenge is developing weakly supervised methods combining two-dimensional image texture with depth-aligned one-dimensional well-logs. Here, we introduce a weakly supervised multimodal segmentation framework that refines threshold-guided pseudo-labels through learned models. This preserves the annotation-free character of classical thresholding and clustering workflows while extending them with denoising, confidence-aware pseudo-supervision, and physically structured fusion. We establish that threshold-guided learned refinement provides the most robust improvement over raw thresholding, denoised thresholding, and latent clustering baselines. Multimodal performance depends strongly on fusion strategy: direct concatenation provides limited gains, whereas depth-aware cross-attention, gated fusion, and confidence-aware modulation substantially improve agreement with the weak supervisory reference. The strongest model, confidence-gated depth-aware cross-attention (CG-DCA), consistently outperforms threshold-based, image-only, and earlier multimodal baselines. Targeted ablations show its advantage depends specifically on confidence-aware fusion and structured local depth interaction rather than model complexity alone. Cross-well analyses confirm this performance is broadly stable. These results establish a practical, scalable framework for annotation-free segmentation, showing multimodal improvement is maximized when auxiliary logs are incorporated selectively and depth-aware.
Figures & tables
Figure 1: Construction of the weak supervisory signal and aligned multimodal inputs for the initial Antilope25 interval. (a) From left to right: raw global Multi-Otsu thresholding, denoised global Multi-Otsu thresholding, adaptive/local thresholding, cleaned pseudo-labels, and the corresponding confidence map. (b) From left to right: acoustic image amplitude (AMP), caliper (CAL), gamma ray (GR), bulk density (DEN), neutron porosity (NEU), compressional slowness (DTC), and deep resistivity (RES90). The log channels were interpolated to the image depth grid, normalized, and laterally replicated into image-shaped auxiliary channels.
Figure 2: Initial multimodal segmentation findings for the representative Antilope25 interval. (a) Input acoustic borehole image in decibel units. (b) Autoencoder-denoised reconstruction. (c) Reconstruction error map. (d) Cleaned pseudo-label map. (e) Raw global Multi-Otsu thresholding. (f) Global Multi-Otsu thresholding applied to the denoised image. (g) AE + KMeans baseline. (h) Final multimodal M-TGS segmentation using the denoised acoustic image and aligned auxiliary logs. (i) Overlay of the AE → Otsu segmentation. (j) Overlay of the image-only M-TGS segmentation. (k) Overlay of the multimodal M-TGS segmentation. (l) Confusion matrix comparing the pseudo-label reference with the multimodal M-TGS segmentation.
Figure 3: Botorosa47 multimodal case. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Bottom row: (e) raw thresholding, (f) denoised thresholding, (g) image-only refined segmentation, and (h) multimodal refined segmentation. The interval is characterized by laterally continuous banded structure. Both refined models produce segmentation maps that are markedly more coherent than the thresholding baselines, while the multimodal refiner yields the strongest overall result, achieving an agreement of 0.9133 with the pseudo-label reference.
Figure 4: Heavy-rerun interval from Antilope25 displaying robust refinement in a distinct morphology. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Bottom row: (e) raw thresholding, (f) denoised thresholding, (g) image-only refined segmentation, and (h) multimodal refined segmentation. The interval is dominated by vertically elongated structures. Both optimized models substantially outperform the thresholding references.
Figure 5: Heavy-rerun interval from Antilope25 illustrating a challenging case for multimodal fusion. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Bottom row: (e) raw thresholding, (f) denoised thresholding, (g) image-only refined segmentation, and (h) multimodal refined segmentation (Concat). The interval contains a localized anomalous structure embedded within a weakly textured, vertically striped background. The image-only refiner successfully isolates the compact anomaly, whereas simple multimodal concatenation amplifies background noise.
Figure 6: Architectural evolution of the multimodal refinement framework. Top , full schematic of the confidence-gated depth-aware cross-attention (CG-DCA) refiner, showing the image and log encoders, local depth window, cross-attention block, gating pathway, confidence-modulated fusion, decoder, and prediction stage. Bottom , conceptual comparison of the multimodal progression considered in this study: (a) early concatenation baseline, (b) ungated depth-aware cross-attention (DCA), and (c) confidence-gated DCA (CG-DCA).
Figure 7: Selected-case interval from Coala88 illustrating the progression from threshold-guided refinement to selectively fused multimodal refinement. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Middle row: (e) raw thresholding, (f) denoised thresholding, (g) pseudo-labels, and (h) concatenation-based multimodal refinement. Bottom row: (i) DCA, (j) G-DCA, (k) CG-DCA, and (l) CG-DCA overlay.
Figure 8: Selected-case interval from Botorosa47 illustrating the progression from threshold-guided refinement to the Phase 4 multimodal family. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Middle row: (e) raw thresholding, (f) denoised thresholding, (g) pseudo-labels, and (h) concatenation-based multimodal refinement. Bottom row: (i) DCA, (j) G-DCA, (k) CG-DCA, and (l) CG-DCA overlay. In this interval, the main visual difference is between the earlier baselines and the DCA-family methods: the latter better preserve the laterally continuous banding visible in the image and reduce the diffuse class spread seen in the earlier multimodal formulation.
Figure 9: Selected-case interval from Antilope25 illustrating the progression from threshold-guided refinement to the Phase 4 multimodal family. Top row: (a) input acoustic image, (b) autoencoder-denoised reconstruction, (c) reconstruction error, and (d) confidence map. Middle row: (e) raw thresholding, (f) denoised thresholding, (g) pseudo-labels, and (h) concatenation-based multimodal refinement. Bottom row: (i) DCA, (j) G-DCA, (k) CG-DCA, and (l) CG-DCA overlay. In this interval, the principal structure is a broad central dark body bounded by strong lateral transitions. Relative to the earlier baselines, the DCA-family methods recover this geometry more cleanly, and the progression from concat multimodal to DCA, G-DCA, and CG-DCA is visually more apparent than in the Botorosa47 case.
Figure 10: Targeted ablation and cross-well robustness of the final multimodal framework. (a) G-DCA segmentation. (b) G-DCA gate map. (c) CG-DCA gate map. (d) CG-DCA segmentation. Panels (a) – (d) illustrate how the confidence-gated variant produces a more structured and spatially selective multimodal control pattern than G-DCA in the representative Antilope25 heavy-top interval. (e) Full CG-DCA with depth-window radius r=2 . (f) CG-DCA without confidence-weighted loss. (g) CG-DCA without confidence-aware fusion. (h) CG-DCA without either confidence-weighted loss or confidence-aware fusion. (i) Full CG-DCA with r=0 , that is, same-depth-only log interaction. (j) CG-DCA without the learned gate. (k) Effective uncertainty-driven fusion map. (l) Confidence-weight map used for pixelwise loss weighting. Together, panels (e) – (l) show that the strongest degradation arises when confidence-aware fusion is removed, whereas removing confidence-weighted loss alone or removing the gate has a smaller and more interval-dependent effect. (m) Held-out-well style performance profile summarizing mean agreement on each well for all compared methods. (n) Cross-well robustness heatmap showing agreement against the pseudo-label reference across wells and methods.
Method
Mean over wells
Std. over wells
Min well
Max well
Range
Raw Otsu
0.6138
0.0366
0.5719
0.6594
0.0875
AE → Otsu
0.7026
0.0230
0.6726
0.7265
0.0539
AE+KMeans
0.4713
0.0168
0.4544
0.4887
0.0343
Image-only
0.7339
0.0218
0.7138
0.7603
0.0466
Concat multimodal
0.7518
0.0226
0.7134
0.7701
0.0567
DCA
0.8044
0.0328
0.7574
0.8425
0.0851
Table 1: Cross-well robustness summary for all compared methods. Agreement is reported against the pseudo-label reference and therefore measures internal consistency within the annotation-free framework rather than externally validated geological accuracy. Mean over wells is the primary ranking criterion; the remaining columns summarize well-to-well variability.
Variant
Overall mean
Std.
Broad mean
Heavy mean
Δ vs full (r=2)
Full wins
Full win rate
CG-DCA full (r=2)
0.9172
0.0888
0.8783
0.9755
0.0000
–
–
CG-DCA no conf-loss (r=2)
0.9187
0.0837
0.8842
0.9705
0.0016
3/10
0.30
CG-DCA no conf-fusion (r=2)
0.8904
0.0984
0.8509
0.9497
-0.0268
10/10
1.00
CG-DCA no conf-loss/fusion (r=2)
0.8914
0.1006
0.8495
0.9543
-0.0258
10/10
1.00
CG-DCA full (r=0)
0.9181
0.0867
0.8792
0.9765
0.0010
6/10
0.60
CG-DCA no gate (r=2)
0.9207
0.0861
0.8821
0.9786
0.0035
3/10
0.30
Table 2: Targeted ablation summary for the CG-DCA family. Means are reported over the 10 targeted ablation intervals. The final columns summarize pairwise comparison against the full CG-DCA model with r=2 when such a comparison is defined. Negative Δ indicates lower agreement than the full model.
Borehole archives commonly contain core tray photographs and corresponding digital log reports, but no native pixel-level crack annotations. We investigate two complementary approaches for extracting defect-spacing information from these archives. First, structured spacing categories recovered from the report text layer provide weak interval-level labels for classification. A DINO encoder trained on unlabeled core crops supplies domain-specific representations, and a manually verified subset is used to identify label inconsistencies. Second, we manually annotate 5,087 extracted core-row images and evaluate fully supervised crack-segmentation models. Our gated U-Net combines PiDiNet edge maps with Mask R-CNN masks through a learned spatial gating mechanism. This configuration achieves an F1 score of 0.860 and a crack-class IoU of 0.754, the highest result among the evaluated segmentation configurations. Deterministic post-processing converts predicted crack locations into defect-spacing categories. Separate rule-based branches estimate core-relative bedding angles and lithological color descriptors; their predictions agree with log-report references on 75.4% and 84.7% of 1,200 evaluated images, respectively. Because these references are extracted from existing reports, the reported values measure agreement with recorded geological observations rather than independent physical accuracy. The resulting framework combines report-derived weak supervision for spacing classification with fully supervised segmentation for image-based crack localization.
Usama Imdad, Ali Khan, Luke Lu +2
aLahore University of Management Sciences, Pakistan · ePI-Neuron, Australia · bSydney Water, Australia +2
In multimodal crack segmentation for industrial facilities, the key challenge is preventing missing modalities from degrading pixel-level performance while maintaining low computational cost. Existing methods struggle to address semantic degradation caused by missing modalities. We propose Compass, a lightweight network for robust crack segmentation under arbitrary missing modalities. Compass comprises Degradation Simulation Distillation (DSD), Needle Block, and Evidential Topology-Preserving Fusion (ETPF). DSD constructs a degradation simulation stream that mimics more severe missing conditions and performs reciprocal distillation with the original stream, decoupling complete perception from degradation adaptation. Within DSD, Feature-Aware Prototype Transmitter (FAPT) performs modality agnostic prototype-guided feature completion to maintain semantic integrity under incomplete modality conditions. As a lightweight backbone, Needle injects crack-direction cues into WKV modulation and combines connectivity-aware gating with anisotropic context probing for structure-aware modeling. ETPF fuses multimodal features via Dempster-Shafer evidential combination with uncertainty-gated decoding, preserving crack topology while suppressing unreliable features. Experiments on three datasets demonstrate state-of-the-art (SOTA) performance under diverse missing modality scenarios. Even with 90% depth modality missing on CrackDepth, Compass achieves F1 of 0.8216 and mIoU of 0.8434 with only 2.58M parameters. The code is available at https://github.com/Karl1109/Compass.
Hui Liu, Chen Jia, Fan Shi +3
Engineering Research Center of Learning-Based Intelligent System (Ministry of Education) Tianjin University of Technology Tianjin, China
Medical experts often manually segment images to obtain diagnostic statistics and discard the resulting annotations. We aim to train segmentation models to alleviate this burden, but constrained to the retained summary statistics (e.g., the area of the annotated region). Empirical results suggest that statistics alone are insufficient for this task, but adding weak information in the form of a few pixels within the area of interest significantly improves performance. We use a novel loss function that combines terms for image reconstruction quality, matching to summary statistics, and overlap between the predicted foreground and the weak supervisory signal. Experiments on standard image, ultrasound (breast cancer), and Computed Tomography (CT) scan (kidney tumors) data demonstrate the utility and potential of the approach.
Omkar Kulkarni, Edward Raff, Tim Oates
University of Maryland, Baltimore County, Baltimore MD, USA · Crowdstrike, Austin TX, USA