HullProbe: Measuring Interventional Part Reliance in Maritime Vessel Classifiers

Faisal Mohammad; Mohamed Younis

cs.CVOct 18, 2026

HullProbe: Measuring Interventional Part Reliance in Maritime Vessel Classifiers

Authors: Faisal Mohammad, Mohamed Younis

Organizations: University of Maryland, Baltimore County · Johns Hopkins University Applied Physics Laboratory

Abstract

Recognizing a vessel's type from an image supports maritime surveillance and sanctions monitoring, but a classifier can reach high accuracy by leaning on the generic hull instead of the more distinctive superstructure, and is then fooled when a vessel is re-hulled or occluded. De-biasing methods claim to remove such reliance, yet the field certifies them with worst-group accuracy, a proxy that needs group labels and only infers reliance from accuracy. We measure the reliance directly. HullProbe swaps a registered anatomical part of a vessel for another instance's and reads how much that part drives the decision, using a same-class swap to control for compositing artifacts. Because a surface vessel is a hull and a superstructure arranged along one axis, its parts form a 2x3 grid whose 2^6 = 64 swaps we evaluate exhaustively, and a single Walsh transform returns each part's interventional influence and its interaction spectrum. On planted shortcuts, where the true reliance is known, the measure tracks worst-group accuracy (r = -0.94 over 45 configurations), and replacing its learned segmenter with human annotations changes the hull-share by less than the audit's resolution. On the 26 vessel types of MARVEL, the classifier places about 46% of its corrected sensitivity on the generic hull yet follows the superstructure when the two conflict, and the group-label-free de-biasers available there do not detectably reduce the hull's share. The findings hold across three architectures and a second maritime dataset, and the method extends to fine-grained vehicle recognition on Stanford Cars.

Figures & tables

Figure 1: Two orthogonal decompositions give a $2{\times}3$ part grid. From a vessel (top-left), a learned segmenter splits it vertically into {hull, superstructure} (top-right) and its PCA pose axis splits it longitudinally into canonical {forward, mid, aft} thirds (bottom-left); their product is a grid of six anatomically defined cells (bottom-right, each named in its legend), each individually registrable and swappable for a donor’s.

Approach	Direct	Interventional	Plausible	Group-label-free	Exact
Worst-group accuracy [ 47 ]	○ (proxy)	○	–	○	–
Gradient saliency (Grad-CAM [ 50 ] )	○	○	–	●	–
Occlusion / removal (RISE [ 42 ] , ROAR [ 24 ] , SAGE [ 8 ] )	◐	●	○	●	○
Counterfactual / generative edits [ 19 , 6 , 48 ]	◐	●	●	●	○
Functional decomposition (Shapley [ 35 ] , Sobol [ 55 ] )	●	◐	○	●	○
HullProbe (ours)	●	●	◐	●	● (64)

Table 1: Where HullProbe sits among approaches to measuring feature reliance (Sec. 2 ). Columns are the five properties discussed in Sec. 2 ; ● has it, ◐ partial, ○ lacks it, – not applicable. Exact = exhaustive over the

64

-point intervention set, not sampled over subsets (the donor law is still estimated).

Figure 2: HullProbe overview. A recipient $x$ and donor $d$ are decomposed into a $2{\times}3$ grid of six anatomical cells (hull/superstructure $\times$ forward/mid/aft). The swap operator $\phi(x,d,z)$ replaces any cell subset $z\in\{0,1\}^{6}$ of the recipient with the donor’s registered cells (here a full-hull swap, $z{=}(1,1,1,0,0,0)$ ), producing a plausible composite. Evaluating the frozen classifier $f$ over all $2^{6}{=}64$ patterns turns its behavior into a function $g(z)$ on the Boolean cube; one exact Walsh–Hadamard transform then yields the functional-ANOVA spectrum $W_{k}$ , the per-cell influence $\mathrm{Inf}_{p}$ and its hull-share ( $46\%$ ), and the semantic- $L_{0}$ radius.

Figure 3: Operator and control. For a recipient (left), a cross-class hull swap (middle) changes the hull’s class while keeping the superstructure; the same-class swap (right) carries the identical seams/inpainting but preserves the label, and is the artifact control subtracted in Eq. ( 7 ). Both are anatomically plausible vessels, though not photorealistic.

Figure 4: Convergent validity on planted shortcuts (Sec. 4.2 ). Interventional reliance vs. worst-group accuracy, colored by shortcut. (a) The measurement principle: three appearance recolors (hull color, background, watermark) track WGA (pooled

r{=}{-}0.97

35

configs). (b) The part-swap operator itself: three anatomical hull-grafts are planted and measured with the real

6

-cell swap, tracking WGA across

45

configs (

3

shortcuts

\times

3

seeds

\times

5

de-biasers; pooled

r{=}{-}0.941

). Each point is one de-biaser

\times

seed; the dashed line is the least-squares fit.

appearance recolor (validates the principle)
shortcut	ERM $\downarrow$	DFR $\downarrow$	corr(WGA, rel.)
hull color	$0.39$	$\mathbf{0.12}$	$-0.966$
background	$0.43$	$\mathbf{0.14}$	$-0.904$
watermark †	$0.08$	$\mathbf{0.02}$	$-0.891$
anatomical part-swap (validates the operator)
Veh-Carrier / Ro-ro	$0.58$	$\mathbf{0.09}$	$-0.945$

Table 2: Replication across six planted shortcuts (Sec. 4.2 ). Interventional reliance (flip rate) for the untreated ERM model and the best de-biaser (DFR), and the correlation between worst-group accuracy and reliance across all de-biaser

\times

seed configurations. Appearance shortcuts validate the measurement principle; anatomical shortcuts validate the swap operator itself (plant and measure both use the real

6

-cell hull graft, listed by the two donor hulls). Watermark † is a known null (no shortcut learned

\Rightarrow

near-zero measured reliance; its correlation, taken over that near-constant range, is not informative).

Panel of Figure 5 — Figure 5: Nearly additive anatomy and single-swap fragility. (a) Variance by interaction degree: $W_{1}{=}0.63$ of behavior is single-cell main effects and $0.80$ is degree $\leq 2$ , so per-cell attribution captures most of the decision (per-cell influence in App. Table 9 ). (b) Semantic- $L_{0}$ distribution: the fewest realistic part swaps that flip the class; $44\%$ of vessels flip on one swap ( $r{=}1$ ), the rest need $\geq 2$ , and $8\%$ never flip within the six cells.

method	hull-share	$\Delta$ vs ERM	$\Pr[r{=}1]$
ERM	$0.394{\scriptstyle\pm.003}$	—	$0.513$
CutMix	$0.396{\scriptstyle\pm.011}$	$+0.002{\scriptstyle\pm.018}$	$\mathbf{0.420}$
hull-rand.	$0.408{\scriptstyle\pm.011}$	$+0.012{\scriptstyle\pm.008}$	$0.548$
JTT	$0.399{\scriptstyle\pm.024}$	$+0.002{\scriptstyle\pm.019}$	$0.505$
LfF	$0.417{\scriptstyle\pm.019}$	$+0.022{\scriptstyle\pm.019}$	$0.538$
AFR	$0.399{\scriptstyle\pm.011}$	$+0.006{\scriptstyle\pm.007}$	$0.478$

Table 3: De-biasing audit (3 seeds, mean

\pm

std). Hull-share is the balance axis;

\Delta

is the paired change vs. ERM (positive

=

more hull-reliant);

\Pr[r{=}1]

is the concentration axis. No method reduces hull-share; CutMix trades concentration for unchanged balance; bg-swap worsens both. ∗ :

|\Delta|>2\sigma

. Each

\Delta

is averaged per seed over the recipients both that method and ERM classify correctly, so it is a tighter paired estimate and need not equal the difference of the two pooled hull-share columns (which use each method’s own recipient pool).

all held-out ( $n{=}194$ )
mask source	hull-share	raw	$W_{1}$	$W_{\leq 2}$	$\Pr[r{=}1]$
segmenter (both)	$0.49$ $[0.45,0.53]$	$0.47$	$0.67$	$0.84$	$0.35$
annotation (both)	$0.48$ $[0.44,0.53]$	$0.46$	$0.67$	$0.84$	$0.35$
annotation (recipient only)	$0.48$ $[0.44,0.52]$	$0.46$	$0.67$	$0.84$	$0.35$
$\Delta$ (both)	$-0.009$ $[-0.041,0.023]$	$-0.015$	$+0.004$	$0.000$	$0.000$
held-out and in MARVEL’s test split ( $n{=}17$ )

Table 4: Segmentation error: annotations vs. the segmenter (App. A ). The audit on the segmenter’s

202

held-out vessels with its predicted hull/superstructure maps, with the annotations, and with annotations for the recipient only; recipients, donors, silhouette, and classifier are identical across arms. Brackets are

95\%

recipient-bootstrap intervals;

\Delta

rows are paired (annotation minus segmenter). Left: pooled readouts. Right: per-cell corrected influence

\mathrm{Inf}_{p}^{\text{corr}}

on all held-out vessels.

variant	corrected hull-share	raw
reference (compatible, $K{=}4$ )	$0.51$ $[.40,.63]$	$0.47$
random donors ( $K{=}4$ )	$0.52$ $[.40,.66]$	$0.46$
compatible, $K{=}2$	$0.50$ $[.35,.63]$	$0.45$
compatible, $K{=}8$	$0.51$ $[.42,.62]$	$0.46$
seg. eroded ( $-3$ px)	$0.61$ $[.51,.72]$	$0.56$
seg. dilated ( $+3$ px)	$0.46$ $[.36,.57]$	$0.43$

Table 5: Sensitivity of the corrected hull-share [

95\%

bootstrap CI] to donor and segmentation choices (App. B ; fixed

61

-recipient subsample, which runs at

0.50

–

0.52

, above the

0.46

full-pool headline but within its bootstrap interval). Robust to donor selection and count; the corrected value is invariant to donor quality (random vs. compatible), validating the artifact control; segmentation boundary shifts move it in the expected direction.

robustness check	corr(WGA, reliance)
pooled Pearson $r$	$-0.941$
Spearman $\rho$	$-0.933$
per-shortcut ( $3$ )	$-0.945,\ -0.964,\ -0.928$
leave-one-out (by shortcut)	$-0.946,\ -0.932,\ -0.950$
cluster bootstrap $95\%$ CI	$[-0.964,\ -0.916]$

Table 6: The operator-level correlation is robust to the nested design (

45

configs; Sec. 4.2 ): reliance vs. worst-group accuracy stays strongly negative under every check, so it is not an artifact of the shared shortcuts and seeds.

method	WGA	re-swap flip	indep. flip
ERM	$0.03{\scriptstyle\pm.03}$	$0.79{\scriptstyle\pm.05}$	$0.23{\scriptstyle\pm.05}$
SUBG	$0.05{\scriptstyle\pm.08}$	$0.75{\scriptstyle\pm.12}$	$0.21{\scriptstyle\pm.03}$
GroupDRO	$0.21{\scriptstyle\pm.07}$	$0.54{\scriptstyle\pm.14}$	$0.18{\scriptstyle\pm.04}$
JTT	$0.13{\scriptstyle\pm.08}$	$0.65{\scriptstyle\pm.09}$	$0.17{\scriptstyle\pm.03}$
DFR	$\mathbf{0.65}{\scriptstyle\pm.07}$	$\mathbf{0.08}{\scriptstyle\pm.03}$	$\mathbf{0.12}{\scriptstyle\pm.06}$

Table 7: Circularity control (ResNet-50,

3

seeds, mean

\pm

std; App. C ). For each de-biaser, the operator’s re-swap flip (hull

\to

the other trained donor) and an independent-mechanism flip (hull

\to

a third, unseen donor through the identical compositor). Both fall from ERM to DFR and are positively correlated across methods (mean

r{=}0.73

), so the operator reads hull reliance that survives a change of donor, not donor or compositing memorization.

Figure 6: Hull sensitivity vs. sufficiency (Sec. 5.3 ). Top: under a hull/superstructure conflict (a cross-class donor hull grafted on, superstructure kept), the classifier follows the superstructure (the true class)

73\%

of the time and adopts the donor hull’s class only

4.8\%

. Bottom: per class, the hull sensitivity (share) far exceeds the rate at which the hull is actually sufficient to set the prediction. Every class sits well below the

y{=}x

line, so sensitivity should not be read as harmful reliance.

Table 14

method	hull	sup.	total	$\Delta$ hull	$\Delta$ total
ERM	$0.461$	$0.602$	$1.063$	—	—
CutMix	$0.455$	$0.552$	$1.006$	$-0.008{\scriptstyle\pm.031}$	$-0.059{\scriptstyle\pm.058}$
hull-rand.	$0.468$	$0.562$	$1.030$	$+0.006{\scriptstyle\pm.027}$	$-0.032{\scriptstyle\pm.044}$
JTT	$0.463$	$0.580$	$1.042$	$-0.002{\scriptstyle\pm.034}$	$-0.022{\scriptstyle\pm.034}$
LfF	$0.494$	$0.567$	$1.061$	$+0.031{\scriptstyle\pm.041}$	$-0.006{\scriptstyle\pm.054}$
AFR	$0.460$	$0.590$	$1.050$	$+0.005{\scriptstyle\pm.013}$	$-0.002{\scriptstyle\pm.012}$

Table 11: Absolute influence is flat, not just the ratio (App. F ;

3

seeds, mean

\pm

std over seeds; same audit as Table 3 ). Mean absolute hull, superstructure, and total cross-class flip-influence per method, with the paired change vs. ERM in absolute hull and total influence. No group-label-free method reduces absolute hull influence, and the small total changes are within seed noise; only background-swap (positive control) moves, raising both.

Figure 7: Two axes move independently (visualizes Table 3 , Sec. 5.4 ). Left: hull-share (which parts); ∗ significant vs. ERM. Right:

\Pr[r{=}1]

(how many parts). CutMix lowers fragility (right) without lowering hull reliance (left); bg-swap worsens both.

de-biaser	Grad-CAM	IG	RISE
ERM	$0.34$	$0.21$	$0.12$
SUBG	$0.32$	$0.18$	$0.12$
JTT	$0.36$	$0.23$	$0.12$
GroupDRO	$0.33$	$0.18$	$0.12$
DFR	$0.31$	$0.20$	$0.12$
range across de-biasers	$0.05$	$0.05$	$0.00$

Table 12: Correlational attribution locates the shortcut but cannot measure reliance. Fraction of attribution mass on the planted hull region (controlled hull-color shortcut,

5

de-biasers

\times

3

seeds,

120

images each; hull area

{\approx}0.12

). Every method stays nearly flat across de-biasers (range

{\leq}0.05

), whereas our interventional hull-color flip spans

0.27

over the same models, so attribution reports where, not how much.

Figure 8: A non-maritime instantiation (Sec. 5.5 ). Four Stanford Cars of distinct body styles ( left of each pair) and their natural $5$ -part decomposition ( right ): {body (gray), front-face (red), greenhouse (blue), body-side (green), wheels (yellow)}. Parts come from an open-vocabulary GroundedSAM prompt and the geometric PCA pose; the hull/superstructure segmenter and the forward/mid/aft grid are not used. Over the resulting $2^{5}$ -swap cube the greenhouse (roof and glazing) carries the majority of corrected interventional influence, well above the front grille.

Figure 9: The car swap operator (Sec. 5.5 ). A recipient car, its natural $5$ -part decomposition, and a cross-class greenhouse swap that flips the predicted class, shown against the label-preserving same-class control that isolates the compositing artifact. This is the car analogue of the vessel operator in Fig. 3 .

Figure 10: Cars: nearly additive anatomy and greenhouse dominance (App. H ). (a) Walsh variance by interaction degree over the

2^{5}

cube: main effects hold

W_{1}{=}0.94

and degree-

\leq 2

holds

0.99

, so per-part attribution captures almost all of the decision. (b) Per-part flip-influence, raw (gray) and artifact-corrected (green): the greenhouse dominates, while the wheels and body fall to

\approx 0

once the same-class control adjusts for the compositing floor.

part	cross $\mathrm{Inf}^{\times}$	same $\mathrm{Inf}^{=}$	corr. $\mathrm{Inf}^{\text{corr}}$
greenhouse	$0.680$	$0.205$	$\mathbf{0.476}$
body-side	$0.210$	$0.073$	$0.137$
front-face	$0.148$	$0.103$	$0.045$
body	$0.000$	$0.000$	$0.000$
wheel	$0.034$	$0.056$	$-0.022$

Table 13: Cars: per-part flip-influence (

80

recipients,

2^{5}

cube; App. H ). Raw cross-class

\mathrm{Inf}^{\times}

, same-class control

\mathrm{Inf}^{=}

, and corrected

\mathrm{Inf}^{\text{corr}}

. The greenhouse carries the majority of corrected influence; wheels and body reduce to

\approx 0

under the control, as class-agnostic parts should.

Figure 11: Convergent validity ports to cars (Sec. 5.5 ). Three graded color shortcuts (

3\times 3

seeds

\times 5

de-biasers,

45

configs): group-label-free recolor-flip reliance tracks worst-group accuracy across the full range (pooled

r{=}{-}0.97

;

-0.97

without DFR), ERM leaning on the shortcut and DFR removing it. The car swap operator is shown in Fig. 9 .

shortcut	strength	ERM (WGA/flip)	DFR (WGA/flip)	corr
painted	strong	$0.09/0.82$	$0.72/0.08$	$-0.962$
background	medium	$0.47/0.30$	$0.82/0.05$	$-0.923$
watermark †	weak	$0.74/0.05$	$0.84/0.02$	$-0.743$

Table 14: Cars convergent validity, per shortcut (ResNet-50,

3

seeds each; App. H ). Three graded shortcuts span the worst-group-accuracy range; for each, WGA rises and the group-label-free recolor-flip reliance falls from ERM to DFR. Pooled

r{=}{-}0.973

over all

45

configurations (

-0.973

excluding DFR).

component	vessels (MARVEL, SeaShips)	cars (Stanford Cars)	requirement for a new domain
part set $P$	$2{\times}3$ : {hull, sup} $\times$ {fwd, mid, aft}, $\|P\|{=}6$	5 named parts, $\|P\|{=}5$	exhaustive partition into $\lesssim 8$ parts present on every instance
part extractor	SegFormer-B2 fine-tuned on $1{,}014$ masks (hull/sup); PCA thirds	GroundedSAM prompt + fixed token-to-part map; no annotation	a learned or promptable segmenter, or a geometric split
registration	closed-form PCA similarity transform	same	a canonical frame with part-to-part correspondence (rigid, elongated, consistent view)
validity filter	MARVEL: PCA aspect $\geq 1.40$ ; SeaShips: every cell $\geq 150$ px ( $76\%$ yield)	$\geq 3$ parts covering $\geq 45\%$ , aspect $\in[1.35,6]$ ( $97\%$ yield)	a rule rejecting views where parts are missing or unregistrable
donor pool	$K{=}4$ aspect-compatible cross-class $+\,2$ same-class per recipient	same	$\geq 3$ held-out instances per class
classifier access	black box, predicted labels (probabilities for the soft variant)	same	no gradients, retraining, or group labels

Table 15: What it takes to port HullProbe (App. H ). Components a new domain must supply, as instantiated for vessels and cars. Only the part decomposition and the registration are domain-specific; the swap operator, artifact control, cube evaluation, and readouts are shared.

Figure 12: Zero-shot transfer to SeaShips (second dataset). Both the MARVEL-trained SegFormer segmenter (middle, the hull/superstructure $\times$ forward/mid/aft decomposition) and the swap operator ( bottom , a cross-class hull swap, hull $\leftarrow$ donor type ) apply to SeaShips vessels with no retraining, staying clean across distinct ship types. This is the qualitative counterpart to the numerical replication (Sec. 5.5 ). Segmentation is zero-shot, so composites are noisier than on MARVEL, which the same-class control absorbs.

Image Classification

Image classification assigns semantic labels to images, enabling recognition of objects, scenes, conditions, and fine-grained categories.

See all 381 papers

Aug 27, 2026

Hull First, Wake Second: Wake-Reliance Suppression for Robust Maritime Vessel Detection

Maritime vessel detectors often face scenes where hulls are small, low-contrast, or blurred, while wakes are longer and easier to detect. This creates a wake-reliance problem: detectors may miss slow or stationary vessels with weak wakes, or produce false positives on wake-like water clutter. We propose HullWake, a hull-first wake-second framework for robust maritime vessel detection. HullWake separates proposal-centered hull evidence from directional wake context, extracts wake cues with bidirectional proposal-anchored corridors, and suppresses wake-dominant predictions through wake response supervision, wake-attenuated consistency, wake-only confidence suppression, and hull--wake decorrelation. We also introduce a wake-oriented evaluation protocol covering weak/no-wake vessels, wake-like hard negatives, worst-group AP, and confidence drop after wake attenuation. Experiments are conducted on Curated-Wake, a wake-oriented maritime dataset of about 10,000 images curated from Ships/Vessels in Aerial Images, the SMD benchmark, and SeaDronesSee, with newly added detection- and segmentation-level wake annotations. Compared with box-only detectors and mask-supervised segmentation baselines, HullWake improves overall AP, weak/no-wake robustness, wake-like false positives, worst-group AP, and confidence stability after wake attenuation.

Yefan Wang, Xingyu Wang, Ruibiao Zhu +1

University of Shanghai for Science and Technology, Shanghai, China · University of Science and Technology Liaoning, Liaoning, China · School of Computing, College of Systems and Society, The Australian National University, ACT, Australia +1

Ship Detection Object Detection Spurious Correlation Robustness Aerial Object Detection

Ask PDF

Sep 23, 2026

Strip Convolution and Direction-Aware Exclusion Loss for Oriented Ship Detection

Oriented ship detection in very high resolution (VHR) remote sensing imagery remains challenging due to elongated hull geometry and dense target distributions in complex port scenes. Existing methods typically address geometric representation and duplicate suppression separately. To jointly tackle these issues, we propose an oriented ship detector with two complementary components. The C3k2_Strip module employs orthogonal strip convolutions to better capture elongated hull structures, while the Class-Aware Direction-Aware Exclusion Loss (CA-DAEL) suppresses redundant predictions using class, direction, and confidence cues. Experiments on HRSC2016 and DIOR-R achieve 78.45% and 53.71% mAP50:95, respectively, with only 2.91M parameters. On HRSC2016, the proposed method improves mAP50:95 by 6.32 percentage points over the YOLOv11-OBB baseline, demonstrating its effectiveness for accurate oriented ship detection.

Bin Chen, Yuanyuan Liu, Peng Yang +1

School of Information and Software Engineering, East China Jiaotong University, Nanchang 330013, China. · Jiangxi Vocational University of Foreign Studies, Nanchang 330099, China.

Remote Sensing Ship Detection Object Detection

Ask PDF

Jun 19, 2026

Sea-Scan: High-Accuracy, ML-based Dark Vessel Detection and Localisation via Weakly Supervised DAS Monitoring

We present an ML-based vessel detection and localization system, trained with weak supervision from imperfect AIS labels, that achieves a 97.8% detection rate at 1.98% false-trigger rate, successfully identifies dark-vessel events from unlabeled data.

Tian Tian, Agastya Raj, Lara Flanagan +2

School of Computer Science and Statistics, IRIS Research Group, ADAPT Research Centre, Trinity College Dublin, Ireland · School of Engineering, ADAPT Research Centre, Trinity College Dublin, Ireland

Distributed Acoustic Sensing Object Detection Ship Detection Maritime Surveillance Maritime Transportation

Ask PDF

Abstract

Image Classification

Explore similar work

Hull First, Wake Second: Wake-Reliance Suppression for Robust Maritime Vessel Detection

Strip Convolution and Direction-Aware Exclusion Loss for Oriented Ship Detection

Sea-Scan: High-Accuracy, ML-based Dark Vessel Detection and Localisation via Weakly Supervised DAS Monitoring