HullProbe: Measuring Interventional Part Reliance in Maritime Vessel Classifiers

Faisal Mohammad; Mohamed Younis

cs.CVOct 18, 2026

HullProbe: Measuring Interventional Part Reliance in Maritime Vessel Classifiers

Authors: Faisal Mohammad, Mohamed Younis

Organizations: University of Maryland, Baltimore County · Johns Hopkins University Applied Physics Laboratory

Abstract

Recognizing a vessel's type from an image supports maritime surveillance and sanctions monitoring, but a classifier can reach high accuracy by leaning on the generic hull instead of the more distinctive superstructure, and is then fooled when a vessel is re-hulled or occluded. De-biasing methods claim to remove such reliance, yet the field certifies them with worst-group accuracy, a proxy that needs group labels and only infers reliance from accuracy. We measure the reliance directly. HullProbe swaps a registered anatomical part of a vessel for another instance's and reads how much that part drives the decision, using a same-class swap to control for compositing artifacts. Because a surface vessel is a hull and a superstructure arranged along one axis, its parts form a 2x3 grid whose 2^6 = 64 swaps we evaluate exhaustively, and a single Walsh transform returns each part's interventional influence and its interaction spectrum. On planted shortcuts, where the true reliance is known, the measure tracks worst-group accuracy (r = -0.94 over 45 configurations), and replacing its learned segmenter with human annotations changes the hull-share by less than the audit's resolution. On the 26 vessel types of MARVEL, the classifier places about 46% of its corrected sensitivity on the generic hull yet follows the superstructure when the two conflict, and the group-label-free de-biasers available there do not detectably reduce the hull's share. The findings hold across three architectures and a second maritime dataset, and the method extends to fine-grained vehicle recognition on Stanford Cars.

Figures & tables

Figure 1: Two orthogonal decompositions give a $2{\times}3$ part grid. From a vessel (top-left), a learned segmenter splits it vertically into {hull, superstructure} (top-right) and its PCA pose axis splits it longitudinally into canonical {forward, mid, aft} thirds (bottom-left); their product is a grid of six anatomically defined cells (bottom-right, each named in its legend), each individually registrable and swappable for a donor’s.

Approach	Direct	Interventional	Plausible	Group-label-free	Exact
Worst-group accuracy [ 47 ]	○ (proxy)	○	–	○	–
Gradient saliency (Grad-CAM [ 50 ] )	○	○	–	●	–
Occlusion / removal (RISE [ 42 ] , ROAR [ 24 ] , SAGE [ 8 ] )	◐	●	○	●	○
Counterfactual / generative edits [ 19 , 6 , 48 ]	◐	●	●	●	○
Functional decomposition (Shapley [ 35 ] , Sobol [ 55 ] )	●	◐	○	●	○
HullProbe (ours)	●	●	◐	●	● (64)

Table 1: Where HullProbe sits among approaches to measuring feature reliance (Sec. 2 ). Columns are the five properties discussed in Sec. 2 ; ● has it, ◐ partial, ○ lacks it, – not applicable. Exact = exhaustive over the

64

-point intervention set, not sampled over subsets (the donor law is still estimated).

Figure 2: HullProbe overview. A recipient $x$ and donor $d$ are decomposed into a $2{\times}3$ grid of six anatomical cells (hull/superstructure $\times$ forward/mid/aft). The swap operator $\phi(x,d,z)$ replaces any cell subset $z\in\{0,1\}^{6}$ of the recipient with the donor’s registered cells (here a full-hull swap, $z{=}(1,1,1,0,0,0)$ ), producing a plausible composite. Evaluating the frozen classifier $f$ over all $2^{6}{=}64$ patterns turns its behavior into a function $g(z)$ on the Boolean cube; one exact Walsh–Hadamard transform then yields the functional-ANOVA spectrum $W_{k}$ , the per-cell influence $\mathrm{Inf}_{p}$ and its hull-share ( $46\%$ ), and the semantic- $L_{0}$ radius.

Figure 3: Operator and control. For a recipient (left), a cross-class hull swap (middle) changes the hull’s class while keeping the superstructure; the same-class swap (right) carries the identical seams/inpainting but preserves the label, and is the artifact control subtracted in Eq. ( 7 ). Both are anatomically plausible vessels, though not photorealistic.

Figure 4: Convergent validity on planted shortcuts (Sec. 4.2 ). Interventional reliance vs. worst-group accuracy, colored by shortcut. (a) The measurement principle: three appearance recolors (hull color, background, watermark) track WGA (pooled

r{=}{-}0.97

35

configs). (b) The part-swap operator itself: three anatomical hull-grafts are planted and measured with the real

6

-cell swap, tracking WGA across

45

configs (

3

shortcuts

\times

3

seeds

\times

5

de-biasers; pooled

r{=}{-}0.941

). Each point is one de-biaser

\times

seed; the dashed line is the least-squares fit.

appearance recolor (validates the principle)
shortcut	ERM $\downarrow$	DFR $\downarrow$	corr(WGA, rel.)
hull color	$0.39$	$\mathbf{0.12}$	$-0.966$
background	$0.43$	$\mathbf{0.14}$	$-0.904$
watermark †	$0.08$	$\mathbf{0.02}$	$-0.891$
anatomical part-swap (validates the operator)
Veh-Carrier / Ro-ro	$0.58$	$\mathbf{0.09}$	$-0.945$

Table 2: Replication across six planted shortcuts (Sec. 4.2 ). Interventional reliance (flip rate) for the untreated ERM model and the best de-biaser (DFR), and the correlation between worst-group accuracy and reliance across all de-biaser

\times

seed configurations. Appearance shortcuts validate the measurement principle; anatomical shortcuts validate the swap operator itself (plant and measure both use the real

6

-cell hull graft, listed by the two donor hulls). Watermark † is a known null (no shortcut learned

\Rightarrow

near-zero measured reliance; its correlation, taken over that near-constant range, is not informative).

Panel of Figure 5 — Figure 5: Nearly additive anatomy and single-swap fragility. (a) Variance by interaction degree: $W_{1}{=}0.63$ of behavior is single-cell main effects and $0.80$ is degree $\leq 2$ , so per-cell attribution captures most of the decision (per-cell influence in App. Table 9 ). (b) Semantic- $L_{0}$ distribution: the fewest realistic part swaps that flip the class; $44\%$ of vessels flip on one swap ( $r{=}1$ ), the rest need $\geq 2$ , and $8\%$ never flip within the six cells.

method	hull-share	$\Delta$ vs ERM	$\Pr[r{=}1]$
ERM	$0.394{\scriptstyle\pm.003}$	—	$0.513$
CutMix	$0.396{\scriptstyle\pm.011}$	$+0.002{\scriptstyle\pm.018}$	$\mathbf{0.420}$
hull-rand.	$0.408{\scriptstyle\pm.011}$	$+0.012{\scriptstyle\pm.008}$	$0.548$
JTT	$0.399{\scriptstyle\pm.024}$	$+0.002{\scriptstyle\pm.019}$	$0.505$
LfF	$0.417{\scriptstyle\pm.019}$	$+0.022{\scriptstyle\pm.019}$	$0.538$
AFR	$0.399{\scriptstyle\pm.011}$	$+0.006{\scriptstyle\pm.007}$	$0.478$

Table 3: De-biasing audit (3 seeds, mean

\pm

std). Hull-share is the balance axis;

\Delta

is the paired change vs. ERM (positive

=

more hull-reliant);

\Pr[r{=}1]

is the concentration axis. No method reduces hull-share; CutMix trades concentration for unchanged balance; bg-swap worsens both. ∗ :

|\Delta|>2\sigma

. Each

\Delta

is averaged per seed over the recipients both that method and ERM classify correctly, so it is a tighter paired estimate and need not equal the difference of the two pooled hull-share columns (which use each method’s own recipient pool).

all held-out ( $n{=}194$ )
mask source	hull-share	raw	$W_{1}$	$W_{\leq 2}$	$\Pr[r{=}1]$
segmenter (both)	$0.49$ $[0.45,0.53]$	$0.47$	$0.67$	$0.84$	$0.35$
annotation (both)	$0.48$ $[0.44,0.53]$	$0.46$	$0.67$	$0.84$	$0.35$
annotation (recipient only)	$0.48$ $[0.44,0.52]$	$0.46$	$0.67$	$0.84$	$0.35$
$\Delta$ (both)	$-0.009$ $[-0.041,0.023]$	$-0.015$	$+0.004$	$0.000$	$0.000$
held-out and in MARVEL’s test split ( $n{=}17$ )

Table 4: Segmentation error: annotations vs. the segmenter (App. A ). The audit on the segmenter’s

202

held-out vessels with its predicted hull/superstructure maps, with the annotations, and with annotations for the recipient only; recipients, donors, silhouette, and classifier are identical across arms. Brackets are

95\%

recipient-bootstrap intervals;

\Delta

rows are paired (annotation minus segmenter). Left: pooled readouts. Right: per-cell corrected influence

\mathrm{Inf}_{p}^{\text{corr}}

on all held-out vessels.

variant	corrected hull-share	raw
reference (compatible, $K{=}4$ )	$0.51$ $[.40,.63]$	$0.47$
random donors ( $K{=}4$ )	$0.52$ $[.40,.66]$	$0.46$
compatible, $K{=}2$	$0.50$ $[.35,.63]$	$0.45$
compatible, $K{=}8$	$0.51$ $[.42,.62]$	$0.46$
seg. eroded ( $-3$ px)	$0.61$ $[.51,.72]$	$0.56$
seg. dilated ( $+3$ px)	$0.46$ $[.36,.57]$	$0.43$

Table 5: Sensitivity of the corrected hull-share [

95\%

bootstrap CI] to donor and segmentation choices (App. B ; fixed

61

-recipient subsample, which runs at

0.50

–

0.52

, above the

0.46

full-pool headline but within its bootstrap interval). Robust to donor selection and count; the corrected value is invariant to donor quality (random vs. compatible), validating the artifact control; segmentation boundary shifts move it in the expected direction.

robustness check	corr(WGA, reliance)
pooled Pearson $r$	$-0.941$
Spearman $\rho$	$-0.933$
per-shortcut ( $3$ )	$-0.945,\ -0.964,\ -0.928$
leave-one-out (by shortcut)	$-0.946,\ -0.932,\ -0.950$
cluster bootstrap $95\%$ CI	$[-0.964,\ -0.916]$

Table 6: The operator-level correlation is robust to the nested design (

45

configs; Sec. 4.2 ): reliance vs. worst-group accuracy stays strongly negative under every check, so it is not an artifact of the shared shortcuts and seeds.

method	WGA	re-swap flip	indep. flip
ERM	$0.03{\scriptstyle\pm.03}$	$0.79{\scriptstyle\pm.05}$	$0.23{\scriptstyle\pm.05}$
SUBG	$0.05{\scriptstyle\pm.08}$	$0.75{\scriptstyle\pm.12}$	$0.21{\scriptstyle\pm.03}$
GroupDRO	$0.21{\scriptstyle\pm.07}$	$0.54{\scriptstyle\pm.14}$	$0.18{\scriptstyle\pm.04}$
JTT	$0.13{\scriptstyle\pm.08}$	$0.65{\scriptstyle\pm.09}$	$0.17{\scriptstyle\pm.03}$
DFR	$\mathbf{0.65}{\scriptstyle\pm.07}$	$\mathbf{0.08}{\scriptstyle\pm.03}$	$\mathbf{0.12}{\scriptstyle\pm.06}$

Table 7: Circularity control (ResNet-50,

3

seeds, mean

\pm

std; App. C ). For each de-biaser, the operator’s re-swap flip (hull

\to

the other trained donor) and an independent-mechanism flip (hull

\to

a third, unseen donor through the identical compositor). Both fall from ERM to DFR and are positively correlated across methods (mean

r{=}0.73

), so the operator reads hull reliance that survives a change of donor, not donor or compositing memorization.

Figure 6: Hull sensitivity vs. sufficiency (Sec. 5.3 ). Top: under a hull/superstructure conflict (a cross-class donor hull grafted on, superstructure kept), the classifier follows the superstructure (the true class)

73\%

of the time and adopts the donor hull’s class only

4.8\%

. Bottom: per class, the hull sensitivity (share) far exceeds the rate at which the hull is actually sufficient to set the prediction. Every class sits well below the

y{=}x

line, so sensitivity should not be read as harmful reliance.

Table 14

method	hull	sup.	total	$\Delta$ hull	$\Delta$ total
ERM	$0.461$	$0.602$	$1.063$	—	—
CutMix	$0.455$	$0.552$	$1.006$	$-0.008{\scriptstyle\pm.031}$	$-0.059{\scriptstyle\pm.058}$
hull-rand.	$0.468$	$0.562$	$1.030$	$+0.006{\scriptstyle\pm.027}$	$-0.032{\scriptstyle\pm.044}$
JTT	$0.463$	$0.580$	$1.042$	$-0.002{\scriptstyle\pm.034}$	$-0.022{\scriptstyle\pm.034}$
LfF	$0.494$	$0.567$	$1.061$	$+0.031{\scriptstyle\pm.041}$	$-0.006{\scriptstyle\pm.054}$
AFR	$0.460$	$0.590$	$1.050$	$+0.005{\scriptstyle\pm.013}$	$-0.002{\scriptstyle\pm.012}$

Table 11: Absolute influence is flat, not just the ratio (App. F ;

3

seeds, mean

\pm

std over seeds; same audit as Table 3 ). Mean absolute hull, superstructure, and total cross-class flip-influence per method, with the paired change vs. ERM in absolute hull and total influence. No group-label-free method reduces absolute hull influence, and the small total changes are within seed noise; only background-swap (positive control) moves, raising both.

Figure 7: Two axes move independently (visualizes Table 3 , Sec. 5.4 ). Left: hull-share (which parts); ∗ significant vs. ERM. Right:

\Pr[r{=}1]

(how many parts). CutMix lowers fragility (right) without lowering hull reliance (left); bg-swap worsens both.

de-biaser	Grad-CAM	IG	RISE
ERM	$0.34$	$0.21$	$0.12$
SUBG	$0.32$	$0.18$	$0.12$
JTT	$0.36$	$0.23$	$0.12$
GroupDRO	$0.33$	$0.18$	$0.12$
DFR	$0.31$	$0.20$	$0.12$
range across de-biasers	$0.05$	$0.05$	$0.00$

Table 12: Correlational attribution locates the shortcut but cannot measure reliance. Fraction of attribution mass on the planted hull region (controlled hull-color shortcut,

5

de-biasers

\times

3

seeds,

120

images each; hull area

{\approx}0.12

). Every method stays nearly flat across de-biasers (range

{\leq}0.05

), whereas our interventional hull-color flip spans

0.27

over the same models, so attribution reports where, not how much.

Figure 8: A non-maritime instantiation (Sec. 5.5 ). Four Stanford Cars of distinct body styles ( left of each pair) and their natural $5$ -part decomposition ( right ): {body (gray), front-face (red), greenhouse (blue), body-side (green), wheels (yellow)}. Parts come from an open-vocabulary GroundedSAM prompt and the geometric PCA pose; the hull/superstructure segmenter and the forward/mid/aft grid are not used. Over the resulting $2^{5}$ -swap cube the greenhouse (roof and glazing) carries the majority of corrected interventional influence, well above the front grille.

Figure 9: The car swap operator (Sec. 5.5 ). A recipient car, its natural $5$ -part decomposition, and a cross-class greenhouse swap that flips the predicted class, shown against the label-preserving same-class control that isolates the compositing artifact. This is the car analogue of the vessel operator in Fig. 3 .

Figure 10: Cars: nearly additive anatomy and greenhouse dominance (App. H ). (a) Walsh variance by interaction degree over the

2^{5}

cube: main effects hold

W_{1}{=}0.94

and degree-

\leq 2

holds

0.99

, so per-part attribution captures almost all of the decision. (b) Per-part flip-influence, raw (gray) and artifact-corrected (green): the greenhouse dominates, while the wheels and body fall to

\approx 0

once the same-class control adjusts for the compositing floor.

part	cross $\mathrm{Inf}^{\times}$	same $\mathrm{Inf}^{=}$	corr. $\mathrm{Inf}^{\text{corr}}$
greenhouse	$0.680$	$0.205$	$\mathbf{0.476}$
body-side	$0.210$	$0.073$	$0.137$
front-face	$0.148$	$0.103$	$0.045$
body	$0.000$	$0.000$	$0.000$
wheel	$0.034$	$0.056$	$-0.022$

Table 13: Cars: per-part flip-influence (

80

recipients,

2^{5}

cube; App. H ). Raw cross-class

\mathrm{Inf}^{\times}

, same-class control

\mathrm{Inf}^{=}

, and corrected

\mathrm{Inf}^{\text{corr}}

. The greenhouse carries the majority of corrected influence; wheels and body reduce to

\approx 0

under the control, as class-agnostic parts should.

Figure 11: Convergent validity ports to cars (Sec. 5.5 ). Three graded color shortcuts (

3\times 3

seeds

\times 5

de-biasers,

45

configs): group-label-free recolor-flip reliance tracks worst-group accuracy across the full range (pooled

r{=}{-}0.97

;

-0.97

without DFR), ERM leaning on the shortcut and DFR removing it. The car swap operator is shown in Fig. 9 .

shortcut	strength	ERM (WGA/flip)	DFR (WGA/flip)	corr
painted	strong	$0.09/0.82$	$0.72/0.08$	$-0.962$
background	medium	$0.47/0.30$	$0.82/0.05$	$-0.923$
watermark †	weak	$0.74/0.05$	$0.84/0.02$	$-0.743$

Table 14: Cars convergent validity, per shortcut (ResNet-50,

3

seeds each; App. H ). Three graded shortcuts span the worst-group-accuracy range; for each, WGA rises and the group-label-free recolor-flip reliance falls from ERM to DFR. Pooled

r{=}{-}0.973

over all

45

configurations (

-0.973

excluding DFR).

component	vessels (MARVEL, SeaShips)	cars (Stanford Cars)	requirement for a new domain
part set $P$	$2{\times}3$ : {hull, sup} $\times$ {fwd, mid, aft}, $\|P\|{=}6$	5 named parts, $\|P\|{=}5$	exhaustive partition into $\lesssim 8$ parts present on every instance
part extractor	SegFormer-B2 fine-tuned on $1{,}014$ masks (hull/sup); PCA thirds	GroundedSAM prompt + fixed token-to-part map; no annotation	a learned or promptable segmenter, or a geometric split
registration	closed-form PCA similarity transform	same	a canonical frame with part-to-part correspondence (rigid, elongated, consistent view)
validity filter	MARVEL: PCA aspect $\geq 1.40$ ; SeaShips: every cell $\geq 150$ px ( $76\%$ yield)	$\geq 3$ parts covering $\geq 45\%$ , aspect $\in[1.35,6]$ ( $97\%$ yield)	a rule rejecting views where parts are missing or unregistrable
donor pool	$K{=}4$ aspect-compatible cross-class $+\,2$ same-class per recipient	same	$\geq 3$ held-out instances per class
classifier access	black box, predicted labels (probabilities for the soft variant)	same	no gradients, retraining, or group labels

Table 15: What it takes to port HullProbe (App. H ). Components a new domain must supply, as instantiated for vessels and cars. Only the part decomposition and the registration are domain-specific; the swap operator, artifact control, cube evaluation, and readouts are shared.

Figure 12: Zero-shot transfer to SeaShips (second dataset). Both the MARVEL-trained SegFormer segmenter (middle, the hull/superstructure $\times$ forward/mid/aft decomposition) and the swap operator ( bottom , a cross-class hull swap, hull $\leftarrow$ donor type ) apply to SeaShips vessels with no retraining, staying clean across distinct ship types. This is the qualitative counterpart to the numerical replication (Sec. 5.5 ). Segmentation is zero-shot, so composites are noisier than on MARVEL, which the same-class control absorbs.

Image Classification

Image classification assigns semantic labels to images, enabling recognition of objects, scenes, conditions, and fine-grained categories.

See all 381 papers

Hull First, Wake Second: Wake-Reliance Suppression for Robust Maritime Vessel Detection
Aug 27, 2026Yefan Wang, Xingyu Wang, Ruibiao Zhu +1Ship Detection Object Detection
University of Shanghai for Science and Technology, Shanghai, China · University of Science and Technology Liaoning, Liaoning, China · School of Computing, College of Systems and Society, The Australian National University, ACT, Australia +1
Strip Convolution and Direction-Aware Exclusion Loss for Oriented Ship Detection
Sep 23, 2026Bin Chen, Yuanyuan Liu, Peng Yang +1Remote Sensing Ship Detection
School of Information and Software Engineering, East China Jiaotong University, Nanchang 330013, China. · Jiangxi Vocational University of Foreign Studies, Nanchang 330099, China.
Sea-Scan: High-Accuracy, ML-based Dark Vessel Detection and Localisation via Weakly Supervised DAS Monitoring
Jun 19, 2026Tian Tian, Agastya Raj, Lara Flanagan +2Distributed Acoustic Sensing Object Detection
School of Computer Science and Statistics, IRIS Research Group, ADAPT Research Centre, Trinity College Dublin, Ireland · School of Engineering, ADAPT Research Centre, Trinity College Dublin, Ireland

Aug 27, 2026J/K move · Enter open · S save

Hull First, Wake Second: Wake-Reliance Suppression for Robust Maritime Vessel Detection

Yefan Wang, Xingyu Wang, Ruibiao Zhu, Yusen Wu

University of Shanghai for Science and Technology, Shanghai, China · University of Science and Technology Liaoning, Liaoning, China · School of Computing, College of Systems and Society, The Australian National University, ACT, Australia+1

Ship Detection Object Detection Spurious Correlation Robustness Aerial Object Detection

Open paper AskPDF arXiv

Maritime vessel detectors often face scenes where hulls are small, low-contrast, or blurred, while wakes are longer and easier to detect. This creates a wake-reliance problem: detectors may miss slow or stationary vessels with weak wakes, or produce false positives on wake-like water clutter. We propose HullWake, a hull-first wake-second framework for robust maritime vessel detection. HullWake separates proposal-centered hull evidence from directional wake context, extracts wake cues with bidirectional proposal-anchored corridors, and suppresses wake-dominant predictions through wake response supervision, wake-attenuated consistency, wake-only confidence suppression, and hull--wake decorrelation. We also introduce a wake-oriented evaluation protocol covering weak/no-wake vessels, wake-like hard negatives, worst-group AP, and confidence drop after wake attenuation. Experiments are conducted on Curated-Wake, a wake-oriented maritime dataset of about 10,000 images curated from Ships/Vessels in Aerial Images, the SMD benchmark, and SeaDronesSee, with newly added detection- and segmentation-level wake annotations. Compared with box-only detectors and mask-supervised segmentation baselines, HullWake improves overall AP, weak/no-wake robustness, wake-like false positives, worst-group AP, and confidence stability after wake attenuation.

Abstract

Image Classification

Explore similar work

Hull First, Wake Second: Wake-Reliance Suppression for Robust Maritime Vessel Detection

Strip Convolution and Direction-Aware Exclusion Loss for Oriented Ship Detection

Sea-Scan: High-Accuracy, ML-based Dark Vessel Detection and Localisation via Weakly Supervised DAS Monitoring