Spatial Supervision Without Attribution Optimization: Improving Post-Hoc Class Activation Maps via Box-Guided Evidence Routing

Wenhao Liang; Liangwei Nathan Zheng; Lin Yue; Wei Emma Zhang; Mingyu Guo; Weitong Chen

cs.CVOct 5, 2026

Spatial Supervision Without Attribution Optimization: Improving Post-Hoc Class Activation Maps via Box-Guided Evidence Routing

Authors: Wenhao Liang, Liangwei Nathan Zheng, Lin Yue, Wei Emma Zhang, Mingyu Guo, Weitong Chen

Organizations: Adelaide University, Adelaide, Australia

Abstract

Post-hoc class activation maps (CAMs) are a standard tool for inspecting the evidence behind an image classifier's predictions, yet nothing in ordinary training encourages these maps to be spatially appropriate. We study whether inexpensive spatial supervision can improve a classifier's own predicted-class Grad-CAM without ever optimizing an attribution map. Box-Guided Evidence Routing (BGER) trains a lightweight gate on the final feature map under box or mask supervision and routes classification through the gated features, while Grad-CAM is computed separately at the pre-gate representation, so the evaluated map never enters the training objective. With a BCE routing loss, BGER raises MaxBoxAccV2 from $0.584$ to $0.715$ on CUB-200-2011 and from $0.757$ to $0.832$ on Stanford Dogs at comparable accuracy. Matched controls attribute most of the ResNet-50 gain to the spatial supervision reshaping the backbone rather than to routing itself: when classification bypasses the gate, most of the improvement remains, and detaching gradients through the gate leaves the ResNet-50 result nearly unchanged. The same detachment preserves most of the gain in two DenseNet-121 chest X-ray settings but removes the apparent gain on Swin-T, and directly supervising the CAM reaches stronger localization at a larger accuracy cost. Overall, spatial supervision can improve separately evaluated post-hoc CAMs, but both the mechanism and the size of the benefit depend on the architecture and the evaluation setting.

Figures & tables

Figure 1: Box-Guided Evidence Routing with BCE-BGER. (1) The box or mask $B$ supervises only the internal gate $M$ via $\mathcal{L}_{\mathrm{route}}=\mathrm{BBCE}(M,B)$ ; classification is routed through $H_{e}=M\odot H$ , and the evaluated predicted-class Grad-CAM $A(H)$ is read at the pre-gate features $H$ and never enters the loss. The CAM is not optimized directly, but the class-score gradient still traverses the learned route; backbone-specific detach tests quantify this contribution (Tab. 5 ; App. Tab. 23 , 24 ). (2) Real route-to-CAM examples (seed-0 Grad-CAM at the pre-gate features; GT box green). CUB : predicted-class Grad-CAM—CE grounds on bark outside the bird (EIB $0.41$ ), BCE-BGER on the creeper ( $0.65$ ). Medical : RSNA positive-finding Grad-CAM lesion grounding on a positive boxed study (conditioned on the abnormality logit, not the predicted class)—CE spreads outside the radiologist box (EIB $0.23$ ), BCE-BGER concentrates more inside ( $0.40$ ); both classify correctly. The panel’s colour-bar label names the shared mechanism (Grad-CAM at pre-gate $H$ ); the conditioning class differs by row as stated here. To avoid a best case, this is the median -improvement study among $491$ eligible positives under a pre-stated rule (App. G ). (3) Headline MaxBoxAccV2; medical is a controlled extension, not clinical validation.

Family	Method	Supervised object	Acc	ECE15	EIB	PG	MaxBoxAccV2
Label-only	CE	—	$.749{\pm}.005$	$.017{\pm}.004$	$.788$	$.966$	$.584{\pm}.001$
Direct attribution	CAM-Align	CAM proxy (energy)	$.747{\pm}.003$	$.022{\pm}.003$	$\mathbf{.866}$	$.996$	$.651{\pm}.005$
	CAM-Dice	CAM proxy ( $\sigma(S)$ , Dice)	$.748{\pm}.003$	$.016{\pm}.007$	$.786$	$.984$	$.696{\pm}.006$
	CAM-BCE	CAM proxy ( $\sigma(S)$ , BCE)	$.737{\pm}.005$	$.030{\pm}.005$	$.837$	$\mathbf{.997}$	$\mathbf{.769}{\pm}.008$
Feature routing	Energy-BGER	internal route (energy–area)	$.750{\pm}.001$	$\underline{.015}{\pm}.002$	$.859$	$.992$	$.658{\pm}.001$
	Dice-BGER	internal route (soft Dice)	$.749{\pm}.001$	$\mathbf{.013}{\pm}.002$	$.829$	$.991$	$.690{\pm}.005$

Table 1: Objective-matched CUB comparison on the full test set (

N{=}5{,}794

; three seeds; predicted-class Grad-CAM), grouped by supervision family. Direct-proxy methods supervise an attribution-family map; route methods supervise an internal feature route and never place an attribution map in the loss. BCE-BGER uses class-balanced binary cross-entropy for route supervision; CAM-BCE applies the same loss to a CAM proxy and reaches higher localization at lower accuracy (full frontier: Fig. 2 ).

Figure 2: CE-relative accuracy–localization trade-offs on CUB. Axes show accuracy change from CE in percentage points and MaxBoxAccV2 change from CE. Small markers are observed seed-0 weight-sweep outcomes connected in increasing-

\lambda

order; large markers with error bars are the validation-selected

\lambda{=}2

means and standard deviations of paired CE-relative differences across three matched seeds. Gate-route BCE-BGER yields

+.130

MaxBoxAccV2 at approximately

-0.1

pp accuracy, whereas direct-proxy CAM-BCE yields

+.184

at approximately

-1.2

pp. Lines connect discrete trained models only; neither family dominates over the tested grids.

Variant	Class. path	Acc	ECE15	MaxBoxV2	$\Delta$ CE	Ret.
CE	$H$	$.749{\pm}.005$	$.017{\pm}.004$	$.584{\pm}.001$	—	$0\%$
Frozen backbone	$M\odot H$	$.731{\pm}.006$	$.026{\pm}.017$	$.597{\pm}.000$	$+.013$	$10\%$
Shuffled boxes	$M\odot H$	$.734{\pm}.007$	$.018{\pm}.008$	$.619{\pm}.003$	$+.035$	$27\%$
Aux. BCE, no routing	$H$	$.751{\pm}.003$	$.012{\pm}.003$	$.683{\pm}.001$	$+.099$	$76\%$
Residual-BCE	$H\odot(1{+}\eta M)$	$.755{\pm}.003$	$.019{\pm}.007$	$.714{\pm}.003$	$+.130$	$99\%$
BCE-BGER	$M\odot H$	$.748{\pm}.001$	$.016{\pm}.004$	$.715{\pm}.005$	$+.131$	$100\%$

Table 2: BCE-BGER mechanism controls (CUB full test, three seeds, same evaluator as Tab. 1 ). “Gain ret.” is the fraction of the CE

\to

BCE-BGER MaxBoxAccV2 gain retained; it is descriptive, and the interventions do not compose additively. Aux. BCE, no routing applies the identical gate and box loss but classifies from the ungated

\mathrm{GAP}(H)

; it isolates shared-backbone reshaping from routing and recovers most of the gain. “Class. path” is the representation the classifier reads. Full protocol and equal-annotation controls: App. E , J .

Figure 3: Region-restricted corruption for BCE-BGER (CUB, fixed

N{=}1{,}000

subset, three seeds, matched noise+blur). BCE-BGER retains more accuracy than CE when only the background is corrupted (a) and loses more when only the annotated object is corrupted (b), consistent with greater object-region sensitivity. Corruption probes sensitivity, not causal reliance.

Data	Method	AUROC	AUPRC	ECE15	EIB	MaxBoxV2
VinDr	CE	$.851{\pm}.021$	$.768{\pm}.027$	$.138{\pm}.012$	$.147{\pm}.012$	$.142{\pm}.015$
	CAM-Align	$.878{\pm}.027$	$.805{\pm}.028$	$\mathbf{.085}{\pm}.013$	$.390{\pm}.006$	$.255{\pm}.044$
	BCE-BGER	$.862{\pm}.016$	$.805{\pm}.023$	$.125{\pm}.020$	$.263{\pm}.029$	$.209{\pm}.011$
	CAM-BCE	$\mathbf{.885}{\pm}.011$	$\mathbf{.820}{\pm}.013$	$.103{\pm}.031$	$\mathbf{.402}{\pm}.015$	$\mathbf{.361}{\pm}.002$
RSNA	CE	$.985{\pm}.002$	$.983{\pm}.003$	$.027{\pm}.007$	$.189{\pm}.036$	$.200{\pm}.066$
	CAM-Align	$.986{\pm}.001$	$.983{\pm}.001$	$.024{\pm}.011$	$.565{\pm}.017$	$.361{\pm}.018$

Table 3: Controlled chest-X-ray lesion grounding (three seeds, mean

\pm

std; DenseNet-121). Both protocols use a single abnormality logit; the CAM is conditioned on that logit and spatial metrics are computed on positive boxed test images only, so—unlike the predicted-class CUB evaluation—these rows read the map for a known-present finding. Discriminative performance is reported as AUROC/AUPRC because both protocols are class-imbalanced (accuracy, PG, and AdaECE in App. Tab. 10 ). Italics denote direct CAM-proxy supervision; BCE-BGER and CAM-BCE use the objective-matched balanced BCE (

\lambda_{\mathrm{route}}{=}2

Readout	CE	BCE-BGER	CAM-BCE
Grad-CAM	$.593$	$.728$	.778
Grad-CAM++	$.602$	$.736$	.804
HiResCAM	$.593$	$.728$	.778
Score-CAM-128	$.590$	$.734$	.798

Table 4: Localization (MaxBoxAccV2) across CAM readouts on the fixed

N{=}500

CUB subset (three seeds, identical images and checkpoints). BCE-BGER improves over CE under all four readouts, including gradient-free Score-CAM-128; CAM-BCE (italic, direct supervision) remains higher, consistent with Fig. 2 . This establishes consistency across readout rules, not attribution-method independence, because all operators read the same box-supervised representation. Full EIB/PG in App. Tab. 28 .

(a) CUB backbone mechanism arms
Backbone	CE	Aux	BGER	det.	corr
ResNet-50	$.584$	$.683$	$\mathbf{.715}$	$.712$	$.9999$
Swin-T	$.280$	$.317$	$.896$	$.256$	$-.16$
ConvNeXt-T	$.291$	$.288$	$.253$	$.255$	$.974$

Table 5: (a) Predicted-class Grad-CAM MaxBoxAccV2 on the full CUB test set (three seeds): CE, auxiliary BCE without routing (Aux), BCE-BGER, BCE-BGER with the gate gradient detached (det.), and the mean per-image standard-vs-detached map correlation (corr). (b) Positive-finding Grad-CAM MaxBoxAccV2 on positive boxed chest-X-ray studies (three seeds): standard vs. detached MaxBoxAccV2, map correlation, and the fraction of the CE

\to

BCE-BGER gain retained. Read values within each panel, not across them. Panel (b) tests only whether the reported localization gain survives removing the gate-gradient derivative; it does not isolate representation reshaping from routing as the matched ResNet-50 no-routing arm does. On ConvNeXt-T both localization scores stay below CE while the standard and detached maps remain highly correlated, so the negative result is not a detachment artifact.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Supervised obj.	Class. path	Evaluated obj.	Attr. in loss	Class cond.	Audit sep.	Detach audit
CAM/Grad-CAM	none	GAP	the CAM	—	—	—	—
WSOL	image label	GAP	true-cls map	no	yes	no	no
CAM-loss	feat. vs. CAM	GAP	CAM	yes	yes	no	no
CGC	CAM consist.	GAP	Grad-CAM	yes	yes	no	no
SAOL	output attn.	attn. logits	attention	no	no	no	no
ABN	attn. branch	attn. feat.	attention	yes	yes	no	no

Appendix

Table 6: Nearest methods by supervised object, classification path, and evaluated object (citations in Related Work): CAM/Grad-CAM ( Zhou et al. 2016 ; Selvaraju et al. 2017 ) , WSOL ( Choe et al. 2020 ) , CAM-loss ( Wang et al. 2021 ) , CGC ( Pillai et al. 2022 ) , SAOL ( Kim et al. 2020 ) , ABN ( Fukui et al. 2019 ) , guided attention ( Li et al. 2018 ) , RRR ( Ross et al. 2017 ) , explanation-aware training ( Schramowski et al. 2020 ) , bounded logit attention ( Baumhauer et al. 2021 ) , Grad-CAM-guided channel-spatial attention ( Xu et al. 2021 ) , and explanation-supervised biomedical training ( Faruqui and Dubey 2026 ) . “Attr. in loss”: an attribution-family map (CAM, saliency, or input-gradient) appears in the objective; “Class cond.”: the supervised spatial object is class-conditioned; “Audit sep.”: the evaluated map is a distinct computational object from the supervised one; “Detach audit”: an explicit intervention tests that the evaluated gain survives removing the supervised object’s gradient path. Among the compared methods, BGER uniquely combines a class-agnostic supervised route, a separately computed pre-gate predicted-class CAM, and an explicit detach intervention testing audit validity.

Dataset	Method	NLL $\downarrow$	Brier $\downarrow$
CUB	CE	$.922{\pm}.009$	$.352{\pm}.005$
	CAM-Align †	$0.936$	$0.354$
	Energy-BGER	$\mathbf{.920}{\pm}.011$	$\mathbf{.350}{\pm}.003$
	BCE-BGER	$.920{\pm}.008$	$.353{\pm}.002$
	CAM-BCE	$.942{\pm}.007$	$.362{\pm}.004$
Pet	CE	$.243{\pm}.006$	$.109{\pm}.000$

Appendix

Table 7: NLL and Brier scores (lower is better; three seeds unless noted). BCE-BGER matches CE-level proper scoring on CUB and Dogs and leads on RSNA; CAM-BCE pays a small CUB cost consistent with its accuracy trade-off. † CUB CAM-Align is seed-0 only.

Method	MaxBoxAccV2 (seeds 0/1/2)
CE	$.585$	$.585$	$.584$
CAM-Align	$.653$	$.654$	$.646$
Energy-BGER	$.656$	$.655$	$.657$
Dice-BGER	$.686$	$.689$	$.696$
BCE-BGER	$.717$	$.709$	$.718$
CAM-BCE	$.771$	$.775$	$.760$

Appendix

Table 8: Per-seed full-test MaxBoxAccV2 for the CUB comparison. Ordering is complete across seeds:

\min

BCE-BGER

(.709)

>

\max

Dice-BGER

(.696)

>

\max

Energy-BGER

(.657)

;

\min

CAM-BCE

(.760)

>

\max

BCE-BGER

(.718)

. On accuracy the direction reverses:

\min

BCE-BGER

(.747)

>

\max

CAM-BCE

(.740)

. Seed variation on the secondary metrics is likewise small: standard deviations over the same runs are EIB

.001/.001/.001/.004/.002/.011

and PG

.002/.000/.001/.002/.001/.000

in row order, so the EIB and PG means of Tab. 1 are stable to the precision reported there.

Variant	Acc	ECE15	EIB	PG	MaxBoxV2	Gain ret.
CE	$.749$	$.017$	$.788$	$.966$	$.584{\pm}.001$	$0\%$
Frozen backbone	$.731$	$.026$	$.794$	$.976$	$.597{\pm}.000$	$10\%$
Shuffled boxes	$.734$	$.018$	$.829$	$.992$	$.619{\pm}.003$	$27\%$
Residual-BCE	$.755$	$.019$	$.814$	$.987$	$.714{\pm}.003$	$99\%$
BCE-BGER	$.748$	$.016$	$.840$	$.995$	$.715{\pm}.005$	$100\%$

Appendix

Table 9: BCE-BGER mechanism controls (CUB full test, three seeds, mean values; MaxBoxAccV2 with seed std). “Gain ret.” = fraction of the CE

\to

BCE-BGER MaxBoxAccV2 gain retained; descriptive, and the interventions do not compose additively. HCU (confidence

\geq.9

, EIB

<.5

): CE

1.24{\pm}.02\%

, frozen

0.67{\pm}.17\%

, shuffled

0.50{\pm}.11\%

, BCE-BGER

0.06{\pm}.01\%

. Routing increment vs. no-routing:

+.032

(per-seed

.034/.025/.036

;

95\%

t

-interval

[.018,.046]

; no accuracy cost). Exclusivity is not load-bearing: residual

.714{\pm}.003

vs. exclusive

.715{\pm}.005

(

\Delta{=}{+}.0005

[-.008,.009]

); the weaker Energy objective shows a small exclusive gain (

+.021

). Source: routing_increment_seed_analysis.csv .

Data	Method	AdaECE $\downarrow$	PG $\uparrow$	MaxBoxV2 $\uparrow$
VinDr	CE	$.136{\pm}.014$	$.226{\pm}.015$	$.142{\pm}.015$
	CAM-Align	$\mathbf{.083}{\pm}.011$	$.550{\pm}.005$	$.255{\pm}.044$
	BCE-BGER	$.125{\pm}.020$	$.404{\pm}.032$	$.209{\pm}.011$
	CAM-BCE	$.102{\pm}.031$	$\mathbf{.563}{\pm}.014$	$\mathbf{.361}{\pm}.002$
RSNA	CE	$.027{\pm}.007$	$.411{\pm}.157$	$.200{\pm}.066$
	CAM-Align	$.025{\pm}.006$	$.777{\pm}.013$	$.361{\pm}.018$

Appendix

Table 10: Complete chest-X-ray metrics for the main-text comparison (three seeds, mean

\pm

std). Italics denote direct CAM-proxy supervision. Accuracy, ECE15, and EIB appear in main Tab. 3 .

Family	$\lambda$	Val Acc	Sel.	Test Acc	Test V2
Gate (BCE)	$0.25$	—	ends	$.756$	$.654$
	$0.5$	$.722$		$.741$	$.670$
	$1$	$.737$		$.753$	$.699$
	$2$	$.727$	$\checkmark$	$.747$	$.717$
	$4$	—	ends	$.742$	$.716$
Proxy (Dice)	$.125$	$.745$		$.744$	$.594$

Appendix

Table 11: Frontier sweep protocol (CUB, seed 0 per point; test used once per point). “Sel.” marks each family’s validation-selected weight, confirmed with three seeds in the main table; “ends” marks post-hoc frontier endpoints outside the selection grid; “infeas.” failed the validation-accuracy constraint.

Metric	Comparison	$\Delta$	$95\%$ CI	$\neq 0$
MaxBoxV2	vs. CE	$+.130$	$[.126,.134]$	yes
	vs. Dice	$+.024$	$[.022,.027]$	yes
	vs. Energy	$+.057$	$[.053,.059]$	yes
	vs. CAM-Align	$+.064$	$[.060,.067]$	yes
	vs. CAM-BCE	$-.054$	$[-.058,-.051]$	yes
	Dice vs. Energy	$+.032$	$[.029,.035]$	yes

Appendix

Table 12: Paired image-level bootstrap for the CUB comparison (full test,

10{,}000

resamples, three-seed means within each resample). Unlabeled rows are BCE-BGER

-

(method).

Method	Box use	Route type	Acc	ECE	EIB	MaxBoxV2
CE	none	—	$.749{\pm}.005$	$.016{\pm}.004$	$.790{\pm}.001$	$.588{\pm}.002$
Bg-blackout	input mask	input-space	$.723{\pm}.004$	$.021{\pm}.004$	$.788{\pm}.007$	$.595{\pm}.008$
Bg-random	input noise	input-space	$.680{\pm}.004$	$.039{\pm}.005$	$.789{\pm}.002$	$.593{\pm}.009$
Object-crop	input crop	input-space	$.618{\pm}.013$	$.017{\pm}.006$	$.816{\pm}.003$	$.592{\pm}.003$
ABN-box	feature	attention branch	$.738{\pm}.004$	$.024{\pm}.004$	$.807{\pm}.001$	$.606{\pm}.006$
w/o box loss	none	exclusive route	$.748{\pm}.003$	$.018{\pm}.002$	$.800{\pm}.000$	$.585{\pm}.001$

Appendix

Table 13: Equal-annotation controls under the original Energy recipe (CUB, three seeds; this protocol’s evaluator reads CE at

.588

vs.

.584

in Tab. 1 , see text). Feature routing produces the largest shift among the tested equal-annotation non-proxy controls.

Variant	Acc	ECE	Grad-CAM EIB	Grad-CAM MaxBoxAccV2	Gate EIB
Energy-BGER (full)	0.748	0.015	0.862	0.667	0.796
w/o box loss	0.748	0.018	0.800	0.585	0.487
w/o area loss	0.748	0.015	0.858	0.670	0.788
residual routing	0.751	0.034	0.815	0.645	0.822
frozen backbone	0.732	0.034	0.808	0.603	0.722
shuffled-box control	0.741	0.018	0.855	0.608	0.795

Appendix

Table 14: Component and topology controls for the original Energy objective on CUB. These results diagnose routing design and should not be read as BCE-BGER ablations.

Figure 4: Route-loss dose response on CUB (seed-0 sweep points; Swin

\lambda_{\mathrm{box}}\in\{1,2\}

are three-seed means). Gate concentration rises smoothly on ResNet-50 and sharply between

\lambda_{\mathrm{box}}{=}1.5

and

2

on Swin-T; dashed lines mark each architecture’s CE Grad-CAM baseline.

Swin-T transformer
Method	Acc $\uparrow$	ECE15 $\downarrow$	EIB $\uparrow$	MaxBoxAccV2 $\uparrow$
CE	$.786{\pm}.005$	$.015{\pm}.004$	$.360{\pm}.003$	$.280{\pm}.002$
CAM-Align	$\mathbf{.791}{\pm}.002$	$\mathbf{.013}{\pm}.002$	$.413{\pm}.003$	$.343{\pm}.010$
Energy-BGER ( $\lambda_{\mathrm{box}}{=}1.0$ )	$.783{\pm}.002$	$.014{\pm}.003$	$.372{\pm}.005$	$.278{\pm}.003$
Energy-BGER ( $\lambda_{\mathrm{box}}{=}2.0$ )	$.789{\pm}.005$	$.145{\pm}.006$	$\mathbf{.567}{\pm}.008$	$\mathbf{.585}{\pm}.021$
+ temperature scaling	$.789{\pm}.005$	$.018{\pm}.004$	$.567{\pm}.008$	$.585{\pm}.021$

Appendix

Table 15: Architecture diagnostics on CUB (three seeds, Energy route). Stronger Energy routing raises the standard audited map on Swin-T, but audit validity is not established for these Energy-route checkpoints (the BCE-BGER detach audit in Tab. 23 shows Swin standard-map gains can be gate-gradient imprinting); ConvNeXt-T shows a smaller gain with an accuracy cost. Operating points are selected on validation data (Fig. 4 , Tab. 16 ).

$\lambda_{\mathrm{box}}$	val Acc	val Gate EIB	test Acc	test EIB	test MaxBoxV2
0 (CE)	.778	—	.815	.526	.290
1	.777	.826	.806	.438	.295
2	.773	.853	.802	.511	.336
4	.765	.883	.799	.516	.338
8	.770	.893	.784	.562	.342

Appendix

Table 16: ConvNeXt-T route-loss sweep on CUB (seed 0).

\lambda_{\mathrm{box}}{=}0

is CE;

\lambda_{\mathrm{box}}{=}4

is selected from validation gate concentration and accuracy.

Figure 5: Cross-dataset localization (three-seed means): each row is one dataset–metric, and the four methods are points on a common

[0.3,1]

axis (no per-axis rescaling), so within-row method comparison is valid and no cross-metric magnitude is implied. Complete estimates are in Tabs. 1 and 17 .

Dataset	Method	Evidence target	Acc	ECE	EIB/EIM	PG	MaxBoxAccV2
Oxford-IIIT Pet	CE	foreground mask	$.930{\pm}.000$	$.022{\pm}.005$	$.677{\pm}.002$	$.891{\pm}.004$	$\mathbf{.739}{\pm}.002$
Oxford-IIIT Pet	CAM-Align	foreground mask	$.927{\pm}.003$	$.019{\pm}.002$	$\mathbf{.761}{\pm}.003$	$\mathbf{.910}{\pm}.002$	$.653{\pm}.005$
Oxford-IIIT Pet	Energy-BGER	foreground mask	$\mathbf{.931}{\pm}.000$	$\mathbf{.016}{\pm}.003$	$.704{\pm}.006$	$.904{\pm}.006$	$.684{\pm}.006$
Oxford-IIIT Pet	Energy gate $M$	diagnostic gate	–	–	$.340{\pm}.001$	$.945{\pm}.003$	$.526{\pm}.002$
Stanford Dogs	CE	dog box	$\mathbf{.880}{\pm}.005$	$.045{\pm}.003$	$.890{\pm}.000$	$.984{\pm}.000$	$.757{\pm}.004$
Stanford Dogs	CAM-Align	dog box	$.875{\pm}.002$	$\mathbf{.041}{\pm}.003$	$\mathbf{.929}{\pm}.001$	$\mathbf{.987}{\pm}.000$	$.773{\pm}.006$

Appendix

Table 17: Natural-image extensions with Energy-BGER (three seeds). BCE-BGER Dogs results are reported in the main paper.

Figure 6: Representative route-to-CAM examples for Energy-BGER on CUB, Stanford Dogs, and Oxford-IIIT Pet. Columns show the input with the target box, the CE predicted-class Grad-CAM, and the Energy-BGER predicted-class Grad-CAM (seed 0); the white contour marks the map’s $0.5$ level. Each row is the largest CE $\to$ BGER EIB-gain case found in a fixed-stride scan of the test split, so Energy-BGER concentrates the audited map inside the target relative to CE.

Method	AUROC $\uparrow$	AUPRC $\uparrow$	Acc $\uparrow$	ECE15 $\downarrow$
CE	$.851{\pm}.021$	$.768{\pm}.027$	$.808{\pm}.008$	$.138{\pm}.012$
CAM-Align	$\mathbf{.878}{\pm}.027$	$\mathbf{.805}{\pm}.028$	$.828{\pm}.007$	$\mathbf{.085}{\pm}.013$
Energy-BGER	$.867{\pm}.025$	$\mathbf{.805}{\pm}.025$	$\mathbf{.835}{\pm}.011$	$.104{\pm}.022$

Appendix

Table 18: VinDr-CXR results with Energy-BGER. BCE-BGER and CAM-BCE results are reported in the main paper.

Method	AUROC $\uparrow$	AUPRC $\uparrow$	Acc $\uparrow$	ECE15 $\downarrow$
CE	$.985{\pm}.002$	$.983{\pm}.003$	$.947{\pm}.003$	$.027{\pm}.007$
CAM-Align	$\mathbf{.986}{\pm}.001$	$.983{\pm}.001$	$.946{\pm}.003$	$.024{\pm}.011$
Energy-BGER	$\mathbf{.986}{\pm}.002$	$\mathbf{.984}{\pm}.002$	$\mathbf{.953}{\pm}.003$	$\mathbf{.021}{\pm}.016$

Appendix

Table 19: RSNA results with Energy-BGER (three seeds, validation-selected

\rho=0.02

). BCE-BGER and CAM-BCE results are reported in the main paper.

Table 26

Dataset	Method	AUROC $\uparrow$	ECE15 $\downarrow$	EIM $\uparrow$	PG $\uparrow$	MaxBoxV2 $\uparrow$
ICH (brain CT)	CE	$.834$	$.072$	$.018$	$.030$	$.034$
	CAM-Align	$.836$	$.061$	$.018$	$.033$	$.044$
	Energy-BGER	$.845$	$.074$	$.022$	$.052$	$.060$
VinDr-Mammo	CE	$.792$	$.140$	$.024$	$.021$	$.008$
	CAM-Align	$.779$	$.107$	$.090$	$.160$	$.053$
	Energy-BGER	$.772$	$.109$	$.059$	$.076$	$.029$

Appendix

Table 22: Exploratory Energy-BGER medical extensions (three-seed means; DenseNet-121; boxed abnormality versus no finding). Spatial metrics use positive boxed test images.

Backbone	CE	Aux	BGER	BGER det.	corr
ResNet-50	$.584$	$.683$	$.715{\pm}.005$	$.712{\pm}.006$	$.9999$
Swin-T	$.280$	$.317$	$.896{\pm}.000$	$.256{\pm}.001$	$-.16{\pm}.04$
ConvNeXt-T	$.291$	$.288$	$.253{\pm}.002$	$.255{\pm}.002$	$.974{\pm}.004$

Appendix

Table 23: Complete architecture mechanism arms with detach audit (CUB, three seeds, full test, MaxBoxAccV2): CE, auxiliary BCE without routing (Aux), routed BCE-BGER, BCE-BGER with the gate gradient detached, and the mean per-image standard-vs-detached map correlation. The detach-based gate-imprinting check passes on ResNet-50 and ConvNeXt-T but fails on Swin-T, and only ResNet-50 combines a passing check with a positive gain. DenseNet-121 medical audits appear separately in Tab. 24 ; source files and reproduction commands are listed in the submitted artifact manifest.

Data	std V2	det V2	corr	retained
RSNA	$.427$	$.413$	$.957$	$94\%$
VinDr	$.208$	$.194$	$.940$	$78\%$

Appendix

Table 24: DenseNet-121 detach audit on the medical BCE-BGER checkpoints (positive boxed test images, three seeds): standard vs. detached MaxBoxAccV2, mean per-image map correlation, and the fraction of the CE

\to

BCE-BGER gain retained after detachment. Both models remain substantially stable under detachment; VinDr’s lower retention reflects its small absolute gain (

.208{-}.142

), while the map correlation (

.94

) shows the map itself is preserved. The “std V2” column is recomputed by this audit and agrees with Tab. 3 to within rounding (VinDr

.2082

vs.

.2089

; RSNA

.4268

vs.

.4269

). Source files and reproduction commands are listed in the submitted artifact manifest.

Figure 7: Original Energy-objective region-restricted diagnostic under the standard ImageNet-C suite. Each point is the mean over 10 corruption types and 3 seeds; error bars are the standard deviation across seeds. Energy-BGER is relatively less sensitive to background corruption and more sensitive to object corruption.

$\tau$	$\theta$	CE	CAM-Align	Energy-BGER
.7	.3	$.07{\pm}.05$	$.00{\pm}.00$	$.00{\pm}.00$
.7	.5	$2.20{\pm}.36$	$.07{\pm}.05$	$.13{\pm}.05$
.7	.7	$15.40{\pm}.08$	$4.07{\pm}.17$	$4.37{\pm}.17$
.8	.3	$.07{\pm}.05$	$.00{\pm}.00$	$.00{\pm}.00$
.8	.5	$1.50{\pm}.24$	$.07{\pm}.05$	$.10{\pm}.00$
.8	.7	$12.60{\pm}.51$	$3.20{\pm}.36$	$3.53{\pm}.12$

Appendix

Table 25: Threshold sensitivity of the original Energy-BGER HCU diagnostic on CUB (

N=1{,}000

, three seeds).

Method	RemoveBoxDrop $\uparrow$	KeepBgConf $\downarrow$	EvidenceGap $\uparrow$	Del-AUC n $\downarrow$
CE	$.746{\pm}.005$	$.012{\pm}.000$	$.599{\pm}.001$	$.082{\pm}.001$
CAM-Align	$\mathbf{.759}{\pm}.006$	$\mathbf{.005}{\pm}.000$	$\mathbf{.624}{\pm}.002$	$\mathbf{.065}{\pm}.001$
Energy-BGER	$.752{\pm}.003$	$\mathbf{.005}{\pm}.000$	$.615{\pm}.002$	$.067{\pm}.000$

Appendix

Table 26: Attribution-free region-restricted confidence diagnostics for the original Energy objective on CUB (three seeds). Deletion-AUC is attribution-guided and listed only as a secondary measure.

Figure 8: Energy-BGER route-to-CAM examples (seed 0). Columns show the input and GT box, CE and Energy-BGER predicted-class Grad-CAM, and diagnostic gate $M$ ; contours span $0.3$ – $0.9$ .

Figure 9: Energy-BGER ablations. Removing the box loss weakens both predicted-class Grad-CAM and gate localization.

Figure 10: Focus stability under weak augmentations.

Figure 11: BCE-BGER MaxBoxAccV2 across four CAM operators on the fixed

N=500

CUB subset (three checkpoints; error bars show seed standard deviation).

Operator	Method	EIB $\uparrow$	MaxBoxV2 $\uparrow$
Grad-CAM (main)	CE	$.788{\pm}.001$	$.584{\pm}.001$
	CAM-Align	$\mathbf{.866}{\pm}.001$	$.651{\pm}.005$
	Energy-BGER	$.859{\pm}.001$	$\mathbf{.658}{\pm}.001$
HiResCAM	CE	$.788{\pm}.001$	$.584{\pm}.001$
	CAM-Align	$\mathbf{.866}{\pm}.001$	$.651{\pm}.004$
	Energy-BGER	$.863{\pm}.001$	$\mathbf{.656}{\pm}.001$

Appendix

Table 27: Full-test operator check for Energy-BGER (CUB,

N=5{,}794

, predicted-class maps, ResNet-50, three seeds).

Method	Operator	EIB $\uparrow$	PG $\uparrow$	MaxBoxV2 $\uparrow$
CE	Grad-CAM	$.795{\pm}.001$	$.957{\pm}.005$	$.593{\pm}.003$
	Grad-CAM++	$.794{\pm}.001$	$.955{\pm}.002$	$.602{\pm}.001$
	HiResCAM	$.795{\pm}.001$	$.957{\pm}.005$	$.593{\pm}.003$
	Score-CAM ∗	$.772{\pm}.001$	$.975{\pm}.009$	$.590{\pm}.001$
BCE-BGER	Grad-CAM	$.846{\pm}.002$	$.991{\pm}.002$	$.728{\pm}.003$
	Grad-CAM++	$.837{\pm}.003$	$.988{\pm}.002$	$.736{\pm}.003$

Appendix

Table 28: Operator consistency on the fixed

N=500

CUB subset (three seeds, identical images; ∗ Score-CAM-128 approximation). BCE-BGER improves over matched CE on every operator; CAM-BCE is highest on MaxBoxAccV2, consistent with the accuracy–localization frontier.

Table 39

Boxes	$n$	Acc	ECE15	EIB	MaxBoxV2	HCU
10%	400	$.749{\pm}.001$	$.020{\pm}.002$	$.836{\pm}.002$	$.630{\pm}.003$	$.19\%$
25%	1200	$.751{\pm}.003$	$.019{\pm}.004$	$.848{\pm}.001$	$.646{\pm}.003$	$.15\%$
50%	2400	$.747{\pm}.002$	$.019{\pm}.003$	$.854{\pm}.001$	$.652{\pm}.000$	$.07\%$
100%	4794	$.751{\pm}.001$	$.017{\pm}.001$	$.861{\pm}.001$	$.663{\pm}.004$	$.04\%$

Appendix

Table 31: Annotation-fraction curve (CUB, ResNet-50): the box-routing loss is applied to a class-stratified fraction of training images (nested subsets); all images keep class labels. Spatial metrics on a fixed 2,500-image test subset; d = seed-0 diagnostic, others three seeds. CE reference (no boxes): MaxBoxAccV2

.588

, EIB

.790

, HCU

1.10\%

. Ten percent of boxes already recovers most of the transfer; gains grow monotonically with budget at CE-level accuracy and ECE.

Figure 12: Energy-BGER route-to-CAM transfer versus CUB training-box fraction. The dashed line is CE; error bars show seed standard deviation where three seeds are available.

Method	Supervision	Map	Acc	EIB	PG	MaxBoxAccV2
SAT ( Wu et al. 2023 )	image labels	WSOL map	.815	.628	.901	.923
CATR ( Chen et al. 2023 )	image labels	WSOL attn.	.842	.607	.865	.783
TS-CAM ( Gao et al. 2021 )	image labels	WSOL CAM	.729	.730	.991	.781

Appendix

Table 32: External CUB WSOL references (single official checkpoint each), evaluated locally under our image/box protocol. These output dedicated true-class WSOL maps and are not comparable to the classifier’s predicted-class Grad-CAM in Tab. 1 ; listed as localization references. ECE15 is NA (calibration not reliably exported).

Table 43

Explore similar work

Aug 12, 2026cs.CV

Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations

Class activation mapping (CAM) is one of the most widely used visual explanation families in explainable artificial intelligence. Its purpose is intuitive: it converts internal model evidence into a heatmap that highlights the image regions, convolutional channels, tokens, or patches that support a target class or concept. Since the first CAM formulation in 2016, the field has moved far beyond global-average-pooled CNN classifiers. CAM-style methods now include gradient-based post-hoc explanations, gradient-free score and ablation methods, high-resolution upscaling, weakly supervised localization and segmentation, transformer token attribution, causal and debiasing methods, and foundation-model-era approaches that use CLIP, DINO, SAM, or feature-distribution comparisons. This review synthesizes a strict corpus of 57 method-centered papers published from 2016 onward. The paper develops a taxonomy that separates methods by attribution mechanism, architectural dependence, and evaluation objective. It then reviews gradient-based CAMs, recent and hybrid CAM-style methods, and model-based or architecture-aware methods. Across the corpus, the main trend is clear: the field is shifting from explaining one class score in one low-resolution CNN layer toward comparative, multi-layer, probabilistic, token-aware, and foundation-model-aware explanations. At the same time, evaluation remains fragmented. Faithfulness, localization, robustness, computational cost, and human trust are often measured with different protocols. The review therefore emphasizes not only what each method contributes, but also which gap it leaves open and which later methods attempt to close that gap.

AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini +2

Department of Computer Engineering, University of Birjand, Iran · Department of Applied Mathematics and Computer Science Visual Computing, Technical University of Denmark, Denmark

Transformer Interpretability Feature Attribution Explainable Image Classification

Ask PDF

Jun 3, 2026cs.CV

Spatially Grounded Concept Bottleneck Models via Part-Factorized Attention

Concept bottleneck models (CBMs) predict a layer of human-named attributes before predicting a class, which makes their decisions auditable. On fine-grained recognition tasks the concept heads are usually free to attend anywhere in the image, so a head named for one body region can be satisfied by evidence on another. This work studies a part-factorized CBM that removes that freedom by construction. The method has three components built on a frozen DINOv3 vision transformer. A learned foreground gate, trained on DINOv3 patch features, suppresses background patches inside the part attention. A set of part queries cross-attends to patch features and each of the 312 CUB attributes is routed, through a fixed concept-to-part map, to read only from the part token its name implies. A learnable two-dimensional Gaussian prior, injected additively in log space into the attention logits, breaks the permutation symmetry among part queries; its means are initialized from the dataset-average keypoint location of each part, which requires no per-image keypoint supervision at training or test time. On CUB-200-2011 the spatial-prior model matches a fully supervised baseline (88.85% versus 88.95% top-1) while raising pointing accuracy by 16 points (52.6% versus 36.4%). Replacing bounding-box supervision with a PCA foreground target and combining it with the Gaussian prior removes all per-image supervision and reaches 88.6% top-1 at about 70% pointing accuracy. A keypoint-fraction sweep shows that 0.5% of the training set (about 27 images) suffices to initialize the prior with no measurable loss. Removing part identity entirely is the harder case: without any spatial prior, pointing accuracy collapses to

2.9\%

Dhanesh Ramachandram

Vector Institute

Concept Bottleneck Models Fine-Grained Image Classification Attention Mechanisms Image Classification

Ask PDF

Jul 26, 2026cs.CV

Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations

Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.

Xianghao Jiao, Ruoyu Chen, Wei Wang +6

School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · University of Chinese Academy of Sciences, Beijing, China · School of Automation, Nanjing University of Information Science and Technology +2

Feature Attribution Image Classification

Ask PDF