Shortcut-Resistant CAM Distillation for Long-Tailed Recognition

Wenhai Wan; Teng Zhang; Shao-Yuan Li; Xinrui Wang; Qiang-Sheng Hua; Songcan Chen

cs.LGOct 21, 2026

Shortcut-Resistant CAM Distillation for Long-Tailed Recognition

Authors: Wenhai Wan, Teng Zhang, Shao-Yuan Li, Xinrui Wang, Qiang-Sheng Hua, Songcan Chen

Organizations: National Engineering Research Center for Big Data Technology and System · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics ·

Abstract

Real-world datasets often follow a long-tailed distribution, making generalization to tail classes difficult. We revisit this problem through the lens of shortcut learning, where models prefer the easiest predictive cues (e.g., background or textures) over object-centric semantics, especially under scarce and biased supervision. We find that this tendency is amplified for tail classes: limited examples often share similar contexts, making non-semantic signals highly correlated and thus tempting shortcuts, whereas head classes with diverse appearances and environments encourage more stable object-focused representations. Motivated by this observation, we propose Shortcut-Resistant CAM Distillation (SRCD), a plug-and-play framework that transfers object-focused explanations from head to tail classes. SRCD operates in the Class Activation Map (CAM) space, where a CAM provides a class-specific spatial evidence map for a prediction. SRCD aggregates CAMs from a small set of head-class candidates into a shortcut-resistant teacher using an energy-model weighting based on coherence and concentration, and distills it to the tail-class CAM. We provide a theoretical analysis that quantifies shortcut reliance as shortcut-region evidence mass in CAM space and shows that SRCD suppresses tail shortcuts. Extensive experiments on long-tailed benchmarks consistently improve strong baselines. The code is available at https://github.com/Haifeng3/SRCD.

Figures & tables

Figure 1: Shortcut learning in long-tailed recognition visualized by CAMs. For a head-class example ( Tench ), the model attends to the object region, while for a tail-class example ( Speedboat ), attention drifts to spurious cues (e.g., ocean background), leading to shortcut-dominated predictions and degraded tail generalization.

Figure 2: CAM similarity analysis. LT–Balanced (GT) denotes the cosine similarity between CAMs produced by the long-tailed model and the balanced model for the same examples under their ground-truth classes. LT–Balanced (Head-Proxy) computes CAMs for tail examples using the most probable head-class predictions from the long-tailed model and compares them to the balanced-model CAMs. Shaded regions indicate standard deviation.

Figure 3: SRCD pipeline. We generate class-conditioned CAMs from feature maps and classifier weights, form a shortcut-resistant teacher CAM by energy-weighting a set of head-proxy CAMs (with stop-gradient used only in this proxy branch), and distill the teacher to the ground-truth class CAM via $\mathcal{L}_{\mathrm{SRCD}}$ to reduce shortcut/background attention alongside $\mathcal{L}_{\mathrm{cls}}$ .

Method	CIFAR100			CIFAR10
	IR=100	IR=50	IR=10	IR=100	IR=50	IR=10
BCL ( Zhu et al., 2022 )	51.76	56.51	67.90	83.61	86.13	90.10
+SRCD	53.64 $\uparrow$ 1.88	56.81 $\uparrow$ 0.30	68.22 $\uparrow$ 0.32	82.97 $\downarrow$ 0.64	87.87 $\uparrow$ 1.74	91.23 $\uparrow$ 1.13
GCL ( Li et al., 2022b )	46.50	51.72	61.79	80.56	84.74	89.65
+SRCD	47.07 $\uparrow$ 0.57	52.32 $\uparrow$ 0.60	60.23 $\downarrow$ 1.56	81.11 $\uparrow$ 0.55	85.69 $\uparrow$ 0.95	88.90 $\downarrow$ 0.75
CE+FCC ( Li et al., 2023 )	40.20	45.93	57.80	73.80	79.57	87.75

Table 1: Training-from-scratch results on CIFAR-10-LT and CIFAR-100-LT under imbalance ratios IR

\in\{100,50,10\}

(Top-1 accuracy, %). +SRCD denotes integrating SRCD into each baseline; superscripts show absolute changes (

\uparrow

\downarrow

for gain/drop).

Training from scratch
Method	ImageNet-LT				iNaturalist
	Head	Med	Tail	All	Head	Med	Tail	All
BCL	65.83	53.16	36.27	55.67	73.47	71.99	68.82	70.80
+SRCD	66.29 $\uparrow$ 0.46	54.47 $\uparrow$ 1.31	39.56 $\uparrow$ 3.29	57.00 $\uparrow$ 1.33	74.26 $\uparrow$ 0.79	72.65 $\uparrow$ 0.66	70.33 $\uparrow$ 1.51	71.89 $\uparrow$ 1.09
GCL	65.04	52.77	35.69	55.51	73.75	72.28	68.34	70.76
+SRCD	64.83 $\downarrow$ 0.21	53.62 $\uparrow$ 0.85	37.10 $\uparrow$ 1.41	55.69 $\uparrow$ 0.18	74.28 $\uparrow$ 0.53	74.02 $\uparrow$ 1.74	70.31 $\uparrow$ 1.97	72.58 $\uparrow$ 1.82

Table 2: Results on ImageNet-LT and iNaturalist, reporting Top-1 accuracy (%) on Head/Med/Tail splits and All. The upper block is training from scratch, and the lower block is fine-tuning foundation models. +SRCD denotes integrating SRCD into each baseline; superscripts show absolute changes (

\uparrow

\downarrow

for gain/drop).

Panel of Figure 4 — Figure 4: Ablation study of SRCD w.r.t. $\beta$ .

Panel of Figure 5 — Figure 5: Ablation study of $K$ and energy-based weighting.

Coherence $\mathcal{E}_{h}$	Concentration $\mathcal{E}_{n}$	Acc.
✓	✗	54.02
✗	✓	53.64
✓	✓	55.95

Table 3: Ablation study on the teacher-weighting cues

\mathcal{E}_{h}

and

\mathcal{E}_{n}

in SRCD, evaluated on ImageNet-LT with the FCC.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Acc.
BCL	67.25
BCL + SRCD	69.88

Appendix

Table 4: Accuracy (%) on the semantically mismatched subset of iNaturalist tail samples, where the selected head proxies are taxonomically unrelated to the target tail classes.

Dataset	$\#$ classes	$\#$ training data	$\#$ test data	imbalance ratio
CIFAR10-LT ( Cao et al., 2019 )	10	50,000	10,000	{100, 50,10}
CIFAR100-LT ( Cao et al., 2019 )	100	50,000	10,000	{100, 50,10}
ImageNet-LT ( Liu et al., 2019 )	1,000	115,846	50,000	256
iNaturalist 2018 ( Van Horn et al., 2018 )	8,142	437,513	24,426	500
Places-LT ( Liu et al., 2019 )	365	62,500	36,500	996

Appendix

Table 5: Statistics of datasets.

Method	Head	Med	Tail	All
GLMC	63.39	60.95	34.63	53.91
+SRCD	63.88 $\uparrow$ 0.49	62.37 $\uparrow$ 1.42	36.90 $\uparrow$ 2.27	55.26 $\uparrow$ 1.35
CE+H2T	50.85	48.91	24.53	42.27
+SRCD	51.93 $\uparrow$ 1.08	51.17 $\uparrow$ 2.26	28.16 $\uparrow$ 3.63	44.53 $\uparrow$ 2.26
BNS	62.40	59.70	31.90	52.40
+SRCD	62.93 $\uparrow$ 0.53	60.56 $\uparrow$ 0.86	35.52 $\uparrow$ 3.62	53.88 $\uparrow$ 1.48

Appendix

Table 6: Split-wise results on CIFAR100-LT under IR=100, reporting Top-1 accuracy (%) on Head/Med/Tail splits and All. +SRCD denotes integrating SRCD into each baseline; superscripts show absolute changes (

\uparrow

\downarrow

for gain/drop).

Method	Head	Med	Tail	All
GLMC	64.73	62.10	38.19	58.87
+SRCD	65.51 $\uparrow$ 0.78	63.53 $\uparrow$ 1.43	41.86 $\uparrow$ 3.67	60.43 $\uparrow$ 1.56
CE+H2T	51.42	50.96	29.93	47.58
+SRCD	51.47 $\uparrow$ 0.05	49.83 $\downarrow$ 1.13	30.21 $\uparrow$ 0.28	46.97 $\downarrow$ 0.61
BNS	61.30	58.70	37.20	55.90
+SRCD	61.20 $\downarrow$ 0.10	58.94 $\uparrow$ 0.24	38.17 $\uparrow$ 0.97	56.12 $\uparrow$ 0.22

Appendix

Table 7: Split-wise results on CIFAR100-LT under IR=50. reporting Top-1 accuracy (%) on Head/Med/Tail splits and All. +SRCD denotes integrating SRCD into each baseline; superscripts show absolute changes (

\uparrow

\downarrow

for gain/drop).

Training from scratch
Method	Places-LT
	Head	Med	Tail	All
BCL	41.98	42.36	34.91	40.77
+SRCD	41.85 $\downarrow$ 0.13	42.79 $\uparrow$ 0.43	36.23 $\uparrow$ 1.32	41.18 $\uparrow$ 0.41
GLMC	42.09	42.88	35.27	41.12
+SRCD	42.28 $\uparrow$ 0.19	43.05 $\uparrow$ 0.17	37.11 $\uparrow$ 1.84	41.62 $\uparrow$ 0.50

Appendix

Table 8: Results on Places-LT, reporting Top-1 accuracy (%) on Head/Med/Tail splits and All. The upper block is training from scratch, and the lower block is fine-tuning foundation models. +SRCD denotes integrating SRCD into each baseline; superscripts show absolute changes (

\uparrow

\downarrow

for gain/drop).

Method	Head	Med	Tail	All
BNS	62.40	59.70	31.90	52.40
BNS + Center Mask	61.76 $\downarrow$ 0.64	58.27 $\downarrow$ 1.43	28.68 $\downarrow$ 3.22	50.61 $\downarrow$ 1.79
BNS + SRCD	62.93 $\uparrow$ 0.53	60.56 $\uparrow$ 0.86	35.52 $\uparrow$ 3.62	53.88 $\uparrow$ 1.48

Appendix

Table 9: Comparison with a center-mask baseline on CIFAR100-LT under IR=100. The center-mask baseline replaces the adaptive head-proxy teacher with a fixed centered mask.

Method	Time/iter (ms)	Total time (h)	Throughput (img/s)	$\Delta$ Time/iter
GLMC	142	19.7	1842	–
GLMC + SRCD	153	22.6	1707	+7.7%

Appendix

Table 10: Training overhead on ImageNet with ResNet-50 (method GLMC). Reported Total time is the amortized wall-clock time over the full training run.

Figure 6: CAM visualizations on multiple tail-class examples. For each example (row), the first three columns show head-proxy CAM candidates, and the last column shows the tail ground-truth class CAM. High-quality proxies tend to capture object-centric evidence consistent with the ground truth, while low-quality ones highlight scattered or background-dominated regions.

Figure 7: Effect of SRCD on CAM localization across multiple examples. Left: without SRCD. Right: with SRCD. SRCD promotes more object-centric and spatially coherent activations while reducing background/context responses, indicating stronger shortcut resistance.

Method	FG/BG Ratio $\uparrow$	Acc. $\uparrow$
LIFT	0.85	78.37
LIFT + SRCD	1.56	83.16

Appendix

Table 11: Mask-based shortcut analysis on the long-tailed VOC 2012 subset. FG/BG Ratio measures the CAM evidence mass inside the foreground mask relative to the background region. A larger ratio indicates weaker shortcut reliance.

Panel of Figure 8 — Figure 8: CAM visualizations on the long-tailed VOC 2012 subset. Compared with the baseline, SRCD shifts attention from background or shortcut regions toward foreground object regions, which is consistent with the mask-based FG/BG analysis.

\mathcal{H}(x_{i})=\operatorname{TopK}_{c\in\mathcal{Y}_{\mathrm{head}}}p_{i}(c).

Appendix

Algorithm 1 Shortcut-Resistant CAM Distillation (SRCD)

Long-Tailed Classification

Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.

See all 55 papers

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning
Sep 29, 2026Shenghan Chen, Yiming Liu, Zhipeng Deng +6Long-Tailed Classification Long-Tail Learning
Westlake University, Hangzhou, China · Shandong University, Jinan, China · Hokkaido University, Sapporo, Japan +1
VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition
Sep 4, 2026Jiangang Zhu, Zheng Wang, Bin Zhu +2Long-Tailed Classification Ensemble Learning
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China +2
OSDTW: Optimal Shared Depth and Task Weighting for Long-Tailed Recognition
May 24, 2026Chang Chu, Qingyue Zhang, Shao-Lun Huang +1Long-Tailed Classification Shared Representation Learning
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Shenzhen Zkosemi Semiconductor Technology Co., Ltd, Shenzhen, China

Sep 29, 2026J/K move · Enter open · S save

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning

Shenghan Chen, Yiming Liu, Zhipeng Deng, Haolin Wang+5

Westlake University, Hangzhou, China · Shandong University, Jinan, China · Hokkaido University, Sapporo, Japan+1

Long-Tailed Classification Long-Tail Learning

Open paper AskPDF arXiv

Balancing performance trade-offs on long-tailed data distributions remains a long-standing challenge in visual recognition. Existing methods mainly improve tail classes through re-balancing, representation learning, or data augmentation, but the underlying cause of tail class degradation is still insufficiently explored. In this paper, we find that standard long-tailed training induces background-biased representation and optimization: tail classes suffer larger background distribution shifts and become increasingly driven by background gradients. This reveals that tail degradation is not merely caused by insufficient samples, but also by the learning of irrelevant background features. To tackle this issue, we propose Object-Focused Background Debiasing (OFBD), a framework that mitigates background bias from both distribution and optimization perspectives. Specifically, Foreground-guided CutMix preserves target-related foregrounds while diversifying complementary backgrounds, and Background-guided Feature Rectification suppresses background-biased features without learnable parameters or additional training. Extensive experiments show that our method improves overall accuracy, achieves significant tail-class gains, and can serve as a plug-in for mainstream long-tailed methods without external data or pretrained recognition models. The code is available at: https://ofbd-neurips2026-longtail-learning.github.io/

Abstract

Long-Tailed Classification

Explore similar work

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

OSDTW: Optimal Shared Depth and Task Weighting for Long-Tailed Recognition