SciLT: Long-tailed Image Classification under Scientific Image Domains

Jiahao Chen; Bing Su

cs.CVApr 4, 2026

SciLT: Long-tailed Image Classification under Scientific Image Domains

Authors: Jiahao Chen, Bing Su

Organizations: Renmin University of China

Abstract

Long-tailed recognition has benefited from foundation models and fine-tuning paradigms, yet existing studies and benchmarks are mainly confined to natural image domains, where pre-training and fine-tuning data share similar distributions. In contrast, scientific images exhibit distinct visual characteristics and supervision signals, raising questions about the effectiveness of fine-tuning foundation models in such settings. In this work, we investigate scientific long-tailed recognition under a purely visual and fine-tuning paradigm. Experiments on three scientific benchmarks show that fine-tuning foundation models yields limited gains, and reveal that penultimate-layer features play an important role, particularly for tail classes. Motivated by these findings, we propose SciLT, a framework that exploits multi-level representations through adaptive feature fusion and dual-supervision learning. By jointly leveraging penultimate- and final-layer features, SciLT achieves balanced performance across head and tail classes. Extensive experiments demonstrate that SciLT consistently outperforms existing methods, establishing a strong and practical baseline for scientific long-tailed recognition and providing valuable guidance for adapting foundation models to scientific data with substantial domain shifts.

Figures & tables

Figure 1: Differences in fine-tuning foundation models for downstream tasks on natural and scientific images. Fine-tuning achieves strong generalization performance on natural images (highlighted in blue), whereas its effectiveness on scientific images (highlighted in green) remains underexplored, due to the discrepancy in visual characteristics between scientific and natural images.

	ImageNet-LT	Places365-LT	iNat2018
cRT	47.3	36.7	65.2
MiSLAS	52.7	40.4	71.6
PaCo	57.0	41.2	73.2
LiVT	60.9	41.2	76.1
LPT	–	50.1 (22%)	76.1 (0%)
LIFT	77.0 (26%)	51.5 (25%)	79.1 (4%)

Table 1: Performance comparison on long-tailed benchmarks. Relative improvements are measured over the best of cRT, MiSLAS, PaCo, and LiVT. Gray rows denote fine-tuning from the foundation model.

Panel of Figure 2 — Figure 2: Relative gain on (a) Places365-LT and (b) iNaturalist2018 datasets with “Many", “Medium", and “Few" classes, respectively. RAC and LPT results on iNaturalist2018 are partially unavailable and therefore not fully plotted.

	NIH-Chest		Blood		ISIC
	OvAcc	Macro	OvAcc	Macro	OvAcc	Macro
CE	41.0	13.6	95.1	94.7	75.3	58.1
CB	30.9	17.3	93.0	95.0	58.9	55.1
LDAM	41.0	13.5	95.4	95.8	73.8	47.3
LA	22.2	24.6	93.8	95.8	66.5	65.9
Focal	40.7	13.4	94.3	94.1	74.5	56.8

Table 2: Performance comparison on long-tailed scientific datasets. Rows shaded in light gray indicate results obtained via fine-tuning from the foundation model.

Panel of Figure 3 — Figure 3: The performance curve on (a) NIH-Chest and (b) ISIC datasets with CE and LA training from scratch and fine-tuning. The class indices are sorted based on the number of samples belonging to each class. Curves are smoothed for better visualization.

	NIH-Chest		Blood		ISIC
	OvAcc	Macro	OvAcc	Macro	OvAcc	Macro
CE ${\dagger}$	40.3	11.9	97.9	97.2	77.7	55.2
LA ${\dagger}$	19.9	20.2	96.8	97.7	70.3	68.8

Table 3: Performance comparison on different long-tailed scientific datasets.

{\dagger}

denotes the performance of the penultimate layer.

		Many	Medium	Few
NIH-Chest	CE	31.89	1.33	0.00
	LA	15.58	35.63	11.33
	CE ${\dagger}$	32.78	2.75	0.06
	LA ${\dagger}$	16.60	32.39	14.24

Table 4: The results on NIH-Chest with “Many", “Medium", and “Few" (split via the number of samples of each class).

	NIH-Chest	Blood	ISIC
CE (penultimate-last)	0.98	1.02	0.96
LA (penultimate-last)	1.01	1.12	0.97

Table 5: Wasserstein distance between the feature distributions of the penultimate and last layers under different training criteria.

	MEL	NV	BCC	AK	BKL	DF	VASC	SCC	BScore
cRT	62.3	89.1	80.4	52.8	64.2	48.5	86.9	42.7	71.2
MiSLAS	63.8	88.4	79.6	58.7	65.3	56.2	89.5	45.1	72.8
PaCo	65.1	89.7	81.2	60.5	66.0	58.4	91.6	46.3	73.6
BCL	65.8	88.9	80.8	61.8	66.4	57.6	92.4	47.0	73.3
LPT	66.4	89.5	81.0	62.7	66.1	60.2	93.6	47.8	74.0
LIFT	58.4	77.2	70.7	68.8	64.0	86.4	95.7	58.0	71.7

Table 6: Results on ISIC datasets. We present the class-wise accuracy and the BScore.

	Baso.	Eosino.	Lympho.	Mono.	Neutro.	BScore
cRT	99.8	99.5	98.7	91.0	98.0	97.5
MiSLAS	100.0	99.6	98.8	92.4	97.7	97.6
PaCo	99.8	99.5	98.6	92.8	97.5	97.4
BCL	100.0	99.7	98.9	93.1	97.7	97.7
LPT	100.0	99.6	98.8	93.4	97.6	97.6
LIFT	100	99.7	98.4	96.2	97.0	97.9

Table 7: Results on Blood datasets. We present the class-wise accuracy and the BScore. Baso., Eosino., Lympho., Mono., and Neutro. denote basophil, eosinophil, lymphocyte, monocyte, and neutrophil, respectively.

	Many	Medium	Few	OvAcc	Macro	BScore
cRT	29.6	8.7	2.1	34.5	14.1	20.0
MiSLAS	30.4	12.1	3.7	34.0	15.8	21.6
PaCo	31.6	12.8	4.0	34.7	16.2	22.1
BCL	31.2	14.2	4.7	34.2	17.3	23.0
LPT	32.4	14.9	5.3	35.3	17.7	23.6
LIFT	15.58	35.63	11.33	19.7	20.8	20.2

Table 8: Results on NIH-Chest with “Many", “Medium", and “Few" (split via the number of samples of each class).

	OvAcc	Macro	BScore
w/o Fusion	34.9	15.1	21.1
SciLT	36.3	18.8	24.8

Table 9: Comparison of SciLT with the no-fusion configuration on the derived multiclass NIH-Chest task.

Selected depths $\mathcal{S}$	OvAcc	Macro	BScore
$\{N\}$	19.7	20.8	20.2
$\{N-1,N\}$	36.3	18.8	24.8
$\{N-2,N-1,N\}$	35.7	18.4	24.3
$\{N-3,N-2,N-1,N\}$	36.2	18.7	24.7

Table 10: Ablation of the selected depth set on NIH-Chest. The two-depth row is the current SciLT configuration.

Long-Tailed Classification

Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.

See all 55 papers

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition
Sep 4, 2026Jiangang Zhu, Zheng Wang, Bin Zhu +2Long-Tailed Classification Ensemble Learning
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China +2
CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning
May 2, 2026Ruichi Zhang, Chikai Shang, Jiacheng Yang +4Long-Tailed Classification Logit Adjustment
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · College of Computer Science and Software Engineering, Shenzhen University, China · Institute of High Performance Computing, A*STAR, Singapore
SciQNet: Two-Stage Multimodal Adaptation for Scientific Image Quality Assessment
Aug 6, 2026Yin-Loon Khor, Yi-Jie Wong, Jing Jie Tan +1VLM Adaptation Multimodal Pretraining
Universiti Malaya · Universiti Tunku Abdul Rahman · National University of Singapore

Sep 4, 2026J/K move · Enter open · S save

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Jiangang Zhu, Zheng Wang, Bin Zhu, Yi-Ping Phoebe Chen+1

Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China+2

Long-Tailed Classification Ensemble Learning Consistency Regularization

Open paper AskPDF arXiv

Multi-expert models have become the dominant paradigm for long-tailed learning, largely attributed to their presumed ability to benefit from expert diversity. However, we revisit this central assumption and reveal that diversity induced by logit adjustment or explicit regularizers does not guarantee better ensemble accuracy. Our work suggests that multi-expert models benefit more from variance reduction than diversity maximization. We introduce VICAL, a VIcinal Consistency ALignment framework that improves long-tailed recognition not by enforcing expert diversity, but by reducing prediction variance. Specifically, our approach comprises two key components: Self-Consistency Learning and Deep Ensemble Distillation. Self-Consistency Learning discourages reliance on unstable high-frequency information, smoothing the local loss landscape and mitigating overfitting, especially for tail classes. Deep Ensemble Distillation promotes cross-expert low-frequency semantic agreement using a low-resolution view, thereby sidestepping optimization conflicts with established knowledge. Extensive experiments on CIFAR-LT, ImageNet-LT, and iNaturalist 2018 show that VICAL consistently outperforms state-of-the-art methods, validating the effectiveness of our consistency-driven design. Our code is available at VICAL.

Abstract

Long-Tailed Classification

Explore similar work

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning

SciQNet: Two-Stage Multimodal Adaptation for Scientific Image Quality Assessment