Long-tail class incremental learning (LT CIL) remains highly challenging because the scarcity of samples in tail classes not only hampers their learning but also exacerbates catastrophic forgetting under continuously evolving and imbalanced data distributions. To tackle these issues, we exploit the informativeness and scalability of language knowledge. Specifically, we analyze the LT CIL data distribution to guide large language models (LLMs) in generating a stratified language tree that hierarchically organizes semantic information from coarse to fine grained granularity. Building upon this structure, we introduce stratified adaptive language guidance, which leverages learnable weights to merge multi-scale semantic representations, thereby enabling dynamic supervisory adjustment for tail classes and alleviating the impact of data imbalance. Furthermore, we introduce stratified alignment language guidance, which exploits the structural stability of the language tree to constrain optimization and reinforce semantic visual alignment, thereby alleviating catastrophic forgetting. Extensive experiments on multiple benchmarks demonstrate that our method achieves state of the art performance.
Figures & tables
Fig. 1: Experiments were conducted on CIFAR100, imbalance rate ρ=0.01 and 10 tasks. We evaluated accuracy across head classes, tail classes, and the complete dataset to assess the influence of semantic information on LT-CIL.
Fig. 2: The overall framework of our method. A is the stratified language tree generation, B is the stratified adaptive language guidance, and C is the stratified alignment language guidance.
Fig. 3: Illustration of Prompt Template 2 and SL-Tree.
Method
CIFAR100
ImageNet-R
CUB200
ρ=0.1
ρ=0.01
ρ=0.1
ρ=0.01
ρ=0.1
Baseline
62.1
62.1
68.2
68.2
45.6
+ SL-Tree
62.5
62.5
69.0
69.0
47.9
+ Lcls
19.8
17.4
23.8
22.4
20.1
+ Lkd
64.7
60.9
69.5
65.8
46.4
+ Lalg
67.4
62.4
72.9
69.4
48.7
TABLE IV: Ablation study on different dataset with 10 tasks. + SL-Tree denotes taking the mean of all text features, while + updated α denotes using adaptive weights.
Fig. 4: Tail classes accuracy of different tasks on ImageNet-R, ρ=0.01 after 10 tasks.
Fig. 5: a) shows robustness of different LLMs and b) shows robustness of different prompt templates. All experiments were conducted on ImageNet-R with ρ=0.01 after 10 tasks.
Tasks
Method
Upper Bound
CLIP Zero-Shot
Adapter Fine-tuning
Ours
20
75.6
63.1
19.6
68.2
50
75.6
63.1
17.4
66.7
TABLE V: Experiments on ImageNet-LT.
Fig. 6: Density map of weights center. Experiments conducted on Imagenet-R, ρ=0.01 , and 10 tasks.
Fig. 7: Performance difference between two adjacent layers. Experiments conducted on Imagenet-R, ρ=0.01 , and 10 tasks.
Fig. 8: Effects on Tail Classes. Experiments conducted on Imagenet-R, ρ=0.01 , and 10 tasks.
Fig. 9: t-SNE visualization of different classes. Experiments conducted on ImageNet-R, ρ=0.01 , and 10 tasks. a) zero-shot; b) fine-tuned adapter; c) fine-tuned adapter with knowledge distillation (KD); d) fine-tuned adapter with KD and the proposed stratified alignment language guidance.
Fig. 10: Similarity heatmap of text featuress. Experiments conducted on ImageNet-R, ρ=0.01 , and 10 tasks, a) is the fixed text; b) duplicating the fixed text; c) is the mean of text from different layers of SL-Tree; d) is the linearly weighted text from different layers of SL-Tree.
Fig. 11: Experimental consumption. Experiments conducted on Imagenet-R, ρ=0.01 , and 10 tasks.
Fig. 12: Sensitivity analysis of hyperparameters. Experiments conducted on ImageNet-R, ρ=0.01 , and 10 tasks.
Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China +2