A Theoretical Bridge Between Long-Tailed Recognition and Continual Learning
Organizations: Department of Electrical and Computer Engineering & Ingenuity Labs Research Institute, Queen’s University, Kingston, Canada
Abstract
We theoretically and empirically establish a previously unexplored connection between Long-Tailed Recognition (LTR) and Continual Learning (CL). Specifically, we show that training on a long-tailed dataset with imbalance factor drives model parameters into an neighborhood of the dominant-class solution, under both uniform and exponentially decaying cardinality distributions. Building on this result, we establish an objective-level reformulation of LTR as a sequential learning problem and prove that the CL objective upper-bounds the balanced LTR loss up to a constant determined by the residual Head loss. Motivated by this connection, we introduce Continual Learning for Long-Tailed Recognition (CLTR), a principled framework that leverages standard off-the-shelf CL methods to sequentially learn Head and Tail classes while mitigating catastrophic forgetting. Extensive experiments on CIFAR100-LT, CIFAR10-LT, ImageNet-LT, and Caltech256 validate our theoretical predictions and demonstrate strong performance across diverse LTR benchmarks. In the foundation-model regime, CLTR with DualPrompt on a CLIP backbone outperforms standard adaptation baselines and is competitive with specialized methods using external semantic supervision. Our work bridges LTR and CL both theoretically and empirically, providing a principled approach for addressing long-tailed learning using standard CL strategies.
Figures & tables
| Model | |||
|---|---|---|---|
| 100 | 50 | 10 | |
| Baseline [ 16 ] | 38.3 | 43.9 | 55.7 |
| Baseline + CB [ 16 ] | 39.6 | 45.3 | 58.0 |
| Focal loss [ 41 ] | 38.4 | 44.3 | 55.8 |
| Focal+CB [ 16 ] | 39.6 | 45.2 | 58.0 |
| -norm [ 21 ] | 47.7 | 52.5 | 63.8 |
| Method | IF | ||
|---|---|---|---|
| 100 | 50 | 10 | |
| Zero-shot CLIP | 64.5 | 64.5 | 64.5 |
| Linear probe | 60.0 | 67.2 | 75.6 |
| Full fine-tuning | 73.4 | 74.8 | 81.0 |
| Prompt learning | 60.4 | 63.1 | 66.9 |
| Naive sequential | 48.0 | 51.2 | 53.1 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Algorithm | LR | Opt. | Momentum | LR Scheduler | CL Loss Weight | Epochs | Steps |
|---|---|---|---|---|---|---|---|
| LwF | 0.001 | SGD | 0.9 | – | 0.01 | 5 | 2 |
| EWC | 0.01 | SGD | 0.9 | – | 10 | 90 | 2 |
| Modified EWC | 0.01 | SGD | 0.9 | – | 1000 | 90 | 2 |
| GPM | 0.001 | SGD | 0 | Cosine Anneal LR | – | 100 | 2 |
| SGP | 0.001 | SGD | 0.9 | Cosine Anneal LR | – | 100 | 2 |
| FOSTER (phase 1) | 0.1 | SGD | 0.9 | Cosine Anneal LR | – | 170 | 4 |
| Method | Backbone | Trainable params | Opt. | LR | Epochs | Tasks |
|---|---|---|---|---|---|---|
| Zero-shot CLIP | ViT-B/32 (frozen) | 0 | – | – | – | – |
| Linear probe | ViT-B/32 (frozen) | head only | AdamW | 50 | 1 | |
| Full fine-tuning | ViT-B/32 | all | AdamW | 20 | 1 | |
| Prompt learning | ViT-B/32 (frozen) | visual prompts | SGD | 40 | 10 | 1 |
| Naive sequential | ViT-B/32 | all | AdamW | 20 | 2 | |
| DualPrompt | ViT-B/32 (LN adapted) | G/E prompts + LN + cosine head | AdamW | 40–60 | 2 |
| Method | ImageNet-LT | CIFAR100-LT | ||
|---|---|---|---|---|
| CLTR (LwF) | 47.6 | 45.1 | 49.3 | 58.7 |
| CLTR (EWC) | 48.9 | 44.4 | 50.3 | 58.8 |
| CLTR (Modified EWC) | 49.1 | 45.9 | 51.0 | 60.7 |
| CLTR (GPM) | 51.7 | 48.3 | 54.7 | 64.7 |
| CLTR (FOSTER) | 52.7 | 48.7 | 54.4 | 63.6 |
| Method | Tasks | |
|---|---|---|
| 5 | 10 | |
| EEIL [ 92 ] | 38.5 | 37.5 |
| EEIL+2sLWS [ 88 ] | 39.0 | 37.6 |
| LUCIR [ 90 ] | 42.7 | 42.2 |
| PODNET [ 91 ] | 44.1 | 44.0 |
| PODNET+2sLWS [ 88 ] | 44.4 | 44.4 |
| Method | Backbone | |
|---|---|---|
| Inc.V4 | Res.101 | |
| [ 93 ] | 84.1 | 85.3 |
| [ 93 ] | 85.8 | 87.2 |
| [ 93 ] | 85.3 | 87.2 |
| DELTA [ 93 ] | 86.8 | 88.7 |
| GBN [ 94 ] | - | 86.9 |
| Training order | Top-1 accuracy |
|---|---|
| Tail Head | 36.0 |
| Random | 32.3 |
| Head Tail (CLTR order) | 50.7 |
| Method | Min. margin | Satisfied |
|---|---|---|
| Naive sequential | ||
| CLTR (LwF) | ||
| CLTR (EWC) |
| Head:Tail split | Local IFs | Top-1 (%) |
|---|---|---|
| 20:80 | [2.43, 39.40] | 46.0 |
| 50:50 | [9.8, 9.6] | 49.7 |
| 80:20 | [41.67, 2.40] | 44.7 |
| Method | Time (s) | Rel. | FLOPs (T) | Rel. | Mem. (MB) |
|---|---|---|---|---|---|
| LTR (WD) | 0.986 | 227.0 | 1.000 | 334 | |
| LTR (WD+MaxNorm) | 1.000 | 227.0 | 1.000 | 334 | |
| CLTR (no CL term) | 1.145 | 252.1 | 1.111 | 334 | |
| CLTR (LwF) | 1.143 | 264.4 | 1.164 | 374 | |
| CLTR (EWC) | 1.250 | 254.3 | 1.120 | 340 | |
| CLTR (SGP) | 1.349 | 252.5 | 1.112 | 343 |
| Method (ViT-B) | |||
|---|---|---|---|
| 10 | 50 | 100 | |
| Baseline [ 16 ] | 66.02 | 54.78 | 50.59 |
| BCE [ 16 ] | 64.63 | 50.65 | 45.50 |
| CB [ 16 ] | 66.30 | 56.18 | 50.63 |
| CLTR (TPL) | 69.74 | 59.71 | 55.94 |
| Method | Head | Tail | Balanced |
|---|---|---|---|
| Balanced training ( ) | 77.42 | 66.18 | 71.80 |
| Long-tailed joint training | 81.53 | 33.20 | 57.37 |
| Naive sequential | 0.00 | 32.78 | 16.39 |
| CLTR (LwF) | 79.21 | 41.69 | 60.45 |
| CLTR (GPM) | 77.39 | 44.72 | 61.06 |
Long-Tailed Classification
Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.
See all 59 papers