Equilibrium contrastive learning for imbalanced image classification
Authors: Sumin Roh, Harim Kim, Ho Yun Lee, Il Yong Chun
Organizations: Department of Electrical and Computer Engineering (ECE), Sungkyunkwan University (SKKU), Suwon 16419, South Korea · Department of Radiology, Samsung Medical Center (SMC), Seoul 06351, South Korea · Department of Radiology, SMC, Seoul 06351, South Korea · School of Medicine, SKKU, Suwon 16419, South Korea · Departments of ECE, Artificial Intelligence, Advanced Display Engineering, and Semiconductor Convergence Engineering, Display Convergence Engineering, SKKU, Suwon 16419, South Korea · Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon 16419, South Korea
Contrastive learning (CL) is a predominant technique in image classification, but they showed limited performance with an imbalanced dataset. Recently, several supervised CL methods have been proposed to promote an ideal regular simplex geometric configuration in the representation space-characterized by intra-class feature collapse and uniform inter-class mean spacing, especially for imbalanced datasets. In particular, existing prototype-based methods include class prototypes, as additional samples to consider all classes. However, the existing CL methods suffer from two limitations. First, they do not consider the alignment between the class means/prototypes and classifiers, which could lead to poor generalization. Second, existing prototype-based methods treat prototypes as only one additional sample per class, making their influence depend on the number of class instances in a batch and causing unbalanced contributions across classes. To address these limitations, we propose Equilibrium Contrastive Learning (ECL), a supervised CL framework designed to promote geometric equilibrium, where class features, means, and classifiers are harmoniously balanced under data imbalance. The proposed ECL framework uses two main components. First, ECL promotes the representation geometric equilibrium (i.e., a regular simplex geometry characterized by collapsed class samples and uniformly distributed class means), while balancing the contributions of class-average features and class prototypes. Second, ECL establishes a classifier-class center geometric equilibrium by aligning classifier weights and class prototypes. We ran experiments with three long-tailed datasets, the CIFAR-10(0)-LT, ImageNet-LT, and the two imbalanced medical datasets, the ISIC 2019 and our constructed LCCT dataset. Results show that ECL outperforms existing SOTA supervised CL methods designed for imbalanced classification.
Figures & tables
Fig. 1 : Illustration of E quilibrium C ontrastive L earning ( ECL ). ECL promotes the following three key geometric properties that collectively establish geometric equilibrium in the representation space: 1) the collapse of within-class representations (denoted by \mdsmblkcircle ) to their class means (denoted by ◊ ), 2) equidistant spacing among class means, and 3) alignment between linear classifier weights (denoted by skyblue sticks) and class means. It can simplify the geometries in representation and classifier, leading to simple nearest-centroid classifications in the representation space. The simple geometries are useful for improving the generalization capability.
Fig. 2: Illustration of three key properties to simplify geometries via ECL under class imbalance (red and green denote features from different majority classes, and blue denotes features from the minority class). (a) Intra-class feature collapse: all the intra-class representations (denoted by \mdsmblkcircle ) collapse to their corresponding class means (denoted by ◊ ). (b) Inter-class mean geometric equilibrium: distances and angles among class means become equal, forming a symmetric geometric structure. (c) CC-GE: Classifier–class center geometric equilibrium: the linear classifiers (denoted by skyblue sticks) and class means converge to each other with rescaling.
Fig. 3: Visualization of feature distribution on the unit sphere obtained by BCL with the LCCT dataset zooming in to the upper half for better visualization. The △ and □ symbols indicate the class means and classifier weights, while the red and green points represent samples from the majority and minority classes, respectively. The results show that the CC-GE property is not induced.
Fig. 4 : The overall ECL architecture. The proposed ECL consists of three core modules, (a) contrastive representation learning with proposed LBC-ECL in Section III-C1 , (b) classifier-class center alignment with proposed LCC-GE in Section III-C2 , and (c) classifier learning with LLC introduced in Section III-C3 . (a) The proposed LBC-ECL formulation equalizes the contribution between class-average features and prototypes in promoting the intra-class feature collapse and geometric equilibrium among class means. We utilize the linearly transformed classifier weights as class prototypes, and ℓ2 -normalize the representations {zi(2),zi(3):∀i} . The symbols Sim. and Avg. denote a similarity computation between two vectors and an averaging operator, respectively. (b) The proposed LCC-GE loss aligns the classifier weights and class prototypes to promote geometric equilibrium between classifier weights and class center representations. (c) For classifier learning, we use a logit-compensated variant of the cross-entropy loss, LLC .
Methods
CIFAR-10-LT
CIFAR-100-LT
ρ =100
50
10
ρ =100
50
10
CB-Focal [ 36 ]
74.5
79.2
87.1
39.6
45.1
57.9
BBN [ 37 ]
79.8
81.1
88.3
42.5
47.0
59.1
ResLT [ 38 ]
82.4
85.1
89.7
48.2
52.7
62.0
Hybrid-SC [ 39 ]
81.4
85.3
91.1
46.7
51.8
63.0
TSC [ 6 ]
79.7
82.9
88.7
42.8
46.3
57.6
TABLE I : Top-1 accuracy (%) comparisons between different supervised (C)L methods with CIFAR-10-LT and CIFAR-100-LT datasets.
Methods
Many
Medium
Few
All
CB-Focal [ 36 ]
39.6
32.7
16.8
33.2
LDAM [ 40 ]
60.4
46.9
30.7
49.8
LADE [ 41 ]
65.1
48.9
33.4
53.0
ResLT [ 38 ]
63.6
55.7
38.9
56.1
KCL [ 3 ]
61.8
49.4
30.9
51.5
TSC [ 6 ]
63.5
49.7
30.4
52.4
TABLE II : Top-1 accuracy (%) comparisons between different (C)L methods with ImageNet-LT dataset.
Methods
ISIC 2019 ( ρ≈54 )
LCCT ( ρ≈3 )
TSC [ 6 ]
50.23
63.6
CoNe [ 14 ]
63.57
74.93
GLMC [ 5 ]
70.49
59.33
BCL [ 7 ]
83.76
80.24
GPaCo [ 10 ]
85.66
74.52
ECL (ours)
87.31
92.36
TABLE III : Top-1 accuracy (%) comparisons between different supervised CL methods with ISIC 2019 and LCCT datasets.
Methods
CIFAR-10-LT ( ρ=100 )
LCCT
FC↓
MS↑
SD↓
FC↓
MS↑
SD↓
TSC [ 6 ]
0.95
0.30
1.42
0.91
0.56
0.97
CoNe [ 14 ]
1.20
0.44
0.62
0.47
0.62
0.38
GLMC [ 5 ]
0.86
0.89
0.71
0.88
0.71
0.69
BCL [ 7 ]
0.64
0.62
0.55
0.41
0.82
0.73
GPaCo [ 10 ]
0.82
0.75
0.42
0.75
0.98
0.27
TABLE IV : Comparisons of FC , MS , and SD values (see their definitions in Section IV-C ) between different supervised CL methods with CIFAR-10-LT and LCCT datasets (the ↓ and ↑ symbols indicate that the lower the better and higher the better, respectively).
Fig. 5 : Visualization of feature distribution on the unit hypersphere for different supervised CL methods with LCCT dataset; {x∈[−1,1],y∈[−1,1]} . If class-wise features are clustered (see Fig. 2 (a)) to form a regular simplex configuration (see Fig. 2 (b)) and their class means are aligned with the classifier weights (see Fig. 2 (c)), then good representations and classifier are learned. Legends: denotes minority class samples; denotes majority class samples; denotes minority class mean; denotes majority class mean; denotes minority classifier weight; and denotes majority classifier weight.
LBC-ECL
LCC-GE
LLC
CIFAR-10-LT ( ρ=100 )
LCCT
( 4 )
( 6 )
( 7 )
Top-1 acc.
FC↓
MS↑
SD↓
Top-1 acc.
FC↓
MS↑
SD↓
✗
✗
✓
82.27 %
0.90
0.39
0.85
79.20 %
0.88
0.52
0.74
✗
✓
✓
87.26 %
0.81
0.37
0.80
81.62%
0.86
0.56
0.72
✓
✗
✓
88.61 %
0.64
0.65
0.86
88.52 %
0.37
0.76
0.71
✓
✓
✓
92.13 %
0.36
0.94
0.35
92.36 %
0.23
1.61
0.21
TABLE V : Ablation study for the primary components of proposed ECL (the ↓ and ↑ symbols indicate that the lower the better and and higher the better, respectively).
Methods
Many
Medium
Few
All
BCL [ 7 ] (baseline)
67.9
54.2
36.6
57.1
Variant of ECL by replacing
67.6
54.1
39.6
57.3
LBC-ECL ( 4 ) w/ LBCL ( 2 )
ECL (ours)
68.6
56.3
40.1
58.8
TABLE VI : Top-1 accuracy (%) comparisons between ECL and its variant by replacing LBC-ECL ( 4 ) with LBCL ( 2 ), with ImageNet-LT dataset.
Methods
CIFAR-10-LT ( ρ=100 )
LCCT
Training time
Top-1 acc.
Training time
Top-1 acc.
Means ( 1 )
- †
- †
11 h
81.61 %
Nonlinearly transf. weights of clf. [ 7 ]
2.5 h
90.68 %
7.5 h
90.06 %
Linearly transf. weights of clf. (ours)
2 h
92.13 %
7 h
92.36 %
TABLE VII : Performance comparisons between different forms of prototypes P in ( 6 ) with different datasets.
Methods
CIFAR-10-LT ( ρ=100 )
LCCT
Top-1 acc.
FC↓
MS↑
SD↓
Top-1 acc.
FC↓
MS↑
SD↓
Nonlinear
90.65%
0.38
0.88
0.36
89.71%
0.36
1.27
0.33
Linear
92.13%
0.36
0.94
0.25
92.36%
0.23
1.61
0.21
TABLE VIII : Comparisons of Top- 1 accuracy, FC , MS , and SD values between a nonlinear classifier and a linear classifier with CIFAR-10-LT and LCCT datasets.
Figure 14Figure 15Figure 16Figure 17
λBC-ECL
λLC
λCC-GE
CIFAR-10-LT
0.5
1
1
91.9
0.5
0.5
1
91.5
0.5
1
3
92.0
0.5
0.5
3
92.1
1
1
1
91.3
1
0.5
1
91.7
TABLE S1 : Top-1 accuracy comparisons of ECL under varying balancing parameter combinations with CIFAR-10-LT ( ρ=100 ).
Methods
Precision
Recall
BCL [ 7 ]
86.65
84.14
PaCO [ 9 ]
77.21
73.72
GPaCo [ 10 ]
79.73
77.56
ECL
94.41
91.60
TABLE S2 : Comparisons of precision and recall with CIFAR- 10 -LT dataset ( ρ=100 ).
Fig. S1: Confusion matrices for ECL on two different medical image datasets. The bar on the right indicates the classification accuracy.
Fig. S2: ROC and PR curves of ECL with the LCCT dataset.
Fig. S3: ROC and PR curves of ECL with the ISIC 2019 dataset.
In this paper, we provide a computable characterization of the geometry of optimal representations in Contrastive Learning (CL) when the classes are imbalanced. When classes are balanced and the representation dimension is greater than the number of classes, it is well-known that the optimal representations exhibit Neural Collapse (NC), i.e., representations from the same class collapse to their class means and the class means form an Equiangular Tight Frame (ETF). For imbalanced classes and a large, generalized family of CL losses, we prove that the optimal representations of all samples from the same class collapse to their class means and their geometry exhibits an angular symmetry structure that is determined by the relative class proportions. In general, we show that the geometry can be determined by solving a convex optimization problem. Exploiting this symmetry structure, we analytically investigate a special case where class imbalance is extreme and prove that CL exhibits a phenomenon called Minority Collapse (MC) where all samples from the minority classes (classes with small probabilities) collapse into a single vector, whenever the class imbalance exceeds a threshold, which in turn depends on the regularity properties of the CL loss used and on the number of negative samples. Numerical results are provided to illustrate these phenomena and corroborate the theoretical results. We conclude by identifying a number of open problems.
Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1
Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute +1
Medical image classification often suffers from class imbalance due to the inherent disparities in disease incidence. Existing approaches, such as class resampling and loss reweighting, mainly improve learning within the observed feature distribution, but do not explicitly enlarge the latent support region of tail classes. As a result, tail-class representations remain overly compact and are easily encroached upon by head classes, leading to biased decision boundaries. In this work, we propose Recurrent Contrastive Learning (RCL) for imbalanced medical image classification. RCL progressively expands the support region of tail classes by recurrently reusing historical feature states across training phases. Specifically, we adopt DINOv3 with LoRA adapters as the backbone to provide robust feature embeddings. We then devise a Temporal Memory Queue (TMQ) to preserve corpus-level features across training phases and provide diversified global references for contrastive learning. Based on TMQ, we construct Temporal Anchors (TARs) to form an anchor field around tail classes. This field enlarges the support region of tail classes, suppresses head-class encroachment, and improves inter-class separation. Extensive experiments on three imbalanced medical datasets demonstrate that RCL achieves consistent improvements over strong baselines. The code is available at https://github.com/dndins/RCL.
Zhiyuan Zhu, Xinling Meng, Junxuan Yu +15
School of Biomedical Engineering, Medical School, Shenzhen University, China · Ultra-X AI Lab, Shenzhen University, Guangdong, China · School of Biomedical Engineering, South-Central Minzu University, Wuhan, China +4
Contrastive learning (CL) aims to preserve relational structure between samples by learning representations that reflect a similarity graph. Yet, the geometry of the resulting embeddings remains poorly understood. Here we show that weighted InfoNCE objectives can be interpreted as Distance Geometry Problems, where the weighting scheme specifies the target geometry to be realized by the representation. This viewpoint yields exact characterizations of the optimal embeddings for several supervised and weakly supervised objectives. In supervised classification, both SupCon and Soft SupCon (a dense relaxation of it where pairs from distinct classes have small non-zero similarity) collapse samples within each class to a single prototype. However, while balanced SupCon recovers the classical regular simplex geometry, class imbalance breaks this symmetry: SupCon induces non-uniform inter-class similarities depending on class sizes, whereas Soft SupCon preserves a regular simplex geometry regardless of class imbalance. In continuous-label settings, our framework reveals a different failure mode: y-Aware CL generally cannot attain its entropic optimum unless the labels lie on a hypersphere, exposing a mismatch between Euclidean label weights and spherical latent similarity. By contrast, geometrically consistent choices such as Euclidean-Euclidean weighting or X-CLR admit unique optimal embeddings. Our results show that the choice of weighting scheme determines whether contrastive learning is geometrically realizable, degenerate, or inconsistent, providing a principled framework for designing contrastive objectives.
Raphael Vock, Edouard Duchesnay, Benoit Dufumier
GAIA Lab, NeuroSpin, CEA, CNRS · Université Paris-Saclay, Gif-sur-Yvette, France