Equilibrium contrastive learning for imbalanced image classification

Sumin Roh; Harim Kim; Ho Yun Lee; Il Yong Chun

cs.CVFeb 10, 2026

Equilibrium contrastive learning for imbalanced image classification

Authors: Sumin Roh, Harim Kim, Ho Yun Lee, Il Yong Chun

Organizations: Department of Electrical and Computer Engineering (ECE), Sungkyunkwan University (SKKU), Suwon 16419, South Korea · Department of Radiology, Samsung Medical Center (SMC), Seoul 06351, South Korea · Department of Radiology, SMC, Seoul 06351, South Korea · School of Medicine, SKKU, Suwon 16419, South Korea ·

Abstract

Contrastive learning (CL) is a predominant technique in image classification, but they showed limited performance with an imbalanced dataset. Recently, several supervised CL methods have been proposed to promote an ideal regular simplex geometric configuration in the representation space-characterized by intra-class feature collapse and uniform inter-class mean spacing, especially for imbalanced datasets. In particular, existing prototype-based methods include class prototypes, as additional samples to consider all classes. However, the existing CL methods suffer from two limitations. First, they do not consider the alignment between the class means/prototypes and classifiers, which could lead to poor generalization. Second, existing prototype-based methods treat prototypes as only one additional sample per class, making their influence depend on the number of class instances in a batch and causing unbalanced contributions across classes. To address these limitations, we propose Equilibrium Contrastive Learning (ECL), a supervised CL framework designed to promote geometric equilibrium, where class features, means, and classifiers are harmoniously balanced under data imbalance. The proposed ECL framework uses two main components. First, ECL promotes the representation geometric equilibrium (i.e., a regular simplex geometry characterized by collapsed class samples and uniformly distributed class means), while balancing the contributions of class-average features and class prototypes. Second, ECL establishes a classifier-class center geometric equilibrium by aligning classifier weights and class prototypes. We ran experiments with three long-tailed datasets, the CIFAR-10(0)-LT, ImageNet-LT, and the two imbalanced medical datasets, the ISIC 2019 and our constructed LCCT dataset. Results show that ECL outperforms existing SOTA supervised CL methods designed for imbalanced classification.

Figures & tables

Fig. 1 : Illustration of E quilibrium C ontrastive L earning ( ECL ). ECL promotes the following three key geometric properties that collectively establish geometric equilibrium in the representation space: 1) the collapse of within-class representations (denoted by $\mdsmblkcircle$ ) to their class means (denoted by $\lozenge$ ), 2) equidistant spacing among class means, and 3) alignment between linear classifier weights (denoted by skyblue sticks) and class means. It can simplify the geometries in representation and classifier, leading to simple nearest-centroid classifications in the representation space. The simple geometries are useful for improving the generalization capability.

Panel of Figure 2 — Fig. 2: Illustration of three key properties to simplify geometries via ECL under class imbalance (red and green denote features from different majority classes, and blue denotes features from the minority class). (a) Intra-class feature collapse: all the intra-class representations (denoted by $\mdsmblkcircle$ ) collapse to their corresponding class means (denoted by $\lozenge$ ). (b) Inter-class mean geometric equilibrium: distances and angles among class means become equal, forming a symmetric geometric structure. (c) CC-GE: Classifier–class center geometric equilibrium: the linear classifiers (denoted by skyblue sticks) and class means converge to each other with rescaling.

Panel of Figure 3 — Fig. 3: Visualization of feature distribution on the unit sphere obtained by BCL with the LCCT dataset zooming in to the upper half for better visualization. The $\triangle$ and $\square$ symbols indicate the class means and classifier weights, while the red and green points represent samples from the majority and minority classes, respectively. The results show that the CC-GE property is not induced.

Fig. 4 : The overall ECL architecture. The proposed ECL consists of three core modules, (a) contrastive representation learning with proposed $\mathcal{L}_{\text{BC-ECL}}$ in Section III-C1 , (b) classifier-class center alignment with proposed $\mathcal{L}_{\text{CC-GE}}$ in Section III-C2 , and (c) classifier learning with $\mathcal{L}_{\text{LC}}$ introduced in Section III-C3 . (a) The proposed $\mathcal{L}_{\text{BC-ECL}}$ formulation equalizes the contribution between class-average features and prototypes in promoting the intra-class feature collapse and geometric equilibrium among class means. We utilize the linearly transformed classifier weights as class prototypes, and $\ell_{2}$ -normalize the representations $\{\mathbf{z}_{i}^{(2)},\mathbf{z}_{i}^{(3)}:\forall i\}$ . The symbols Sim. and Avg. denote a similarity computation between two vectors and an averaging operator, respectively. (b) The proposed $\mathcal{L}_{\text{CC-GE}}$ loss aligns the classifier weights and class prototypes to promote geometric equilibrium between classifier weights and class center representations. (c) For classifier learning, we use a logit-compensated variant of the cross-entropy loss, $\mathcal{L}_{\text{LC}}$ .

Methods	CIFAR-10-LT			CIFAR-100-LT
Methods	$\rho$ =100	50	10	$\rho$ =100	50	10
CB-Focal [ 36 ]	74.5	79.2	87.1	39.6	45.1	57.9
BBN [ 37 ]	79.8	81.1	88.3	42.5	47.0	59.1
ResLT [ 38 ]	82.4	85.1	89.7	48.2	52.7	62.0
Hybrid-SC [ 39 ]	81.4	85.3	91.1	46.7	51.8	63.0
TSC [ 6 ]	79.7	82.9	88.7	42.8	46.3	57.6

TABLE I : Top-1 accuracy (%) comparisons between different supervised (C)L methods with CIFAR-10-LT and CIFAR-100-LT datasets.

Methods	Many	Medium	Few	All
CB-Focal [ 36 ]	39.6	32.7	16.8	33.2
LDAM [ 40 ]	60.4	46.9	30.7	49.8
LADE [ 41 ]	65.1	48.9	33.4	53.0
ResLT [ 38 ]	63.6	55.7	38.9	56.1
KCL [ 3 ]	61.8	49.4	30.9	51.5
TSC [ 6 ]	63.5	49.7	30.4	52.4

TABLE II : Top-1 accuracy (%) comparisons between different (C)L methods with ImageNet-LT dataset.

Methods	ISIC 2019 ( $\rho\approx 54$ )	LCCT ( $\rho\approx 3$ )
TSC [ 6 ]	50.23	63.6
CoNe [ 14 ]	63.57	74.93
GLMC [ 5 ]	70.49	59.33
BCL [ 7 ]	83.76	80.24
GPaCo [ 10 ]	85.66	74.52
ECL (ours)	87.31	92.36

TABLE III : Top-1 accuracy (%) comparisons between different supervised CL methods with ISIC 2019 and LCCT datasets.

Methods	CIFAR-10-LT ( $\rho=100$ )			LCCT
Methods	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$
TSC [ 6 ]	0.95	0.30	1.42	0.91	0.56	0.97
CoNe [ 14 ]	1.20	0.44	0.62	0.47	0.62	0.38
GLMC [ 5 ]	0.86	0.89	0.71	0.88	0.71	0.69
BCL [ 7 ]	0.64	0.62	0.55	0.41	0.82	0.73
GPaCo [ 10 ]	0.82	0.75	0.42	0.75	0.98	0.27

TABLE IV : Comparisons of FC , MS , and SD values (see their definitions in Section IV-C ) between different supervised CL methods with CIFAR-10-LT and LCCT datasets (the

\downarrow

and

\uparrow

symbols indicate that the lower the better and higher the better, respectively).

Panel of Figure 9 — Fig. 5 : Visualization of feature distribution on the unit hypersphere for different supervised CL methods with LCCT dataset; $\{x\in[-1,1],y\in[-1,1]\}$ . If class-wise features are clustered (see Fig. 2 (a)) to form a regular simplex configuration (see Fig. 2 (b)) and their class means are aligned with the classifier weights (see Fig. 2 (c)), then good representations and classifier are learned. Legends: denotes minority class samples; denotes majority class samples; denotes minority class mean; denotes majority class mean; denotes minority classifier weight; and denotes majority classifier weight.

$\mathcal{L}_{\text{BC-ECL}}$	$\mathcal{L}_{\text{CC-GE}}$	$\mathcal{L}_{\text{LC}}$	CIFAR-10-LT ( $\rho=100$ )				LCCT
( 4 )	( 6 )	( 7 )	Top-1 acc.	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$	Top-1 acc.	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$
✗	✗	✓	82.27 %	0.90	0.39	0.85	79.20 %	0.88	0.52	0.74
✗	✓	✓	87.26 %	0.81	0.37	0.80	81.62%	0.86	0.56	0.72
✓	✗	✓	88.61 %	0.64	0.65	0.86	88.52 %	0.37	0.76	0.71
✓	✓	✓	92.13 %	0.36	0.94	0.35	92.36 %	0.23	1.61	0.21

TABLE V : Ablation study for the primary components of proposed ECL (the

\downarrow

and

\uparrow

symbols indicate that the lower the better and and higher the better, respectively).

Methods	Many	Medium	Few	All
BCL [ 7 ] (baseline)	67.9	54.2	36.6	57.1
Variant of ECL by replacing	67.6	54.1	39.6	57.3
$\mathcal{L}_{\text{BC-ECL}}$ ( 4 ) w/ $\mathcal{L}_{\text{BCL}}$ ( 2 )	67.6	54.1	39.6	57.3
ECL (ours)	68.6	56.3	40.1	58.8

TABLE VI : Top-1 accuracy (%) comparisons between ECL and its variant by replacing

\mathcal{L}_{\text{BC-ECL}}

( 4 ) with

\mathcal{L}_{\text{BCL}}

( 2 ), with ImageNet-LT dataset.

Methods	CIFAR-10-LT ( $\rho=100$ )		LCCT
Methods	Training time	Top-1 acc.	Training time	Top-1 acc.
Means ( 1 )	- †	- †	11 h	81.61 %
Nonlinearly transf. weights of clf. [ 7 ]	2.5 h	90.68 %	7.5 h	90.06 %
Linearly transf. weights of clf. (ours)	2 h	92.13 %	7 h	92.36 %

TABLE VII : Performance comparisons between different forms of prototypes

\mathbf{P}

in ( 6 ) with different datasets.

Methods	CIFAR-10-LT ( $\rho=100$ )				LCCT
Methods	Top-1 acc.	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$	Top-1 acc.	$\text{FC}^{\downarrow}$	$\text{MS}^{\uparrow}$	$\text{SD}^{\downarrow}$
Nonlinear	90.65%	0.38	0.88	0.36	89.71%	0.36	1.27	0.33
Linear	92.13%	0.36	0.94	0.25	92.36%	0.23	1.61	0.21

TABLE VIII : Comparisons of Top-

1

accuracy, FC , MS , and SD values between a nonlinear classifier and a linear classifier with CIFAR-10-LT and LCCT datasets.

$\lambda_{\text{BC-ECL}}$	$\lambda_{\text{LC}}$	$\lambda_{\text{CC-GE}}$	CIFAR-10-LT
0.5	1	1	91.9
0.5	0.5	1	91.5
0.5	1	3	92.0
0.5	0.5	3	92.1
1	1	1	91.3
1	0.5	1	91.7

TABLE S1 : Top-1 accuracy comparisons of ECL under varying balancing parameter combinations with CIFAR-10-LT (

\rho=100

Methods	Precision	Recall
BCL [ 7 ]	86.65	84.14
PaCO [ 9 ]	77.21	73.72
GPaCo [ 10 ]	79.73	77.56
ECL	94.41	91.60

TABLE S2 : Comparisons of precision and recall with CIFAR-

10

-LT dataset (

\rho=100

Panel of Figure 20 — Fig. S1: Confusion matrices for ECL on two different medical image datasets. The bar on the right indicates the classification accuracy.

Panel of Figure 21 — Fig. S2: ROC and PR curves of ECL with the LCCT dataset.

Panel of Figure 22 — Fig. S3: ROC and PR curves of ECL with the ISIC 2019 dataset.

Class-Imbalanced Learning

Class-imbalanced learning develops classifiers that recognize rare classes without sacrificing performance on common ones.

See all 254 papers

Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets
May 11, 2026Thuan Nguyen, Shuchin Aeron, D. Richard Brown +1Class-Imbalanced Learning Contrastive Learning
Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute +1
Recurrent Contrastive Learning for Imbalanced Medical Image Classification
Aug 4, 2026Zhiyuan Zhu, Xinling Meng, Junxuan Yu +15Class-Imbalanced Learning Contrastive Learning
School of Biomedical Engineering, Medical School, Shenzhen University, China · Ultra-X AI Lab, Shenzhen University, Guangdong, China · School of Biomedical Engineering, South-Central Minzu University, Wuhan, China +4
A Unified Geometric Framework for Weighted Contrastive Learning
May 13, 2026Raphael Vock, Edouard Duchesnay, Benoit DufumierContrastive Learning Neural Representation Geometry
GAIA Lab, NeuroSpin, CEA, CNRS · Université Paris-Saclay, Gif-sur-Yvette, France

May 11, 2026J/K move · Enter open · S save

Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets

Thuan Nguyen, Shuchin Aeron, D. Richard Brown, Prakash Ishwar

Department of Engineering, Engineering Technology East Tennessee State University · Department of Electrical and Computer Engineering Tufts University · Department of Electrical and Computer Engineering Worcester Polytechnic Institute+1

Class-Imbalanced Learning Contrastive Learning Representation Collapse Neural Representation Geometry Neural Collapse

Open paper AskPDF arXiv

In this paper, we provide a computable characterization of the geometry of optimal representations in Contrastive Learning (CL) when the classes are imbalanced. When classes are balanced and the representation dimension is greater than the number of classes, it is well-known that the optimal representations exhibit Neural Collapse (NC), i.e., representations from the same class collapse to their class means and the class means form an Equiangular Tight Frame (ETF). For imbalanced classes and a large, generalized family of CL losses, we prove that the optimal representations of all samples from the same class collapse to their class means and their geometry exhibits an angular symmetry structure that is determined by the relative class proportions. In general, we show that the geometry can be determined by solving a convex optimization problem. Exploiting this symmetry structure, we analytically investigate a special case where class imbalance is extreme and prove that CL exhibits a phenomenon called Minority Collapse (MC) where all samples from the minority classes (classes with small probabilities) collapse into a single vector, whenever the class imbalance exceeds a threshold, which in turn depends on the regularity properties of the CL loss used and on the number of negative samples. Numerical results are provided to illustrate these phenomena and corroborate the theoretical results. We conclude by identifying a number of open problems.

Abstract

Class-Imbalanced Learning

Explore similar work

Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets

Recurrent Contrastive Learning for Imbalanced Medical Image Classification

A Unified Geometric Framework for Weighted Contrastive Learning