VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Jiangang Zhu; Zheng Wang; Bin Zhu; Yi-Ping Phoebe Chen; Jingjing Chen

cs.CVSep 4, 2026

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Authors: Jiangang Zhu, Zheng Wang, Bin Zhu, Yi-Ping Phoebe Chen, Jingjing Chen

Organizations: Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China · Singapore Management University, Singapore, Singapore ·

Abstract

Multi-expert models have become the dominant paradigm for long-tailed learning, largely attributed to their presumed ability to benefit from expert diversity. However, we revisit this central assumption and reveal that diversity induced by logit adjustment or explicit regularizers does not guarantee better ensemble accuracy. Our work suggests that multi-expert models benefit more from variance reduction than diversity maximization. We introduce VICAL, a VIcinal Consistency ALignment framework that improves long-tailed recognition not by enforcing expert diversity, but by reducing prediction variance. Specifically, our approach comprises two key components: Self-Consistency Learning and Deep Ensemble Distillation. Self-Consistency Learning discourages reliance on unstable high-frequency information, smoothing the local loss landscape and mitigating overfitting, especially for tail classes. Deep Ensemble Distillation promotes cross-expert low-frequency semantic agreement using a low-resolution view, thereby sidestepping optimization conflicts with established knowledge. Extensive experiments on CIFAR-LT, ImageNet-LT, and iNaturalist 2018 show that VICAL consistently outperforms state-of-the-art methods, validating the effectiveness of our consistency-driven design. Our code is available at VICAL.

Figures & tables

Figure 1: We use three different diversity metrics, i.e. , Q-statistics, correlation, and diversity accuracy, to measure the relationship between model diversity and ensemble accuracy on CIFAR-100-LT. Each point denotes an independently trained model, and

\lambda

controls the differences in logit adjustment strength across experts. A larger

\lambda

indicates more diverse experts by controlling the logit adjustment intensity for each expert. Varying

\lambda

changes expert agreement but produces no clear improvement in ensemble accuracy. (The Pearson correlation coefficients are

0.11

0.11

, and

-0.13

for Q-statistics, correlation, and diversity accuracy.)

Panel of Figure 2 — Figure 2: Loss landscapes as a function of the interpolation factor $\gamma$ $\in$ [0,1] between two augmented views $\mathrm{v_{1}}$ and $\mathrm{v_{2}}$ on CIFAR-100-LT (IF=100) with a single expert: (a) vanilla self-distillation , (b) SC . SC induces a smoother loss curve as $\gamma$ varies, especially for the tail classes. (c) Accuracy gains over the SC baseline when DED uses a full-resolution or low-resolution input.

Figure 3: The framework of our proposed VICAL, consisting of Self-Consistency (SC) Learning and Deep Ensemble Distillation (DED). SC utilizes the interpolated view $\widetilde{\mathrm{v}}$ to discourage reliance on unstable high-frequency patterns, significantly improving robustness of tail classes. DED strictly confines the cross-expert alignment to low-frequency semantic agreement via a low-resolution view $\mathrm{v}_{s}$ , avoiding optimization conflicts with the SC objective. DED further filters conflicting transfers while retaining complementary expert knowledge through the Conflicting Knowledge Filter.

Method	CIFAR-100-LT				CIFAR-10-LT				ImageNet-LT
Method	200	100	50	10	200	100	50	10	256
LDAM-DRW [ 3 ]	38.5	42.0	46.6	58.7	74.7	77.0	81.0	88.2	45.8
BCL ‡ [ 51 ]	-	51.9	56.6	64.9	-	84.3	87.2	91.1	57.1
PaCo ‡ [ 8 ]	47.8	52.0	56.0	64.2	82.3	85.4	88.0	91.5	57.2
ProCo ‡ [ 10 ]	-	52.8	57.1	65.5	-	85.9	88.2	91.9	58.0
RIDE [ 36 ]	44.6	48.0	51.7	61.8	77.8	81.2	83.7	86.3	56.8

Table 1: Top-1 accuracy on CIFAR-LT using ResNet-32 as backbone and on ImageNet-LT using ResNeXt-50 as backbone. We report the results of 400 and 200 epochs for CIFAR-LT and ImageNet-LT, respectively.

{\ddagger}

denotes models trained with strong augmentation [ 8 ] . The best result is shown in bold, and the second-best result is underlined.

Table 5

Methods	Multi-Expert	Intra-Expert Stabilization	Inter-Expert Alignment	Objective Decoupling	Distillation Gate
PCL [ 18 ]	$\times$	Predictive Consistency	$\times$	$\times$	$\times$
GLMC [ 11 ]	$\times$	Inter-Class Mixup	$\times$	$\times$	$\times$
LTRL [ 46 ]	$\times$	Reflective Learning	$\times$	$\times$	$\times$
RIDE [ 36 ]	✓	$\times$	Explicit Divergence	$\times$	$\times$
BalPoE [ 1 ]	✓	Inter-class Mixup	Explicit Divergence	$\times$	$\times$
MDCS [ 47 ]	✓	Vanilla Self-Distillation	Explicit Divergence	$\times$	Confident Sampling

Table 5: Mechanism comparisons of VICAL against existing consistency-driven and multi-expert methods. Unlike prior approaches that rely on explicit divergence or coupled full-resolution distillation, VICAL decouples intra-expert local smoothing from inter-expert global semantic agreement, which avoids optimization conflicts.

Panel of Figure 4 — Figure 4: Ablation study of the hyperparameters (a) $\eta$ and (b) $\beta$ on CIFAR-100-LT (IF=100). We report the best results achieved by the online and target models. (c) By the terminal training phase, the histogram of the conflicting knowledge set $\mathbb{D}_{c}$ .

Panel of Figure 5 — Figure 5: Model sensitivity to additive noise aligned with different Fourier basis vectors on the CIFAR-100-LT validation set: (a) Head and (b) Tail . The center represents low-frequency , while the edges represent high-frequency . For head classes, VICAL is slightly more sensitive to extreme high-frequency noise. In contrast, VICAL substantially improves tail class robustness in the low-frequency region.

$p^{\mathcal{T}}$	$p^{\mathcal{S}}$	Ratio(%)	Many	Med	Few	All
-	-	-	74.3	59.1	36.6	57.7
All	All	100	74.8	59.8	38.4	58.6
Correct	-	73	74.6	59.8	37.9	58.4
Wrong	Wrong	25	73.6	59.3	38.4	58.0
Wrong	Correct	2	67.7	55.1	34.2	53.2
Ours	Ours	98	75.5	60.1	38.7	59.1

Table 6: We compare the results of several Conflicting Knowledge Filters on the CIFAR-100-LT (IF=100). The baseline is VICAL w/o DED. The Ratio denotes the percentage of samples used for DED throughout the training.

Method	All			Many			Med			Few
Method	Acc.	Bias	Var	Acc.	Bias	Var	Acc.	Bias	Var	Acc.	Bias	Var
CE	31.6	0.60	0.47	57.3	0.28	0.35	28.2	0.61	0.51	6.3	0.94	0.57
RIDE [ 36 ]	40.5	0.50	0.42	60.5	0.28	0.30	38.7	0.50	0.44	20.1	0.74	0.52
MDCS [ 47 ]	46.1	-	0.36	-	-	0.24	-	-	0.38	-	-	0.46
MDCS †	48.2	0.43	0.37	68.9	0.22	0.24	55.1	0.34	0.34	30.4	0.63	0.48
$\lambda$ =2	49.9	0.42	0.35	74.6	0.16	0.19	57.0	0.33	0.31	29.3	0.65	0.48

Table 7: Comparisons of the mean accuracy, bias, and variance of baselines and our proposed method on CIFAR-100-LT (IF=100), following the previous setup [ 36 , 47 ] .

\dagger

represents our reproduced results using the official code on the specified datasets for a fair comparison.

\lambda=0

refers to our proposed method without SC and DED.

Figure 6: Heatmaps of the entropy profiles as a function of the interpolation factor $\gamma$ $\in$ [0,1] between two full-resolution views $\mathrm{v}_{1}$ and $\mathrm{v}_{2}$ on the CIFAR-100-LT test set with a single expert. Head samples ( Top ) are more susceptible to $\gamma$ , whereas tail samples ( Bottom ) have higher entropy overall. SC reduces the entropy of interpolated samples, particularly for tail classes.

Figure 7: We use three different diversity metrics, i.e. , Q-statistics, Correlation, and Diversity accuracy, to measure the relationship between model diversity and ensemble accuracy on CIFAR-100-LT with an explicit diversity regularizer . Each point represents an independently trained multi-expert model.

\kappa

controls the explicit diversity regularizer intensities. A larger

\kappa

value encourages each expert to align with the ensemble prediction

\overline{p}

, while a negative

\kappa

forces predictions to diverge from this ensemble consensus. Ensemble accuracy shows a positive correlation with both Q-statistics and the correlation coefficient

\rho

. (The Pearson correlation coefficients are

0.76

0.76

, and

-0.31

for Q-statistics, Correlation, and Diversity accuracy.)

Panel of Figure 8 — Figure 8: (a) Ensemble accuracy and mean individual expert accuracy across different $\kappa$ . A large positive $\kappa$ yields better individual accuracy, thereby leading to an improved ensemble performance. (b) Inter-expert logit variance on the CIFAR-100-LT test set. We compute the variance of logits across experts. Lower values indicate stronger cross-expert consistency.

Panel of Figure 9 — Figure 9: Ablations of (a) the teacher logit aggregation weight $\varphi$ and (b) the Beta-distribution parameter $\alpha$ on CIFAR-100-LT (IF=100).

Table 15

Figure 10: Comparisons of our method and NCL++ on CIFAR-100-LT (IF=100) with different numbers of experts. While NCL++ achieves superior accuracy in the tail classes, this is accompanied by a high cost in many and medium-shot performance.

Panel of Figure 11 — Figure 11: Grad-CAM visualizations for three experts on iNaturalist 2018.

Method	Params	Many	Med	Few	All
BalPoE [ 1 ]	66.1M	68.2	57.2	44.9	59.8
BalPoE ‡ [ 1 ]	66.1M	-	-	-	61.6
NCL++ ‡ [ 33 ]	50.1M	-	-	-	60.9
RIDE [ 36 ]	38.3M	67.6	53.5	35.9	56.4
SADE [ 45 ]	38.3M	66.5	57.0	43.5	58.8
ML [ 26 ]	38.3M	70.2	56.7	39.1	59.5

Table 10: Top-1 accuracy on ImageNet-LT using ResNeXt-50 as backbone. We report the results of 200 epochs and the trainable parameters of all compared multi-expert models.

{\ddagger}

denotes models trained with RandAugment [ 6 ] .

Method	Prior	Forward-LT					Uni.	Backward-LT
Method	Prior	50	25	10	5	2	1	2	5	10	25	50
LADE	$\times$	56.0	55.5	52.8	51.0	48.0	45.6	43.2	40.0	38.3	35.5	34.0
RIDE	$\times$	63.0	59.9	57.0	53.6	49.4	48.0	42.5	38.1	35.4	31.6	29.2
SADE	$\times$	58.4	57.0	54.4	53.1	50.1	49.4	45.2	42.6	39.7	36.7	35.0
BalPoE	$\times$	65.1	63.1	60.8	58.4	54.8	52.0	48.6	44.6	41.8	38.0	36.1
BalPoE †	$\times$	71.4	69.4	66.1	63.6	59.7	56.4	53.0	49.0	46.2	41.9	39.5

Table 11: Top-1 accuracy on multiple test distributions for ResNet-32 trained on CIFAR-100-LT (IF=100). Prior denotes whether the test set prior is used.

\dagger

refers to our reproduced results, and other results are from [ 1 ] .

Method	Prior	Forward-LT					Uni.	Backward-LT
Method	Prior	50	25	10	5	2	1	2	5	10	25	50
LADE	$\times$	63.4	62.1	59.9	57.4	54.6	52.3	49.9	46.8	44.9	42.7	40.7
RIDE	$\times$	67.6	66.3	64.0	61.7	58.9	56.3	54.0	51.0	48.7	46.2	44.0
SADE	$\times$	65.5	64.4	63.6	62.0	60.0	58.8	56.8	54.7	53.1	51.1	49.8
BalPoE	$\times$	67.6	66.3	65.2	63.3	61.5	59.8	58.1	55.7	54.3	52.2	50.8
Ours	$\times$	72.1	70.9	69.3	67.3	65.0	62.9	60.7	57.7	55.8	53.3	51.3

Table 12: Top-1 accuracy on multiple test distributions for ResNeXt-50 trained on ImageNet-LT. Prior denotes whether the test set prior is used, and

*

refers to implicit test prior estimation.

Long-Tailed Classification

Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.

See all 55 papers

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification
Aug 11, 2026Gawon LimLong-Tailed Classification Ensemble Learning
School of Information Sciences University of Illinois Urbana-Champaign
CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning
May 2, 2026Ruichi Zhang, Chikai Shang, Jiacheng Yang +4Long-Tailed Classification Logit Adjustment
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · College of Computer Science and Software Engineering, Shenzhen University, China · Institute of High Performance Computing, A*STAR, Singapore
OFBD: Object-Focused Background Debiasing for Long-Tailed Learning
Sep 29, 2026Shenghan Chen, Yiming Liu, Zhipeng Deng +6Long-Tailed Classification Long-Tail Learning
Westlake University, Hangzhou, China · Shandong University, Jinan, China · Hokkaido University, Sapporo, Japan +1

Aug 11, 2026J/K move · Enter open · S save

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

Gawon Lim

School of Information Sciences University of Illinois Urbana-Champaign

Long-Tailed Classification Ensemble Learning Product of Experts Classification Mixture-of-Experts Models

Open paper AskPDF arXiv

Long-tailed classification poses a reliability challenge because models trained on imbalanced data are unevenly reliable across frequent and underrepresented classes. While existing methods address imbalance through re-balancing, adjustment, representation learning, or multi-expert modeling, they rarely estimate which expert should be trusted for each class. This paper proposes CLEAR (Class-wise reLiability-aware Expert Aggregation for long-tailed Recognition), a modular ensemble framework for long-tailed classification. CLEAR generates diverse experts through threshold-based structured sampling while preserving the full label space, then estimates a class-wise trust score for each expert using a smoothed class-wise precision formulation. During inference, expert predictions are combined through class-wise generalized product-of-experts aggregation, allowing different experts to be emphasized for different classes. Experiments on CIFAR-100-LT, ImageNet-LT, and Places-LT across multiple backbones show that CLEAR achieves competitive overall accuracy and particularly strong few-shot performance. These results support class-wise expert reliability as a useful design principle for long-tailed ensemble learning.

Abstract

Long-Tailed Classification

Explore similar work

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning