VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Jiangang Zhu; Zheng Wang; Bin Zhu; Yi-Ping Phoebe Chen; Jingjing Chen

cs.CVSep 4, 2026

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

Authors: Jiangang Zhu, Zheng Wang, Bin Zhu, Yi-Ping Phoebe Chen, Jingjing Chen

Organizations: Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China · Singapore Management University, Singapore, Singapore ·

Abstract

Multi-expert models have become the dominant paradigm for long-tailed learning, largely attributed to their presumed ability to benefit from expert diversity. However, we revisit this central assumption and reveal that diversity induced by logit adjustment or explicit regularizers does not guarantee better ensemble accuracy. Our work suggests that multi-expert models benefit more from variance reduction than diversity maximization. We introduce VICAL, a VIcinal Consistency ALignment framework that improves long-tailed recognition not by enforcing expert diversity, but by reducing prediction variance. Specifically, our approach comprises two key components: Self-Consistency Learning and Deep Ensemble Distillation. Self-Consistency Learning discourages reliance on unstable high-frequency information, smoothing the local loss landscape and mitigating overfitting, especially for tail classes. Deep Ensemble Distillation promotes cross-expert low-frequency semantic agreement using a low-resolution view, thereby sidestepping optimization conflicts with established knowledge. Extensive experiments on CIFAR-LT, ImageNet-LT, and iNaturalist 2018 show that VICAL consistently outperforms state-of-the-art methods, validating the effectiveness of our consistency-driven design. Our code is available at VICAL.

Figures & tables

Figure 1: We use three different diversity metrics, i.e. , Q-statistics, correlation, and diversity accuracy, to measure the relationship between model diversity and ensemble accuracy on CIFAR-100-LT. Each point denotes an independently trained model, and

\lambda

controls the differences in logit adjustment strength across experts. A larger

\lambda

indicates more diverse experts by controlling the logit adjustment intensity for each expert. Varying

\lambda

changes expert agreement but produces no clear improvement in ensemble accuracy. (The Pearson correlation coefficients are

0.11

0.11

, and

-0.13

for Q-statistics, correlation, and diversity accuracy.)

Panel of Figure 2 — Figure 2: Loss landscapes as a function of the interpolation factor $\gamma$ $\in$ [0,1] between two augmented views $\mathrm{v_{1}}$ and $\mathrm{v_{2}}$ on CIFAR-100-LT (IF=100) with a single expert: (a) vanilla self-distillation , (b) SC . SC induces a smoother loss curve as $\gamma$ varies, especially for the tail classes. (c) Accuracy gains over the SC baseline when DED uses a full-resolution or low-resolution input.

Figure 3: The framework of our proposed VICAL, consisting of Self-Consistency (SC) Learning and Deep Ensemble Distillation (DED). SC utilizes the interpolated view $\widetilde{\mathrm{v}}$ to discourage reliance on unstable high-frequency patterns, significantly improving robustness of tail classes. DED strictly confines the cross-expert alignment to low-frequency semantic agreement via a low-resolution view $\mathrm{v}_{s}$ , avoiding optimization conflicts with the SC objective. DED further filters conflicting transfers while retaining complementary expert knowledge through the Conflicting Knowledge Filter.

Method	CIFAR-100-LT				CIFAR-10-LT				ImageNet-LT
Method	200	100	50	10	200	100	50	10	256
LDAM-DRW [ 3 ]	38.5	42.0	46.6	58.7	74.7	77.0	81.0	88.2	45.8
BCL ‡ [ 51 ]	-	51.9	56.6	64.9	-	84.3	87.2	91.1	57.1
PaCo ‡ [ 8 ]	47.8	52.0	56.0	64.2	82.3	85.4	88.0	91.5	57.2
ProCo ‡ [ 10 ]	-	52.8	57.1	65.5	-	85.9	88.2	91.9	58.0
RIDE [ 36 ]	44.6	48.0	51.7	61.8	77.8	81.2	83.7	86.3	56.8

Table 1: Top-1 accuracy on CIFAR-LT using ResNet-32 as backbone and on ImageNet-LT using ResNeXt-50 as backbone. We report the results of 400 and 200 epochs for CIFAR-LT and ImageNet-LT, respectively.

{\ddagger}

denotes models trained with strong augmentation [ 8 ] . The best result is shown in bold, and the second-best result is underlined.

Table 5

Methods	Multi-Expert	Intra-Expert Stabilization	Inter-Expert Alignment	Objective Decoupling	Distillation Gate
PCL [ 18 ]	$\times$	Predictive Consistency	$\times$	$\times$	$\times$
GLMC [ 11 ]	$\times$	Inter-Class Mixup	$\times$	$\times$	$\times$
LTRL [ 46 ]	$\times$	Reflective Learning	$\times$	$\times$	$\times$
RIDE [ 36 ]	✓	$\times$	Explicit Divergence	$\times$	$\times$
BalPoE [ 1 ]	✓	Inter-class Mixup	Explicit Divergence	$\times$	$\times$
MDCS [ 47 ]	✓	Vanilla Self-Distillation	Explicit Divergence	$\times$	Confident Sampling

Table 5: Mechanism comparisons of VICAL against existing consistency-driven and multi-expert methods. Unlike prior approaches that rely on explicit divergence or coupled full-resolution distillation, VICAL decouples intra-expert local smoothing from inter-expert global semantic agreement, which avoids optimization conflicts.

Panel of Figure 4 — Figure 4: Ablation study of the hyperparameters (a) $\eta$ and (b) $\beta$ on CIFAR-100-LT (IF=100). We report the best results achieved by the online and target models. (c) By the terminal training phase, the histogram of the conflicting knowledge set $\mathbb{D}_{c}$ .

Panel of Figure 5 — Figure 5: Model sensitivity to additive noise aligned with different Fourier basis vectors on the CIFAR-100-LT validation set: (a) Head and (b) Tail . The center represents low-frequency , while the edges represent high-frequency . For head classes, VICAL is slightly more sensitive to extreme high-frequency noise. In contrast, VICAL substantially improves tail class robustness in the low-frequency region.

$p^{\mathcal{T}}$	$p^{\mathcal{S}}$	Ratio(%)	Many	Med	Few	All
-	-	-	74.3	59.1	36.6	57.7
All	All	100	74.8	59.8	38.4	58.6
Correct	-	73	74.6	59.8	37.9	58.4
Wrong	Wrong	25	73.6	59.3	38.4	58.0
Wrong	Correct	2	67.7	55.1	34.2	53.2
Ours	Ours	98	75.5	60.1	38.7	59.1

Table 6: We compare the results of several Conflicting Knowledge Filters on the CIFAR-100-LT (IF=100). The baseline is VICAL w/o DED. The Ratio denotes the percentage of samples used for DED throughout the training.

Method	All			Many			Med			Few
Method	Acc.	Bias	Var	Acc.	Bias	Var	Acc.	Bias	Var	Acc.	Bias	Var
CE	31.6	0.60	0.47	57.3	0.28	0.35	28.2	0.61	0.51	6.3	0.94	0.57
RIDE [ 36 ]	40.5	0.50	0.42	60.5	0.28	0.30	38.7	0.50	0.44	20.1	0.74	0.52
MDCS [ 47 ]	46.1	-	0.36	-	-	0.24	-	-	0.38	-	-	0.46
MDCS †	48.2	0.43	0.37	68.9	0.22	0.24	55.1	0.34	0.34	30.4	0.63	0.48
$\lambda$ =2	49.9	0.42	0.35	74.6	0.16	0.19	57.0	0.33	0.31	29.3	0.65	0.48

Table 7: Comparisons of the mean accuracy, bias, and variance of baselines and our proposed method on CIFAR-100-LT (IF=100), following the previous setup [ 36 , 47 ] .

\dagger

represents our reproduced results using the official code on the specified datasets for a fair comparison.

\lambda=0

refers to our proposed method without SC and DED.

Figure 6: Heatmaps of the entropy profiles as a function of the interpolation factor $\gamma$ $\in$ [0,1] between two full-resolution views $\mathrm{v}_{1}$ and $\mathrm{v}_{2}$ on the CIFAR-100-LT test set with a single expert. Head samples ( Top ) are more susceptible to $\gamma$ , whereas tail samples ( Bottom ) have higher entropy overall. SC reduces the entropy of interpolated samples, particularly for tail classes.

Figure 7: We use three different diversity metrics, i.e. , Q-statistics, Correlation, and Diversity accuracy, to measure the relationship between model diversity and ensemble accuracy on CIFAR-100-LT with an explicit diversity regularizer . Each point represents an independently trained multi-expert model.

\kappa

controls the explicit diversity regularizer intensities. A larger

\kappa

value encourages each expert to align with the ensemble prediction

\overline{p}

, while a negative

\kappa

forces predictions to diverge from this ensemble consensus. Ensemble accuracy shows a positive correlation with both Q-statistics and the correlation coefficient

\rho

. (The Pearson correlation coefficients are

0.76

0.76

, and

-0.31

for Q-statistics, Correlation, and Diversity accuracy.)

Panel of Figure 8 — Figure 8: (a) Ensemble accuracy and mean individual expert accuracy across different $\kappa$ . A large positive $\kappa$ yields better individual accuracy, thereby leading to an improved ensemble performance. (b) Inter-expert logit variance on the CIFAR-100-LT test set. We compute the variance of logits across experts. Lower values indicate stronger cross-expert consistency.

Panel of Figure 9 — Figure 9: Ablations of (a) the teacher logit aggregation weight $\varphi$ and (b) the Beta-distribution parameter $\alpha$ on CIFAR-100-LT (IF=100).

Table 15

Figure 10: Comparisons of our method and NCL++ on CIFAR-100-LT (IF=100) with different numbers of experts. While NCL++ achieves superior accuracy in the tail classes, this is accompanied by a high cost in many and medium-shot performance.

Panel of Figure 11 — Figure 11: Grad-CAM visualizations for three experts on iNaturalist 2018.

Method	Params	Many	Med	Few	All
BalPoE [ 1 ]	66.1M	68.2	57.2	44.9	59.8
BalPoE ‡ [ 1 ]	66.1M	-	-	-	61.6
NCL++ ‡ [ 33 ]	50.1M	-	-	-	60.9
RIDE [ 36 ]	38.3M	67.6	53.5	35.9	56.4
SADE [ 45 ]	38.3M	66.5	57.0	43.5	58.8
ML [ 26 ]	38.3M	70.2	56.7	39.1	59.5

Table 10: Top-1 accuracy on ImageNet-LT using ResNeXt-50 as backbone. We report the results of 200 epochs and the trainable parameters of all compared multi-expert models.

{\ddagger}

denotes models trained with RandAugment [ 6 ] .

Method	Prior	Forward-LT					Uni.	Backward-LT
Method	Prior	50	25	10	5	2	1	2	5	10	25	50
LADE	$\times$	56.0	55.5	52.8	51.0	48.0	45.6	43.2	40.0	38.3	35.5	34.0
RIDE	$\times$	63.0	59.9	57.0	53.6	49.4	48.0	42.5	38.1	35.4	31.6	29.2
SADE	$\times$	58.4	57.0	54.4	53.1	50.1	49.4	45.2	42.6	39.7	36.7	35.0
BalPoE	$\times$	65.1	63.1	60.8	58.4	54.8	52.0	48.6	44.6	41.8	38.0	36.1
BalPoE †	$\times$	71.4	69.4	66.1	63.6	59.7	56.4	53.0	49.0	46.2	41.9	39.5

Table 11: Top-1 accuracy on multiple test distributions for ResNet-32 trained on CIFAR-100-LT (IF=100). Prior denotes whether the test set prior is used.

\dagger

refers to our reproduced results, and other results are from [ 1 ] .

Method	Prior	Forward-LT					Uni.	Backward-LT
Method	Prior	50	25	10	5	2	1	2	5	10	25	50
LADE	$\times$	63.4	62.1	59.9	57.4	54.6	52.3	49.9	46.8	44.9	42.7	40.7
RIDE	$\times$	67.6	66.3	64.0	61.7	58.9	56.3	54.0	51.0	48.7	46.2	44.0
SADE	$\times$	65.5	64.4	63.6	62.0	60.0	58.8	56.8	54.7	53.1	51.1	49.8
BalPoE	$\times$	67.6	66.3	65.2	63.3	61.5	59.8	58.1	55.7	54.3	52.2	50.8
Ours	$\times$	72.1	70.9	69.3	67.3	65.0	62.9	60.7	57.7	55.8	53.3	51.3

Table 12: Top-1 accuracy on multiple test distributions for ResNeXt-50 trained on ImageNet-LT. Prior denotes whether the test set prior is used, and

*

refers to implicit test prior estimation.

Long-Tailed Classification

Long-tailed classification studies how to recognize rare classes when training data are dominated by a few frequent classes.

See all 55 papers

Aug 11, 2026

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

Long-tailed classification poses a reliability challenge because models trained on imbalanced data are unevenly reliable across frequent and underrepresented classes. While existing methods address imbalance through re-balancing, adjustment, representation learning, or multi-expert modeling, they rarely estimate which expert should be trusted for each class. This paper proposes CLEAR (Class-wise reLiability-aware Expert Aggregation for long-tailed Recognition), a modular ensemble framework for long-tailed classification. CLEAR generates diverse experts through threshold-based structured sampling while preserving the full label space, then estimates a class-wise trust score for each expert using a smoothed class-wise precision formulation. During inference, expert predictions are combined through class-wise generalized product-of-experts aggregation, allowing different experts to be emphasized for different classes. Experiments on CIFAR-100-LT, ImageNet-LT, and Places-LT across multiple backbones show that CLEAR achieves competitive overall accuracy and particularly strong few-shot performance. These results support class-wise expert reliability as a useful design principle for long-tailed ensemble learning.

Gawon Lim

School of Information Sciences University of Illinois Urbana-Champaign

Long-Tailed Classification Ensemble Learning Product of Experts Classification Mixture-of-Experts Models

Ask PDF

May 2, 2026

CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning

Long-tailed distributions are common in real-world recognition tasks, where a few head classes have many samples while most tail classes have very few. Recently, fine-tuning foundation models for long-tailed learning has gained attention due to their excellent performance. However, most existing methods focus solely on mitigating long-tailed distribution bias while overlooking concept confusion caused by the long-tailed distribution. In this paper, we study this problem and attribute it to the mutual exclusivity of single-label supervision under long-tailed distributions, which suppresses feature sharing among related classes and amplifies the dominance of head classes, leading to disrupted inter-class discriminability. To address this, we propose CUE, Concept-aware mUlti-label Expansion, which introduces multi-label concept signals to preserve disrupted inter-class relationships. Specifically, CUE constructs concept sets by (i) extracting instance-level visual cues from zero-shot CLIP and (ii) generating class-level semantic cues with LLM; the two cues are incorporated via separately weighted Binary Logit-Adjustment (BLA) auxiliary losses and jointly optimized with the baseline Logit-Adjustment (LA) loss. Experiments on several long-tailed benchmarks, CUE achieves balanced and strong performance, surpassing recent state-of-the-art methods. Code is available at: https://github.com/zhangruichi/CUE.

Ruichi Zhang, Chikai Shang, Jiacheng Yang +4

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · College of Computer Science and Software Engineering, Shenzhen University, China · Institute of High Performance Computing, A*STAR, Singapore

Long-Tailed Classification Logit Adjustment Vision Foundation Model Adaptation

Ask PDF

Sep 29, 2026

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning

Balancing performance trade-offs on long-tailed data distributions remains a long-standing challenge in visual recognition. Existing methods mainly improve tail classes through re-balancing, representation learning, or data augmentation, but the underlying cause of tail class degradation is still insufficiently explored. In this paper, we find that standard long-tailed training induces background-biased representation and optimization: tail classes suffer larger background distribution shifts and become increasingly driven by background gradients. This reveals that tail degradation is not merely caused by insufficient samples, but also by the learning of irrelevant background features. To tackle this issue, we propose Object-Focused Background Debiasing (OFBD), a framework that mitigates background bias from both distribution and optimization perspectives. Specifically, Foreground-guided CutMix preserves target-related foregrounds while diversifying complementary backgrounds, and Background-guided Feature Rectification suppresses background-biased features without learnable parameters or additional training. Extensive experiments show that our method improves overall accuracy, achieves significant tail-class gains, and can serve as a plug-in for mainstream long-tailed methods without external data or pretrained recognition models. The code is available at: https://ofbd-neurips2026-longtail-learning.github.io/

Shenghan Chen, Yiming Liu, Zhipeng Deng +6

Westlake University, Hangzhou, China · Shandong University, Jinan, China · Hokkaido University, Sapporo, Japan +1

Long-Tailed Classification Long-Tail Learning

Ask PDF

Abstract

Long-Tailed Classification

Explore similar work

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning