Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning

Ye Wang; Zixuan Wu; Lifeng Shen; Jiang Xie; Xiaoling Wang; Hong Yu; Guoyin Wang

cs.LGMar 16, 2026

Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning

Authors: Ye Wang, Zixuan Wu, Lifeng Shen, Jiang Xie, Xiaoling Wang, Hong Yu, Guoyin Wang

Organizations: Key Laboratory of Cyberspace Big Data Intelligent Security, Ministry of Education; School of Artificial Intelligence, Chongqing University of Post and Telecommunications, Chongqing 400065, China. · School of Computer Science and Technology, East China Normal University, Shanghai, 200062, China. · Chongqing Key Laboratory of Brain-Inspired Cognitive Computing and Educational Rehabilitation for Children with Special Needs, Chongqing Normal University, Chongqing, 401331, China.

Abstract

Imbalanced data distribution remains a critical challenge in sequential learning, leading models to easily recognize frequent categories while failing to detect minority classes adequately. The Mixture-of-Experts model offers a scalable solution, yet its application is often hindered by parameter inefficiency, poor expert specialization, and difficulty in resolving prediction conflicts. To Master the Minority classes effectively, we propose the Uncertainty-based Multi-Expert fusion network (UME) framework. UME is designed with three core innovations: First, we employ Ensemble LoRA for parameter-efficient modeling, significantly reducing the trainable parameter count. Second, we introduce Sequential Specialization guided by Dempster-Shafer Theory (DST), which ensures effective specialization on the challenging-tailed classes. Finally, an Uncertainty-Guided Fusion mechanism uses DST's certainty measures to dynamically weigh expert opinions, resolving conflicts by prioritizing the most confident expert for reliable final predictions. Extensive experiments across four public hierarchical text classification datasets demonstrate that UME achieves state-of-the-art performance. We achieve a performance gain of up to 17.97% over the best baseline on individual categories, while reducing trainable parameters by up to 10.32%. The findings highlight that uncertainty-guided expert coordination is a principled strategy for addressing challenging-tailed sequence learning. Our code is available at https://github.com/CQUPTWZX/Multi-experts.

Figures & tables

Panel of Figure 1 — Fig. 1 : As the classification hierarchy becomes deeper, the number of samples per label decreases significantly, leading to a pronounced challenging-tailed problem and a sharp drop in the classification performance of minority classes.

Fig. 2 : The structure of our proposed model UME. The uncertainty of every single expert is attained for multi-expert learning. Multi-expert Joint Uncertainty is dynamically ensembled by the degree of uncertainty among experts.

Panel of Figure 3 — Fig. 3 : Class distribution analysis across four benchmark datasets. The histograms use a logarithmic scale to visualize the severe long-tailed nature and extreme class imbalance ratios (IR) present in WOS, RCV1-V2, AAPD, and BGC.

Dataset	$\left\|Y\right\|$	$Avg\left(y_{i}\right)$	Depth	# Train	# Dev	# Test
WOS	141	2.0	2	30,070	7,518	9,397
RCV1-V2	103	3.24	4	18,520	4,629	781,265
AAPD	54	2.41	2	43,872	10,968	1,000
BGC	146	3.01	4	58,800	14,700	18,394

TABLE I : The statistical details of datasets.

\left|Y\right|

represents the number of unique labels in each dataset.

Avg\left(y_{i}\right)

the average value of the labels for each entry across the dataset. Depth indicates the depth of the hierarchical structure in the dataset. The last three columns of the table include the number of instances in the training, validation, and testing sets, respectively.

Model	WOS		RCV1-V2		AAPD		BGC		NYT
	Micro-F1	Macro-F1	Micro-F1	Macro-F1	Micro-F1	Macro-F1	Micro-F1	Macro-F1	Micro-F1	Macro-F1
Hierarchy-Aware Models
HiAGM (ACL2020)	85.11 $\pm$ 1.4	80.02 $\pm$ 0.7	83.17 $\pm$ 1.8	62.35 $\pm$ 1.9	75.54 $\pm$ 2.1	58.86 $\pm$ 1.2	76.61 $\pm$ 0.9	57.27 $\pm$ 0.7	—	—
HTCInfoMax (NAACL2021)	84.86 $\pm$ 0.6	79.12 $\pm$ 0.2	83.26 $\pm$ 0.8	60.66 $\pm$ 1.5	77.42 $\pm$ 0.7	57.72 $\pm$ 0.7	74.26 $\pm$ 0.4	56.81 $\pm$ 1.0	—	—
HiMatch (ACL2021)	85.04 $\pm$ 0.9	80.37 $\pm$ 1.3	84.42 $\pm$ 0.9	63.99 $\pm$ 1.1	76.70 $\pm$ 0.3	59.86 $\pm$ 0.5	76.63 $\pm$ 0.8	57.87 $\pm$ 1.0	—	—
Pretrained Language Models

TABLE II : The comparison of different models on WOS, RCV1-V2, AAPD, BGC and NYT.

Parameters	Description
r	LoRA Rank. Default: 64
lr	Learning rate
batch	Batch size
epoch	Default: 10
early-stop	Epoch before early stop
update	Gradient accumulate steps

TABLE III : Parameter description and details

Category	Count	Ours	HiTIN	HJCL	HGCLR	HPT
g157	1991	55.6 5%	55.25%	51.28%	49.42%	50.77%
c16	1871	58.04 %	54.62%	57.96%	53.82%	55.41%
gwelf	1818	48.01%	51.65 %	49.87%	45.21%	45.21%
e311	1658	79.79 %	61.46%	58.74%	61.82%	59.18%
c331	1179	77.26%	67.77%	79.55 %	54.20%	74.36%
e143	1172	77.39 %	65.61%	66.20%	65.87%	65.31%

TABLE IV : The performance of different models on challenging-tailed categories in the RCV1-V2 dataset. We report results regarding Win-Lose numbers.

Datasets	Top N Tailed Labels	HGCLR	HPT	HiTIN	HJCL	HILL	Ours
WOS	N=8	28.47	27.78	37.83	41.67	42.08	46.60
	N=16	48.67	49.04	53.16	57.67	51.78	62.66
	N=32	61.49	65.36	61.43	71.62	68.44	74.70
RCV1-V2	N=8	18.92	16.88	20.24	22.40	18.38	26.93
	N=16	41.83	41.37	47.21	43.22	46.72	50.45
	N=32	56.67	54.77	57.98	57.20	61.46	63.69

TABLE V : The Macro-F1 performance on tailed labels for the Top N least frequent labels across five datasets.

Panel of Figure 9 — Fig. 4 : The confusion matrix highlights the contrast between HGCLR and multi-experts in terms of prediction accuracy.

Fig. 5 : Two examples illustrate the effectiveness of uncertainty-based multi-expert fusion. Compared with baselines, our classification results are more reliable.

Fig. 6 : The classification error rate distribution across different Conflict Metric ( $C^{m}$ ) intervals. $N$ denotes the number of samples in each bin.

Panel of Figure 12 — Fig. 7 : Bar charts show the different performances with varying numbers of experts among four datasets.

Experts ( $M$ )	Tail Micro-F1	Avg. Conflict ( $C^{last}$ )	Last Expert Util.
3	76.21	0.1724	20.26
4	75.32	0.1597	1.80
5	74.05	0.1175	1.28

TABLE VI : Analysis of expert capacity and utilization on the WOS dataset with expert numbers (

M

). The metrics include Tail Micro-F1 (computed on the bottom 20% least frequent classes), the Average Conflict of the final expert (

C^{last}

), and the proportion of samples routed to the final expert (defined as weight

>0.5

Fig. 8 : The participation percentage among experts in the head, medium, and tail categories on RCV1-V2.

Panel of Figure 15 — Fig. 9 : Sensitivity analysis on WOS and RCV1-V2.

Ablation Study	WOS	RCV1-V2	AAPD	BGC
w/o $\mathcal{L}_{evidence}$	87.52	86.61	78.86	79.66
w/o ( $\mathcal{L}_{evidence}\&\mathcal{L}_{ml}$ )	86.98	86.37	78.15	79.27
w/o ( $\mathcal{L}_{evidence}\&\mathcal{L}_{ml}$ $\&{\hat{\mathcal{L^{C}}}}\&{\mathcal{L}^{cl}}$ )	85.91	85.72	77.28	78.51
$\mbox{-}r.p.$ Direct Integration	87.18	85.73	78.53	79.51
$\mbox{-}r.p.$ Gated MoE	86.74	86.12	77.91	79.08
UME	87.71	86.88	79.21	80.46

TABLE VII : Ablation study results for Micro-F1.

Fig. 10 : Compare model parameters, training time and Macro-F1 scores on the WOS dataset.

Model	WOS		RCV1-V2
Model	Micro-F1	Macro-F1	Micro-F1	Macro-F1
GPT-4	59.74	37.80	56.12	36.54
LlaMa-3	61.91	27.33	54.67	28.50
UME (Ours)	87.71	82.29	86.88	69.70

TABLE VIII : Performance comparison between LLMs and our UME framework on WOS and RCV1-V2 datasets. The LLMs are evaluated using the zero-shot prompt templates shown.

Prompt Template

You are an expert hierarchical text classifier. Your task is to classify the given text into the correct category path from the predefined taxonomy. [Taxonomy Definition] The categories are defined in a hierarchy formatted as:

Level1>Level2>Level3

- The symbol “

>

” represents the parent-child relationship. - You must output the full path from the root to the specific leaf category. [Allowed Category Paths] (Insert your full list of category paths here, e.g., Technology

>

>

NLP) [Constraints] 1. Output ONLY the category path. No explanation, no preamble. 2. If the text fits multiple paths, output the most specific and relevant one. 3. Use the exact string format provided in the [Allowed Category Paths]. [Input Text] {{text_content}} [Prediction]

TABLE IX : Prompt template for hierarchical text classification.

Hierarchical Classification

Hierarchical classification uses relationships among labels to predict categories at multiple levels, from broad groups to fine-grained classes.

See all 89 papers

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification
Aug 11, 2026Gawon LimLong-Tailed Classification Ensemble Learning
School of Information Sciences University of Illinois Urbana-Champaign
VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition
Sep 4, 2026Jiangang Zhu, Zheng Wang, Bin Zhu +2Long-Tailed Classification Ensemble Learning
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Zhejiang University of Technology, Hangzhou, China +2
DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
May 28, 2026Hyuck Lee, Taemin Park, Heeyoung KimClass-Imbalanced Learning Long-Tail Learning
AI Research, Krafton, Teheran-ro 231, Centerfield EAST, Gangnam-gu, Seoul, 06142, Republic of Korea · Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea

Aug 11, 2026J/K move · Enter open · S save

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

Gawon Lim

School of Information Sciences University of Illinois Urbana-Champaign

Long-Tailed Classification Ensemble Learning Product of Experts Classification Mixture-of-Experts Models

Open paper AskPDF arXiv

Long-tailed classification poses a reliability challenge because models trained on imbalanced data are unevenly reliable across frequent and underrepresented classes. While existing methods address imbalance through re-balancing, adjustment, representation learning, or multi-expert modeling, they rarely estimate which expert should be trusted for each class. This paper proposes CLEAR (Class-wise reLiability-aware Expert Aggregation for long-tailed Recognition), a modular ensemble framework for long-tailed classification. CLEAR generates diverse experts through threshold-based structured sampling while preserving the full label space, then estimates a class-wise trust score for each expert using a smoothed class-wise precision formulation. During inference, expert predictions are combined through class-wise generalized product-of-experts aggregation, allowing different experts to be emphasized for different classes. Experiments on CIFAR-100-LT, ImageNet-LT, and Places-LT across multiple backbones show that CLEAR achieves competitive overall accuracy and particularly strong few-shot performance. These results support class-wise expert reliability as a useful design principle for long-tailed ensemble learning.

Abstract

Hierarchical Classification

Explore similar work

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition

DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning