FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learning

Ziquan Wang; Haobo Wang; Ke Chen; Lei Feng; Gang Chen

cs.LGFeb 28, 2026

FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learning

Authors: Ziquan Wang, Haobo Wang, Ke Chen, Lei Feng, Gang Chen

Organizations: the School of Software Technology, Zhejiang University, Hangzhou 310027, China · the State Key Laboratory of Blockchain and Data Security, Zhejiang University, Zhejiang 310027, China, and also with the Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Binjiang 310000, China · the School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · the College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China

Abstract

Machine Learning often involves various imprecise labels, leading to diverse weakly supervised settings. While recent methods aim for universal handling, they usually suffer from complex manual pre-work, ignore the relationships between associated labels, or are unable to batch process due to computational design flaws, resulting in long running times. To address these limitations, we propose a novel general framework that efficiently infers latent true label distributions across various weak supervisions. Our key idea is to express the label brute-force search process as a probabilistic transition of label variables, compressing diverse weakly supervised DFS tree structures into a shared Bayesian network. From this, we derived a latent probability calculation algorithm based on generalized belief propagation and proposed two joint acceleration strategies: 1) introducing a low-rank assumption to approximate the transition matrix, reducing time complexity; 2) designing an end-to-end state evolution module to learn batch-scale transition matrices, facilitating multi-category batch processing. In addition, the equivalence of our method with the EM algorithm in most scenarios is further demonstrated. Extensive experiments show that our method achieves SOTA results under most weakly supervised settings, and achieves up to hundreds of times faster acceleration in running time compared to other general methods.

Figures & tables

Strategy	Method	No Pre-Work	Multi-label Correlation	Worst-Case Time Complexity
Inductive Properties of Base Loss	Centroid Loss [ 22 ]	✗	✗	$-$ a
	Count Loss [ 24 ]	✓	✗	$\mathcal{O}(K^{2})$
Calculation of Latent Probobilities	UUM [ 21 ]	✓	✗	$\mathcal{O}(C^{K})$
	IMP [ 23 ]	✗	✗	$\mathcal{O}(K\|W\|^{2})$ b
	FastBUS	✓	✓	$\mathcal{O}(K\|W\|)$

TABLE I: Comparison Between Several Works and FastBUS for General Weak Supervision.

Fig. 1: Comparison of run time and accuracy with recent SOTA methods on general weakly supervised learning as the batch size is 4 and the average number of instances in each bag is 20. The runtime advantage becomes more pronounced as the batch size, number of classes, or instances increases.

Fig. 2: DFS tree for Multi-Instance Learning when the number of instances in one bag is 3 for only one class. Note that nodes of the same color belong to the same layer.

Weak Supervision	Meanings of $Z^{k}$
LProp , PosUnl , UnlUnl	number of positive samples for the first k instances in the bag
PairComp	more positive than the first instance or not
PairSim	similar to the first instance or not
…	…
PartialL	the k-th category is considered as a candidate label or not

TABLE II: Meanings of Node Variables in DFS Tree or Bayesian Network for Different Weak Supervision

Fig. 3: A loopy Bayesian network model for general weak supervision. Different colors denote distinct label types. (a) Left panel: Visualization of a probabilistic graph with 3 instances and 3 labels, where all feasible instance configurations are mined through chain-structured dependencies, while loops explicitly characterize multi-label correlations. Right panel: Factor graph representation corresponding to the chain (b) and loop (c) components of the left network, employing generalized belief propagation (GBP) for efficient probabilistic inference.

\bm{\mu}_{\cdot\rightarrow\bm{W}}=g_{w,o}(\bm{\mu}^{(1)}_{K}{})\cdot\bm{T}_{class}

Algorithm 1 GBP Calculation Process.

Fig. 4: The end-to-end state evolution module to learn transition matrices $\bm{U}(\bm{x}^{[K]})$ . The core innovation points are marked with underlines.

Multi-Instance Learning
Dataset	CIFAR10		CIFAR100		STL10
$N$	$800$	$1200$	$1000$	$2500$	$500$	$2000$
$K$	$\mathcal{N}(8,1)$	$\mathcal{N}(5,1)$	$\mathcal{N}(10,2)$	$\mathcal{N}(8,1)$	$\mathcal{N}(12,2)$	$\mathcal{N}(5,1)$
Count Loss	$64.95~$ $\pm 0.61$	$70.31~$ $\pm 0.83$	$29.50~$ $\pm 1.29$	$46.87$ $\pm 0.82$	$49.84~$ $\pm 2.62$	$73.05~$ $\pm 1.37$
UUM	$14.90~$ $\pm 0.62$	$16.23~$ $\pm 0.66$	$1.50~$ $\pm 0.62$	$1.58$ $\pm 0.51$	$16.95~$ $\pm 2.84$	$14.45~$ $\pm 2.26$
GLWS	$68.97~$ $\pm 0.79$	$74.04~$ $\pm 0.67$	$25.77~$ $\pm 1.26$	$44.86$ $\pm 1.92$	$53.95~$ $\pm 0.95$	$76.28~$ $\pm 1.31$

TABLE III: Accuracy on Aggregation Supervision for Three Datasets. All Results Are Averaged over Five runs.

Pairwise Comparison
Dataset	CIFAR-10		CIFAR-100		STL-10
${N}$	$7500$	$10000$	$5000$	$7500$	$2000$	$6000$
Prior	$0.65$	$0.45$	$0.65$	$0.45$	$0.65$	$0.45$
PcABS	$70.19$ $\pm 0.23$	$72.97$ $\pm 0.28$	$55.87$ $\pm 0.39$	$60.13$ $\pm 0.53$	$70.98$ $\pm 0.52$	$72.16$ $\pm 0.43$
PcReLU	$71.64$ $\pm 0.46$	$74.97$ $\pm 0.34$	$56.88$ $\pm 0.31$	$63.24$ $\pm 0.61$	$72.03$ $\pm 0.37$	$74.66$ $\pm 0.36$
PcTeacher	$60.29$ $\pm 0.35$	$60.62$ $\pm 0.56$	$57.97$ $\pm 0.47$	$59.06$ $\pm 0.31$	$70.21$ $\pm 0.23$	$70.36$ $\pm 0.48$

TABLE IV: Accuracy on Hard-Label Pairwise Supervision for Three datasets. All Results Are Averaged over Five Runs.

Similarity Confidence
Dataset	CIFAR-10		CIFAR-100		STL-10
${N}$	$4000$	$10000$	$3000$	$10000$	$3000$	$10000$
Prior	$0.45$	$0.65$	$0.45$	$0.65$	$0.45$	$0.65$
ScABS	$61.26$ $\pm 0.32$	$68.68$ $\pm 0.51$	$60.52$ $\pm 0.21$	$60.02$ $\pm 0.47$	$66.08$ $\pm 0.17$	$71.32$ $\pm 0.29$
SCReLU	$60.21$ $\pm 0.51$	$68.51$ $\pm 0.43$	$60.45$ $\pm 0.17$	$60.04$ $\pm 0.38$	$68.02$ $\pm 0.26$	$70.98$ $\pm 0.19$
ScNNABS	$61.11$ $\pm 0.46$	$68.50$ $\pm 0.72$	$60.46$ $\pm 0.16$	$60.02$ $\pm 0.52$	$69.41$ $\pm 0.32$	$71.06$ $\pm 0.58$

TABLE V: Accuracy on Soft-Label Pairwise Supervision for Three datasets. All Results Are Averaged over Five Runs.

Partial Label
Dataset	CIFAR-10		CIFAR-100		STL-10
${N}$	20000		20000		20000
Ratio	$0.30$	$0.70$	$0.03$	$0.10$	$0.05$	$0.50$
CC	$91.57$ $\pm 0.15$	$83.58$ $\pm 0.36$	$56.47$ $\pm 0.24$	$53.74$ $\pm 0.18$	$85.72$ $\pm 0.45$	$82.05$ $\pm 0.72$
PRODEN	$91.86$ $\pm 0.21$	$85.97$ $\pm 0.35$	$57.19$ $\pm 0.16$	$53.73$ $\pm 0.42$	$85.88$ $\pm 0.39$	$82.49$ $\pm 0.73$
PiCO	$92.49$ $\pm 0.48$	$87.37$ $\pm 0.53$	$57.28$ $\pm 0.39$	$55.07$ $\pm 0.44$	$86.07$ $\pm 0.64$	$82.85$ $\pm 0.47$

TABLE VI: Accuracy on Imperfect Supervision for Three datasets. All Results Are Averaged over Five Runs.

Positive Unlabeled
Dataset	CIFAR-10		CIFAR-100		STL-10
${N_{pos}}$	$250$	$800$	$1500$	$3000$	$250$	$800$
CVIR	$87.76$ $\pm 1.15$	$91.19$ $\pm 0.94$	$80.63$ $\pm 0.57$	$\mathbf{84.25}$ $\pm 0.34$	$71.76$ $\pm 1.48$	$76.90$ $\pm 1.17$
DIST PU	$81.67$ $\pm 2.87$	$83.94$ $\pm 1.69$	$74.68$ $\pm 1.21$	$77.19$ $\pm 0.85$	$72.87$ $\pm 1.46$	$74.64$ $\pm 0.74$
NN PU	$\mathbf{88.12}$ $\pm 0.94$	$89.86$ $\pm 0.68$	$76.22$ $\pm 0.89$	$77.96$ $\pm 0.51$	$71.23$ $\pm 0.72$	$79.46$ $\pm 0.37$
U PU	$73.75$ $\pm 1.82$	$86.24$ $\pm 0.83$	$67.64$ $\pm 0.76$	$72.14$ $\pm 0.69$	$71.52$ $\pm 0.82$	$75.35$ $\pm 0.55$

TABLE VII: Accuracy on Incomplete Supervision for Three datasets. All Results Are Averaged over Five Runs.

Fig. 5: Runtime under varying instances per bag, batch sizes, and class counts. Results show our method maintains stable runtime on CIFAR-10/100 across these parameters, while others slow down. This efficiency arises from our unified Bayesian network, accelerated inference, and batch-level state evolution module, supporting a simultaneous multi-class batch process. In specific settings, we achieve up to 400x speedup against Count Loss and GLWS.

Settings	MultiIns	LProp	PairComp	PairSim	SimConf	ConfDiff
MLP	$72.65$	$84.71$	$70.53$	$69.38$	$68.33$	$60.25$
Ours w/o att.	$72.29$	$85.16$	$71.56$	$69.03$	$71.56$	$64.84$
Ours w/ ReLU	$15.98$	$84.93$	$60.18$	$70.06$	$60.08$	$60.27$
Ours w/ GELU	$73.37$	$84.81$	$60.83$	$70.34$	$60.75$	$60.74$
Ours	$74.58$	$86.14$	$73.81$	$70.57$	$72.31$	$70.65$

TABLE VIII: Ablation Experiments using Different Modules to Learn Matrix

\bm{U}

. Report Accuracy after Running All Results over Five Times.

Gaussian Belief Propagation

See all 10 papers

Aug 7, 2026

Recent advances in weakly supervised learning: New supervision paradigms, assumption relaxations, and practical solutions

Deep learning has achieved great success in recent years thanks to the availability of high-quality, well-annotated training data. However, this requirement is often not met in real-world applications. Weakly supervised learning aims to train an accurate model with incomplete, inexact, or inaccurate supervision. In this chapter, we will discuss recent advances in this field, including new supervision paradigms, relaxed assumptions, and practical solutions. First, we introduce a new weakly supervised binary classification problem called confidence-difference classification and propose consistent approaches to solve it. Next, we investigate complementary-label learning, a weakly supervised multi-class classification problem. Our proposed approaches are based on more relaxed assumptions about the data generation process than existing consistent approaches. Lastly, we present an evaluation framework for partial-label learning, another popular multi-class weakly supervised learning problem, in order to promote fair and realistic evaluation of algorithms in this field.

Wei Wang, Gang Niu, Masashi Sugiyama

RIKEN, Tokyo, Japan · The University of Tokyo, Chiba, Japan

Weakly Supervised Learning Learning with Noisy Labels

Ask PDF

Nov 28, 2025

A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees

Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire. However, many existing methods are tailored to specific supervision patterns -- such as positive-unlabeled (PU), unlabeled-unlabeled (UU), complementary-label (CLL), partial-label (PLL), or similarity-unlabeled annotations -- and rely on post-hoc corrections to mitigate instability induced by indirect supervision. We propose a principled, unified framework that bypasses such post-hoc adjustments by directly formulating a stable surrogate risk grounded in the structure of weakly supervised data. The formulation naturally subsumes diverse settings -- including PU, UU, CLL, PLL, multi-class unlabeled, and tuple-based learning -- under a single optimization objective. We further establish a non-asymptotic generalization bound via Rademacher complexity that clarifies how supervision structure, model capacity, and sample size jointly govern performance. Beyond this, we analyze the effect of class-prior misspecification on the bound, deriving explicit terms that quantify its impact, and we study identifiability, giving sufficient conditions -- most notably via supervision stratification across groups -- under which the target risk is recoverable. Extensive experiments show consistent gains across class priors, dataset scales, and class counts -- without heuristic stabilization -- while exhibiting robustness to overfitting.

Miao Zhang, Junpeng Li, Changchun Hua +1

Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China

Weakly Supervised Learning Statistical Learning Theory Positive-Unlabeled Learning Generalization Bounds

Ask PDF

May 25, 2026

Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark

Weakly supervised anomaly detection (WSAD) has developed in three primary directions: incomplete, inexact, and inaccurate supervision. However, these directions remain isolated, lacking a unified framework to assess whether they address unique challenges or share fundamental mechanisms. This paper introduces WSADBench, the first benchmark that unifies evaluation across distinct weakly supervised scenarios, benchmarking diverse approaches from specialized WSAD methods to advanced tabular foundation models. WSADBench establishes standardized protocols to evaluate 36 algorithms across 4 modalities by systematically varying label quantity, granularity, and quality, revealing the performance boundaries of various methods. Based on over 700K experiments, WSADBench reveals four critical insights: (i) Strong intrinsic correlations exist between these weak supervision scenarios, challenging the isolation of current research directions. (ii) Specialized WSAD algorithms excel only in extreme label-scarcity regimes but are quickly dominated by tabular foundation models and general classification methods as supervision increases or in OOD scenarios. (iii) Unlabeled data shows inconsistent utility across settings, with marginal gains compared to label refinement. (iv) Models exhibit asymmetric sensitivity to different types of label noise. We release WSADBench as an open-source benchmark with code and datasets to facilitate future WSAD research: https://github.com/SUFE-AILAB/WSADBench.

Xu Yao, Siyuan Zhou, Zhenbo Wu +6

Shanghai University of Finance and Economics · Ant Group · Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics

Learning with Noisy Labels Anomaly Detection

Ask PDF

Abstract

Gaussian Belief Propagation

Explore similar work

Recent advances in weakly supervised learning: New supervision paradigms, assumption relaxations, and practical solutions

A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees

Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark