cs.LGAug 2, 2026

Factorized AdaBoost.MH Achieves the Same Convergence Rate as AdaBoost.MH

Authors: Xin ZouJingyuan Xu

Abstract

{AdaBoost.MH} reduces multi-class classification to a collection of binary subproblems and enjoys the classical boosting-type convergence guarantee under a weak learning condition. A more structured variant, Factorized {AdaBoost.MH}, uses base classifiers of the form h(x)=αvφ(x)\mathbf{h}(x)=α\mathbf{v} \bm{\varphi}(x), where a single binary classifier φ\bm{\varphi} is shared across all classes and the label dependence is carried by a vote vector v{±1}K\mathbf{v} \in\{\pm1\}^K. This factorization is algorithmically attractive and achieves better performance in practice, but its convergence depends on whether one can always choose a vote vector with sufficiently large induced binary weight mass. Previous work resolved this question with a lower bound max{1/n,1/2K}\max\{1/n,1/\sqrt{2K}\}, which still leaves a dimension-dependent slowdown relative to the original {AdaBoost.MH} analysis. In this paper, we sharpen this combinatorial step. For the minimax quantity Wn,K\mathfrak{W}_{n,K} governing the factorized edge, we prove Wn,K=Cmin{n+1,K}\mathfrak{W}_{n,K} = C_{\min\{n+1,K\}}, where Cq=1C_q=1 for q=1q=1, Cq=q/(3q4)C_q=q/(3q-4) for even q2q\ge2, and Cq=(q+1)/(3q1)C_q=(q+1)/(3q-1) for odd q2q\ge2. Since Cq1/3C_q\downarrow 1/3, our bounds show that Wn,K=Θ(1)\mathfrak{W}_{n,K}=Θ(1) uniformly over nn and KK. Consequently, Factorized {AdaBoost.MH} achieves the same boosting-type convergence rate as {AdaBoost.MH} up to a universal constant factor, removing the previously suggested additional dependence on nn or KK in the number of boosting rounds.

Explore similar work

CardsList
  1. Multiclass Linear Perceptrons with Multiplicative Margins

    Aug 30, 2026Dmitri Rachkovskij, Evgeny Osipov, Olexander Volkov +2Multiclass ClassificationBias Learning