stat.MLOct 31, 2026

Representative Learning for Distributed Support Vector Machines

Authors: Joseph A. Casey, Boxiang Wang, Keren Li

Organizations: Department of Mathematics, University of Alabama at Birmingham · Department of Statistics and Actuarial Science, University of Iowa

Abstract

Distributed support vector machine (SVM) learning is challenging under heterogeneous local data and limited communication. We develop a Representative Learning framework based on Lifted Empirical Risk Minimization, replacing each covariate by a local Gaussian distribution. For linear SVMs, the resulting expected hinge loss has an exact closed form, is smooth for nondegenerate lifting covariances, and converges to the ordinary hinge loss as the covariance vanishes. The lifted formulation enables clouded representatives that encode both location and covariance. We study a model-independent Mean Representative (MR) and a Score-Matching Representative (SMR) that matches the class-wise lifted gradient at its construction parameter. We show that SMR covariance scaling is stable with respect to within-group heterogeneity and that the gradient discrepancy between full data and frozen representatives is controlled by the maximum within-group diameter. Under strong convexity and smoothness, this yields an inexact-gradient convergence guarantee. Experiments under heterogeneous class allocation show that lifted representatives remain competitive with centralized solutions, while SMR more closely recovers the centralized lifted-SVM optimizer. We further extend the framework to nonlinear SVMs through a shared kernel dictionary, with experiments on Ionosphere and Australian Credit Approval showing analogous behavior. These results provide a distributional representative framework for communication-efficient distributed SVM learning under heterogeneity.

Figures & tables

Explore similar work

CardsList
  1. Bringing Generative Learning to Representation Learning: Self-Supervised Transfer Learning as Distribution Matching

    Feb 20, 2025Yuling Jiao, Wensen Ma, Defeng Sun +2Visual Representation LearningDistribution Matching

  2. Semantic-based Distributed Learning for Diverse and Discriminative Representations

    Apr 20, 2026Zhuojun Tian, Chaouki Ben Issaid, Mehdi BennisRepresentation LearningDistributed Optimization

  3. Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning

    May 31, 2026Tehila Dahan, Bassel Hamoud, Roie Reshef +2Communication-Efficient Distributed TrainingDistributed Optimization