Heaviside Low-Rank Support Matrix Machine

Xianchao Xiu; Shenghao Sun; Xinrong Li; Jiyuan Tao

cs.LGFeb 28, 2026

Heaviside Low-Rank Support Matrix Machine

Authors: Xianchao Xiu, Shenghao Sun, Xinrong Li, Jiyuan Tao

Organizations: School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200400, China. · National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University, Shenyang, 110819, China. · Department of Mathematics and Statistics, Loyola University Maryland, Baltimore, MD 21210, USA.

Abstract

Support matrix machine (SMM) is an emerging classification framework that directly handles matrix-structured observations, thereby avoiding the spatial correlations destroyed by vectorization. However, most existing SMM variants rely on convex or nonconvex surrogate loss functions, which may lead to high sensitivity to noise. To address this issue, we propose a novel Heaviside low-rank SMM model called HL-SMM, which leverages the Heaviside loss instead of the common hinge or ramp losses for robustness. Moreover, the low-rank constraint is adopted to accurately characterize the inherent global structure. In theory, we analyze the Karush-Kuhn-Tucker (KKT) points and rigorously prove the sufficient and necessary conditions. In algorithms, we develop an effective proximal alternating minimization (PAM) scheme, where all subproblems have closed-form solutions. Extensive experiments on benchmark datasets validate that the proposed HL-SMM achieves superior classification accuracy and robustness compared to state-of-the-art methods.

Figures & tables

Panel of Figure 1 — Figure 1: Coefficient matrices obtained by different loss functions.

	$\displaystyle\underset{\mathbf{W}}{\min}$	$\displaystyle f(\mathbf{W},{\mathbf{z}}^{k},b^{k})+\frac{\tau_{1}}{2}\\|\mathbf{W}-\mathbf{W}^{k}\\|_{\textrm{F}}^{2}$
	$\displaystyle\text{s.t.}$	$\displaystyle\operatorname{rank}(\mathbf{W})\leq r.$

Algorithm 1 Optimization algorithm for problem ( 11 )

Panel of Figure 2 — Figure 2: Robustness comparison under 20% Gaussian noise.

Panel of Figure 3 — Figure 3: Parameter sensitivity analysis of $r$ and $\beta$ .

Panel of Figure 4 — Figure 4: Convergence curves of the proposed algorithm.

Classification

Classification maps observations to discrete labels, including familiar, unknown, hierarchical, or multiple categories.

See all 401 papers

Robust and sparse support vector machine via hybrid truncated loss for supervised classification
Jun 4, 2026Yuliang Yang, Chen Chen, Yuxiang Liu +1Multi-View Learning Robust Loss Functions
School of Science, Beijing Forestry University, No.35 Qinghua East Road, 100083 Haidian, Beijing, China · Translational Cancer Research Center, Peking University First Hospital, No. 8 Xishiku Street, 100034 Xicheng, Beijing, China
Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model
Sep 30, 2026Yugo NakayamaSupport Vector Machines Classification
Independent researcher
Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation
May 6, 2026Chengyu Cui, Gongjun XuNonconvex Optimization
Department of Statistics, University of Michigan

Jun 4, 2026J/K move · Enter open · S save

Robust and sparse support vector machine via hybrid truncated loss for supervised classification

Yuliang Yang, Chen Chen, Yuxiang Liu, Huiru Wang

School of Science, Beijing Forestry University, No.35 Qinghua East Road, 100083 Haidian, Beijing, China · Translational Cancer Research Center, Peking University First Hospital, No. 8 Xishiku Street, 100034 Xicheng, Beijing, China

Multi-View Learning Robust Loss Functions Support Vector Machines

Open paper AskPDF arXiv

The support vector machine (SVM) is a widely used classifier, but choosing an appropriate loss function remains difficult. Convex losses such as the hinge loss and least-squares loss are sensitive to outliers, while bounded non-convex losses often lead to high computational cost. To address this, we propose a hybrid truncated loss function (

L_{\mathrm{ht}}

) that is both sparse and bounded, and build the

L_{\mathrm{ht}}

-SVM model for single-view classification. We introduce the P-stationary point and use it to establish the first-order necessary and sufficient optimality conditions. Based on these conditions, we design an alternating direction method of multipliers with a working-set strategy that reduces computational cost and achieves global convergence. We further extend

L_{\mathrm{ht}}

-SVM to multi-view learning by adding structural information and view weights, resulting in Mv

L_{\mathrm{ht}}

-SVM, which follows both the consensus and complementarity principles. Experiments on synthetic, real-world, and image datasets show that

L_{\mathrm{ht}}

-SVM achieves higher accuracy with fewer support vectors and better noise robustness than five single-view methods, while Mv

L_{\mathrm{ht}}

-SVM outperforms six multi-view methods in accuracy, precision, recall, and F1-score.

Abstract

Classification

Explore similar work

Robust and sparse support vector machine via hybrid truncated loss for supervised classification

Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model

Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation