cs.LGAug 12, 2026

Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

Authors: Kai Qi, Xinji Huang, Hongchun Wang

Abstract

In real-world scenarios, the training data usually contains redundant features, label noise and feature noise, which provide severe challenges for the efficiency of machine learning methods. Since standard support vector machine (SVM) adopts l2l_2-norm penalty and hinge loss function, it lacks the ability of selecting significant features and is sensitive to noise. To address these issues, this paper proposes a novel asymmetric, robust, bounded, sparse and smooth (aR) loss function for l1l_1-norm penalized geometric twin SVM (aRSGTSVM) to handle classification and regression tasks. The l1l_1-norm penalty can achieve the feature selection. The proposed aR loss function can not only effectively mitigate the impact of label noise, but also significantly enhance the stability to resampling noise, i.e., the zero-mean feature noise around the boundary hyperplanes. Furthermore, a statistical analysis of the robustness of aRSGTSVM was also conducted using the influence function. Since aRSGTSVM involves nonconvex and nonsmooth optimization, we develop a fast and stable proximal gradient descent based solving algorithm. Compared with related state-of-the-art methods, experimental results demonstrate the superiority of the proposed aRSGTSVM on both synthetic and UCI datasets. Furthermore, we apply aRSGTSVM to index tracking tasks, where results for tracking the different indices in the China stock market show that it can achieve satisfactory performance.

Explore similar work

CardsList
Jun 4, 2026cs.LG

Robust and sparse support vector machine via hybrid truncated loss for supervised classification

The support vector machine (SVM) is a widely used classifier, but choosing an appropriate loss function remains difficult. Convex losses such as the hinge loss and least-squares loss are sensitive to outliers, while bounded non-convex losses often lead to high computational cost. To address this, we propose a hybrid truncated loss function (LhtL_{\mathrm{ht}}) that is both sparse and bounded, and build the LhtL_{\mathrm{ht}}-SVM model for single-view classification. We introduce the P-stationary point and use it to establish the first-order necessary and sufficient optimality conditions. Based on these conditions, we design an alternating direction method of multipliers with a working-set strategy that reduces computational cost and achieves global convergence. We further extend LhtL_{\mathrm{ht}}-SVM to multi-view learning by adding structural information and view weights, resulting in MvLhtL_{\mathrm{ht}}-SVM, which follows both the consensus and complementarity principles. Experiments on synthetic, real-world, and image datasets show that LhtL_{\mathrm{ht}}-SVM achieves higher accuracy with fewer support vectors and better noise robustness than five single-view methods, while MvLhtL_{\mathrm{ht}}-SVM outperforms six multi-view methods in accuracy, precision, recall, and F1-score.
Yuliang Yang, Chen Chen, Yuxiang Liu +1
Support Vector MachinesMulti-ViewLoss FunctionSupervised LearningConvex Optimization
AskPDF
Jun 20, 2026cs.LG

Alternate loss functions and regression models that achieve robustness to outliers by modulating the learning rate

Most real-world datasets used for training supervised learning models are contaminated with noisy data and outliers leading to large prediction errors. This paper proposes a new approach for achieving robustness where the learning rate is modulated by a factor that is sensitive to outliers. In this approach a reduction of the learning rate is shown to be achieved by using alternate loss functions that are infinitely differentiable, strictly convex or quasiconvex and more closely approximate the absolute error than Huber and log-cosh losses. A comparison of the performance of regression models trained with different loss functions on a wide variety of benchmarks and datasets is presented to demonstrate the superior performance of the Square Root Loss (SRL) and Smooth Mean Absolute Error (SMAE) losses proposed in this paper. Two new robust linear regression models are presented. Highly vectorized robust parameter update formulae that take advantage of modern GPUs for both stochastic and batch gradient descent are presented.
Mathew Mithra Noel, Arindam Banerjee, Yug D. Oswal +2
OutliersLoss FunctionLearning RatesGradient DescentMean Squared Error
AskPDF
May 19, 2026cs.LG

Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.
Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe Giguère
Noisy LabelsLoss FunctionCross-Entropy LossesSymmetryNeural Network
AskPDF