Support Vector Machines

Also known as SVM

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 29

Oct 7, 2026eess.SP

Second-order optimization of variable projection SVM models and road abnormality detection

We introduce a novel second-order optimization framework for minimizing so-called variable projection functionals. We demonstrate that the proposed framework is especially usefulfor the training of variable projection based kernel methods. In particular, the problem of efficiently training variable projection support vector machines (VP-SVMs) is considered. We show the effectiveness of the proposed training methodology in a real-world application, namely we demonstrate how second-order trust region algorithms can be used to train VPSVM models to recognize road surface abnormalities based on 1D signals obtained from a tire sensor.

Andrea Angino, Matthias Voigt, Rolf Krause +1

Faculty of Mathematics and Computer Science, UniDistance Suisse, Brig, Switzerland · CEMSE Division, KAUST, Thuwal, Saudi Arabia · Eötvös Loránd University, Faculty of Informatics, Department of Numerical Analysis

Second-Order Optimization Support Vector Machines Anomaly Detection Kernel Methods

Oct 6, 2026stat.ML

High-Dimensional Statistical Inference for Sparse Support Vector Machines

Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample size and number of features grow proportionally. The main challenge is the nonsmooth hinge loss, which prevents direct application of debiasing arguments developed for smooth classification losses. We overcome this difficulty by representing the

L_1

-penalized support vector machine (SVM) as a linear program and identifying the hinge-loss subgradient through its dual variables. This yields a computationally accessible debiased estimator whose coordinates are asymptotically Gaussian under the proportional asymptotic regime. The resulting distributional characterization provides confidence intervals and hypothesis tests for individual features and enables false-discovery-rate-controlled variable selection. Extensive simulations examine calibration, power, and variable-selection performance under a range of covariance structures, including strongly correlated designs. An analysis of high-dimensional breast cancer gene-expression data illustrates how the proposed inference can distinguish statistically significant features from variables selected by the original sparse SVM.

Peng Zeng, Hanwen Huang

Department of Mathematics and Statistics Auburn University, Auburn, AL 36849 · Department of Biostatistics, Data Science and Epidemiology Medical College of Georgia, Augusta University, Augusta, GA, 30912

Support Vector Machines Feature Selection

Sep 30, 2026stat.ML

Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model

In this paper, we consider asymptotic properties of the support vector machine (SVM) in high-dimension, low-sample-size (HDLSS) settings under a spiked model. The existing theory of the SVM in the HDLSS context relies on the geometric representation of HDLSS data, which requires that the eigenvalues of the covariance matrices are not dominant. We first show that the geometric representation does not hold under the spiked model. We show that the Gram matrix of HDLSS data converges in distribution to a random matrix, namely, the HDLSS data converge to a random configuration in a finite-dimensional space whose dimension is given by the number of the spikes. We show that the misclassification rates of the SVM do not tend to zero, that is, the SVM does not hold the consistency property. We also show that the bias-corrected SVM (BC-SVM) does not give preferable performance in this setting because the bias term itself should be modified. In order to overcome such difficulties, we propose a spike-corrected SVM (SC-SVM). We show that the SC-SVM holds the consistency property when the sample size goes to infinity, and that the growth of the sample size is essential in the sense that any projection-based procedure fails when the sample size is fixed. Finally, we check the performance of the classifiers by numerical simulations.

Yugo Nakayama

Independent researcher

Support Vector Machines Classification Statistical Learning Theory Data-Efficient Learning

Sep 15, 2026stat.ML

Sharp margin-based generalization bounds for realizable SVM

Let the exact homogeneous hard-margin support vector machine be trained on

m

independent observations from a Borel probability law on a real Hilbert space. We prove that, with score zero counted as an error, there is a universal numerical constant

C

such that

\Pp\left( γ_m>0,\quad \Risk(u_m)> \frac{C}{m} \left( K_m+\log\frac1δ \right) \right) \le δ.

Here

γ_m

is the empirical homogeneous margin,

u_m

is the exact minimum-norm unit-margin separator,

r_m

is the largest training radius, and

K_m:=r_m^2\norm{u_m}^2=r_m^2/γ_m^2

on

\{γ_m>0\}

. The proof is driven by a deterministic deletion problem. Given vectors

x_1,\ldots,x_n

in the unit ball, delete a set

B

of constraints and let

u_B

be the closest point to the origin that satisfies every retained unit-margin constraint. Suppose that

\norm{u_B}^2\le k

and that every deleted vector has nonpositive score under

u_B

. We prove that a family of such deletion sets of cardinality

q

has size at most

\exp(8k+2q)

. The conceptual step is an exact identity obtained from the KKT representation of

u_B

. For a random deletion set, the identity converts the mean squared spread of the separators into a weighted sum of score deficits. It therefore forces a coordinate whose deletion status separates the two conditional means by a quantitatively large amount. Revealing that coordinate decreases the conditional separator variance enough to control the binary entropy of the split. An entropy induction gives the deletion count, and an exact factorial ghost-sample identity converts that count into the stated high-probability SVM bound.

Steve Hanneke, Aryeh Kontorovich

Support Vector Machines Statistical Learning Theory Generalization Bounds

Aug 12, 2026cs.LG

Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

In real-world scenarios, the training data usually contains redundant features, label noise and feature noise, which provide severe challenges for the efficiency of machine learning methods. Since standard support vector machine (SVM) adopts

l_2

-norm penalty and hinge loss function, it lacks the ability of selecting significant features and is sensitive to noise. To address these issues, this paper proposes a novel asymmetric, robust, bounded, sparse and smooth (aR) loss function for

l_1

-norm penalized geometric twin SVM (aRSGTSVM) to handle classification and regression tasks. The

l_1

-norm penalty can achieve the feature selection. The proposed aR loss function can not only effectively mitigate the impact of label noise, but also significantly enhance the stability to resampling noise, i.e., the zero-mean feature noise around the boundary hyperplanes. Furthermore, a statistical analysis of the robustness of aRSGTSVM was also conducted using the influence function. Since aRSGTSVM involves nonconvex and nonsmooth optimization, we develop a fast and stable proximal gradient descent based solving algorithm. Compared with related state-of-the-art methods, experimental results demonstrate the superiority of the proposed aRSGTSVM on both synthetic and UCI datasets. Furthermore, we apply aRSGTSVM to index tracking tasks, where results for tracking the different indices in the China stock market show that it can achieve satisfactory performance.

Kai Qi, Xinji Huang, Hongchun Wang

National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing, 401331, China. · School of Mathematical Sciences, Chongqing Normal University, Chongqing, 401331, China.

Noisy-Label Learning Robust Loss Functions Support Vector Machines Noise Robustness

Aug 4, 2026cs.LG

Beyond the Gegenbauer Paradigm: q-Orthogonal Kernels for Machine Learning

The performance of Support Vector Machines (SVMs) critically depends on the kernel function choice, which enables implicit mapping of data into high-dimensional feature spaces. While classical kernels like Radial Basis Function (RBF) remain popular, orthogonal polynomial kernels offer mathematically interpretable alternatives that can incorporate structured prior knowledge. This work extends the orthogonal polynomial kernel paradigm by introducing a novel family based on discrete

q

-Hermite I polynomials, a class of

q

-orthogonal polynomials that generalize classical Hermite polynomials through a deformation parameter

q

. We formally define the q-Hermite kernel and establish its validity under Mercer's theorem. The kernel's inherent boundedness properties naturally prevent annihilation and explosion effects without requiring explicit scaling mechanisms. Extensive experiments across 20 benchmark datasets demonstrate that the proposed kernel achieves competitive performance compared to both classical kernels and other orthogonal polynomial kernels, while offering advantages in numerical stability and computational simplicity. Our results confirm that

q

-orthogonal polynomials constitute a promising direction for kernel design, bridging mathematical elegance with practical machine learning applications, that provides conceptual and algorithmic resources that may be further extended to emerging quantum computing paradigms. To facilitate full reproducibility, we provide the complete implementation and experimental pipeline in an open-access GitHub repository at https://github.com/Kokechacho/SVMs-QSVMs.

Álvaro Sánchez-Paniagua Ríos, Juan P. Llerena, Alberto Lastra +2

Physics and Mathematics Department, University of Alcalá, Spain · Computer Science and Artificial Intelligence Department, University of Alcalá, Spain · Mathematics Department, Universidad Autónoma de Madrid, Spain

Support Vector Machines Kernel Methods

Aug 2, 2026cs.LG

Data-Driven Pinball-Loss Selection for Vertically Distributed Elastic-Net SVMs

The pinball-loss support vector machine is robust, but its asymmetry parameter is usually fixed in advance. We propose a data-driven elastic-net support vector machine that learns simplex-constrained weights over candidate pinball losses while retaining one classifier. The weighted loss is equivalent to a pinball loss with a data-dependent effective parameter. An empirical oracle inequality shows that, when weight regularization and simplex truncation vanish, the classifier objective at a global minimizer does not exceed that of the best fixed candidate; otherwise, the excess is explicitly bounded. For high-dimensional data, we develop a column-partitioned variable-splitting solver. It converges with a best-iterate

O(1/T)

squared-step residual rate. Under common initialization and global parameters, any column partition produces, in exact arithmetic, the same iterates and solution as centralized training. Experiments assess predictive behavior, numerical equivalence, and multi-process scalability.

Xiaofei Wu, Kai Qi, Rongmei Liang

Yunnan Key Laboratory of Statistical Modeling and Data Analysis, School of Mathematics and Statistics, Yunnan University, East Outer Ring South Road, University Town, Chenggong District, Kunming, Yunnan, China · National Center for Applied Mathematics in Chongqing, Chongqing Normal University, No. 37 University Town Middle Road, Shapingba District, Chongqing, China · Department of Statistics and Data Science, Southern University of Science and Technology, 1088 Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong 518055, China

Distributed Optimization Adaptive Loss Weighting Support Vector Machines

Jul 21, 2026cs.LG

Formulation-Level Auto-Tuning for QUBO-Based Machine Learning: A Case Study Across Multiple Quantum-Inspired Annealers

This paper presents an Optuna-based formulation-level auto-tuning framework for support vector machines (SVMs) implemented on multiple quantum-inspired annealers. In an annealing-based SVM, continuous dual variables are discretized and converted into a quadratic unconstrained binary optimization (QUBO) model. This transformation introduces three coupled classes of parameters: representation parameters-the encoding base B and bit depth K-which determine numerical range, resolution, and QUBO size; the RBF kernel parameter γ, which determines classifier geometry; and the equality-constraint penalty ξ, which controls feasibility and coefficient balance. We formulate their joint selection as a mixed discrete-continuous black-box optimization problem. The framework has two optimization levels: an inner annealer minimizes the generated QUBO, while an outer Optuna loop reconstructs the formulation in every trial and maximizes validation accuracy. The same solver-agnostic procedure is applied to Fixstars Amplify Annealing Engine, Toshiba SQBM+, and Fujitsu Digital Annealer using TPE and Gaussian-process samplers and is compared with conventional grid search. Experiments on linear and nonlinear classification tasks with 0-20% label noise show mean gains over grid search of approximately 0.8 and 2.1 percentage points, respectively. The results demonstrate that formulation quality and backend capability must be evaluated jointly and that task-level feedback can compensate for discretization, penalty imbalance, and backend-dependent approximate optimization.

Naoya Mizuki, Takahiro Katagiri, Daichi Mukunoki +1

Graduate School of Informatics, Nagoya University · Information Technology Center, Nagoya University

Black-Box Optimization Quantum-Inspired ML Quadratic Unconstrained Binary Optimization Support Vector Machines Hyperparameter Optimization

Jul 19, 2026cs.LG

An Iterative Geometric Approach to Optimizing Separating Hyperplanes

Given a binary-labeled linearly separable dataset, and the objective is to compute the maximum-margin separating hyperplane, also known as the hard-margin Support Vector Machine (SVM) classifier. This paper investigates whether, if given an initial separating hyperplane, can it be exploited to reach this unique optimum more efficiently. We present a geometric approach that gradually improves the alignment of the hyperplane, starting from an initial separating hyperplane, while preserving separation and continuously increasing its margin until convergence to the global optimum. At each iteration, the method considers only local information, namely the current active set, and aims to re-align the hyperplane according to the optimal separating hyperplane of this reduced subset. Consequently, the original convex quadratic optimization problem is addressed through a sequence of smaller subproblems. The paper presents the algorithm in detail, together with a preliminary experimental evaluation and several theoretical findings. The results suggest that, when an initial separating hyperplane is available, the proposed method can be competitive on larger datasets and, in some cases, can outperform state-of-the-art approaches that solve the optimization problem directly.

Akos Hajnal

Laboratory of Parallel and Distributed Systems Institute for Computer Science and Control (SZTAKI) · John von Neumann Faculty of Informatics Obuda University Budapest, Hungary

Support Vector Machines Constrained Optimization

Jun 22, 2026cs.LG

Differential Spectral Damping Gap Adaptive Regularization for Ill-Conditioned Kernel Methods

Kernel methods requiring matrix inversion -- particularly Least-Squares Twin Support Vector Machines (LSTSVM) -- suffer from exponential eigenvalue decay in their system matrices, producing severely ill-conditioned problems where standard Tikhonov regularization applies uniform damping regardless of eigenvector reliability. We propose Differential Spectral Damping (DSD), a regularization formula that adapts its penalty to localized eigengap structure: preserving eigenvectors with large spectral gaps (reliable per Davis-Kahan perturbation theory) while aggressively suppressing those with small gaps (directionally corrupted beyond recovery). We motivate DSD through a principled design procedure grounded in the Davis-Kahan

\sin(Θ)

theorem, systematically deriving the requirements for a reliability-aware damping function and selecting the exponential form for its smoothness, differentiability, and natural saturation properties. Through rigorous paired testing with fairly optimized baselines (including gradient-optimized Tikhonov receiving equal optimization opportunity), we demonstrate that DSD improves LSTSVM classification accuracy by +4.8 percentage points on real-world GINA (

d=970

, Cohen's

d = 4.49

,

p < 0.0001

), +10.4 percentage points at

d=200

, and +2.6 percentage points on Madelon (

d=500

) -- all using only principled spectral initialization while Tikhonov receives grid search. For pre-image reconstruction on manifold data, DSD ties Tikhonov at high perturbation noise (

p=0.99

) but slightly underperforms at lower noise levels; both reduce naive inversion error by

66\times

. We characterize the precise operating regime (

d \geq 100

, condition number

> 10^3

) and document where simpler methods suffice, providing practitioners with clear deployment guidance.

Praveg Vashishtha

Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · Sopra Banking Software India

Spectral Regularization Support Vector Machines Kernel Methods

Jun 20, 2026cs.LG

Sequential Minimal Optimization Algorithm for One-Class Support Vector Machines With Privileged Information

One of the powerful techniques in data modeling is accounting for features that are available at the training stage, but are not available when the trained model is used to classify or predict test data -- the Learning Using Privileged Information paradigm (LUPI). Sequential Minimal Optimization (SMO) methods have been developed for supervised Support Vector Machines (SVM), unsupervised one-class SVM, and SVM with privileged information (SVM+). The missing brick in this research has long been a one-class SVM with privileged information (OC-SVM+). In this paper, we propose an SMO algorithm for OC-SVM+ that significantly outperforms non-sequential algorithms for training the OC-SVM+ model. Its finite-time convergence is established. The experiments show how privileged information affects a descriptive domain in the space of original features. Comparative benchmark tests demonstrate that our algorithm is superior over interior point algorithms.

Andrey Lange, Dmitry Smolyakov, Evgeny Burnaev

Skolkovo Institute of Science and Technology (Skoltech), Moscow 121205, Russia · Federal Research Center ‘‘Computer Science and Control" of Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia · Institute for Information Transmission Problems of Russian Academy of Sciences (IITP RAS), 127994 Moscow, Russia +1

Support Vector Machines One-Class Classification Learning Using Privileged Information

Jun 14, 2026cs.LG

Decomposing one-class support vector machine into an ensemble of one-data support vector machines

One-class classification (OCC) is a classification problem in which the training data contains only one class. The one-class support vector machine (OCSVM) is one of the most competitive OCC algorithms. However, OCSVM has scalability issues with large-scale datasets. This paper proposes the acceleration strategy of OCSVM. The idea is to decompose the dataset into samples and train OCSVM models for single data points. Subsequently, ensemble learning is applied to combine all models to compute the OCSVM model for the dataset. In addition, further acceleration is achieved through a data-reduction strategy with an OCSVM model trained on the average of the training samples. The experiment compared the proposal and traditional OCSVM using the Python package. The proposed strategy is faster than traditional OCSVM, while achieving similar classification results. Moreover, the proposed strategy can create one-to-one correspondence between samples and models. Source code is uploaded at https://github.com/ToshiHayashi/ODSVM

Toshitaka Hayashi, Dalibor Cimr, Hamido Fujita +1

Faculty of Science, University of Hradec Kralove, Hradecká 1285, Hradec Kralove, 50003, Czech Republic · Malaysia-Japan International Institute of Technology (MJIIT), Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, Kampung Datuk Keramat, Kuala Lumpur, 54100, Malaysia · Regional Research Center, Iwate Prefectural University, Sugo 152-52, Takizawa, 020-0693, Iwate, Japan

Support Vector Machines Classification One-Class Classification

Jun 4, 2026cs.LG

TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection

TorchKM is an open-source library for kernel machines, including support vector machines, kernel logistic regression, and kernel quantile regression, with GPU acceleration. The library features a scikit-learn-style API and is designed to exploit GPU-friendly linear algebra, accelerating the full training and model-selection pipeline through intelligent reuse of matrix operations. Benchmarks show competitive predictive performance with substantial speedups over standard baselines. The efficiency and programmable design also make TorchKM a kernel-learning component for AI-driven workflows. Code and documentation are available at https://github.com/YikaiZhang95/torchkm, and the package can be easily installed via PyPI.

Yikai Zhang, Gaoxiang Jia, Jie Ding +1

Department of Statistics and Actuarial Science, University of Iowa, Iowa City, IA 52242, USA · Individual Researcher · AIScientists, Inc. (MorphMind), 245 Main Street, Cambridge, MA 02142, USA

Model Selection Kernel Regression GPU Acceleration Support Vector Machines Kernel Methods

Jun 4, 2026cs.LG

Robust and sparse support vector machine via hybrid truncated loss for supervised classification

The support vector machine (SVM) is a widely used classifier, but choosing an appropriate loss function remains difficult. Convex losses such as the hinge loss and least-squares loss are sensitive to outliers, while bounded non-convex losses often lead to high computational cost. To address this, we propose a hybrid truncated loss function (

L_{\mathrm{ht}}

) that is both sparse and bounded, and build the

L_{\mathrm{ht}}

-SVM model for single-view classification. We introduce the P-stationary point and use it to establish the first-order necessary and sufficient optimality conditions. Based on these conditions, we design an alternating direction method of multipliers with a working-set strategy that reduces computational cost and achieves global convergence. We further extend

L_{\mathrm{ht}}

-SVM to multi-view learning by adding structural information and view weights, resulting in Mv

L_{\mathrm{ht}}

-SVM, which follows both the consensus and complementarity principles. Experiments on synthetic, real-world, and image datasets show that

L_{\mathrm{ht}}

-SVM achieves higher accuracy with fewer support vectors and better noise robustness than five single-view methods, while Mv

L_{\mathrm{ht}}

-SVM outperforms six multi-view methods in accuracy, precision, recall, and F1-score.

Yuliang Yang, Chen Chen, Yuxiang Liu +1

School of Science, Beijing Forestry University, No.35 Qinghua East Road, 100083 Haidian, Beijing, China · Translational Cancer Research Center, Peking University First Hospital, No. 8 Xishiku Street, 100034 Xicheng, Beijing, China

Multi-View Learning Robust Loss Functions Support Vector Machines

Jun 4, 2026cs.LG

Intercomparison of Machine Learning Algorithms for Remote Sensing-based In-season Crop Mapping

In-season crop type mapping is critical for food security in the face of increasingly extreme climate-related threats to crops. Currently, the USDA Cropland Data Layer provides crop type labels at 30m resolution and is available the February after harvest, but no product exists that maps crop types before harvest with satisfactory accuracy that would allow emergency managers to respond to crop threats in near real time. Furthermore, the relative advantages of a wide range of algorithms have not been evaluated in a way that accounts for interannual variability, until this study. Here, Harmonized Landsat-Sentinel surface reflectance imagery time series and crop rotation history information are combined to map corn in Iowa and almonds in California at 30m resolution accurately by early June in unseen years, with robust quantification of uncertainty due to phenology and crop distribution. Thousands of model configurations across ten machine learning algorithms were compared using a year-wise cross-validation and a suite of metrics. Hyperparameter search revealed Support Vector Machines to be the most successful algorithm overall, with a mean F1 score of 0.74 (0.59) across five unseen validation years for almonds by early June in California (corn by early June in Iowa). Interannual variation was a large source of uncertainty, but patterns showed the potential to further improve performance with ensemble approaches or ancillary data. Future work may extend these methods to include multiclass maps of all crop types, CONUS-wide maps, and in-season crop yield forecasting.

August Posch, Jitendra Kumar, Forrest M. Hoffman +1

Institute for Experiential Artificial Intelligence and Sustainability and Data Sciences Lab, Northeastern University, 216 Massachusetts Avenue, Boston, MA 02155 USA · Environmental Sciences Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37830 USA · Computational Science and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37830 USA

Agricultural Remote Sensing Remote Sensing Support Vector Machines

Jun 3, 2026cs.LG

Graph-Guided Universum Learning in Generalized Eigenvalue Proximal SVMs for Alzheimer's Disease Classification

Early and accurate detection of Alzheimer's disease (AD) is important for timely intervention and disease management. Generalized Eigenvalue Proximal Support Vector Machine (GEPSVM) and its Universum-based variants have shown promising results for AD classification. However, existing methods treat Universum samples as independent points and do not consider the geometric relationships among them. This paper proposes two graph-guided Universum learning models, namely UG-GEPSVM and IUG-GEPSVM, for AD versus cognitively normal (CN) classification using structural MRI data. In the proposed framework, mild cognitive impairment (MCI) subjects are used as Universum data to provide intermediate information between AD and CN classes. A graph is constructed over the Universum samples using Gaussian similarity, Minimum Spanning Tree connectivity, and multi-hop propagation. From this graph, a Laplacian matrix is derived that captures the geometric structure of the MCI samples. This Laplacian-based regularization is incorporated into the learning process in place of the conventional independent Universum penalty term. UG-GEPSVM integrates this regularization into the generalized eigenvalue formulation, while IUG-GEPSVM extends the numerically stable improved GEPSVM framework using a standard eigenvalue formulation. Experiments on ADNI MRI dataset variants using ICA- and PCA-based features at five different noise levels show that both proposed models consistently outperform existing GEPSVM and Universum-based methods. UG-GEPSVM achieves the highest average AUC of 88.07% and maintains stable performance under increasing noise levels. Statistical tests further confirm the significance of the observed improvements.

Yogesh Kumar, Vrushank Ahire, Mudasir Ganaie

Dept. of Computer Science and Engineering, IIT Ropar, Punjab 140001, India

Alzheimer's Disease Medical Image Classification Support Vector Machines Alzheimer's Disease Classification

Jun 2, 2026cs.LG

Analytical Evaluation of DCA Convergence Properties for Minimizing Prediction Functions of Gaussian RBF Support Vector Regression

For nonconvex optimization problems whose objective is the prediction function of a trained Support Vector Regression (SVR) model with the Gaussian radial basis function (RBF) kernel (RBF-SVR), we present a framework that applies the difference of convex functions (DC) algorithm (DCA) by exploiting the analytical structure of the RBF kernel to construct an explicit DC decomposition. Specifically, we derive in closed form both the lower bound

μ

of the strong convexity parameter of the DC components and the upper bound

L

of the gradient Lipschitz constant of the subproblem. Both

μ

and

L

are determined solely by the post-training dual-coefficient sum

C_α

and the RBF kernel parameter

γ

, together with the DC decomposition parameter

ρ

, and they share a common leading term

C_αρ

. Through numerical experiments on six benchmark functions, we show that

C_αρ

is the primary single quantity characterizing both the convergence properties and the initial-point dependence of DCA, and further demonstrate that it decomposes into two independent pathways,

C \to C_α

and

γ\to ρ

, with its primary variation governed by the SVR hyperparameters

(C, γ)

. Together, these results allow the convergence properties of DCA on RBF-SVR to be assessed in advance through the single scalar quantity

C_αρ

: approximately from

(C, γ)

before training, and exactly in closed form after training.

Yohei Kakimoto, Yuto Omae, Hirotaka Takahashi

Nihon University · Tokyo City University

Optimization Convergence Analysis Support Vector Machines Kernel Methods

May 29, 2026stat.ML

Out-of-Distribution generalization of quantile regression with heavy tailed inputs: an SVM approach

We study quantile regression in an extrapolation regime where the covariate takes unusually large values. Under regular variation assumptions, extreme observations can be effectively characterized through their angular components, enabling learning strategies that focus on the angle of the most extreme observations. This approach is formalized through the minimization of an asymptotic conditional risk that localizes learning in the tail of the covariate distribution. We propose a novel Support Vector Machine (SVM) framework for extreme quantile regression, leveraging reproducing kernel Hilbert spaces to handle high-dimensional and nonlinear settings. Our method also accommodates unbounded response variables and avoids restrictive transformations. We establish finite-sample learning guarantees under mild regularity assumptions. The proposed framework unifies ideas from statistical learning and multivariate extremes, providing a tractable and theoretically grounded approach to extrapolation. We complement our theoretical findings with an empirical study on river flow data from the Danube, demonstrating the practical relevance of our methods.

Baptiste Leroux, Clément Dombry, Anne Sabourin

Université Paris Cité, CNRS, MAP5, F-75006 Paris, France · Université Marie et Louis Pasteur, CNRS, LmB (UMR 6623), F-25000 Besançon, France · Université Paris Cité, Université Paris Saclay, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli, F-75006, Paris, France

Quantile Regression OOD Generalization Support Vector Machines Extreme Value Theory

May 21, 2026cs.LG

Adaptive Measurement Allocation for Learning Kernelized SVMs Under Noisy Observations

Kernel methods are typically formulated under the assumption of exact, noise-free access to the Gram matrix. However, in emerging settings such as quantum machine learning, each kernel entry must be inferred from noisy observations, and its accuracy depends on how a limited measurement budget is allocated. Despite this, existing approaches overwhelmingly rely on uniform allocation, which equalizes estimator variance but ignores the highly non-uniform dependence of kernelized classifiers on the Gram matrix. In this work, we introduce an adaptive measurement-allocation strategy for learning kernelized Support Vector Machines (SVMs) from noisy Bernoulli observations. Our approach combines two complementary principles: (i) geometric sensitivity, capturing how perturbations of individual kernel entries affect the classifier margin, and (ii) active-set instability, quantifying the probability of discrete changes in support-vector membership induced by measurement noise. These signals define a task-aware allocation scheme that concentrates measurements on the most decision-critical regions of the kernel matrix. We provide a theoretical analysis showing that the benefit of adaptive allocation is governed by the heterogeneity of the induced kernel importance structure, leading to distinct regimes in which adaptive or uniform strategies are preferable. Empirical evaluations on synthetic datasets demonstrate that adaptive allocation significantly improves support-vector recovery, margin estimation, and decision-function accuracy under fixed measurement budgets. A dual-coefficient stability criterion further enables early stopping, achieving near-optimal performance while using only a fraction of the measurement cost. Additional experiments on quantum kernels derived from real-world data reveal a regime-dependent behavior aligned with known phenomena such as kernel concentration. Together...

Artur Miroszewski

Φ-lab, European Space Agency (ESA/ESRIN), Frascati, Italy

Adaptive Sampling Support Vector Machines Kernel Methods

May 9, 2026cs.LG

Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning

In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing benchmark dataset, with Preethi et al. (2025)specifically ranking it last among the algorithms they tested, reporting an R2 of only 0.60. This paper examines whether the previously reported performance reflects experimental configuration choices rather than an inherent algorithmic limitation. A structured experimental workflow is applied: ten domain-motivated derived features are constructed from the eight raw inputs, an exploratory ensemble feature importance analysis identifies the most predictive candidates, and a randomised search over hyperparameter combinations with three-fold cross-validation selects the optimal SVR configuration within a leakage-safe scikit-learn Pipeline. A formal four-stage ablation study isolates the contribution of each component: scaling alone accounts for +0.744 in R2 (from -0.054 to 0.690), feature engineering adds +0.026 (to 0.716), and hyperparameter tuning contributes +0.008 (to 0.723). The resulting tuned SVR achieves a test R2 of 0.723, a 0.123-point absolute improvement over the previously reported SVR result (from 0.60 to 0.723, approximately 20% relative gain). In the ten-model comparison, the tuned SVR ranks fourth with R2 = 0.723, below XGBoost (0.832), Random Forest (0.814) and Gradient Boosting (0.783), while substantially outperforming simpler baselines. Ten-fold cross-validation yields a mean R2 of 0.703 (95% CI: [0.630, 0.775]), confirming robust generalisation. The observed improvement from R2 = 0.60 to R2 = 0.723 is associated primarily with proper feature scaling within a unified preprocessing pipeline, with domain-motivated feature engineering and systematic hyperparameter tuning, providing further incremental gains.

Emmanuel Adutwum

Soka University of America, Aliso Viejo, California, United States.

Support Vector Machines Hyperparameter Optimization Tabular Regression

May 8, 2026cs.CL

A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews

This paper presents a comparative study of classical machine learning and deep learning methods for sentiment classification on the IMDb movie reviews dataset. The machine learning pipeline uses TF-IDF features and PyCaret AutoML to evaluate Logistic Regression, Naïve Bayes, and Support Vector Machine, while the deep learning pipeline implements BiLSTM and BiLSTM with an attention mechanism. Experimental results show that classical machine learning, especially SVM, achieves the best performance with an accuracy of 0.8530, outperforming the deep learning models in this study. The BiLSTM with Attention model improves over the standard BiLSTM and reaches an accuracy of 0.706, indicating better contextual modeling. The paper concludes that although deep learning can capture sequential dependencies, classical machine learning remains a strong baseline when combined with effective feature engineering such as TF-IDF, particularly under limited data and computational resources.

Erma Daniar Safitri, Lia Hana Ichisasmita, Citra Agustin +3

Department of Data Science Institut Teknologi Sumatera Lampung Selatan, 35365, Indonesia · Department of Informatics Engineering Institut Teknologi Sumatera Lampung Selatan, 35365, Indonesia

Sentiment Analysis Support Vector Machines

May 5, 2026cs.LG

GraphSVR: A Graph Convolutional Support Vector Regression Framework for Robust Spatiotemporal Air Pollution Forecasting

Urban air quality forecasting is challenging because pollutant concentrations are nonlinear, nonstationary, spatiotemporally dependent, and often affected by anomalous observations caused by traffic congestion, industrial emissions, and seasonal meteorological variability. This study proposes a Graph Convolutional Support Vector Regression (GraphSVR) framework for robust spatiotemporal forecasting of urban air pollution. The model combines graph convolutional learning to capture inter-station spatial dependence with support vector regression to model nonlinear temporal dynamics while reducing sensitivity to outlier observations. The proposed framework is evaluated using air quality records from 37 monitoring stations in Delhi and 18 stations in Mumbai, representing inland and coastal metropolitan environments in India. Forecasting performance is assessed across multiple horizons and compared with established temporal and spatiotemporal benchmarks. The results show that GraphSVR consistently improves predictive accuracy and maintains stable performance across seasons and outlier-prone pollution episodes. Statistical test further confirms the reliability of the proposed approach across the datasets. Furthermore, the conformal prediction approach is integrated with GraphSVR to generate calibrated prediction intervals, enhancing its practical value for uncertainty-aware air quality monitoring and public health decision-making.

Nourin Jahan, Muhammed Navas T, Tanujit Chakraborty +1

Department of Mathematics, School of Advanced Sciences, Vellore Institute of Technology, Vellore, Tamil Nadu, India · SAFIR, Sorbonne University, Abu Dhabi, UAE · SCAI - Sorbonne Cluster for Artificial Intelligence, Paris, France +1

Air Quality Prediction Support Vector Machines Spatiotemporal Forecasting Spatiotemporal GNNs

May 5, 2026cs.CL

A Comparison of Traditional Machine Learning Algorithms and LSTM-Based Deep Learning Models for Email Sentiment Analysis

The rapid growth of electronic communication has necessitated more robust systems for email classification and sentiment detection. This study presents a comparative performance analysis between traditional machine learning algorithms and deep learning architectures, specifically focusing on Support Vector Machines (SVMs), Logistic Regression, Naive Bayes, and Long Short-Term Memory (LSTM). Utilizing Word2Vec embeddings for feature representation, our experimental results indicate that the SVM model with a linear kernel achieves the highest efficiency and accuracy, reaching a peak performance of 98.74%. While the LSTM model demonstrates exceptional recall capabilities in detecting spam-related sentiments, it requires significantly more computational time compared to discriminative statistical models. Detailed evaluations via confusion matrices further reveal that traditional classifiers remain highly robust for dense vector spaces. This research concludes that for email detection tasks, SVM offers the most optimal balance between predictive precision and processing speed. These findings provide critical insights for developing high-performance automated email filtering systems in professional and academic environments.

Virdio Samuel Saragih, Baruna Abirawa, Kartini Lovian Simbolon +3

Faculty of Science Sumatra Institute of Technology

Sentiment Analysis Text Classification Support Vector Machines Long Short-Term Memory Networks

Apr 28, 2026stat.ML

Elite-Driven Support Vector Machines for Classification

Support vector machines (SVMs) are a standard tool for binary classification, but their classical formulations are purely data-driven and offer no direct way to encode trusted benchmark models or structured preferences on selected subsets of the data. We propose Elite-Driven Support Vector Machines (EDSVM), a general framework that augments regularized empirical risk minimization by guiding the slack variables for a curated set of elite observations (typically the union of support vectors from one or more reference SVMs). EDSVM combines the usual slack loss with a deviation penalty that shrinks new slacks toward benchmark slack values, defining a localized, margin-aligned notion of proximity to reference models, unlike global function penalties in knowledge distillation or teacher-student methods, and without requiring privileged features as in SVM+/LUPI. Within this framework we develop two concrete models, C-EDSVM and LS-EDSVM, based respectively on hinge-type and squared-slack losses. For both variants we derive dual quadratic programs that can be implemented with modest modifications of standard SVM solvers, and we give simple sufficient conditions under which the induced margin losses are classification calibrated. Simulation studies and experiments on several UCI benchmarks show that EDSVMs closely track the behaviour induced by reference SVMs while achieving predictive performance that is competitive with, and sometimes better than, C-SVM, LINEX-SVM, and LS-SVM.

Mohammad Jafari Jozani, Bahram Moeinianfar

Department of Statistics, University of Manitoba, Winnipeg, Manitoba, CANADA, R3T2N2

Binary Classification Support Vector Machines Classification Statistical Learning Theory Empirical Risk Minimization

Apr 20, 2026quant-ph

Benchmarking Quantum Kernel Support Vector Machines Against Classical Baselines on Tabular Data: A Rigorous Empirical Study with Hardware Validation

Quantum kernel methods have been proposed as a promising approach for leveraging near-term quantum computers for supervised learning, yet rigorous benchmarks against strong classical baselines remain scarce. We present a comprehensive empirical study of quantum kernel support vector machines (QSVMs) across nine binary classification datasets, four quantum feature maps, three classical kernels, and multiple noise models, totalling 970 experiments with strict nested cross-validation. Our analysis spans four phases: (i) statistical significance testing, revealing that none of 29 pairwise quantum-classical comparisons reach significance at

α= 0.05

; (ii) learning curve analysis over six training fractions, showing steeper quantum slopes on six of eight datasets that nonetheless fail to close the gap to the best classical baseline; (iii) hardware validation on IBM ibm_fez (Heron r2), demonstrating kernel fidelity

r \geq 0.976

across six experiments; and (iv) seed sensitivity analysis confirming reproducibility (mean CV 1.4%). A Kruskal-Wallis factorial analysis reveals that dataset choice dominates performance variance (

\varepsilon^2 = 0.73

), while kernel type accounts for only 9%. Spectral analysis offers a mechanistic explanation: current quantum feature maps produce eigenspectra that are either too flat or too concentrated, missing the intermediate profile of the best classical kernel, the radial basis function (RBF). Quantum kernel training (QKT) via kernel-target alignment yields the single competitive result -- balanced accuracy 0.968 on breast cancer -- but with ~2,000x computational overhead. Our findings provide actionable guidelines for quantum kernel research. The complete benchmark suite is publicly available to facilitate reproduction and extension.

Siavash Kakavand, Christoph Strohmeyer, Michael Schlotter

SHARE at FAU, Schaeffler Technologies AG & Co. KG, Herzogenaurach 91074, Bavaria, Germany

Quantum Feature Maps Quantum Kernel Methods Support Vector Machines Tabular ML Kernel Methods

Apr 16, 2026stat.ML

Structural interpretability in SVMs with truncated orthogonal polynomial kernels

We study post-training interpretability for Support Vector Machines (SVMs) built from truncated orthogonal polynomial kernels. Since the associated reproducing kernel Hilbert space is finite-dimensional and admits an explicit tensor-product orthonormal basis, the fitted decision function can be expanded exactly in intrinsic RKHS coordinates. This leads to Orthogonal Representation Contribution Analysis (ORCA), a diagnostic framework based on normalized Orthogonal Kernel Contribution (OKC) indices. These indices quantify how the squared RKHS norm of the classifier is distributed across interaction orders, total polynomial degrees, marginal coordinate effects, and pairwise contributions. The methodology is fully post-training and requires neither surrogate models nor retraining. We illustrate its diagnostic value on a synthetic double-spiral problem and on a real five-dimensional echocardiogram dataset. The results show that the proposed indices reveal structural aspects of model complexity that are not captured by predictive accuracy alone.

Víctor Soto-Larrosa, Nuria Torrado, Edmundo J. Huertas

Dpto. de Biociencias, Facultad de Ciencias Biomédicas y de la Salud, Universidad Europea de Madrid, C/ Tajo, s/n, 28670 Villaviciosa de Odón, Madrid, Spain. · Dpto. de Matemáticas, Facultad de Ciencias, Universidad Autónoma de Madrid, C/ Francisco Tomás y Valiente, 7, 28049 Madrid, Spain. · Instituto de Ciencias Matemáticas (ICMAT), Campus de Cantoblanco UAM, C/ Nicolás Cabrera, 13–15, 28049 Madrid, Spain. +1

Interpretable ML Support Vector Machines Kernel Methods

Jul 25, 2025cs.LG

OCSVM-Guided Representation Learning for Unsupervised Anomaly Detection

Unsupervised anomaly detection (UAD) aims to detect anomalies without labeled data, a necessity in many machine learning applications where anomalous samples are rare or not available. Most state-of-the-art methods fall into two categories: reconstruction-based approaches, which often reconstruct anomalies too well, and decoupled representation learning with density estimators, which can suffer from suboptimal feature spaces. While some recent methods attempt to couple feature learning and anomaly detection, they often rely on surrogate objectives, restrict kernel choices, or introduce approximations that limit their expressiveness and robustness. To address this challenge, we propose a novel method that couples representation learning the exact One-Class SVM (OCSVM) optimization problem, through a custom loss formulation that directly aligns latent features with the OCSVM decision boundary. The model is evaluated on two tasks: a benchmark based on MNIST-C, and a challenging brain MRI lesion detection task. Unlike most methods that focus on large, hyperintense lesions at the image level, our approach succeeds to target small, non-hyperintense lesions, while we evaluate voxel-wise metrics, addressing a more clinically relevant scenario. Both experiments evaluate a form of robustness to domain shifts, including corruption types in MNIST-C and texture or population age variations in MRI. Results demonstrate performance and robustness of our proposed model, highlighting its potential for general UAD and real-world medical imaging applications. The source code is available at https://github.com/Nicolas-Pinon/uad\_ocsvm\_guided\_repr\_learning.

Nicolas Pinon, Robin Trombetta, Carole Lartizien

MYRIAD

Medical Image Anomaly Detection Unsupervised Representation Learning Anomaly Localization Support Vector Machines Unsupervised Anomaly Detection

Feb 16, 2025quant-ph

Physics-Informed Support Vector Kernels via Green-Function Analogies and Jackson-Chebyshev Spectral Design

Kernel selection for regression of physical observables is often heuristic. We investigate a physics-informed strategy in which functional forms and spectral structures associated with Green's functions motivate kernel selection without requiring an exact identification between a machine-learning kernel and a physical propagator. The principal construction is a Jackson-damped Chebyshev kernel inspired by the kernel polynomial method (KPM); its explicit feature map yields a positive-semidefinite Gram matrix by construction and provides an inspectable spectral prior for structured observables. We evaluate standard and custom SVR models on copper-conductivity proxies, local Dirac-like band dispersion, quartic-oscillator energy levels, photonic-crystal transmission, and Fibonacci-chain transmission using repeated nested validation, learning curves, random-forest and multilayer-perceptron baselines, and low-rank Nyström tests where relevant. The framework is intended for finite-data regression of precomputed observables while boundary conditions remain part of the physical model that generates those observables.

Nan-Hong Kuo, Renata Wong

Department of Physics, National Taiwan University, Taipei, Taiwan · Department of Artificial Intelligence, Chang Gung University, Taoyuan, Taiwan · Department of Neurology, Chang Gung Memorial Hospital, Keelung, Taiwan +1

Support Vector Machines Kernel Methods Physics-Informed ML Spectral Methods

Dec 14, 2024quant-ph

Classification of Financial Data Using Quantum Support Vector Machine

Quantum Support Vector Machine is a kernel-based approach to classification problems. We study the applicability of quantum kernels to financial data, specifically our self-curated Dhaka Stock Exchange (DSEx) Broad Index dataset. To the best of our knowledge, this is the first systematic study of quantum kernels applied to this dataset. Working within the empirical quantum advantage (EQA) framework of Krunic et al., we benchmark several quantum kernels against a classical RBF-kernel SVM baseline, propose the best-performing kernel for this dataset, and relate the observations to the Phase Space Terrain Ruggedness Index metric. We estimate the resources needed to carry out these investigations on a larger scale for future practitioners.

Seemanta Bhattacharjee, MD. Muhtasim Fuad, A. K. M. Fakhrul Hossain

Shahjalal University Of Science and Technology, Sylhet-3114, Bangladesh

Quantitative Finance Hybrid Quantum-Classical ML Quantum Kernel Methods Support Vector Machines Finance