Tabular ML

ML: Machine Learning

Ask about this topic

Momentum

7 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 72

All topics
CardsList
Apr 30, 2026cs.LG

ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data

Learning informative representations from tabular data in remote sensing and environmental science is challenging due to heterogeneity, scarce labels, and redundancy among features. We present ZAYAN (Zero-Anchor dYnamic feAture eNcoding), a self-supervised, feature-centric contrastive framework for tabular data. ZAYAN performs contrastive learning at the feature rather than sample level, removing the need for explicit anchor selection and any reliance on class labels, while encouraging a redundancy-minimized, disentangled embedding space. The framework has two modules: ZAYAN-CL, which pretrains feature embeddings via a zero-anchor contrastive objective with dynamic perturbations and masking, and ZAYAN-T, a Transformer that conditions on these embeddings for downstream classification. Across eight datasets, including six remote-sensing tabular benchmarks and two remote-sensing-driven flood-prediction tables from satellite and GIS products, ZAYAN achieves superior accuracy, robustness, and generalization over tabular deep learning baselines, with consistent gains under label scarcity and distribution shift. These results indicate that feature-level contrastive learning and dynamic feature encoding provide an effective recipe for learning from tabular sensing data.

Al Zadid Sultan Bin Habib, Tanpia Tasnim, Md. Ekramul Islam +1

Lane Dept. of Computer Science and Electrical Engineering* and Dept. of Geology & Geography+, West Virginia University, Morgantown, WV 26506, USA · Department of Computer Science and Engineering, Green University of Bangladesh, Narayanganj-1461, Bangladesh · Department of Computer Science and Engineering, Stamford University Bangladesh, Dhaka-1217, Bangladesh

Contrastive LearningDisentangled Representation LearningRemote SensingTabular MLTabular Representation Learning
AskPDF
Apr 29, 2026stat.ML

SCOPE-FE: Structured Control of Operator and Pairwise Exploration for Feature Engineering

Automatic feature engineering is an effective approach for improving predictive performance in tabular learning. However, expand-and-reduce methods, such as OpenFE, become increasingly computationally expensive as the input dimensionality grows. This limitation arises primarily from the combinatorial explosion of candidate features generated through operator-feature combinations. To address this issue, we propose SCOPE-FE, a structured search space control framework that improves efficiency by reducing the candidate space prior to feature generation. SCOPE-FE jointly regulates two major sources of combinatorial growth: the operator space and feature-pair space. First, OperatorProbing estimates the dataset-specific utility of candidate operators and eliminates low-contribution operators in advance. Second, FeatureClustering employs spectral embedding and fuzzy c-means clustering to group structurally related features, thereby restricting candidate generation to relevant within-cluster combinations. In addition, we introduce ReliabilityScoring, which incorporates variance across subsamples to stabilize pruning decisions. Experiments on ten benchmark datasets demonstrate that SCOPE-FE substantially reduces feature engineering time while maintaining competitive predictive performance relative to existing baselines. The efficiency gains are particularly pronounced for high-dimensional datasets. These results indicate that structured control of the search space is an effective strategy for scalable automatic feature engineering. The code will be made publicly available upon acceptance.

Minhee Park, Seongyeon Son, Yonghyun Lee +1

Tabular MLAutomated Feature Engineering
AskPDF
Apr 28, 2026cs.LG

Measuring the Sensitivity of Classification Models with the Error Sensitivity Profile

The quality of training data is critical to the performance of machine learning models. In this paper, the Error Sensitivity Profile (ESP) is proposed. It quantifies the sensitivity of model performance to errors in a single feature or in multiple features. By leveraging ESP, data-cleaning efforts can be prioritized based on error types and features most likely to affect model performance. To support the computation of this metric, an integrated suite of tools, called \dirty, is created. We conduct an extensive experimental study on two widely used datasets using 14 classification models, revealing that performance degradation is not always predictable from simple correlations with the target variable.

Andrea Maurino

Università degli studi di Milano Bicocca Dipartimento di Informatica, Sistemistica e Comunicazione Viale Sarca 336,20126 Milano, Italy

Training Data CurationTabular MLSensitivity Analysis
AskPDF
Apr 27, 2026quant-ph

Do Quantum Transformers Help? A Systematic VQC Architecture Comparison on Tabular Benchmarks

Variational quantum circuits (VQCs) are a leading approach to quantum machine learning on near-term devices, yet it remains unclear which circuit architecture yields the best accuracy-parameter trade-off on classical tabular data. We present a systematic empirical comparison of four VQC families -- multi-layer fully-connected (FC-VQC), residual (ResNet-VQC), hybrid quantum-classical transformer (QT), and fully quantum transformer (FQT) -- across five regression and classification benchmarks. Our key findings are: \textbf{(i)}~FC-VQCs achieve 90-96% of the R2R^2 of attention-based VQCs while using 40-50% fewer parameters, and consistently outperform equal-capacity MLPs (mean R2=0.829R^2{=}0.829 vs.\ MLP720_{720}'s 0.7530.753 on Boston Housing, 3-seed average); \textbf{(ii)}FC-VQC's Type4 inter-block connectivity provides partial cross-token mixing that approximates the role of attention -- explicit quantum self-attention yields only marginal gains on most datasets while significantly increasing parameter count; \textbf{(iii)}expressibility saturates at circuit depth≈ 3{\approx}\,3, explaining why shallow VQCs already cover the Hilbert space effectively; \textbf{(iv)}~LayerNorm on the fully quantum transformer improves classification accuracy, suggesting normalization is important when all operations are quantum; \textbf{(v)}~in our noise study on Boston Housing, FQT degrades gracefully under depolarizing noise while QT collapses. All results are validated across three random seeds. These findings provide practical architectural guidance for deploying VQCs on near-term quantum hardware.

Chi-Sheng Chen, En-Jui Kuo

Beth Israel Deaconess Medical Center & Harvard Medical School Boston, MA, USA · Department of Electrophysics National Yang Ming Chiao Tung University Hsinchu, Taiwan

Quantum Machine LearningTabular MLVariational Quantum Circuits
AskPDF
Apr 22, 2026cs.LG

TabSHAP

Large Language Models (LLMs) fine-tuned on serialized tabular data are emerging as powerful alternatives to traditional tree-based models, particularly for heterogeneous or context-rich datasets. However, their deployment in high-stakes domains is hindered by a lack of faithful interpretability; existing methods often rely on global linear proxies or scalar probability shifts that fail to capture the model's full probabilistic uncertainty. In this work, we introduce TabSHAP, a model-agnostic interpretability framework designed to directly attribute local query decision logic in LLM-based tabular classifiers. By adapting a Shapley-style sampled-coalition estimator with Jensen-Shannon divergence between full-input and masked-input class distributions, TabSHAP quantifies the distributional impact of each feature rather than simple prediction flips. To align with tabular semantics, we mask at the level of serialized key:value fields (atomic in the prompt string), not individual subword tokens. Experimental validation on the Adult Income and Heart Disease benchmarks demonstrates that TabSHAP isolates critical diagnostic features, achieving significantly higher faithfulness than random baselines and XGBoost proxies. We further run a distance-metric ablation on the same test instances and TabSHAP settings: attributions are recomputed with KL or L1 replacing JSD in the similarity step (results cached per metric), and we compare deletion faithfulness across all three.

Aryan Chaudhary, Prateek Agarwal, Tejasvi Alladi

Department of Computer Science Birla Institute of Technology and Science India

Feature AttributionSHAP Feature AttributionLLM InterpretabilityTabular ML
AskPDF
Apr 22, 2026cs.AI

Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data

Automated feature generation extracts informative features from raw tabular data without manual intervention and is crucial for accurate, generalizable machine learning. Traditional methods rely on predefined operator libraries and cannot leverage task semantics, limiting their ability to produce diverse, high-value features for complex tasks. Recent Large Language Model (LLM)-based approaches introduce richer semantic signals, but still suffer from a restricted feature space due to fixed generation patterns and from the absence of feedback from the learning objective. To address these challenges, we propose a Memory-Augmented LLM-based Multi-Agent System (\textbf{MALMAS}) for automated feature generation. MALMAS decomposes the generation process into agents with distinct responsibilities, and a Router Agent activates an appropriate subset of agents per iteration, further broadening exploration of the feature space. We further integrate a memory module comprising procedural memory, feedback memory, and conceptual memory, enabling iterative refinement that adaptively guides subsequent feature generation and improves feature quality and diversity. Extensive experiments on multiple public datasets against state-of-the-art baselines demonstrate the effectiveness of our approach. The code is available at https://github.com/fxdong24/MALMAS

Fengxian Dong, Zhi Zheng, Xiao Han +5

University of Science and Technology of China · Zhejiang University of Technology · Meituan

Tabular MLMulti-Agent LLMsAutomated Feature EngineeringMemory-Augmented Language Model Agents
AskPDF
Apr 20, 2026quant-ph

Benchmarking Quantum Kernel Support Vector Machines Against Classical Baselines on Tabular Data: A Rigorous Empirical Study with Hardware Validation

Quantum kernel methods have been proposed as a promising approach for leveraging near-term quantum computers for supervised learning, yet rigorous benchmarks against strong classical baselines remain scarce. We present a comprehensive empirical study of quantum kernel support vector machines (QSVMs) across nine binary classification datasets, four quantum feature maps, three classical kernels, and multiple noise models, totalling 970 experiments with strict nested cross-validation. Our analysis spans four phases: (i) statistical significance testing, revealing that none of 29 pairwise quantum-classical comparisons reach significance at α=0.05α= 0.05; (ii) learning curve analysis over six training fractions, showing steeper quantum slopes on six of eight datasets that nonetheless fail to close the gap to the best classical baseline; (iii) hardware validation on IBM ibm_fez (Heron r2), demonstrating kernel fidelity r≥0.976r \geq 0.976 across six experiments; and (iv) seed sensitivity analysis confirming reproducibility (mean CV 1.4%). A Kruskal-Wallis factorial analysis reveals that dataset choice dominates performance variance (ε2=0.73\varepsilon^2 = 0.73), while kernel type accounts for only 9%. Spectral analysis offers a mechanistic explanation: current quantum feature maps produce eigenspectra that are either too flat or too concentrated, missing the intermediate profile of the best classical kernel, the radial basis function (RBF). Quantum kernel training (QKT) via kernel-target alignment yields the single competitive result -- balanced accuracy 0.968 on breast cancer -- but with ~2,000x computational overhead. Our findings provide actionable guidelines for quantum kernel research. The complete benchmark suite is publicly available to facilitate reproduction and extension.

Siavash Kakavand, Christoph Strohmeyer, Michael Schlotter

SHARE at FAU, Schaeffler Technologies AG & Co. KG, Herzogenaurach 91074, Bavaria, Germany

Quantum Feature MapsQuantum Kernel MethodsSupport Vector MachinesTabular MLKernel Methods
AskPDF
Apr 16, 2026cs.LG

Benchmarking Optimizers for MLPs in Tabular Deep Learning

MLP is a heavily used backbone in modern deep learning (DL) architectures for supervised learning on tabular data, and AdamW is the go-to optimizer used to train tabular DL models. Unlike architecture design, however, the choice of optimizer for tabular DL has not been examined systematically, despite new optimizers showing promise in other domains. To fill this gap, we benchmark 15 optimizers on 17 tabular datasets for training MLP-based models in the standard supervised learning setting under a shared experiment protocol. Our main finding is that the Muon optimizer consistently outperforms AdamW, and thus should be considered a strong and practical choice for practitioners and researchers, if the associated training efficiency overhead is affordable. Additionally, we find exponential moving average of model weights to be a simple yet effective technique that improves AdamW on vanilla MLPs, though its effect is less consistent across model variants.

Yury Gorishniy, Ivan Rubachev, Dmitrii Feoktistov +1

α Yandex · β HSE University

Multilayer PerceptronsDeep Learning OptimizationNeural Network OptimizationMuon OptimizerTabular ML
AskPDF
Mar 16, 2026cs.LG

TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins

Data-free knowledge distillation enables model compression without original training data, critical for privacy-sensitive tabular domains. However, existing methods does not perform well on tabular data because they do not explicitly address feature interactions, the fundamental way tabular models encode predictive knowledge. We identify interaction diversity, systematic coverage of feature combinations, as an essential requirement for effective tabular distillation. To operationalize this insight, we propose TabKD, which learns adaptive feature bins aligned with teacher decision boundaries, then generates synthetic queries that maximize pairwise interaction coverage. Across 4 benchmark datasets and 4 teacher architectures, TabKD achieves highest student-teacher agreement in 14 out of 16 configurations, outperforming 5 state-of-the-art baselines. We further show that interaction coverage strongly correlates with distillation quality, validating our core hypothesis. Our work establishes interaction-focused exploration as a principled framework for tabular model extraction.

Shovon Niverd Pereira, Krishna Khadka, Yu Lei

Department of Computer Science and Engineering, The University of Texas at Arlington

Feature Interaction ModelingData-Free Knowledge DistillationTabular MLTeacher-Student LearningKnowledge Distillation
AskPDF
Feb 27, 2026cs.LG

Vectorized Dynamic Histograms for Sparse Oblique Forests

Sparse oblique (SPO), part of the top-ranked configuration of Google's Yggdrasil Decision Forests (YDF), improve the accuracy while maintaining interpretability of Random Forests (RF) and Gradient Boosted Trees (GBT) by scanning a sparse linear combination of features instead of a single feature. Because projections are sampled at runtime, training is significantly slower, as pre-run optimizations such as presorting cannot be used. We overhaul SPO training in YDF, first by fixing inefficiencies in the released version, achieving 2-5x speedup, then by devising two novel methods, one aimed at GBTs and the other at RFs: (1) Hierarchical AVX2 and AVX-512-vectorized histogram filling speeds up typical GBT-depth trees by up to 1.6x in SPO-GBT and 2x for SPO-RF, and (2) runtime-dynamic histograms between vectorized histograms and exact splits, speeds up deeper, RF depth-typical trees (depths 16 to purity) by up to 1.6x with no detectable effect on accuracy. Our optimizations bring SPO-RF training time on par with YDF's axis-aligned RF. We extensively evaluate on 8 natural and 11 synthetic datasets of up to 10.5 million rows and 1.6 million features, from depth 6 to purity, and open-source the implementation.

Ariel Lubonja, Jungsang Yoon, Haoyin Xu +6

Johns Hopkins University Baltimore, Maryland, USA · Google Zurich, Switzerland

Gradient-Boosted Decision TreesTabular MLRandom Forest
AskPDF
Aug 30, 2025cs.LG

Integrated Multivariate Segmentation Tree for Heterogeneous Credit Data Analysis in Small- and Medium-Sized Enterprises

Traditional decision tree models, which rely exclusively on numerical variables, often face challenges in handling high-dimensional data and are limited in their ability to incorporate textual information effectively. To address these limitations, we propose the integrated multivariate segmentation tree (IMST), a comprehensive framework designed to improve credit evaluation for small- and medium-sized enterprises (SMEs) by integrating financial data with textual sources. This method comprises three core stages: (1) transforming textual data into numerical matrices through matrix factorization, (2) selecting salient financial features using Lasso regression, and (3) constructing a multivariate segmentation tree based on either the Gini index or entropy, with weakest-link pruning applied to control model complexity. Experimental results based on a dataset of 1,428 Chinese SMEs demonstrated that IMST achieved an accuracy rate of 88.9%, surpassing both baseline decision trees (87.4%) and conventional models such as support vector machines and neural networks. Furthermore, the proposed model demonstrated superior interpretability and computational efficiency, featuring a more streamlined architecture and improved risk detection capabilities.

Lu Han, Xiuying Wang

School of Management Science and Engineering, Central University of Finance and Economics, Beijing 100081, China · School of Computer Science, University of Sydney, NSW 2006, Australia

Financial ServicesInterpretable MLMultimodal ClassificationDecision Tree LearningTabular MLCredit Scoring
AskPDF
May 1, 2025cs.LG

Predicting Estimated Times of Restoration for Electrical Outages Using Longitudinal Tabular Transformers

Utilities publish Estimated Times of Restoration (ETRs) for customer-facing storm outages, and their accuracy governs whether customers can make sound decisions about food, medical equipment, and relocation. Prior work treats ETR as static tabular regression in which each outage contributes one record, discarding the fact that every development of an outage, from crew assignment through dispatch, suspension, damage assessment and partial restoration, is recorded as a revision. We reformulate ETR prediction as longitudinal tabular regression and introduce a Longitudinal Tabular Transformer (LTT), an axial-attention model that consumes the revisions preceding a prediction and issues a refined estimate at every one. On 242{,}928 storm-attributed outages from a cohort of 526{,}468 filtered events and 10.0 million revisions at six operating companies, LTT reduces customer-weighted asymmetric error at all six, by a median of 36.9,% against the estimates the utilities published during the same storms and 11.3,% against the strongest learned baseline at each. It is the only method improving on the incumbent's satisfaction impact at all six companies while also reducing root mean squared error at all six. Stratification by revision index shows that LTT error is largest at the first revision, where no history is available, and falls monotonically as revisions accumulate.

Bogireddy Sai Prasanna Teja, Valliappan Muthukaruppan, Carls Benjamin

Exelon Corporation, Chicago, IL, USA

Event Time PredictionTransformerTabular MLTabular RegressionPower Systems
AskPDF