KMLP: A Scalable Hybrid Architecture for Web-Scale Tabular Data Modeling

Mingming Zhang; Pengfei Shi; Zhiqing Xiao; Feng Zhao; Guandong Sun; Yulin Kang; Ruizhe Gao; Ningtao Wang

cs.LGFeb 26, 2026

KMLP: A Scalable Hybrid Architecture for Web-Scale Tabular Data Modeling

Authors: Mingming Zhang, Pengfei Shi, Zhiqing Xiao, Feng Zhao, Guandong Sun, Yulin Kang, Ruizhe Gao, Ningtao Wang,

Organizations: Zhejiang University Hangzhou, China · Ant Group Hangzhou, China

Abstract

Predictive modeling on web-scale tabular data with billions of instances and hundreds of heterogeneous numerical features faces significant scalability challenges. These features exhibit anisotropy, heavy-tailed distributions, and non-stationarity, creating bottlenecks for models like Gradient Boosting Decision Trees and requiring laborious manual feature engineering. We introduce KMLP, a hybrid deep architecture integrating a shallow Kolmogorov-Arnold Network (KAN) front-end with a Gated Multilayer Perceptron (gMLP) backbone. The KAN front-end uses learnable activation functions to automatically model complex non-linear transformations for each feature, while the gMLP backbone captures high-order interactions. Experiments on public benchmarks and an industrial dataset with billions of samples show KMLP achieves state-of-the-art performance, with advantages over baselines like GBDTs increasing at larger scales, validating KMLP as a scalable deep learning paradigm for large-scale web tabular data.

Figures & tables

Figure 1. Overview of the KMLP-QTL structure. Tabular data features are first processed by QTL for fine-grained numerical representation. The data then flows through the KAN layer to manage feature heterogeneity and complex interactions, followed by stacked gMLP modules to capture deep non-linear interactions. Batch Normalization and Dropout are included in intermediate layers to enhance performance and stability.

Figure 2. Quantile Transformation with Linear interpolation

Dataset	#Train	#Valid	#Test
Tiny	200K	50K	1B
Small	2M	500K	1B
Medium	20M	5M	1B
Large	200M	50M	1B
Ultra Large	2B	50M	1B

Table 1. Statistical Information of the credit scoring dataset. It is a binary classification dataset. We prepared five training and validation sets with different data scales, while the test set is the same, with a proportion of 0.47% for overdues (label=1) in the test set.

Operator	CP		MT		CD		EG		HI		JA
Operator	AUC	KS	AUC	KS	AUC	KS	AUC	KS	AUC	KS	AUC	KS
CLR	69.18	28.68	93.35	72.15	85.16	55.18	81.84	48.74	80.03	44.49	87.03	58.66
Quantile	64.22	21.58	92.53	69.99	82.37	51.26	64.65	19.56	79.38	44.17	86.67	57.92
PLE	69.20	28.92	90.98	67.43	84.65	54.15	93.73	72.46	79.39	43.47	85.23	55.00
QTL	69.10	27.76	93.72	73.68	85.20	55.93	99.16	91.87	80.88	46.23	87.30	59.51

Table 2. Comparsion of different numerical preprocessing operators in open public datasets. We highlight the best results in bold. QTL achieved the best performance on 5 out of 6 datasets.

Model	KMLP	LightGBM
Training Resources	8 A100 GPUs	10,000 CPUs
Training Time	10 hours	24 hours
Inference Time	2 hours	2 hours

Table 3. Comparison of Model Training and Inference.We conducted a comparison of model training and inference in terms of time and resource consumption. The training was performed on a large-scale dataset (200M), while the inference was executed on an ultra-large test set (1B Test).

Operator	Valid		Test
Operator	AUC	KS	AUC	KS
CLR	95.11	76.34	94.91	75.60
Quantile	95.05	76.51	94.96	75.82
PLE	94.66	75.89	94.73	75.45
QTL	95.20	76.82	95.04	76.08

Table 4. Comparison of different numerical preprocessing operators on the medium-scale financial data.

Methods	Valid		Test
Methods	AUC	KS	AUC	KS
LightGBM	95.10	76.16	94.93	75.63
MLP	94.87	75.53	94.71	75.04
gMLP	94.99	75.99	94.86	75.41
KAN	93.15	73.26	93.99	72.86
SAINT	94.79	75.73	94.64	75.24

Table 5. Experimental results on Medium Scale Financial Data. We highlight the best results in bold, and the second-best results with underscores.

Methods	CP		MT		CD		EG		HI		JA
Methods	AUC	KS	AUC	KS	AUC	KS	AUC	KS	AUC	KS	AUC	KS
LightGBM	68.98	28.27	92.24	69.89	84.97	55.30	95.05	75.76	79.86	44.16	86.09	56.56
MLP	68.41	27.93	93.59	72.64	84.94	54.88	99.4	93.38	79.71	44.00	86.27	57.22
gMLP	68.23	27.19	93.35	72.66	84.75	54.99	99.56	95.06	79.44	43.69	85.62	56.37
KAN	67.39	25.82	92.10	68.49	84.92	54.91	98.07	85.86	79.32	43.50	86.19	56.85
SAINT	67.41	26.71	92.10	69.37	84.35	53.82	98.95	91.2	78.36	41.78	85.3	55.05

Table 6. Experimental results on Open Public Datasets. Both AUC and KS metrics indicate that the higher the value, the stronger the model’s classification capability; we highlight the best results in bold, and the second best results with underscores.

Figure 3. Data Scale Effects. The KS (left) and AUC (right) of LightGBM and KMLP with different sizes of training sets. When the scale of the training data set is relatively small, LightGBM still performs better than KMLP. However, as the scale of the data set increases, KMLP outperformed LightGBM, achieving a 1.76 improvement in the KS value.

Model Variant	Average AUC	Average KS	AUC	KS
	( Open Public Datasets)	(Open Public Datasets)	(Industrial Dataset)	(Industrial Dataset)
KMLP (FULL)	73.62	50.71	95.04	76.08
w.o. QTL	67.12 ( $\downarrow$ 6.50)	37.78 ( $\downarrow$ 12.93)	94.96 ( $\downarrow$ 0.08)	75.82 ( $\downarrow$ 0.26)
w.o. gMLP	73.19 ( $\downarrow$ 0.43)	50.01 ( $\downarrow$ 0.70)	94.71 ( $\downarrow$ 0.33)	75.04 ( $\downarrow$ 1.04)
w.o. KAN	72.99 ( $\downarrow$ 0.63)	49.99 ( $\downarrow$ 0.72)	94.86 ( $\downarrow$ 0.18)	75.41 ( $\downarrow$ 0.67)

Table 7. Ablation Study Performance. The results for the open public datasets are the average across six datasets. The values in parentheses indicate the performance drop compared to the full model, with a downward arrow

\downarrow

representing the decrease.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Abbr	Name	#samples	#features
CP	Click_prediction_small	39948	9
MT	MagicTelescope	13376	10
CD	Credit	16714	10
EG	Eeg-eye-state	14980	14
HI	Higgs	98050	28
JA	Jannis	57580	54

Appendix

Table 8. Statistical Information of open public datasets.

Dataset	Link
CP	https://www.openml.org/d/43901
MT	https://www.openml.org/d/43971
CD	https://www.openml.org/d/45024
EG	https://www.openml.org/d/1471
HI	https://www.openml.org/d/42769
JA	https://www.openml.org/d/41168

Appendix

Table 9. The source of the open public datasets.

Models	Parameter	Values
LightGBM	learning_rate	{0.1,0.01}
	num_leaves	{32, 256,512}
	subsample	{0.8,0.9,1.0}
	colsample_bytree	{0.8, 1.0}
	min_child_samples	{10, 100}
	#Iterations	{500,1000, 5000}

Appendix

Table 10. Hyperparameter Space. We use Grid Search to find the optimal parameters. Transfomer-model include FT-Transformer, TabNet and SAINT.

Tabular ML

Tabular machine learning develops models and tools for learning from structured data arranged in rows and columns.

See all 115 papers

TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning
Jul 6, 2026Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev +1Multilayer Perceptrons Ensemble Learning
1Yandex · 2HSE University
TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data
Jul 11, 2026Jiaqi Luo, Shixin XuEnsemble Learning Low-Rank Adaptation
School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China · Digital innovation research center, Duke Kunshan University, No.8 Duke Avenue, Kunshan, 215000, Jiangsu Province, China
Xiaomi-TabLDM: A Tabular Foundation Model Technical Report
Sep 3, 2026Xiaomi-TabLDM Team, :, Penghui Wang +13Synthetic Data Pretraining Tabular Foundation Models
Xiaomi

Jul 6, 2026J/K move · Enter open · S save

TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko

1Yandex · 2HSE University

Multilayer Perceptrons Ensemble Learning Tabular ML Hyperparameter Optimization

Open paper AskPDF arXiv

In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kind use the same hyperparameters for all underlying MLPs, which requires hyperparameter tuning for achieving the best performance. In this work, we introduce TabPack, an efficient MLP ensemble with strong out-of-the-box performance and reduced reliance on traditional tuning. In a single run, TabPack samples and trains many MLPs with different hyperparameters efficiently in parallel and selects ensemble members on the fly during training. Thus, TabPack only requires specifying ranges from which to sample MLP hyperparameter rather than exact hyperparameter values, which naturally demands less precision for good performance. In experiments on medium-to-large public datasets, TabPack with default settings performs on par with extensively tuned prior methods, thus substantially reducing effort and compute resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook took less time than tuning some baselines on an industry-grade GPU.

Abstract

Tabular ML

Explore similar work

TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

Xiaomi-TabLDM: A Tabular Foundation Model Technical Report