Spectral Path Regression: Directional Chebyshev Harmonics for Interpretable Tabular Learning

Milo Coombs

cs.LGApr 5, 2026

Spectral Path Regression: Directional Chebyshev Harmonics for Interpretable Tabular Learning

Authors: Milo Coombs

Abstract

Classical approximation bases such as Chebyshev polynomials provide principled and interpretable representations, but their multivariate tensor-product constructions scale exponentially with dimension and impose axis-aligned structure that is poorly matched to real tabular data. We address this by replacing tensorised oscillations with directional harmonic modes of the form $\cos(\mathbf{m}^{\top}\arccos(\mathbf{x}))$ , which organise multivariate structure by direction in angular space rather than by coordinate index. This representation yields a discrete spectral regression model in which complexity is controlled by selecting a small number of structured frequency vectors (spectral paths), and training reduces to a single closed-form ridge solve with no iterative optimisation. Experiments on standard continuous-feature tabular regression benchmarks show that the resulting models achieve accuracy competitive with strong nonlinear baselines while remaining compact, computationally efficient, and explicitly interpretable through analytic expressions of learned feature interactions.

Figures & tables

Panel of Figure 1 — Figure 1: Geometric interpretation of Chebyshev polynomials. The input $x\in[-1,1]$ is lifted to an angle $\theta=\arccos(x)$ on the upper semicircle, rotated by a factor $n$ , and projected onto the horizontal axis, yielding $T_{n}(x)=\cos(n\theta)$ .

Panel of Figure 2 — Figure 2: In a multivariate Chebyshev series, each variable is lifted to an angular coordinate, rotated independently by its frequency, and projected onto its coordinate axis. The resulting scalar projections combine multiplicatively, producing a rectangular “shadow” on the base plane whose area equals $\cos(m_{1}\theta_{1})\times\cos(m_{2}\theta_{2})$ .

Panel of Figure 3 — Figure 3: The new basis: directional harmonics defined by projections onto frequency vectors.

Source	Dataset	$N$	$D$	Spectral Paths	Ridge	MLP	XGBoost
UCI	Concrete Strength	1030	8	0.893	0.628	0.864	0.918
UCI	Energy (Heating)	768	8	0.998	0.907	0.995	0.998
UCI	Energy (Cooling)	768	8	0.979	0.888	0.981	0.992
UCI	Superconductivity	21263	81	0.838	0.737	0.891	0.902
UCI	Wine Quality	4898	11	0.352	0.259	0.353	0.423
UCI	Phishing Websites	11055	30	0.807	0.700	0.859	0.848

Table 1: Test

R^{2}

performance across datasets. Best result per dataset in bold.

Figure 4: Training and validation performance of the discrete spectral model on the UCI Concrete dataset as a function of the number of paths $\mathcal{M}$ . The model achieves stable generalisation after around $\mathcal{M}=9$ , where validation $R^{2}$ plateaus and NRMSE σ reaches a minimum.

Figure 5: Feature importance on the Concrete dataset computed from normalised analytic sensitivities.

Figure 6: Test performance as a function of the ridge regularisation parameter $\lambda$ for the spectral path model on the UCI Concrete Compressive Strength dataset. The horizontal axis shows $\lambda$ on a logarithmic scale, while the vertical axis reports test $R^{2}$ . Performance remains relatively stable across several orders of magnitude in $\lambda$ , indicating limited sensitivity to precise regularisation strength and a broad region of near-optimal performance.

Figure 7: Box plot of test $R^{2}$ scores over 10 random train-validation-test splits. The spectral path model shows consistent performance across splits, with no extreme outliers, suggesting robustness to data partitioning and greedy path selection.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Source	Dataset	Spectral Paths	Ridge	MLP	XGBoost
UCI	Concrete	1.0	$\mathbf{3.0\times 10^{-3}}$	28	0.53
UCI	Energy (Heating)	0.4	$\mathbf{2.9\times 10^{-3}}$	28	0.27
UCI	Energy (Cooling)	0.4	$\mathbf{7.5\times 10^{-3}}$	26	0.57
UCI	Superconductivity	6.3	$\mathbf{2.2\times 10^{-2}}$	120	3.7
UCI	Wine Quality	4.4	$\mathbf{7.2\times 10^{-3}}$	16	3.4
UCI	Phishing Websites	5.5	$\mathbf{7.1\times 10^{-3}}$	34	0.37

Appendix

Table 2: Total experiment time in seconds across datasets. Best result per dataset in bold.

Tabular Regression

Tabular regression predicts continuous outcomes from structured rows of features for tasks such as pricing, forecasting, and scientific measurement.

See all 35 papers

TabNSM: Neural Sparse Mixer for Tabular Regression
Aug 18, 2026Ali Eslamian, Qiang ChengTabular Regression Tabular Representation Learning
Department of Computer Science University of Kentucky Lexington, KY 40506 · Department of Computer Science Institute for Biomedical Informatics University of Kentucky Lexington, KY 40506
Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression
Feb 25, 2026Luciano Gerber, Huw LloydTabular Regression Radial Basis Function Networks
Department of Computing and Mathematics, Manchester Metropolitan University, Dalton Building, Chester Street, Manchester M1 5GD, United Kingdom
Ordinary Least Squares as an Attention Mechanism
Apr 13, 2025Philippe Goulet CoulombeTransformer Attention Linear Regression
Département des Sciences Économiques, Université du Québec à Montréal

Aug 18, 2026J/K move · Enter open · S save

TabNSM: Neural Sparse Mixer for Tabular Regression

Ali Eslamian, Qiang Cheng

Department of Computer Science University of Kentucky Lexington, KY 40506 · Department of Computer Science Institute for Biomedical Informatics University of Kentucky Lexington, KY 40506

Tabular Regression Tabular Representation Learning Feature Interaction Modeling Sparse Neural Networks

Open paper AskPDF arXiv

Large-scale, high-dimensional tabular regression remains challenging: tree-based models are robust but lack end-to-end representation learning, while deep models enable flexible feature learning but often incur costly interaction modeling and sensitivity to noisy or redundant features. We propose TabNSM, a scalable regression framework that extends our earlier sparse-attention and mixer architectures. At its core, the Adaptive Sparse Interaction Module (ASIM) integrates foreground feature discovery, sparse local interaction encoding, and Feature-Token Mixing, providing near-linear complexity under fixed sparse configurations. For regression, TabNSM introduces three complementary components: a Multi-Stage Regression Head for progressive prediction refinement; GridLoss, an ordinal-aware soft-binning objective that incorporates target structure into representation learning; and RISE (Reweighted Instance Sampling by Error), a difficulty-aware sampling strategy based on loss-quantile bins. Across nine real-world regression benchmarks, TabNSM delivers strong predictive performance and practical scalability, with particularly consistent gains on high-dimensional and heterogeneous datasets. These results demonstrate that selective interaction modeling, structured regression supervision, and difficulty-aware sampling provide an effective and scalable approach to deep tabular regression.

Abstract

Tabular Regression

Explore similar work

TabNSM: Neural Sparse Mixer for Tabular Regression

Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression

Ordinary Least Squares as an Attention Mechanism