LAIR-Net: Leaky Alignment-Impulse Residual Networks for Tabular Regression

Rahul Goswami; Aryan Bhambu; Bittu Karmakar

stat.MLOct 8, 2026

LAIR-Net: Leaky Alignment-Impulse Residual Networks for Tabular Regression

Authors: Rahul Goswami, Aryan Bhambu, Bittu Karmakar

Organizations: Indian Institute of Technology Guwahati · SAFIR, Sorbonne University Abu Dhabi · Indian Institute of Technology Bombay

Abstract

Deep randomized models fix hidden-layer parameters through random initialization and learn only closed-form readouts, typically adding depth by stacking random trans formations without target-aware control of hidden-state evolution. We propose LAIR Net, the Leaky Alignment-Impulse Residual Network, which mixes a shallow learned anchor into each hidden state through a leaky residual transition. We derive a depth uniform bound on input-perturbation sensitivity and use controlled simulations to attribute gains over a randomized baseline to the anchor rather than recursion or added capacity. Benefits emerge when a nonlinear target structure is learnable at the available noise level and diminish for nearly linear targets or dominant noise. Across 23 benchmark datasets, LAIR-Net achieves the best average rank among eight randomized networks and twelve conventional models, with relative performance associated with the same nonlinear-structure and noise quantities identified in simulation.

Figures & tables

Figure 1: A single LAIR-Net block. The layer input contains the bias, the observed inputs, and the previous hidden state. A fixed random transformation produces

\widetilde{\mathbf{H}}^{(\ell)}

. The leaky transition combines this representation with

\mathbf{H}^{(\ell-1)}

using

\gamma

. The alignment impulse then combines the resulting state with

\mathbf{S}

using

\alpha

Figure 2: The LAIR-Net architecture. The SLAN produces the anchor

\mathbf{S}

, which is supplied to every randomized layer. Each LAIR-Net block applies a fixed random transformation, a leaky transition, and an alignment impulse. A ridge readout is fitted at every depth, and the resulting layerwise predictions are combined using

\mathcal{A}

Figure 3: The region in which the propagation factor of ( 11 ) is below one, for four values of the state gain

c_{1}

. The condition holds above each curve. Shading marks the region for

c_{1}=8

, the most demanding case shown. The curves are the identity

\rho=(1-\alpha)(1+\gamma(c_{1}-1))

rather than a measured quantity, so the figure describes the scope of Theorem 4.4 and not the behaviour of any fitted model.

Dataset	RVFL	dRVFL	edRVFL	edRVFL-SC	Fuzzy RVFL	RVFL-Bag	RVFL-Boost	LAIR-Net
Airfoil	7.943 $\pm$ 2.342	4.128 $\pm$ 1.060	5.589 $\pm$ 1.397	3.391 $\pm$ 0.952	17.553 $\pm$ 3.613	10.315 $\pm$ 1.574	6.716 $\pm$ 1.462	2.208 $\pm$ 0.485
Auto MPG	7.368 $\pm$ 1.936	7.096 $\pm$ 1.885	7.613 $\pm$ 1.886	7.407 $\pm$ 2.537	8.520 $\pm$ 2.138	7.160 $\pm$ 2.107	6.975 $\pm$ 1.662	6.919 $\pm$ 2.074
Autos	0.020 $\pm$ 0.006	0.021 $\pm$ 0.005	0.019 $\pm$ 0.006	0.019 $\pm$ 0.006	0.025 $\pm$ 0.007	0.017 $\pm$ 0.007	0.020 $\pm$ 0.008	0.021 $\pm$ 0.010
Breast Cancer	869.556 $\pm$ 310.974	896.296 $\pm$ 291.201	917.961 $\pm$ 268.373	901.050 $\pm$ 317.573	877.582 $\pm$ 279.775	896.660 $\pm$ 290.495	908.803 $\pm$ 307.653	871.580 $\pm$ 265.293
Challenger	0.495 $\pm$ 0.470	0.466 $\pm$ 0.490	0.975 $\pm$ 1.160	0.510 $\pm$ 0.543	0.418 $\pm$ 0.433	0.480 $\pm$ 0.415	0.329 $\pm$ 0.252	0.389 $\pm$ 0.448
Concrete	56.708 $\pm$ 10.841	56.480 $\pm$ 11.082	48.850 $\pm$ 9.629	50.208 $\pm$ 11.808	92.994 $\pm$ 13.458	55.531 $\pm$ 11.564	54.718 $\pm$ 11.492	40.199 $\pm$ 10.883

Table 1: Test mean squared error against the randomized-network family, mean

\pm

standard deviation over ten folds. Lower is better and the best entry in each row is in bold. The final rows give the average rank within this family and the Holm-adjusted Wilcoxon signed-rank

p

-value against LAIR-Net.

Figure 4: Nemenyi comparison within the randomized-network family. Each point is a mean rank with its interval, and the shaded band marks the region not separated from the best-ranked model at the $0.05$ level, with $\mathrm{CD}=2.19$ over $23$ datasets. Lower is better.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Where the advantage appears. Each curve is the median ratio of edRVFL to LAIR-Net test error over ten seeds at one noise level, with the alignment coefficient fixed at

\alpha=0.5

, and the band is a

95\%

percentile bootstrap over those seeds. The dashed line at one is the crossover. At

\sigma=0.1

the curve crosses between

\kappa=0

and

\kappa=0.25

, at

\sigma=0.5

it reaches one only at

\kappa=0.75

, and at

\sigma=1.0

it does not cross within the grid.

$\kappa$	$\sigma=0.1$	$\sigma=0.5$	$\sigma=1.0$
0.00	0.852	0.614	0.591
0.25	2.106	0.751	0.625
0.50	2.522	0.875	0.661
0.75	2.639	1.005	0.693
1.00	2.991	1.122	0.729

Appendix

Table 2: Median ratio of edRVFL to LAIR-Net test mean squared error over ten seeds, with

\alpha=0.5

fixed. A ratio above one favours LAIR-Net. Rows vary the nonlinear share

\kappa

of Equation ( 13 ); columns vary the noise level

\sigma

Figure 6: Decomposition of the edRVFL-to-LAIR-Net error ratio across the $(\kappa,\sigma)$ grid. The architecture term compares edRVFL with an $\alpha=0$ arm that keeps the residual layer input and the leaky transition; the anchor term compares that arm with the full model. Values above one favour the model on the right of each comparison.

Figure 7: Effect of selecting $\alpha$ on a held-out validation split, with all other hyperparameters fixed. Left, the edRVFL-to-LAIR-Net error ratio with $\alpha$ selected; right, the ratio of the selected model’s error to that of the best $\alpha$ in the grid. A ratio near one on the right indicates that validation recovered the best available setting.

Figure 8: What validation selection of

\alpha

recovers, and what it costs. Left, the selected

\alpha

against the best available one, with marker area proportional to the number of seeds landing on that pair and the dashed line marking exact agreement. Right, the ratio of the selected model’s error to the error at the best available

\alpha

, by nonlinear share, with

95\%

percentile bootstrap bands over seeds. Points on the dashed line on the right lose nothing by selecting.

Figure 9: Alignment selection at $N=250$ . Left, the error of the selected $\alpha$ relative to the best in the grid; right, the edRVFL-to-LAIR-Net ratio at the best $\alpha$ . Effect sizes are smaller than at $N=1000$ and the low-noise cells retain the clearest separation.

Figure 10: Relative performance of LAIR-Net against the best randomized baseline on each benchmark dataset, plotted against the nonlinearity proxy $\widehat{\kappa}$ . The vertical axis is $\log(\mathrm{MSE}_{LAIR-Net}/\mathrm{MSE}_{\text{best randomized}})$ , so points below the horizontal line at zero favour LAIR-Net. The gas dataset is marked as an off-scale point and is discussed in the text.

Dataset	Acronym	N	d	Provenance note
Airfoil Self-Noise	Airfoil	1503	5	( Brooks et al., 1989 )
Auto MPG	Auto MPG	392	7	( Quinlan, 1993a ) ; local rows reflect preprocessing
Automobile	Autos	159	25	( Kibler et al., 1989 ) ; local shape reflects encoding
Breast Cancer Prognostic	Breast Cancer	194	33	( Street et al., 1995 )
Challenger O-Ring	Challenger	23	4	( Draper, 1995 )
Concrete Strength	Concrete	1030	8	( Yeh, 1998 )

Appendix

Table 3: Dataset provenance targets for the benchmark suite. The local shape is the shape used by the preprocessed benchmark files.

Figure 11: Nemenyi comparison against conventional models, drawn as in Figure 4 , with $\mathrm{CD}=3.47$ over $23$ datasets. Lower is better.

Dataset	MLP	DT	RF	AdaBoost	GBM	SVR	KNN	LR	Ridge	Lasso	ElasticNet	LAIR-Net
Airfoil	5.315 $\pm$ 1.512	10.705 $\pm$ 2.487	7.501 $\pm$ 1.896	21.337 $\pm$ 4.179	16.551 $\pm$ 2.630	9.192 $\pm$ 1.916	3.868 $\pm$ 0.786	23.378 $\pm$ 5.534	23.378 $\pm$ 5.532	23.378 $\pm$ 5.532	23.375 $\pm$ 5.530	2.208 $\pm$ 0.485
Auto MPG	7.483 $\pm$ 1.514	12.560 $\pm$ 2.468	8.538 $\pm$ 2.757	11.207 $\pm$ 3.339	9.825 $\pm$ 2.377	6.734 $\pm$ 1.975	7.880 $\pm$ 2.700	11.266 $\pm$ 2.480	11.252 $\pm$ 2.453	11.399 $\pm$ 2.619	11.400 $\pm$ 2.525	6.919 $\pm$ 2.074
Autos	0.027 $\pm$ 0.013	0.034 $\pm$ 0.011	0.029 $\pm$ 0.015	0.031 $\pm$ 0.015	0.027 $\pm$ 0.018	0.028 $\pm$ 0.010	0.031 $\pm$ 0.022	0.027 $\pm$ 0.010	0.024 $\pm$ 0.007	0.027 $\pm$ 0.013	0.027 $\pm$ 0.012	0.021 $\pm$ 0.010
Breast Cancer	939.197 $\pm$ 333.499	1.06e3 $\pm$ 253.065	832.325 $\pm$ 208.431	839.905 $\pm$ 229.339	741.678 $\pm$ 162.746	911.205 $\pm$ 290.348	1.02e3 $\pm$ 236.385	944.958 $\pm$ 364.476	896.263 $\pm$ 308.580	914.506 $\pm$ 276.342	874.418 $\pm$ 257.062	871.580 $\pm$ 265.293
Challenger	0.375 $\pm$ 0.436	0.407 $\pm$ 0.365	0.407 $\pm$ 0.370	0.480 $\pm$ 0.556	0.415 $\pm$ 0.358	0.317 $\pm$ 0.314	0.428 $\pm$ 0.397	0.365 $\pm$ 0.408	0.359 $\pm$ 0.391	0.414 $\pm$ 0.451	0.395 $\pm$ 0.432	0.389 $\pm$ 0.448
Concrete	43.674 $\pm$ 12.915	74.354 $\pm$ 26.260	46.642 $\pm$ 9.922	79.306 $\pm$ 15.171	65.111 $\pm$ 10.004	42.791 $\pm$ 9.954	58.822 $\pm$ 11.054	110.468 $\pm$ 12.515	110.461 $\pm$ 12.455	110.905 $\pm$ 11.686	110.962 $\pm$ 11.709	40.199 $\pm$ 10.883

Appendix

Table 4: Test mean squared error against conventional neural, tree-based, kernel, instance-based and linear models, mean

\pm

standard deviation over ten folds. Lower is better and the best entry in each row is in bold. The final rows give the average rank within this family and the Holm-adjusted Wilcoxon signed-rank

p

-value against LAIR-Net.

Model	Avg. rank	$p$	$p_{\mathrm{Holm}}$
LAIR-Net	4.22	–	–
MLP	6.70	0.0078	0.0389
edRVFL-SC	7.17	0.0037	0.0221
RVFL-Boost	7.35	0.0010	0.0106
RVFL-Bag	8.00	0.0022	0.0157
edRVFL	8.35	0.0001	0.0022

Appendix

Table 5: Average rank and paired Wilcoxon signed-rank tests against LAIR-Net with all nineteen models pooled, over the

23

datasets. Lower rank is better.

p

-values are Holm-adjusted over the eighteen comparisons. Ranks here are not comparable with the within-family tables, since each model is ranked against a different set.

Model	Average rank	Dataset wins
LAIR-Net	2.565	3
LAIR-SLANInit	3.043	7
LAIR-Ortho	3.783	4
LAIR-OrthoBMA	3.913	4
LAIR-NoAlign	4.652	3
LAIR-BestState	4.696	2

Appendix

Table 6: Average ranks and per-dataset wins within the variant family. Lower average rank and higher win count are better.

Shared by all randomized models, including LAIR-Net
Hyperparameter	Search space
$m$	$\{10,20,\ldots,200\}$
Input scale	$\mathcal{U}(0,1)$
$\lambda$	$\log\mathcal{U}(10^{-8},10^{2})$
Activation	$\{\mathrm{ReLU},\tanh\}$
LAIR-Net-specific

Appendix

Table 7: Hyperparameter search spaces for the randomized models.

Model	Parameter	Range
MLP	hidden units	$\{8,16,32,64\}$
	activation	$\{\text{identity},\text{logistic},\text{tanh},\text{ReLU}\}$
	learning-rate schedule	$\{\text{constant},\text{invscaling},\text{adaptive}\}$
	initial learning rate	$\{10^{-1},10^{-2},10^{-3},10^{-4}\}$
Decision tree	criterion	$\{\text{squared},\text{absolute},\text{Friedman}\}$
	splitter	$\{\text{best},\text{random}\}$

Appendix

Table 8: Hyperparameter search space for the conventional models. Ridge and lasso regression are fitted by their own internal cross-validation over the regularisation path rather than by the outer search, and linear regression has no hyperparameter, so none of the three enters this table.

Explore similar work

Random Recursive Models
Sep 30, 2026Jama Hussein Mohamud, Mirco RavanelliEfficient Neural Network Inference LLM Reasoning
Mila – Quebec AI Institute · Université de Montréal · Concordia University
Progressive Approximation in Deep Residual Networks: Theory and Validation
Apr 27, 2026Wei Wang, Xiao-Yong Wei, Qing LiEfficient Neural Network Inference Residual Learning
Department of Computing, the Hong Kong Polytechnic University.
HalfNet: Randomized Neural Networks with Learned Subspace Geometry
Jun 3, 2026Ethem AlpaydinNeural Representation Geometry Random Feature Methods
Independent Researcher Istanbul, Turkey

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Random Recursive Models

Jama Hussein Mohamud, Mirco Ravanelli

Mila – Quebec AI Institute · Université de Montréal · Concordia University

Efficient Neural Network Inference LLM Reasoning Recursive Language Models

Open paper AskPDF arXiv

Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly apply one learned transformation or a prescribed sequence of transformations, restricting computation to a fixed layer order. We introduce the Random Recursive Model (RRM), which maintains a pool of

L

learned layers and performs

T

recursive steps by sampling one layer independently with replacement for each example and step. This enables flexible layer reuse while retaining the parameter efficiency of recurrence. We evaluate RRM on challenging reasoning tasks, where it matches or exceeds the baselines, often with 50-75 % fewer parameters. RRM can vary its depth at inference, including beyond that seen during training, without retraining or adding parameters, improving tasks that benefit from deeper iterative computation. RRM also supports Monte Carlo inference and probabilistic test-time scaling, both of which improve performance without retraining. These insights may open new directions in neural network architecture design.

Abstract

Explore similar work

Random Recursive Models

Progressive Approximation in Deep Residual Networks: Theory and Validation

HalfNet: Randomized Neural Networks with Learned Subspace Geometry