FI-KAN: Fractal Interpolation Kolmogorov-Arnold Networks

Gnankan Landry Regis N'guessan

cs.LGMar 30, 2026

FI-KAN: Fractal Interpolation Kolmogorov-Arnold Networks

Authors: Gnankan Landry Regis N'guessan

Organizations: Axiom Research Group · Dept. of Applied Mathematics and Computational Science, NM-AIST, Arusha, Tanzania · AIMS Research and Innovation Centre, Kigali, Rwanda

Abstract

Kolmogorov-Arnold Networks (KAN) employ B-spline bases on a fixed grid, providing no intrinsic multi-scale decomposition for non-smooth function approximation. We introduce Fractal Interpolation KAN (FI-KAN), which incorporates learnable fractal interpolation function (FIF) bases from iterated function system (IFS) theory into KAN. Two variants are presented: Pure FI-KAN (Barnsley, 1986) replaces B-splines entirely with FIF bases; Hybrid FI-KAN (Navascues, 2005) retains the B-spline path and adds a learnable fractal correction. The IFS contraction parameters give each edge a differentiable fractal dimension that adapts to target regularity during training. On a Holder regularity benchmark ( $α\in [0.2, 2.0]$ ), Hybrid FI-KAN outperforms KAN at every regularity level (1.3x to 33x). On fractal targets, FI-KAN achieves up to 6.3x MSE reduction over KAN, maintaining 4.7x advantage at 5 dB SNR. On non-smooth PDE solutions (scikit-fem), Hybrid FI-KAN achieves up to 79x improvement on rough-coefficient diffusion and 3.5x on L-shaped domain corner singularities. Pure FI-KAN's complementary behavior, dominating on rough targets while underperforming on smooth ones, provides controlled evidence that basis geometry must match target regularity. A fractal dimension regularizer provides interpretable complexity control whose learned values recover the true fractal dimension of each target. These results establish regularity-matched basis design as a principled strategy for neural function approximation.

Figures & tables

Figure 2: FI-KAN: learning basis geometry across the smooth-to-rough spectrum. Top row: FI-KAN combines smooth basis functions (B-spline/hat-type), fractal interpolation with learnable contraction parameters

\mathbf{d}

, and the KAN edge-function graph to produce regularity-matched edge geometry. Middle row: Smooth-to-fractal basis morphing as

d_{i}

increases from

0

0.9

. At

d_{i}=0

the basis is a smooth hat function (

\dim_{B}=1

); as

d_{i}

increases, the basis acquires progressively finer self-affine structure with

\dim_{B}>1

. Bottom row: The two FI-KAN variants. Pure FI-KAN (Barnsley framework) uses all-fractal bases, strongest on rough targets. Hybrid FI-KAN (Navascués framework) retains a spline backbone and adds a fractal correction, providing robustness across both smooth and rough regimes.

Figure 3: Edge function architecture for the three models. (a) KAN : base activation plus B-spline path. (b) Pure FI-KAN (Barnsley): replaces B-splines with fractal interpolation bases

\varphi_{m}(x;\mathbf{d})

. When

\mathbf{d}=\mathbf{0}

, the FIF bases reduce to piecewise linear hat functions. (c) Hybrid FI-KAN (Navascués): retains the B-spline path and adds a fractal correction, implementing the

\alpha

-fractal decomposition

f_{b}^{\alpha}=b+h

. When

\mathbf{d}=\mathbf{0}

and fractal weights vanish, Hybrid reduces to standard KAN.

Figure 4: Fractal basis function

\varphi_{2}(x;\mathbf{d})

(the middle basis on a 5-interval grid) as the contraction parameters

d_{i}

vary uniformly from 0 to 0.9. At

d_{i}=0

, the basis is the standard piecewise linear hat function (

\dim_{\mathrm{B}}=1

). As

d_{i}

increases, the basis acquires increasingly fine-scale fractal structure (

\dim_{\mathrm{B}}>1

) while maintaining the Kronecker property

\varphi_{2}(x_{j})=\delta_{2j}

at all grid points (black dots). Recursion depth

K=8

Figure 5: Pure FI-KAN (Barnsley framework) architecture detail. (a) Network graph: all edges carry fractal interpolation function (FIF) bases

\varphi_{m}(x;\mathbf{d})

with learnable contraction parameters; nodes perform summation. (b) Edge activation: the input

x

splits into a base SiLU path and a fractal path

\sum w_{m}^{\mathrm{frac}}\varphi_{m}(x;\mathbf{d})

, where

\mathbf{d}\in(-1,1)^{N}

controls the fractal character of the basis. When

\mathbf{d}=\mathbf{0}

, the FIF bases reduce to piecewise linear hat functions (order-1 KAN). (c) Fractal basis

\varphi_{2}(x;\mathbf{d})

at four contraction values

d_{i}\in\{0,0.3,0.6,0.9\}

, computed via

K=10

truncated Read–Bajraktarević iterations. The Kronecker property

\varphi_{2}(x_{j})=\delta_{2j}

is preserved at all

d

values (dots). (d) Box-counting dimension

\dim_{B}(\mathbf{d})

as a function of uniform

d_{i}

, with the transition surface

\sum|d_{i}|=1

marked. (e) Fractal dimension regularizer

\mathcal{R}(\mathbf{d})=(\dim_{B}(\mathbf{d})-1)^{2}

: a geometry-aware Occam’s razor that penalizes unnecessary fractal complexity.

Figure 6: Hybrid FI-KAN (Navascués framework) architecture detail. (a) Network graph: edges carry dual paths—solid blue for the B-spline path

b

(classical approximant) and dashed coral for the fractal correction

h

—implementing the

\alpha

-fractal decomposition

f_{b}^{\alpha}=b+h

. (b) Edge activation: the input splits into three paths (base SiLU, B-spline, FIF), combined into the

\alpha

-fractal output

f_{b}^{\alpha}=b+h

. When

\mathbf{d}=\mathbf{0}

and

\mathbf{w}^{\mathrm{frac}}=\mathbf{0}

, the architecture reduces exactly to standard KAN. Initialization sets

w^{\mathrm{frac}}=0

and

d^{\mathrm{raw}}\sim\mathcal{N}(0,0.01)

, so the network starts as a KAN and develops fractal structure only where the data demands it. (c)

\alpha

-fractal decomposition showing the B-spline basis

b

(blue), hybrid outputs

b+h

at increasing

d_{i}

, and the isolated fractal correction

h

(dashed). (d) Recovery property:

\|\mathbf{d}\|\to 0

implies

f_{b}^{\alpha}\to b

(KAN recovery). (e) Fractal energy ratio

\rho=\|w^{\mathrm{frac}}\|_{1}/\|w^{\mathrm{spl}}\|_{1}

across target Hölder exponents: fractal path is active (

\rho\gg 0

) on rough targets and dormant (

\rho\approx 0

) on smooth targets.

		Pure FI-KAN experiment			Hybrid FI-KAN experiment
Target	$\dim_{\mathrm{B}}$	MLP	KAN	Pure	MLP	KAN	Hybrid
polynomial	1.00	1.42e-2	2.97e-3	3.20e-1	1.44e-2	2.97e-3	1.3e-5
exp_sin	1.00	2.18e-1	3.32e-4	4.41e-1	1.68e-1	3.32e-4	7.0e-6
chirp	1.00	4.51e-1	2.95e-1	3.34e-1	4.51e-1	2.95e-1	1.57e-1
weierstrass ${}_{\text{std}}$	1.64	2.20e-1	1.23e-1	1.01e-1	2.20e-1	1.23e-1	4.38e-2
weierstrass ${}_{\text{rough}}$	1.68	4.48e-1	1.98e-1	2.15e-1	4.48e-1	1.98e-1	9.31e-2

Table 1: 1D regression benchmark (test MSE, mean

\pm

std over 5 seeds, grid size

G=8

). Pure FI-KAN parameters: MLP=487, KAN=416, FI-KAN=488. Hybrid FI-KAN parameters: MLP=841, KAN=416, FI-KAN=840. Best result per target shown in bold .

Figure 7: Hybrid FI-KAN vs. KAN on four representative targets spanning the regularity spectrum. Top row: target function (gray), KAN fit (blue dashed), and Hybrid FI-KAN fit (red) with final test MSE. Bottom row: training curves (test MSE vs. epoch, log scale) with improvement ratios. On smooth targets (polynomial, exp_sin), Hybrid FI-KAN converges to orders-of-magnitude lower MSE. On fractal targets (Weierstrass, sawtooth), the fractal correction path captures multi-scale structure that B-splines miss entirely.

$\alpha$	MLP	KAN	Hybrid FI-KAN	Improvement over KAN
0.2	7.42e-1	3.99e-1	3.02e-1	$1.3\times$
0.4	4.59e-1	1.69e-1	6.77e-2	$2.5\times$
0.6	2.94e-1	7.36e-2	1.20e-2	$6.1\times$
0.8	1.94e-1	3.14e-2	2.34e-3	$13.4\times$
1.0	1.34e-1	1.32e-2	6.30e-4	$21.0\times$
1.5	6.99e-2	1.58e-3	4.8e-5	$32.9\times$

Table 2: Hölder regularity sweep (test MSE, Hybrid FI-KAN experiment). Hybrid FI-KAN wins at every regularity level.

Figure 8: Hölder regularity sweep: test MSE as a function of Hölder exponent $\alpha$ for $f_{\alpha}(x)=|x|^{\alpha}$ . Hybrid FI-KAN achieves the lowest MSE at every regularity level, with improvement factors over KAN increasing from $1.3\times$ ( $\alpha=0.2$ ) to $33\times$ ( $\alpha=1.5$ ). The three curves are clearly separated across the entire spectrum, with the $C^{1}$ boundary ( $\alpha=1$ ) marked. Shaded regions show $\pm 1$ std over 5 seeds.

Target	KAN (864p)	Hybrid FI-KAN (840p)	Ratio	Winner
polynomial	1.58e-4	1.2e-5	$13.2\times$	FI-KAN
exp_sin	4.0e-6	6.0e-6	$0.7\times$	KAN
weierstrass ${}_{\text{std}}$	4.58e-2	4.24e-2	$1.1\times$	FI-KAN
sawtooth	4.46e-3	1.81e-3	$2.5\times$	FI-KAN

Table 3: Parameter-matched comparison (Hybrid FI-KAN: 840 params vs. KAN: 864 params). FI-KAN wins 3 of 4 targets; the one KAN win (exp_sin) is nearly tied.

	Test MSE at grid size $G$						Scaling exponent $\gamma$
Target	3	5	8	12	16	20	MLP	KAN	FI-KAN
Hybrid FI-KAN, smooth target: exp_sin
MLP	2.29e-1	2.02e-1	1.68e-1	1.08e-1	6.82e-2	4.16e-2	0.58	3.21	0.18
KAN	5.91e-2	3.10e-3	3.32e-4	9.1e-5	3.9e-5	3.3e-5
FI-KAN	9.0e-5	1.5e-5	6.0e-6	6.0e-6	1.2e-4	1.5e-5
Hybrid FI-KAN, fractal target: sawtooth ( $\dim_{\mathrm{B}}=1.5$ )

Table 4: Scaling laws: MSE vs. parameter count at varying grid sizes (Hybrid FI-KAN). Exponent

\gamma

fitted via log-log regression.

Figure 9: Scaling laws (Hybrid FI-KAN): test RMSE vs. parameter count across three targets spanning the regularity spectrum. Left (exp_sin, smooth): FI-KAN achieves low error from the smallest grid size, saturating early while KAN continues improving with more parameters. Center (Weierstrass, rough): FI-KAN achieves lower RMSE than KAN at all model sizes, with both showing positive scaling. Right (sawtooth, fractal): MLP completely stagnates. FI-KAN maintains a consistent advantage over KAN at every model size. Scaling exponents fitted over the full grid range $G\in\{3,5,8,12,16,20\}$ are reported in Table 4 . Shaded regions show $\pm 1$ std over 5 seeds.

	Weierstrass ( $\dim_{\mathrm{B}}\approx 1.64$ )		Sawtooth ( $\dim_{\mathrm{B}}=1.5$ )
SNR (dB)	KAN	Hybrid FI-KAN	KAN	Hybrid FI-KAN
100.0	1.567e-1	4.52e-2	1.139e-2	1.87e-3
40.0	1.568e-1	4.62e-2	1.140e-2	1.85e-3
30.0	1.569e-1	4.62e-2	1.141e-2	1.84e-3
20.0	1.573e-1	4.74e-2	1.144e-2	1.92e-3
10.0	1.579e-1	5.17e-2	1.155e-2	2.13e-3

Table 5: Noise robustness (test MSE on clean data after training with noisy targets). Both FI-KAN variants outperform KAN across all noise levels on fractal targets.

Figure 10: Noise robustness on fractal targets (Hybrid FI-KAN). Models trained with noisy data, evaluated on clean test data. Left (Weierstrass): FI-KAN maintains a $\sim\!3.5\times$ advantage across all SNR levels, with the gap narrowing gracefully under heavy noise. Right (sawtooth): up to $6.1\times$ advantage at clean SNR, degrading gracefully to $4.7\times$ at extreme noise ( $5$ dB). Shaded regions show $\pm 1$ std over 5 seeds.

Model	Final MSE	Std	Relative to KAN
MLP	1.294	0.008	$1.04\times$ worse
KAN	1.248	0.022	(baseline)
Pure FI-KAN	1.158	0.051	$1.1\times$ better
Hybrid FI-KAN	1.376	0.035	$1.1\times$ worse

Table 6: Continual learning: final MSE after sequential training on multiple tasks.

Figure 11: Continual learning across 5 sequential tasks (Hybrid FI-KAN). Each column shows one training phase; each row shows a different architecture’s output. Top (Data): the current task’s target function (black) with all tasks shown in gray. MLP: catastrophic forgetting collapses the output to a near-constant function. KAN: retains some structure but progressively distorts earlier tasks. FI-KAN: retains peaked structure across phases, with the fractal and spline paths providing implicit modularity. Quantitative MSE for the task sequence including the corrected Takagi–Landsberg target is reported in Table 6 .

	Polynomial ( $\dim_{\mathrm{B}}=1$ )		Weierstrass ( $\dim_{\mathrm{B}}\approx 1.64$ )
$\lambda_{\mathrm{frac}}$	Test MSE	Learned $\dim_{\mathrm{B}}$	Test MSE	Learned $\dim_{\mathrm{B}}$
0	3.74e-1	1.302	9.68e-2	1.150
$10^{-4}$	3.69e-1	1.299	9.63e-2	1.146
$10^{-3}$	3.41e-1	1.294	9.15e-2	1.137
$10^{-2}$	1.23e-1	1.231	7.28e-2	1.053
$10^{-1}$	9.27e-3	1.001	6.09e-2	1.008

Table 7: Regularization sweep (Pure FI-KAN). The regularizer drives the learned fractal dimension toward 1.0 and significantly improves performance on the smooth target.

Figure 12: Regularization sweep (Hybrid FI-KAN) on polynomial (left) and Weierstrass (right) targets. Left (polynomial): test MSE is nearly flat across $\lambda_{\mathrm{frac}}$ , confirming that the spline path already handles smooth targets effectively; the fractal regularizer has little to correct. Right (Weierstrass): stronger regularization ( $\lambda_{\mathrm{frac}}\geq 10^{-2}$ ) reduces MSE by $\sim\!20\%$ , suppressing unnecessary fractal complexity while preserving beneficial multi-scale structure. Shaded regions show $\pm 1$ std. The Hybrid variant is robust to regularization weight choice, in contrast to the Pure variant’s sensitivity ( Table 7 ).

Depth $K$	Test MSE	Std	Wall time (s)
1	3.39e-2	8.35e-4	5.8
2	2.72e-2	8.33e-4	7.1
4	3.03e-2	7.95e-4	9.6
6	4.11e-2	1.45e-3	12.0
8	5.43e-2	3.63e-3	14.4
10	6.26e-2	3.87e-3	16.9

Table 8: Fractal depth analysis (Hybrid FI-KAN, Weierstrass target).

K=2

is optimal; deeper recursion degrades performance and increases training time.

Figure 13: Impact of fractal recursion depth $K$ on Hybrid FI-KAN performance (Weierstrass target). Red (left axis): test MSE (log scale) with $\pm 1$ std error bars. Blue dashed (right axis): average training time. $K=2$ achieves optimal MSE at minimal computational cost. Deeper recursion ( $K>4$ ) monotonically degrades performance while linearly increasing training time, reflecting the optimization difficulty of propagating gradients through multiple sequential multiplicative steps.

Target	MLP	KAN	Pure FI-KAN	Hybrid FI-KAN
Ackley 2D	4.27e-1	2.56e-2	6.03e-2	1.75e-3
Weierstrass 2D	4.32e-1	1.87e-1	1.25e-1	8.95e-2

Table 9: 2D function regression (test MSE, mean

\pm

std over 5 seeds).

Figure 14: 2D function regression (Hybrid FI-KAN). Top row (Ackley): KAN smooths out the multi-modal structure near the origin (MSE 0.0204), while FI-KAN recovers the peaked landscape (MSE 0.0016), a $12.8\times$ improvement. Bottom row (2D Weierstrass): KAN captures only coarse structure (MSE 0.1577), while FI-KAN resolves finer oscillatory detail (MSE 0.0826), a $1.9\times$ improvement. Left column: target surface. Center: KAN prediction. Right: Hybrid FI-KAN prediction.

Problem	Roughness	MLP	KAN	Hybrid FI-KAN	Improv.	Source
Rough-coefficient diffusion $-\frac{d}{dx}(a(x)\frac{du}{dx})=1$ , $a(x)=e^{0.5B_{H}(x)}$
$H_{c}=0.1$	rough coeff	2.20e-3	1.26e-3	1.74e-5	$73\times$	FEM
$H_{c}=0.3$	rough coeff	3.05e-3	1.19e-3	1.50e-5	$79\times$	FEM
$H_{c}=0.5$	rough coeff	2.11e-3	1.15e-3	1.50e-5	$77\times$	FEM
$H_{c}=0.7$	smooth coeff	8.39e-4	1.11e-3	1.71e-5	$65\times$	FEM
L-shaped domain $-\Delta u=1$ , corner singularity $u\sim r^{2/3}\sin(2\theta/3)$

Table 10: Non-smooth PDE solutions (test MSE, mean

\pm

std over 5 seeds). Reference solutions computed via scikit-fem and exact spectral methods. Hybrid FI-KAN wins all 12 experiments. The rough-coefficient diffusion results (

65

–

79\times

) are the strongest in the paper.

Figure 15: L-shaped domain benchmark ( $-\Delta u=1$ , Hölder $2/3$ corner singularity). Left: FEM reference solution computed via scikit-fem (P1 elements, 6144 elements); the singularity at the re-entrant corner is visible as the peaked region near the origin. Center: Test MSE comparison (log scale, $\pm 1$ std over 5 seeds). Hybrid FI-KAN achieves $3.5\times$ lower MSE than KAN. Right: Training curves showing Hybrid FI-KAN converging to $\sim\!10^{-3}$ MSE while KAN plateaus near $3\times 10^{-3}$ .

Target	True $\alpha$	Pure $\dim_{\mathrm{B}}$	Hybrid $\dim_{\mathrm{B}}$
smooth ( $\alpha=2.0$ )	2.0	$1.128\pm 0.012$	$1.000\pm 0.000$
$C^{1}$ ( $\alpha=1.0$ )	1.0	$1.157\pm 0.012$	$1.034\pm 0.007$
Hölder 0.6	0.6	$1.194\pm 0.014$	$1.119\pm 0.007$
Hölder 0.3	0.3	$1.190\pm 0.026$	$1.152\pm 0.018$
Weierstrass	0.64	$1.144\pm 0.008$	$1.149\pm 0.004$

Table 11: Learned fractal dimension (mean

\pm

std over 5 seeds) for targets of varying regularity. The Hybrid variant produces sharper estimates, with smooth targets yielding

\dim_{\mathrm{B}}\approx 1

and rough targets yielding

\dim_{\mathrm{B}}>1.1

Figure 16: Learned fractal dimension vs. true Hölder exponent (Hybrid FI-KAN). The learned $\dim_{\mathrm{B}}$ correlates monotonically with target regularity: smooth targets ( $\alpha=2.0$ ) yield $\dim_{\mathrm{B}}=1.0$ exactly (fractal path inactive), while rough targets ( $\alpha=0.3$ , Weierstrass) yield $\dim_{\mathrm{B}}>1.1$ (fractal path active). The dotted line at $\dim_{\mathrm{B}}=1$ marks the smooth baseline. This monotonic correlation makes the learned fractal dimension a useful, interpretable proxy for target regularity.

Figure 17: Fractal parameter evolution during training (Pure FI-KAN). Top row: individual $d_{i}$ trajectories for layer 0 across three targets. Bottom row: mean $\dim_{\mathrm{B}}$ over training. Left (polynomial): the contraction parameters grow steadily, pushing $\dim_{\mathrm{B}}$ above 1.2; this is the smooth-target overfitting pathology predicted by Corollary 4.8 . Center (Weierstrass): $\dim_{\mathrm{B}}$ stays near 1.0 throughout, indicating the regularizer successfully constrains fractal complexity. Right (sawtooth): $\dim_{\mathrm{B}}$ spikes to $\sim\!1.05$ in early training (exploration phase) then settles back, suggesting an explore-then-consolidate dynamic where the network first probes fractal structure and then retains only what reduces the loss.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	Mean MSE			Params
Target	MLP	KAN	Hybrid	MLP	KAN	Hybrid
$J_{0}$ Bessel	8.62e-2	1.54e-4	2.18e-4	424	208	424
$e^{x}\sin(y)+y^{2}$	7.79e-1	4.32e-2	6.16e-2	609	312	608
$x\cdot y$	4.6e-5	8.3e-5	8.2e-5	609	312	608
10D sum	4.57e-3	1.21e-2	4.03e-4	1081	572	1080
4D composite	2.50	1.38	4.27	639	338	652

Appendix

Table 12: KAN paper toy functions [ 27 ] . These are smooth, low-dimensional targets where B-spline KAN is near-optimal. Hybrid FI-KAN wins 1 of 5; Pure FI-KAN wins 0 of 5. This is consistent with the theoretical analysis: fractal bases provide no advantage on smooth analytic targets.

Target	MLP	KAN	Pure FI-KAN
$J_{0}$ Bessel	8.62e-2	1.54e-4	7.30e-3
$e^{x}\sin(y)+y^{2}$	7.93e-1	4.32e-2	1.98
$x\cdot y$	6.1e-5	8.3e-5	8.22e-4
10D sum	8.17e-3	1.21e-2	3.05e-1
4D composite	2.75	1.38	9.59

Appendix

Table 13: KAN paper toy functions (Pure FI-KAN).

Figure 18: KAN paper toy functions (Hybrid FI-KAN). Bar chart comparison of test MSE across five canonical smooth targets. KAN wins on 3 of 5 targets where B-splines are near-optimal. The one Hybrid FI-KAN win (10D sum, $\sim\!30\times$ improvement) suggests that the fractal correction provides additional representational diversity that benefits higher-dimensional targets even when the underlying function is smooth.

Equation	MLP	KAN	Hybrid FI-KAN	Best
I.6.2 Gaussian	6.46e-1	1.32e-2	9.77e-3	FI-KAN
I.12.11 Lorentz	9.40e-4	2.15e-2	4.53e-2	MLP
I.16.6 Relativity	8.27e-2	7.35e-3	1.97e-3	FI-KAN
I.29.16 Distance	2.00e-2	2.15e-3	2.92e-3	KAN
I.30.3 Diffraction	2.10e-1	1.07e-1	5.45e-2	FI-KAN
I.50.26 Cosine	7.33e-2	1.12e-2	1.33e-2	KAN

Appendix

Table 14: Feynman physics equations (Hybrid FI-KAN). FI-KAN wins 3 of 7 on these smooth physics targets.

Figure 19: Feynman physics equations (Hybrid FI-KAN). Bar chart comparison across seven analytic physics targets. KAN wins 4 of 7, consistent with the expectation that B-splines are well-adapted to analytic targets. The FI-KAN wins (Gaussian, relativity, diffraction) occur on functions with sharp transitions or oscillatory structure that benefit from multi-scale representation.

Figure 20: Scaling laws (Pure FI-KAN): test RMSE vs. parameter count. Left (exp_sin): Pure FI-KAN exhibits a negative scaling exponent: test error increases with model size. This is the pathological behavior predicted by Corollary 4.8 : additional grid points introduce more oscillatory fractal basis functions, making the cancellation required for smooth approximation increasingly fragile. Center (Weierstrass): mild negative scaling. Right (sawtooth): negative scaling at large grid sizes, extending the smooth-approximation obstruction to fractal targets when too many uncontrolled fractal bases are introduced without the spline backbone. At small-to-moderate grid sizes, positive scaling is observed before the pathology sets in.

Figure 21: Pure FI-KAN vs. KAN on four representative targets. Top row: function fits. Bottom row: training curves with improvement ratios. The pattern is complementary to Hybrid FI-KAN ( Fig. 7 ): Pure FI-KAN loses badly on smooth targets (ratio 0.0x on polynomial, exp_sin) but wins on the sawtooth (ratio 2.3x). This asymmetry is the strongest controlled evidence for the regularity-matching hypothesis.

Figure 22: Noise robustness (Pure FI-KAN). The Pure variant also outperforms KAN on fractal targets under noise, though with smaller margins than the Hybrid. Left (Weierstrass): $\sim\!1.5\times$ advantage. Right (sawtooth): up to $2.2\times$ advantage at clean SNR.

Figure 23: Learned fractal dimension vs. true Hölder exponent (Pure FI-KAN). The monotonic correlation is present but weaker than the Hybrid variant ( Fig. 16 ): smooth targets receive $\dim_{\mathrm{B}}\approx 1.13$ rather than 1.0, because the Pure variant is forced to use some fractal structure even when inappropriate. This systematic overestimation further confirms that fractal-only bases are suboptimal for smooth targets.

Figure 24: 2D function regression (Pure FI-KAN). Pure FI-KAN loses on Ackley (smooth outside origin) but captures more fine-scale structure on the 2D Weierstrass target than KAN (MSE 0.1251 vs. 0.1871), consistent with the regularity-matching thesis.

Explore similar work

Mar 5, 2026

Multilevel Training for Kolmogorov Arnold Networks

Algorithmic speedup of training common neural architectures is made difficult by the lack of structure guaranteed by the function compositions inherent to such networks. In contrast to multilayer perceptrons (MLPs), Kolmogorov-Arnold networks (KANs) provide more structure by expanding learned activations in a specified basis. This paper exploits this structure to develop practical algorithms and theoretical insights, yielding training speedup via multilevel training for KANs. To do so, we first establish an equivalence between KANs with spline basis functions and multichannel MLPs with power ReLU activations through a linear change of basis. We then analyze how this change of basis affects the geometry of gradient-based optimization with respect to spline knots. The KANs change-of-basis motivates a multilevel training approach, where we train a sequence of KANs naturally defined through a uniform refinement of spline knots with analytic geometric interpolation operators between models. The interpolation scheme enables a ``properly nested hierarchy'' of architectures, ensuring that interpolation to a fine model preserves the progress made on coarse models, while the compact support of spline basis functions ensures complementary optimization on subsequent levels. Numerical experiments demonstrate that our multilevel training approach can achieve orders of magnitude improvement in accuracy over conventional methods to train comparable KANs or MLPs, particularly for physics informed neural networks. Finally, this work demonstrates how principled design of neural networks can lead to exploitable structure, and in this case, multilevel algorithms that can dramatically improve training performance.

Ben S. Southworth, Jonas A. Actor, Graham Harper +1

Theoretical Division, Los Alamos National Laboratory, USA. · Center for Computing Research, Sandia National Laboratories, USA.

Kolmogorov-Arnold Networks Neural Network Optimization

Ask PDF

Oct 13, 2026

Hybrid Parallel Bessel KAN Model: a Kolmogorov Arnold Network: integrated framework for developing a novel architecture to reconstruct the fused-basis approach via a parameter free quadratic interaction

The Kolmogorov-Arnold Network (KAN) challenges traditional MLPs by placing learnable activation functions on network edges. However, scaling KANs to spatial data exposes limitations in default B-Splines. While hybrid architectures mix multiple basis families on a single edge to resolve this, monolithic superposition causes geometric overlap, parameter explosion, and severe numerical ill-conditioning (

κ(G) \approx 10^{16}

).To overcome this, we propose Parallel BesselKAN, which isolates B-Spline and low-order Bessel branches into parallel processing pipelines integrated via a parameter-free quadratic interaction. This decoupled architecture acts as an efficient internal ensemble, bypassing monolithic stability bottlenecks. It compresses the parameter footprint by

35\%

while achieving

97.99\%

accuracy on MNIST. Although standard MLPs retain a hardware speed advantage for pure vision tasks, Parallel BesselKAN provides a robust, parameter-efficient framework stabilizing mixed-basis networks and paving the way for scalable spatial adaptations like Convolutional KANs.

Mintu Pathak, Meshv Patel, Mohendra Roy +1

Department of Applied Sciences, Tezpur University, Napaam, Tezpur, 784028, Assam, India. · Department of ICT, Pandit Deendayal Energy University, Raysan, Gandhinagar, 382007, Gujarat, India.

Kolmogorov-Arnold Networks Modular Neural Networks

Ask PDF

Feb 18, 2026

FEKAN: Feature-Enriched Kolmogorov-Arnold Networks

Kolmogorov-Arnold Networks (KANs) have recently emerged as a compelling alternative to multilayer perceptrons, offering enhanced interpretability via functional decomposition. However, existing KAN architectures, including spline-, wavelet-, radial-basis variants, etc., suffer from high computational cost and slow convergence, limiting scalability and practical applicability. Here, we introduce Feature-Enriched Kolmogorov-Arnold Networks (FEKAN), a simple yet effective extension that preserves all the advantages of KAN while improving computational efficiency and predictive accuracy through feature enrichment, without increasing the number of trainable parameters. By incorporating these additional features, FEKAN accelerates convergence, increases representation capacity, and substantially mitigates the computational overhead characteristic of state-of-the-art KAN architectures. We investigate FEKAN across a comprehensive set of benchmarks, including function-approximation tasks, physics-informed formulations for diverse partial differential equations (PDEs), and neural operator settings that map between input and output function spaces. For function approximation, we systematically compare FEKAN against a broad family of KAN variants, FastKAN, WavKAN, ReLUKAN, HRKAN, ChebyshevKAN, RBFKAN, and the original SplineKAN. Across all tasks, FEKAN demonstrates substantially faster convergence and consistently higher approximation accuracy than the underlying baseline architectures. We also establish the theoretical foundations for FEKAN, showing its superior representation capacity compared to KAN, which contributes to improved accuracy and efficiency.

Sidharth S. Menon, Ameya D. Jagtap

Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA.

Kolmogorov-Arnold Networks Neural Network Approximation Theory

Ask PDF