Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

K S Sesh Kumar

cs.LGAug 26, 2026

Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

Authors: K S Sesh Kumar

Organizations: Brevan Howard Centre for Financial Analysis Imperial Business School

Abstract

Kolmogorov-Arnold Networks (KANs) replace fixed activations in deep architectures with learnable univariate edge functions, making the choice of edge parametrisation central. Existing variants rely on fixed bases such as splines, polynomials, or Fourier features, which impose a function-space geometry before data are observed. We introduce geometry-constrained KANs, a family of edge activations derived from Banach duality maps in which the geometry itself is learned through a scalar exponent $p > 1$ per edge. This exponent controls the qualitative response: sub-Euclidean values produce sharp, threshold-like behaviour reminiscent of the $\ell_1$ (LASSO) geometry, $p = 2$ recovers the linear regime, and larger values produce flatter responses near the origin. Across 50 symbolic-regression targets ( $40$ from the AI Feynman benchmark plus $10$ synthetic stress tests), geometry-constrained KANs match or beat every fixed-basis baseline on median NRMSE (Banach-KAN $0.030$ , tying Chebyshev and improving on splines); on average rank Banach-KAN is best on the $18$ -equation core ( $2.00$ ) and statistically tied with the strongest spline on the full benchmark ( $2.32$ vs. $2.34$ ). The clearest gains appear under measurement noise: as $σ$ grows from $0$ to $1$ , $\ell^p$ -KAN degrades only $3.7\times$ -- below even a cross-validated spline ( $\approx 11\times$ ) -- while an unregularised spline degrades $21.6\times$ ; Banach-KAN degrades $8.8\times$ , comparable to a tuned spline but far more stable than the unregularised one. Banach-KAN also takes the most per-equation wins in the small-sample regime, with fixed-basis models catching up only as the training set grows. Learned exponents provide an interpretable, relative signal: at a fixed initialisation they reveal a consistent, target-dependent geometric ordering across equation families and input dimensions.

Figures & tables

Figure 1: Fixed bases lock in geometry. All columns use a matched budget of $21$ trainable parameters, and each target is fit from $N=2000$ samples under label noise $\sigma=0.03$ , so every panel is a noisy fit rather than a noiseless interpolation. Different bases fail differently on the same targets, so no single fixed geometry is uniformly appropriate across both; the geometry-adaptive $\ell^{p}$ -KAN adapts to each rather than committing to one. A budget sweep across $\{13,21,41,125\}$ parameters is reported in Appendix M .

Figure 2: Geometry induces activation. Convex potentials define the geometry, their gradients act as activations, and the unit balls illustrate how the geometry varies.

Figure 3: (a) The two duality maps compared: Orlicz duality (

\tanh

) saturates at

\pm 1

, providing bounded expressivity; ball duality (

J_{2.5}

) grows as

|z|^{1.5}

, providing adaptive power-law geometry. (b) Banach-KAN edge functions span diverse shapes by combining both: saturating curves (Orlicz-led), power-law growth (ball-led), non-monotone functions (mixed), asymmetric responses (high

p

Method	Specialisation	Edge form $\phi(x)$	Free parameters per edge
linear	$p=2$ in $\ell^{p}$	$a(wx+b)+d$	$(a,w,b,d)$
tanh	$a_{2}=0$	$a\tanh(wx+b)+d$	$(a,w,b,d)$
lp_fixed	$p$ fixed	$aJ_{p}(wx+b)+d$	$(a,w,b,d)$
lp	learned $p$ in $\ell^{p}$	$aJ_{p}(wx+b)+d$	$(a,w,b,d,\theta)$
banach	full Orlicz $\oplus$ $\ell^{p}$	$a_{1}\tanh(w_{1}x+b_{1})+a_{2}J_{p}(w_{2}x+b_{2})+d$	$(a_{1},a_{2},w_{1},w_{2},b_{1},b_{2},d,\theta)$

Table 1: Members of the geometry-constrained KAN family. Top: special-value members of Banach-KAN. Bottom: standalone variants.

Figure 4: The

C^{1}

duality map approximates the exact map and regularises its gradient. Top: the exact map

J_{p}(z)=\operatorname{sign}(z)|z|^{p-1}

(dashed), the additive-

\varepsilon

smoothing

\operatorname{sign}(z)(|z|+\varepsilon)^{p-1}

, and the adopted

C^{1}

map

z(z^{2}+\varepsilon^{2})^{(p-2)/2}

coincide away from the origin; near

z=0

the additive-

\varepsilon

map has a jump of

2\varepsilon^{p-1}

for

p<2

, whereas the

C^{1}

map passes through the origin smoothly. Bottom: the gradient—the exact

(p-1)|z|^{p-2}

is singular at

z=0

for

p<2

, and the

C^{1}

map replaces it with a bounded, smooth peak of height

\varepsilon^{p-2}

, the quantity the gradient clip acts on. A visible

\varepsilon=0.1

is shown for legibility; the experiments use

\varepsilon=10^{-8}

, for which the curves are indistinguishable outside an

\varepsilon

-neighbourhood of

0

Realisation of $J_{p}$	$\ell^{p}$ degr.	Banach degr.
additive $\varepsilon$ -map (paper)	3.7 $\times$	8.8 $\times$
Adam re-run	3.5 $\times$	8.9 $\times$
$C^{1}$ map (adopted)	3.6 $\times$	8.7 $\times$

Table 2: The

C^{1}

map reproduces the

\varepsilon

-map: noise-degradation ratio (

\sigma{=}0\to 1

) under three realisations of

J_{p}

; clean medians agree within

0.5\%

, so the differentiability correction changes no result-table entry.

Model	Median NRMSE	Wins	Rank	Median NRMSE	Wins	Rank
	Full $50$ equations			Core $18$ equations
Spline G=3	0.037	21	2.34	0.037	7	2.39
Tanh	0.067	0	5.18	0.089	0	5.28
Banach	0.030	11	2.32	0.030	6	2.00
$\ell^{p}$	0.060	0	4.78	0.068	0	4.72
MLP	0.049	8	3.80	0.048	4	3.89

Table 3: Clean-data baseline (

n=500

\sigma=0

). Two column groups: “Full

50

equations” uses the complete benchmark, “Core

18

equations” uses the subset re-used in the noise and small-sample sweeps.

	Noise robustness ( $n=500$ , median NRMSE)					Small-sample wins out of $18$ ( $\sigma=0$ )
Model	$\sigma\!=\!0$	$0.1$	$0.3$	$0.5$	$1.0$	$n\!=\!50$	$100$	$200$	$500$	$1000$
Spline G=3	0.037	0.077	0.197	0.346	0.789	3	2	4	2	6
Spline CV	0.036	0.054	0.132	0.187	0.403	0	3	1	6	1
Tanh	0.089	0.094	0.112	0.185	0.283	0	0	0	0	0
Banach	0.030	0.041	0.098	0.145	0.263	8	7	5	5	3
$\ell^{p}$	0.068	0.077	0.108	0.138	0.249	3	2	0	0	0

Table 4: Left: median NRMSE under noise at

n=500

(best per column in bold ). Right: number of equations won at

\sigma=0

for each sample size (out of

18

5

seeds.

Model (params/edge)	$\sigma{=}0$	$0.3$	$0.5$	$1.0$	Degr.	Rank ( $\sigma$ -sweep)
Banach ( $8$ )	0.029	0.095	0.149	0.259	8.9 $\times$	2.12
$\ell^{p}$ ( $5$ )	0.071	0.110	0.140	0.250	3.5 $\times$	4.03
RBF ( $7$ )	0.031	0.110	0.193	0.411	13.2 $\times$	3.51
Cheby ( $7$ )	0.029	0.119	0.204	0.435	14.9 $\times$	4.29
Jacobi ( $7$ )	0.034	0.118	0.198	0.453	13.5 $\times$	4.51
Wavelet ( $6$ )	0.047	0.137	0.229	0.489	10.4 $\times$	6.40

Table 5: Median NRMSE under noise, unified Adam (Core

18

5

seeds); “Degr.” is the

\sigma{=}0\to 1

ratio. Right column: mean rank over the

\sigma

-sweep

\{0,0.1,0.3,0.5,1.0\}

across the eight edge models.

Edge (params/edge)	Clean NRMSE	Wins/ $18$	Rank (7-model)	Small-sample/ $90$
Banach ( $8$ )	0.029	15	1.17	60
mixture ( $7$ )	0.052	3	2.39	16
rational/Padé ( $8$ )	0.066	0	4.22	2
$\ell^{p}$ ( $5$ )	0.071	0	4.33	6
learnable-power ( $5$ )	0.086	0	4.89	2
$\tanh$ ( $4$ )	0.089	0	5.00	2

Table 6: Parametric-activation controls, clean data, unified Adam (Core

18

5

seeds). “Rank” is the mean rank within the seven-model set; “Small-sample” is wins out of

90

. The Banach–mixture clean gap is

-0.023

(

95\%

[-0.039,-0.001]

) by paired equation-bootstrap.

Model	NRMSE	Wins/ $18$
Banach	0.35	7
$\ell^{p}$	0.42	6
spline	0.73	0
Cheby	0.80	3
RBF	0.86	0
$\tanh$	1.10	2

Table 7: Extrapolation (train inner

60\%

, test out-of-range): median NRMSE and per-model wins out of

18

Dataset	Geometry (readout)	ReLU	Gap
CIFAR-10	Banach $0.881$	$0.911$	$-3.0$ pt
CIFAR-100	Banach $0.598$	$0.634$	$-3.6$ pt
STL-10	$0.44$ – $0.49$	$0.79$	$-30$ pt

Table 8: Test accuracy, matched parameters; final-layer geometry only, ReLU backbone.

Dataset	Domain	$d$	$n$	Spline	Tanh	Banach	$\ell^{p}$	MLP
712_chscase_geyser1	geophysics	2	222	1.02	0.46	0.48	0.46	0.48
678_vis_environmental	environmental	3	111	1.80	0.86	0.99	0.91	0.92
1027_ESL	ordinal	4	488	0.52	0.37	0.40	0.41	0.40
1030_ERA	ordinal	4	1000	0.81	0.81	0.81	0.81	0.81
210_cloud	meteorology	5	108	0.69	0.41	0.51	0.41	0.35
230_machine_cpu	computing	6	209	0.76	0.27	0.31	0.33	0.27

Table 9: Per-dataset median NRMSE (

5

seeds, lower is better, best per row in bold ) on

12

PMLB regression datasets. The geometry-constrained family wins

10/12

datasets; Spline G=3 wins zero. Last two rows aggregate over the table.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	I.12.1 NRMSE	I.34.14 NRMSE	Time/config
lr $=3\times 10^{-3}$ , 5k steps	0.0072	0.0197	55 s
lr $=1\times 10^{-3}$ , 5k steps	0.0138	0.0317	54 s
lr $=3\times 10^{-4}$ , 5k steps	0.0333	0.0637	56 s
lr $=1\times 10^{-2}$ , 10k steps	0.0075	0.0098	93 s

Appendix

Table 10: Preregistered Spline–Adam sweep on two held-out equations (5 seeds each). Step count and learning rate for the main run ( bold ) selected before observing the 18-equation results.

	Tanh	$\ell^{p}$	Banach	Cheby	Spline G=3	MLP
2D	60	75	120	105	304	201
3D	64	80	128	112	314	251
4D	80	100	160	140	380	2851
5D	72	90	144	126	348	2901
6D	84	105	168	147	400	2951

Appendix

Table 11: Parameter counts per model by input dimension.

	Spline G=3	Banach	MLP	Cheby
2D ( $7$ eq.)	7	0	0	0
3D ( $17$ eq.)	8	4	1	4
4D ( $13$ eq.)	2	4	3	4
5D ( $11$ eq.)	3	3	4	1
6D ( $2$ eq.)	1	0	0	1

Appendix

Table 12: Optimal geometry by input dimension (

50

equations, clean data). Entries are equations won.

Model	Mean	Median	Min	Max
Spline G=3	$107.1$	$99.6$	$56.3$	$183.0$
Banach	$28.9$	$19.6$	$15.0$	$83.1$
Cheby	$23.4$	$16.8$	$10.4$	$61.1$
$\ell^{p}$	$19.2$	$12.0$	$9.3$	$65.6$
Tanh	$15.7$	$13.6$	$7.3$	$38.1$
MLP	$9.9$	$8.8$	$7.7$	$25.0$

Appendix

Table 13: Per-training wall-clock (seconds) on a single

22

-core node, Core 18 equations, single seed,

5{,}000

Adam steps. Statistics over

18

equations per model.

$\sigma$	$\lambda=0$	$\lambda=10^{-3}$	$\lambda=10^{-2}$	$\lambda=10^{-1}$	Best $\lambda$
0.0	0.035	0.047	0.128	0.793	$\lambda=0$
0.1	0.077	0.054	0.101	0.763	$10^{-3}$
0.3	0.196	0.167	0.132	0.550	$10^{-2}$
0.5	0.342	0.360	0.187	0.530	$10^{-2}$
1.0	0.788	0.732	0.403	0.467	$10^{-2}$

Appendix

Table 14: Median NRMSE across 18 equations for each

\lambda

. Best

\lambda

per row in bold .

$\sigma$	Best Reg. Spline	Banach-KAN	$\ell^{p}$ -KAN	Tanh-KAN
0.0	11	7	0	0
0.1	4	10	3	1
0.3	3	7	6	2
0.5	3	4	10	1
1.0	0	2	13	3

Appendix

Table 15: Per-equation wins against oracle-tuned regularised spline (18 equations).

	2D (3 eq.)	3D (6 eq.)	4D (4 eq.)	5D (4 eq.)	6D (1 eq.)	Overall
Full Banach-KAN	0.0042	0.0220	0.0775	0.0573	0.0776	0.0298
Banach ${}_{a_{2}=0}$ (tanh-only)	0.0116	0.0294	0.1411	0.1319	0.0975	0.0556
Ratio (tanh-only / full)	$2.74\times$	$1.33\times$	$1.82\times$	$2.30\times$	$1.26\times$	$1.91\times$

Appendix

Table 16:

J_{p}

branch ablation: Banach-KAN with

a_{2}

frozen at 0 (tanh-only) vs. full Banach-KAN, by input dimension (18 core equations, clean data, median NRMSE over 5 seeds).

Equation	Type	$d$	Spline G=3	Tanh	Banach	$\ell^{p}$
I.12.1 Coulomb	poly	2	0.0014	0.0093	0.0041	0.0061
I.14.4 Spring	poly	2	0.0014	0.0095	0.0042	0.0073
I.25.13 Capacitor	rat	2	0.0015	0.0128	0.0082	0.0172
I.34.14 Rel. Doppler	pow	3	0.0091	0.0446	0.0167	0.0321
I.14.3 Grav. PE	poly	3	0.0184	0.0372	0.0153	0.0296
I.18.12 Torque	trig	3	0.0275	0.1291	0.0322	0.1509

Appendix

Table 17: Clean-data NRMSE per equation (18 core equations).

$\sigma$	G=3	G=5	G=10	G=20	Tanh	Banach	$\ell^{p}$	MLP
0.0	0.037	0.064	0.104	0.122	0.089	0.030	0.068	0.048
0.1	0.077	0.085	0.144	0.170	0.094	0.041	0.077	0.073
0.3	0.197	0.245	0.332	0.282	0.112	0.098	0.108	0.148
0.5	0.346	0.374	0.414	0.408	0.185	0.145	0.138	0.200
1.0	0.789	0.805	0.767	0.772	0.283	0.263	0.249	0.381

Appendix

Table 18: Aggregate noise-by-model median NRMSE (18 equations, 5 seeds). Best per row in bold .

$\sigma$	G=3	G=5	G=10	Tanh	Banach	$\ell^{p}$	MLP
0.0	6	2	1	0	6	0	3
0.1	1	0	0	1	11	2	3
0.3	0	0	0	2	8	6	2
0.5	0	0	0	1	5	10	2
1.0	0	0	0	3	2	12	1

Appendix

Table 19: Number of equations (out of 18) where each model achieves the lowest median NRMSE.

Model	NRMSE at $\sigma=0$	NRMSE at $\sigma=1.0$	Degradation
Spline G=3	0.037	0.789	$\mathbf{21.6\times}$
Spline G=5	0.064	0.805	$12.6\times$
Spline G=10	0.104	0.767	$7.4\times$
Spline G=20	0.122	0.772	$6.3\times$
Spline CV ( $G{=}3$ )	0.036	0.403	$11.2\times$
MLP	0.048	0.381	$8.0\times$

Appendix

Table 20: Degradation factor (NRMSE at

\sigma=1.0

/ NRMSE at

\sigma=0

$n$	G=3	G=5	G=10	G=20	Tanh	Banach	$\ell^{p}$	MLP
50	0.246	0.316	0.425	0.670	0.207	0.102	0.190	0.179
100	0.087	0.203	0.348	0.346	0.150	0.075	0.100	0.096
200	0.045	0.098	0.186	0.253	0.121	0.037	0.069	0.069
500	0.037	0.058	0.106	0.123	0.089	0.030	0.068	0.048
1000	0.029	0.046	0.074	0.088	0.058	0.020	0.059	0.036

Appendix

Table 21: Aggregate small-sample median NRMSE (

\sigma=0

, 18 core equations, 5 seeds). Best per row in bold .

$n$	G=3	G=5	G=10	Tanh	Banach	$\ell^{p}$	MLP
50	3	0	0	0	8	3	4
100	5	0	0	0	7	2	4
200	5	2	0	1	7	0	3
500	5	2	1	0	7	0	3
1000	7	0	1	0	6	0	4

Appendix

Table 22: Number of equations (out of 18) where each model wins per sample size at

\sigma=0

(across

8

models including spline grids G=5,10,20).

#	Equation	Type	$d$	$n=50$	$n=500$	$n=1000$
1	I.12.1 Coulomb	poly	2	$\ell^{p}$	G=5	G=3
2	I.14.4 Spring	poly	2	G=3	G=3	G=3
3	I.25.13 Capacitor	rat	2	Banach	G=5	G=3
4	I.34.14 Rel. Doppler	pow	3	Banach	G=3	G=3
5	I.14.3 Grav. PE	poly	3	Banach	Banach	Banach
6	I.18.12 Torque	trig	3	Banach	G=3	Banach

Appendix

Table 23: Per-equation best model in the clean small-sample regime (

\sigma=0

#	Equation	Type	$d$	Clean	Noisy
1	I.12.1	poly	2	B	L
2	I.14.4	poly	2	B	B
3	I.25.13	rat	2	B	T
4	I.34.14	pow	3	B	L
5	I.14.3	poly	3	B	B
6	I.18.12	trig	3	B	B

Appendix

Table 24: Dissection winner per equation by regime. B = Banach, L =

\ell^{p}

, T = Tanh; columns are clean (

\sigma=0

n=500

) and noisy (

\sigma=0.3

n=500

Figure 5: Distribution of learned exponents

p

across

4{,}465

Banach-KAN edges, broken out by input dimension.

Dimension	2D	3D	4D	5D	6D
Edge count	525	1,275	1,040	1,320	305
Fraction with $p<2$ (%)	11	15	16	29	34
Fraction with $p<1.5$ (%)	1.2	1.4	1.8	4.0	5.4

Appendix

Table 25: Fraction of edges with

p<2

and edge counts by dimension.

Equation type	Mean $p$	Std	Range
Trigonometric	2.73	1.49	1.14–22.2
Polynomial	2.64	0.86	1.34–9.0
Power	2.61	0.80	1.37–8.2
Compositional	2.63	0.92	1.04–11.1
Rational	2.50	0.85	1.09–10.3
Exponential	2.49	2.01	1.13–53.2

Appendix

Table 26: Learned exponent

p

aggregated by equation type (Banach-KAN, 50 equations, clean data).

$\sigma$	$\ell^{p}$ mean $p$	$\ell^{p}$ std	Banach mean $p$	Banach std
0.0	2.50	0.15	2.56	0.15
0.1	2.49	0.16	2.59	0.14
0.3	2.52	0.18	2.63	0.14
0.5	2.51	0.18	2.64	0.16
1.0	2.61	0.19	2.74	0.19

Appendix

Table 27: Mean learned

p

across 18 equations at each noise level (with std across equations).

#	Equation	Type	$d$	$p\!=\!1.5$	$p\!=\!2.0$	$p\!=\!2.5$	$p\!=\!3.0$	$p\!=\!4.0$	Learned
1	I.12.1	poly	2	0.081	0.266	0.025	0.048	0.046	0.018
2	I.14.4	poly	2	0.080	0.351	0.031	0.026	0.044	0.022
3	I.25.13	rat	2	0.083	0.405	0.097	0.070	0.076	0.042
4	I.34.14	pow	3	0.102	0.213	0.097	0.088	0.109	0.055
5	I.14.3	poly	3	0.120	0.376	0.116	0.099	0.109	0.061
6	I.18.12	trig	3	0.233	0.714	0.182	0.187	0.171	0.157

Appendix

Table 28:

\ell^{p}

-KAN fixed-

p

vs learned-

p

NRMSE per equation (clean,

n=500

). Best per row in bold .

Figure 6: Two regularity-mismatch failure modes for fixed bases (parameter-matched pedagogical models; train

[0.01,2.0]

n=500

, extrapolate on

(2.0,2.8]

). Left : pendulum correction

R(\theta_{0})

with a log singularity at

\theta_{0}=\pi

. The spline (blue) extrapolates flat past the training interval; the

J_{p}

-stack (red) extrapolates with the correct power-law shape. Extrapolation NRMSE is reported in the legend; absolute values exceed those in Table 30 because of the smaller per-edge parameter budget used here for visual clarity, but the relative ordering matches the full experiments. Right : algebraic cusp

f(x)=\operatorname{sign}(x)|x|^{0.7}

. A single

J_{p}

edge with learned

p\approx 1.7

recovers the cusp; a same-budget spline cannot.

$n$	Spline G=3	Tanh	Banach	$\ell^{p}$	MLP
50	0.0066	0.0053	0.0032	0.0025	0.0136
100	0.0031	0.0043	0.0029	0.0018	0.0130
200	0.0018	0.0038	0.0031	0.0013	0.0090
500	0.0015	0.0044	0.0024	0.0012	0.0068
1000	0.0015	0.0043	0.0023	0.0014	0.0063

Appendix

Table 29: 1D pendulum interpolation NRMSE (median over 5 seeds).

$n$	Spline G=3	Tanh	Banach	$\ell^{p}$	MLP
200	0.675	1.139	0.364	0.394	0.907
500	0.708	1.132	0.356	0.390	0.868
1000	0.690	1.130	0.348	0.394	0.881

Appendix

Table 30: 1D pendulum extrapolation NRMSE (train on

[0.01,2.0]

, test on

[2.0,2.8]

#	ID	Name	Type	$d$	Formula	Core
1	I.12.1	Coulomb (linear)	poly	2	$\mu\,N$	$\checkmark$
2	I.14.4	Spring energy	poly	2	$\tfrac{1}{2}kx^{2}$	$\checkmark$
3	I.25.13	Capacitor voltage	rat	2	$q/C$	$\checkmark$
4	I.34.14	Relativistic Doppler	pow	3	$\omega\sqrt{(1+v/c)/(1-v/c)}$	$\checkmark$
5	I.14.3	Gravitational PE	poly	3	$mgz$	$\checkmark$
6	I.18.12	Torque	trig	3	$rF\sin\theta$	$\checkmark$

Appendix

Table 31: All 50 benchmark equations. Entries with IDs I.#.# , II.#.# , III.#.# are taken from [ 21 ] ; entries prefixed F. are synthetic stress-test targets we constructed for this work. “Core” denotes membership in the 18-equation subset.

Basis	$K{=}12$	$K{=}21$	$K{=}42$	$K{=}126$
geometry-adaptive edge	0.059	0.059	0.059	0.059
Haar wavelet	0.064	0.063	0.063	0.062
B-spline	0.201	0.173	0.109	0.078
RBF	0.186	0.163	0.108	0.079
Fourier	0.267	0.211	0.156	0.103

Appendix

Table 32: Heaviside step, NRMSE by parameter budget (seed

1729

). The geometry-adaptive edge is budget-invariant at the noise floor; smooth bases need budget to catch up.

Model	$\sigma{=}0$	$\sigma{=}0.3$	$\sigma{=}1.0$	Degr.	Small-sample/ $90$
Banach independent ( $8$ /edge)	0.029	0.095	0.259	8.9 $\times$	55
Banach tied ( $6$ /edge)	0.034	0.096	0.274	8.0 $\times$	17
Banach tied, $C^{1}$ ( $6$ /edge)	0.034	0.092	0.269	7.8 $\times$	18

Appendix

Table 33: Tied- vs. independent-affine (median NRMSE); “Degr.” is the

\sigma{=}0\to 1

ratio, “Small-sample” wins out of

90

Explore similar work

May 7, 2026

Geometric Kolmogorov--Arnold Network (GeoKAN)

We introduce Geometric Kolmogorov--Arnold Networks (GeoKANs), a family of geometry-aware KAN-type models in which approximation is carried out in learned, geometry-adapted coordinates rather than in fixed Euclidean input coordinates. GeoKAN achieves this by learning a diagonal Riemannian metric that warps the input before basis expansion and feature mixing. The learned metric provides a geometric inductive bias through local length scaling and volume distortion, and in physics-informed settings it also affects the differential structure seen by the model. Within this framework, we develop three main variants, namely GeoKAN-NNMetric, GeoKAN-

γ

, and LM-KAN. For LM-KAN, we further consider three basis-specific versions, LM-KAN-RBF, LM-KAN-Wav, and LM-KAN-Fourier. These variants allow us to study geometry-aware KAN models both as general function approximators and as surrogates in physics-informed learning. By stretching regions with rapid variation and compressing smoother regions, GeoKAN reallocates representational resolution in a task-dependent manner, allowing the model to place capacity where it is most needed. As a result, GeoKAN is well suited to sharp, stiff, localized, and strongly non-uniform regimes arising in scientific machine learning and differential-equation problems.

Abhijit Sen, Bikram Keshari Parida, Giridas Maiti +2

Department of Physics and Engineering Physics, Tulane University, New Orleans, Louisiana 70118, USA. · Institute of Applied Geosciences, Karlsruhe Institute of Technology, Karlsruhe 76131, Germany.

Neural Surrogate Modeling Metric Learning Kolmogorov-Arnold Networks

Ask PDF

May 4, 2026

KANs need curvature: penalties for compositional smoothness

Kolmogorov-Arnold networks (KANs) offer a potent combination of accuracy and interpretability, thanks to their compositions of learnable univariate activation functions. However, the activations of well-fitting KANs tend to exhibit pathologically high-curvature oscillations, making them difficult to interpret, and standard regularization penalties do not prevent this. Here we derive a basis-agnostic curvature penalty and show that penalized models can maintain accuracy while achieving substantially smoother activations. Accounting for how function composition shapes curvature, we prove an upper bound on the full model's curvature relative to the curvature penalty, and use this to motivate richer forms of penalties. Scientific machine learning is increasingly bottlenecked by the trade-off between accuracy and interpretability. Results such as ours that improve interpretability without sacrificing accuracy will further strengthen KANs as a practical tool for both prediction and insight.

James Bagrow

Mathematics & Statistics, University of Vermont, Burlington, VT, United States · Vermont Complex Systems Center, University of Vermont, Burlington, VT, United States

Neural Network Activation Functions Neural Network Interpretability Kolmogorov-Arnold Networks

Ask PDF

Jul 17, 2026

Kolmogorov--Arnold Networks for Small Language Models

Kolmogorov--Arnold Networks (KANs) replace fixed node activations with learned one-dimensional edge functions, offering an explicit interface for interpretation and a possible alternative to transformer feed-forward networks. We test these claims separately. In a six-layer, 10M-parameter B-spline KAN, we reconstruct all 884,736 feed-forward edges: 87.8% exceed (NLS>0.1) and 0.4% are inactive. Pruning the lowest-activity 20--25% causes negligible loss increase, although structured MLP neuron pruning tolerates comparable sparsity. The audit replicates on BabyLM, but grid-size sweeps show that near-total fPCA compression and high closed-form-fit coverage are properties of the low-capacity grid-2 basis, not universal KAN behavior. For replacement, we evaluate MLP, SwiGLU, grouped Chebyshev, and rational GR-KAN networks on BabyLM. The KAN-family and gated variants improve validation loss over the GELU MLP, but this ordering does not transfer to standardized benchmarks: across ten seeds and 59,875 BLiMP pairs, accuracies span 62.4--63.1%, EWoK remains at chance, and a (+0.7)-point GR-KAN effect on BLiMP reverses on the supplement. Larger tests are also cautionary: parameter-matched MLPEdge underperforms the MLP on Wikitext-103, and 286M-parameter GR-KAN remains below a SwiGLU ClimbMix baseline after stabilization. Thus, small-basis KANs provide a practical, corpus-transferable interface for auditing learned scalar transformations, but the tested replacements show no consistent benchmark, quality, or latency advantage over strong MLP baselines.

Felippe Alves, Renato Vicente

TELUS Digital Research Hub – CIAAM Institute of Mathematics, Statistics and Computer Science University of São Paulo

Small Language Models Neural Network Interpretability Kolmogorov-Arnold Networks Feedforward Neural Networks

Ask PDF