No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation

Taha Bouhsine

cs.LGFeb 22, 2026

No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation

Authors: Taha Bouhsine

Organizations: Azetta AI.

Abstract

We introduce the yat-product, a kernel operator combining quadratic alignment with inverse-square proximity. We prove it is a Mercer kernel, analytic, Lipschitz on bounded domains, and self-regularizing, admitting a unique RKHS embedding. Neural Matter Networks (NMNs) use yat-product as the sole non-linearity, replacing conventional linear-activation-normalization blocks with a single geometrically-grounded operation. This architectural simplification preserves universal approximation while shifting normalization into the kernel itself via the denominator, rather than relying on separate normalization layers. Empirically, NMN-based classifiers match linear baselines on MNIST while exhibiting bounded prototype evolution and superposition robustness. In language modeling, Aether-GPT2 achieves lower validation loss than GPT-2 with a comparable parameter budget while using yat-based attention and MLP blocks. Our framework unifies kernel learning, gradient stability, and information geometry, establishing NMNs as a principled alternative to conventional neural architectures.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of the gradient field and vector field for Dot Product, Euclidean Distance, \tifinaghfont ⵟ -product, and Cosine Similarity (from left to right). The heatmaps illustrate how the \tifinaghfont ⵟ -product, unlike traditional similarity measures, creates a potential well around the weight vector $\mathbf{w}$ , reflecting both alignment and proximity.

Figure 2: Potential well induced by a single \tifinaghfont ⵟ -neuron. High response occurs near

\mathbf{w}

when inputs are both aligned and close, in contrast to unbounded linear hyperplanes.

Panel of Figure 3 — Figure 3: (a) \tifinaghfont ⵟ -product response as a function of angle $\theta$ between $\mathbf{w}$ and $\mathbf{x}$ , demonstrating orthogonality sensitivity ( $K_{\text{\tifinaghfont ⵟ}}=0$ at $\theta=\pi/2$ ). (b) Self-regulation property: response converges to $\|\mathbf{w}\|^{2}\cos^{2}\theta$ as radius $k\to\infty$ , ensuring bounded outputs.

Panel of Figure 4 — Figure 4: Comparison of standard Transformer block (left) and Aether block (right). In Aether-GPT2, \tifinaghfont ⵟ -multi-head attention replaces scaled dot-product attention, and an NMN layer replaces Linear+GeLU, eliminating activation functions and all LayerNorm operations.

Figure 5: Decision boundaries in 2D: linear (left) creates unbounded half-spaces; \tifinaghfont ⵟ -product (right) forms localized regions around prototypes (stars).

	Acc.	$\Delta\\|\mathbf{w}\\|$	$\alpha$
Linear	92.08%	$+$ 13.8%	–
\tifinaghfont ⵟ	92.38%	$-$ 4.5%	$1\to 2.68$

Table 1: MNIST results (10-neuron classifier).

Figure 6: Learned MNIST prototypes from a linear classifier (top) and an \tifinaghfont ⵟ -product classifier (bottom). We visualize the learned class prototypes as 28 $\times$ 28 weight images using a shared diverging colormap and robust limits for fair comparison across classes and methods. \tifinaghfont ⵟ -product prototypes tend to exhibit more localized, structured strokes, consistent with the kernel’s potential-well geometry.

Figure 7: (a) Effect of regularization parameter

\varepsilon

on \tifinaghfont ⵟ -product shape: smaller

\varepsilon

sharpens the response peak. (b) Input perturbation robustness: bounded change in output for bounded input perturbations, demonstrating Lipschitz continuity.

	P@k			PSP@k
Method	P@1	P@3	P@5	PSP@1	PSP@3	PSP@5
\tifinaghfont ⵟ -product	0.6465	0.5114	0.4271	1.1542	0.9664	0.8443
Inner Product	0.6235	0.5041	0.4125	1.2117	1.0215	0.8587

Table 2: Extreme classification results on Eurlex-4K. \tifinaghfont ⵟ -product improves top-

k

precision (P@1–5), while the inner product baseline achieves slightly higher propensity-scored precision (PSP@3, PSP@5).

	GPT-2	Aether
Activation	GeLU	\tifinaghfont ⵟ
LayerNorm	Yes	No
Train Loss (Final)	4.1969	4.0479
Val Loss (Final)	4.6417	4.5747
Val improvement	–	1.45%
Throughput (tok/s)	comparable	comparable

Table 3: GPT-2 vs Aether-GPT2 (2.5B tokens, identical hyperparameters).

Panel of Figure 8 — Figure 8: (a) Eigenvalue spectrum of \tifinaghfont ⵟ -NTK vs ReLU-NTK: faster decay ( $O(k^{-2/d})$ ) indicates smoother function spaces. (b) Reliability diagram comparing GPT-2 and Aether-GPT2: Aether exhibits better calibration, with predictions closer to the diagonal.

Figure 9: Training and validation loss curves (W&B). Solid: training; dashed: validation.

Configuration	Val Loss	Status
Aether (no LayerNorm)	4.5747	Converged
Aether + Pre-LN	$>$ 10	Diverged
Aether + Post-LN	$>$ 10	Diverged

Table 4: Ablation: LayerNorm impact on Aether-GPT2.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Algebraic squashing functions for non-negative \tifinaghfont ⵟ -product scores. These offer bounded, monotonic mappings without exponential saturation.

Component	Linear	\tifinaghfont ⵟ -Product
Forward main	$2Bnd$	$2Bnd$
Forward aux	$Bn$	$Bd+nd+5Bn$
Backward main	$4Bnd$	$4Bnd$
Backward aux	$2Bn$	$6Bn+Bd+nd$
Total	$\Theta(Bnd)$	$\Theta(Bnd)$
Overhead	$1$	$1+\frac{1}{2n}+\frac{1}{2B}+\frac{2}{d}$

Appendix

Table 5: Complexity comparison. Overhead

<5\%

for

d,n\geq 64

B\geq 16

Method	FLOPs	Relative
Linear + ReLU	$2d+1$	$1.00\times$
Linear + GELU	$2d+15$	$\approx 1.03\times$
\tifinaghfont ⵟ -product (naive)	$5d+1$	$\approx 2.5\times$
\tifinaghfont ⵟ -product (optimized)	$4d+4$	$\approx 2.0\times$

Appendix

Table 6: Per-neuron FLOPs. Optimized variant avoids redundant norm computation.

Architecture
Parameter	GPT-2	Aether
Total Params	124M	$\sim$ 124M
Embed Params	39M	39M
Non-Embed Params	85M	$\sim$ 85M
Embed Dim	768	768
MLP Hidden Dim	3072	3072

Appendix

Table 7: GPT-2 vs Aether-GPT2 detailed comparison (2.5B tokens from FineWeb).

Kernel Methods

Kernel methods use similarity functions and reproducing kernel spaces to model nonlinear relationships, quantify uncertainty, and compare data.

See all 230 papers

A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance
May 5, 2026Taha BouhsineReproducing Kernel Hilbert Spaces Kernel Methods
Azetta AI
Product Interaction: An Algebraic Formalism for Deep Learning Architectures
Jan 31, 2026Haonan Dong, Chun-Wun Cheng, Angelica I. Aviles-Rivero
Tsinghua University · University of Cambridge · YMSC, Tsinghua University
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Jun 8, 2026Hu Tan, Kuo Gai, Shihua ZhangNeural Representation Geometry Classification
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China; and Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Hangzhou 310024, China

May 5, 2026J/K move · Enter open · S save

A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance

Taha Bouhsine

Azetta AI

Reproducing Kernel Hilbert Spaces Kernel Methods

Open paper AskPDF arXiv

We introduce the Yat kernel

k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon},\qquad b\ge 0,\ \varepsilon>0,

a rational hidden-unit primitive whose units are Mercer sections over a shared input/weight space. For

b\ge 0

the kernel is PSD; for

b>0

it dominates a scaled inverse-multiquadric (IMQ) in the Loewner order, yielding fixed-kernel universality, characteristicness, and strict positive definiteness on every compact domain. The polynomial numerator opens nonradial alignment channels absent from finite IMQ expansions, witnessed by the directional far-field trace

T_\infty g_\varepsilon(\cdot;\mathbf{w},b)(\mathbf{u})=(\mathbf{u}^\top\mathbf{w})^2

. Algebraically, a second finite difference in the bias recovers any IMQ atom from three positive-bias Yat atoms exactly, sharp at three atoms in every dimension at exact pointwise equality. A trained shared-

(b,\varepsilon)

Yat layer is therefore a finite learned-center expansion in a fixed universal characteristic RKHS, with closed-form norm

\boldsymbolα^\top\mathbf{K}\boldsymbolα

and explicit diagonal

(\|\mathbf{x}\|^2+b)^2/\varepsilon

driving a Rademacher generalization bound.

Abstract

Kernel Methods

Explore similar work

A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance

Product Interaction: An Algebraic Formalism for Deep Learning Architectures

Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic