Norm-optimal model order reduction in reproducing kernel Hilbert spaces

Tamás Dózsa; Andrea Angino; Rolf Krause; Matthias Voigt

math.NAOct 20, 2026

Norm-optimal model order reduction in reproducing kernel Hilbert spaces

Authors: Tamás Dózsa, Andrea Angino, Rolf Krause, Matthias Voigt

Abstract

We consider the problem of model order reduction (MOR) in reproducing kernel Hilbert spaces (RKHSs). We prove generalizations of well-known necessary interpolatory conditions for norm-optimal reduced-order models, originally stated for rational functions in $H_2$ Hardy-Hilbert spaces. We construct an alternative to the iterative rational Krylov (IRKA) algorithm for RKHSs with differentiable kernels to obtain norm-optimal reduced-order models. Then, we introduce generalized Malmquist-Takenaka (GMT) functions that can be used to define orthonormal bases in an arbitrary RKHS. Using GMT functions, we propose unitary transformations between separable RKHSs. This allows us to extend our optimality conditions and subsequent algorithms to MOR problems posed in RKHSs with non-differentiable kernels. The potential and versatility of the proposed MOR framework is demonstrated in several numerical experiments. We perform norm-optimal MOR on various machine learning models such as support vector machines and deep ReLU networks. In addition, we show how the proposed MOR framework can be used to solve seemingly unrelated problems. In particular, we provide an example for fitting high degree polynomials to data and also obtain a FEM-like method for solving partial differential equations (PDEs) with an optimal kernel basis.

Figures & tables

Panel of Figure 1 — Figure 1: The proposed model order reduction pipeline for reproducing kernel Hilbert spaces.

Domains and function spaces
$\mathbb{N}$	Natural numbers including $0$ .
$\Omega\subseteq\mathbb{C}$ , $\mathcal{H}$	Domain of the functions and the original reproducing kernel Hilbert space of functions $\Omega\to\mathbb{C}$ .
$\widehat{\Omega}$ , $\widehat{\mathcal{H}}$	The companion domain and companion reproducing kernel Hilbert space
$\mathbb{D}$ , $\mathbb{C}^{+}$	Open unit complex disk, open right complex half–plane
$H_{2}(\mathbb{D})$ , $H_{2}(\mathbb{C}^{+})$	Hardy–Hilbert spaces on $\mathbb{D}$ , $\mathbb{C}^{+}$
Kernels

Table 1: Summary of the core notation used throughout the paper.

Panel of Figure 2 — Figure 2: The generalized Malmquist–Takenaka transformation pipeline. The full–order model $\mathbf{G}$ and its companion counterpart $\widehat{\mathbf{G}}$ are shown in gray/black; the companion reduced–order model $\widehat{\mathbf{G}}_{r}$ obtained by model order reduction in blue. Inverting the generalized Malmquist–Takenaka transformation directly gives the weak reduced–order model $\mathbf{G}_{r}^{\mathrm{weak}}\in W_{r}$ (green), while projecting $\mathbf{G}$ onto the subspace $V_{r}^{\widetilde{\boldsymbol{\lambda}}}$ determined by the kernel centers of $\widehat{\mathbf{G}}_{r}$ (via $\gamma^{-1}$ ) gives the projected reduced–order model $\mathbf{G}_{r}^{\mathrm{econ}}$ (purple).

Figure 3: Polynomial fits obtained with the different approximation strategies for

r=4

and

n=100

Figure 4: Pointwise square fitting errors obtained with the different approximation strategies for

r=4

and

n=100

	$r=2$	$r=3$	$r=4$	$r=5$	$r=2$	$r=3$	$r=4$	$r=5$
Model	$\mathcal{E}^{\mathfrak{D}}_{\mathrm{MSE}}$				$\mathcal{E}_{\xi}^{r}$
$\mathbf{G}_{r}^{\mathrm{dir}}$	$9.687$	$0.134$	$0.093$	$0.060$	$3.713$	$1.046$	$1.132$	$0.962$
$\mathbf{G}_{r}^{\mathrm{weak}}$	$1.640$	$\mathbf{0.038}$	$\mathbf{2.6{\cdot}10^{-4}}$	$\mathbf{6.5{\cdot}10^{-5}}$	$2.408$	$\mathbf{0.417}$	$\mathbf{0.190}$	$\mathbf{0.113}$
$\mathbf{G}_{r}^{\mathrm{econ}}$	$\mathbf{1.568}$	$0.076$	$2.0{\cdot}10^{-3}$	$1.2{\cdot}10^{-3}$	$\mathbf{2.362}$	$0.438$	$0.255$	$0.189$
$\mathbf{G}_{r}^{\mathrm{LSQ}}$	$9.784$	$9.20$	$3.528$	$3.485$	$10.527$	$10.627$	$17.631$	$17.703$
$\mathbf{G}_{2r}^{\mathrm{LSQ}}$	$3.528$	$1.361$	$0.513$	$0.184$	$17.631$	$47.712$	$157.146$	$532.577$

Table 2: mean squared error and reproducing kernel Hilbert space errors for the polynomial fitting application (

n=100

). The best reduced–order models per reduced order are indicated by boldface font.

$r$	$\mathbf{G}_{r}^{\mathrm{dir}}$	$\max\left\{\mathbf{G}_{r}^{\mathrm{weak}},\mathbf{G}_{r}^{\mathrm{econ}}\right\}$	$\mathbf{G}_{r}^{\mathrm{LSQ}}$	$\mathbf{G}_{2r}^{\mathrm{LSQ}}$
2	$6.10$	$0.53$	$0.02$	$3.05\cdot 10^{-4}$
3	$6.15$	$0.28$	$4.62\cdot 10^{-3}$	$2.96\cdot 10^{-4}$
4	$6.76$	$0.24$	$4.21\cdot 10^{-3}$	$2.52\cdot 10^{-4}$
5	$6.33$	$0.28$	$3.99\cdot 10^{-3}$	$2.72\cdot 10^{-4}$

Table 3: Recorded runtimes (seconds) of the considered MOR schemes.

\mathbf{G}_{r}^{\mathrm{LSQ}}

and

\mathbf{G}_{2r}^{\mathrm{LSQ}}

are obtained with MATLAB’s polyfit routine.

Data	Metric	$\mathbf{G}$	$\mathbf{G}_{30}^{\mathrm{dir}}$	$\mathbf{G}_{100}^{\mathrm{dir}}$	$\mathbf{G}_{200}^{\mathrm{dir}}$	$\mathbf{G}_{30}^{\mathrm{econ}}$	$\mathbf{G}_{100}^{\mathrm{econ}}$	$\mathbf{G}_{200}^{\mathrm{econ}}$
Global	$\mathcal{E}^{r}_{\xi,\mathrm{rel}}$	$0$	$0.4416$	$0.1578$	$0.1322$	$0.4621$	$0.0868$	$0.0257$
Training	$\mathcal{E}_{\mathrm{pt}}$	$0$	$3.0816$	$0.7373$	$0.4306$	$0.6019$	$0.5163$	$0.0571$
	Acc. $(\%)$	$80.50$	$59.65$	$80.48$	$80.48$	$81.56$	$80.48$	$80.50$
	$\Cert_{\mathrm{unif}}$ $(\%)$	–	$0.00$	$1.39$	$3.94$	$0.00$	$14.19$	$92.57$
	$\Cert_{\mathrm{proj}}$ $(\%)$	–	$71.80$	$99.57$	$99.74$	$86.53$	$99.93$	$100.00$
	$\Cert_{\mathrm{true}}$ $(\%)$	–	$75.07$	$99.83$	$99.86$	$92.79$	$99.95$	$100.00$

Table 4: Approximation errors, classification performance, and consistency–certificate applicability for the full–order model, direct full–order models, and projected generalized Malmquist–Takenaka–reduced–order models.

Figure 5: Consistency certificates for the projected generalized Malmquist–Takenaka–reduced–order models with

r=30,100,200

, evaluated on

\mathcal{Z}_{\mathrm{tr}}

(left) and

\mathcal{Z}_{\mathrm{te}}

(right).

Metric	$\mathrm{CR}\,(\%)$	$\mathbf{G}_{r}^{\mathrm{econ}}$	$\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}$	$\mathbf{G}_{r}^{\mathrm{EIM}}$	$\mathbf{F}^{\mathrm{prun}}$	$\mathcal{I}_{r}^{\mathrm{econ}}$	$\mathcal{I}_{r}^{\mathrm{econ\text{-}EIM}}$
$\mathcal{E}^{r}_{\xi,\mathrm{rel}}$	27.27	0.08018	0.03824	0.03824	–	$0.3216$	$-1.1856\times 10^{-8}$
	32.62	0.07276	0.03701	0.03701	–	$0.2936$	$-1.2626\times 10^{-8}$
	37.97	0.07132	0.03215	0.03215	–	$0.3461$	$-1.6617\times 10^{-8}$
	43.32	0.07111	0.03156	0.03156	–	$0.3528$	$-1.7235\times 10^{-8}$
	48.66	0.07109	0.02752	0.02752	–	$0.4121$	$-2.2664\times 10^{-8}$
	54.01	0.07053	0.02659	0.02659	–	$0.4236$	$-2.4156\times 10^{-8}$

Table 5: Proposed and baseline reduced–order models for model order reduction on a ReLU network.

Figure 6: Approximation of the FOM neural network

\mathbf{F}

with the considered kernel expansion based ROMs. Here,

\mathbf{F}_{r}^{\mathrm{econ}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{econ}}(t)+\beta(t)

\mathbf{F}_{r}^{\mathrm{EIM}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{EIM}}(t)+\beta(t)

, and

\mathbf{F}_{r}^{\mathrm{econ\text{-}EIM}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}(t)+\beta(t)

are given in ( 7.14 ). In this figure,

r=70

is considered.

$r$	$\mathbf{G}_{r}^{\mathrm{econ}}$	$\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}$	$\mathbf{G}_{r}^{\mathrm{EIM}}$
50	$1.59\times 10^{-29}$	$1.43\times 10^{-34}$	$1.54\times 10^{-18}$
60	$2.72\times 10^{-36}$	$3.70\times 10^{-41}$	$1.24\times 10^{-21}$
70	$1.30\times 10^{-43}$	$4.23\times 10^{-47}$	$1.74\times 10^{-24}$
80	$4.81\times 10^{-57}$	$6.98\times 10^{-53}$	$2.82\times 10^{-27}$
90	$1.75\times 10^{-62}$	$2.22\times 10^{-58}$	$5.92\times 10^{-30}$
100	$9.49\times 10^{-68}$	$1.73\times 10^{-64}$	$7.35\times 10^{-33}$

Table 6: Maximum equilibrium function magnitude

\left\|\left[\widehat{\mathcal{D}}\left(\gamma\big(\widetilde{\lambda}_{0}\big),\ldots,\gamma\big(\widetilde{\lambda}_{r-1}\big)\right)\right]^{\top}\right\|_{\infty}

in the companion reproducing kernel Hilbert space

H_{2}^{\mathbb{I}}

for different reduced–order models. These results were computed using MATLAB’s variable precision arithmetic using

100

digits.

Figure 7: Solution to the 4th–order Euler beam equation using the proposed adaptive kernel mesh based approach and FEM for

r=5

editflow	\cellcolor red
received	\cellcolor red
revised
accepted	\cellcolor red
communicated (by)	\cellcolor red

Personal data (Author 1)

Table 14

Reduced-Order Modeling

Reduced-order modeling builds compact approximations of complex systems to make simulation, prediction, and control more efficient.

See all 97 papers

Feb 27, 2026

KROM: Kernelized Reduced Order Modeling

We propose KROM, a kernel-based reduced-order framework for fast solution of nonlinear partial differential equations. KROM formulates PDE solution as a minimum-norm (Gaussian-process) recovery problem in an RKHS, and accelerates the resulting kernel solves by sparsifying the precision matrix via sparse Cholesky factorization. A central ingredient is an empirical kernel constructed from a snapshot library of PDE solutions (generated under varying forcings, initial data, boundary data, or parameters). This snapshot-driven kernel adapts to problem-specific structure -- boundary behavior, oscillations, nonsmooth features, linear constraints, conservation and dissipation laws -- thereby reducing the dependence on hand-tuned stationary kernels. The resulting method yields an implicit reduced model: after sparsification, only a localized subset of effective degrees of freedom is used online. We report numerical results for semilinear elliptic equations, discontinuous-coefficient Darcy flow, viscous Burgers, Allen--Cahn, and two-dimensional Navier--Stokes, showing that empirical kernels can match or outperform Matérn baselines, especially in nonsmooth regimes. We also provide error bounds that separate discretization effects, snapshot-space approximation error, and sparse-Cholesky approximation error.

Aras Bacho, Jonghyeon Lee, Houman Owhadi

Department of Computing and Mathematical Sciences, Caltech, CA, USA.

PDE Surrogate Modeling Reduced-Order Modeling Reproducing Kernel Hilbert Spaces

Ask PDF

Oct 22, 2026

Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms

Random projections (RP) offer an effective method for reducing computational and storage overhead while preserving the geometric structure of data. However, existing studies have primarily focused on the optimal generalization performance of specific kernel-regularized algorithms with RP in the well-specified setting and under restrictive conditions. In this paper, we provide a comprehensive and improved analysis of the generalization performance of RP-based spectral algorithms under general conditions \change{without sacrificing statistical efficiency}. By leveraging the embedding property of the reproducing kernel Hilbert space (RKHS) and a refined analysis of the operator similarity, we establish optimal learning rates in Sobolev norms that match the minimax lower bounds up to logarithmic factors. For two common implementations of RP, randomized sketches and Nyström sub-sampling (uniform or leverage-based), we show that the projection dimension required for optimality is proportional to the average or maximal effective dimension, yielding a significant reduction in computational cost while maintaining the statistical efficiency. Our results do not rely on the uniform boundedness assumption on the regression function and are valid across a wide range of source conditions, i.e.,

s> α_0-1/β

, where

s,β

, and

α_0

denote the smoothness index, capacity index, and the embedding index, respectively. In the benign case when

α_0=1/β

, the optimality holds for all

s\in (0,2τ]

with

τ

and

γ

denoting the qualification index and norm index. Experimental results demonstrate both the theoretical validity and the practical effectiveness of the proposed method.

Caixing Wang, Yue Wang

School of Statistics and Data Science, Southeast University, Nanjing, 210000, Jiangsu, China · Faculty of Business for Science and Technology, School of Management, University of Science and Technology of China, Hefei, 230000, Anhui, China

Generalization Bounds Spectral Methods Randomized Sketching Nyström Method

Ask PDF

Apr 27, 2026

Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation

We study online regression with the square loss in a reproducing kernel Hilbert space under a dynamic regret criterion. The learner is compared with a time-varying comparator sequence, and the bounds depend on its path length in the RKHS norm. The proposed method transfers the finite-dimensional discounted Vovk--Azoury--Warmuth approach of Jacobsen & Cutkosky (2024) to the RKHS setting by means of finite-dimensional subspace approximations. For a fixed subspace, we run a VAW-based ensemble of discounted VAW forecasters over a geometric grid of discount factors. The additional approximation error is controlled by the uniform projection error of kernel sections. We then introduce a general orthogonal truncation method: starting from a feature expansion of the kernel, we construct the associated RKHS by introducing an inner product that makes the feature functions orthonormal, and then use the spans of the first basis functions as finite-dimensional approximation spaces. The resulting subspace reduction is applied to several approximation schemes. Explicit feature expansions yield fast-regime bounds for Gaussian and analytic dot-product kernels. Mercer truncations provide a spectral approximation method and lead to dynamic regret bounds in fast and slow regimes, depending on the eigenvalue decay. Finally, we study subspaces spanned by kernel sections and apply this construction to Matérn kernels.

Dmitry B. Rokhlin, Georgiy A. Karapetyants

Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University

Sequential Decision Making Reproducing Kernel Hilbert Spaces Kernel Methods

Ask PDF

Abstract

Reduced-Order Modeling

Explore similar work

KROM: Kernelized Reduced Order Modeling

Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms

Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation