math.NAOct 20, 2026

Norm-optimal model order reduction in reproducing kernel Hilbert spaces

Authors: Tamás Dózsa, Andrea Angino, Rolf Krause, Matthias Voigt

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We consider the problem of model order reduction (MOR) in reproducing kernel Hilbert spaces (RKHSs). We prove generalizations of well-known necessary interpolatory conditions for norm-optimal reduced-order models, originally stated for rational functions in H2H_2 Hardy-Hilbert spaces. We construct an alternative to the iterative rational Krylov (IRKA) algorithm for RKHSs with differentiable kernels to obtain norm-optimal reduced-order models. Then, we introduce generalized Malmquist-Takenaka (GMT) functions that can be used to define orthonormal bases in an arbitrary RKHS. Using GMT functions, we propose unitary transformations between separable RKHSs. This allows us to extend our optimality conditions and subsequent algorithms to MOR problems posed in RKHSs with non-differentiable kernels. The potential and versatility of the proposed MOR framework is demonstrated in several numerical experiments. We perform norm-optimal MOR on various machine learning models such as support vector machines and deep ReLU networks. In addition, we show how the proposed MOR framework can be used to solve seemingly unrelated problems. In particular, we provide an example for fitting high degree polynomials to data and also obtain a FEM-like method for solving partial differential equations (PDEs) with an optimal kernel basis.

Figures & tables

Figure 1: The proposed model order reduction pipeline for reproducing kernel Hilbert spaces.
Domains and function spaces
N\mathbb{N}Natural numbers including 00 .
Ω⊆C\Omega\subseteq\mathbb{C} , H\mathcal{H}Domain of the functions and the original reproducing kernel Hilbert space of functions Ω→C\Omega\to\mathbb{C} .
Ω^\widehat{\Omega} , H^\widehat{\mathcal{H}}The companion domain and companion reproducing kernel Hilbert space
D\mathbb{D} , C+\mathbb{C}^{+}Open unit complex disk, open right complex half–plane
H2(D)H_{2}(\mathbb{D}) , H2(C+)H_{2}(\mathbb{C}^{+})Hardy–Hilbert spaces on D\mathbb{D} , C+\mathbb{C}^{+}
Kernels
Table 1: Summary of the core notation used throughout the paper.
Figure 2: The generalized Malmquist–Takenaka transformation pipeline. The full–order model G\mathbf{G} and its companion counterpart G^\widehat{\mathbf{G}} are shown in gray/black; the companion reduced–order model G^r\widehat{\mathbf{G}}_{r} obtained by model order reduction in blue. Inverting the generalized Malmquist–Takenaka transformation directly gives the weak reduced–order model Grweak∈Wr\mathbf{G}_{r}^{\mathrm{weak}}\in W_{r} (green), while projecting G\mathbf{G} onto the subspace Vrλ~V_{r}^{\widetilde{\boldsymbol{\lambda}}} determined by the kernel centers of G^r\widehat{\mathbf{G}}_{r} (via γ−1\gamma^{-1} ) gives the projected reduced–order model Grecon\mathbf{G}_{r}^{\mathrm{econ}} (purple).
Figure 3: Polynomial fits obtained with the different approximation strategies for r=4r=4 and n=100n=100 .
Figure 4: Pointwise square fitting errors obtained with the different approximation strategies for r=4r=4 and n=100n=100 .
ModelEMSED\mathcal{E}^{\mathfrak{D}}_{\mathrm{MSE}}   Eξr\mathcal{E}_{\xi}^{r}   
 r=2r=2r=3r=3r=4r=4r=5r=5r=2r=2r=3r=3r=4r=4r=5r=5
Grdir\mathbf{G}_{r}^{\mathrm{dir}}9.6879.6870.1340.1340.0930.0930.0600.0603.7133.7131.0461.0461.1321.1320.9620.962
Grweak\mathbf{G}_{r}^{\mathrm{weak}}1.6401.6400.038\mathbf{0.038}2.6⋅10−4\mathbf{2.6{\cdot}10^{-4}}6.5⋅10−5\mathbf{6.5{\cdot}10^{-5}}2.4082.4080.417\mathbf{0.417}0.190\mathbf{0.190}0.113\mathbf{0.113}
Grecon\mathbf{G}_{r}^{\mathrm{econ}}1.568\mathbf{1.568}0.0760.0762.0⋅10−32.0{\cdot}10^{-3}1.2⋅10−31.2{\cdot}10^{-3}2.362\mathbf{2.362}0.4380.4380.2550.2550.1890.189
GrLSQ\mathbf{G}_{r}^{\mathrm{LSQ}}9.7849.7849.209.203.5283.5283.4853.48510.52710.52710.62710.62717.63117.63117.70317.703
G2rLSQ\mathbf{G}_{2r}^{\mathrm{LSQ}}3.5283.5281.3611.3610.5130.5130.1840.18417.63117.63147.71247.712157.146157.146532.577532.577
Table 2: mean squared error and reproducing kernel Hilbert space errors for the polynomial fitting application ( n=100n=100 ). The best reduced–order models per reduced order are indicated by boldface font.
rrGrdir\mathbf{G}_{r}^{\mathrm{dir}}max⁡{Grweak,Grecon}\max\left\{\mathbf{G}_{r}^{\mathrm{weak}},\mathbf{G}_{r}^{\mathrm{econ}}\right\}GrLSQ\mathbf{G}_{r}^{\mathrm{LSQ}}G2rLSQ\mathbf{G}_{2r}^{\mathrm{LSQ}}
26.106.100.530.530.020.023.05⋅10−43.05\cdot 10^{-4}
36.156.150.280.284.62⋅10−34.62\cdot 10^{-3}2.96⋅10−42.96\cdot 10^{-4}
46.766.760.240.244.21⋅10−34.21\cdot 10^{-3}2.52⋅10−42.52\cdot 10^{-4}
56.336.330.280.283.99⋅10−33.99\cdot 10^{-3}2.72⋅10−42.72\cdot 10^{-4}
Table 3: Recorded runtimes (seconds) of the considered MOR schemes. GrLSQ\mathbf{G}_{r}^{\mathrm{LSQ}} and G2rLSQ\mathbf{G}_{2r}^{\mathrm{LSQ}} are obtained with MATLAB’s polyfit routine.
DataMetricG\mathbf{G}G30dir\mathbf{G}_{30}^{\mathrm{dir}}G100dir\mathbf{G}_{100}^{\mathrm{dir}}G200dir\mathbf{G}_{200}^{\mathrm{dir}}G30econ\mathbf{G}_{30}^{\mathrm{econ}}G100econ\mathbf{G}_{100}^{\mathrm{econ}}G200econ\mathbf{G}_{200}^{\mathrm{econ}}
GlobalEξ,relr\mathcal{E}^{r}_{\xi,\mathrm{rel}}000.44160.44160.15780.15780.13220.13220.46210.46210.08680.08680.02570.0257
TrainingEpt\mathcal{E}_{\mathrm{pt}}003.08163.08160.73730.73730.43060.43060.60190.60190.51630.51630.05710.0571
Acc. (%)(\%)80.5080.5059.6559.6580.4880.4880.4880.4881.5681.5680.4880.4880.5080.50
\Certunif\Cert_{\mathrm{unif}} (%)(\%)–0.000.001.391.393.943.940.000.0014.1914.1992.5792.57
\Certproj\Cert_{\mathrm{proj}} (%)(\%)–71.8071.8099.5799.5799.7499.7486.5386.5399.9399.93100.00100.00
\Certtrue\Cert_{\mathrm{true}} (%)(\%)–75.0775.0799.8399.8399.8699.8692.7992.7999.9599.95100.00100.00
Table 4: Approximation errors, classification performance, and consistency–certificate applicability for the full–order model, direct full–order models, and projected generalized Malmquist–Takenaka–reduced–order models.
Figure 5: Consistency certificates for the projected generalized Malmquist–Takenaka–reduced–order models with r=30,100,200r=30,100,200 , evaluated on Ztr\mathcal{Z}_{\mathrm{tr}} (left) and Zte\mathcal{Z}_{\mathrm{te}} (right).
MetricCR (%)\mathrm{CR}\,(\%)Grecon\mathbf{G}_{r}^{\mathrm{econ}}Grecon-EIM\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}GrEIM\mathbf{G}_{r}^{\mathrm{EIM}}Fprun\mathbf{F}^{\mathrm{prun}}Irecon\mathcal{I}_{r}^{\mathrm{econ}}Irecon-EIM\mathcal{I}_{r}^{\mathrm{econ\text{-}EIM}}
Eξ,relr\mathcal{E}^{r}_{\xi,\mathrm{rel}}27.270.080180.038240.03824–0.32160.3216−1.1856×10−8-1.1856\times 10^{-8}
32.620.072760.037010.03701–0.29360.2936−1.2626×10−8-1.2626\times 10^{-8}
37.970.071320.032150.03215–0.34610.3461−1.6617×10−8-1.6617\times 10^{-8}
43.320.071110.031560.03156–0.35280.3528−1.7235×10−8-1.7235\times 10^{-8}
48.660.071090.027520.02752–0.41210.4121−2.2664×10−8-2.2664\times 10^{-8}
54.010.070530.026590.02659–0.42360.4236−2.4156×10−8-2.4156\times 10^{-8}
Table 5: Proposed and baseline reduced–order models for model order reduction on a ReLU network.
Figure 6: Approximation of the FOM neural network F\mathbf{F} with the considered kernel expansion based ROMs. Here, Frecon(t)≔Grecon(t)+β(t)\mathbf{F}_{r}^{\mathrm{econ}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{econ}}(t)+\beta(t) , FrEIM(t)≔GrEIM(t)+β(t)\mathbf{F}_{r}^{\mathrm{EIM}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{EIM}}(t)+\beta(t) , and Frecon-EIM(t)≔Grecon-EIM(t)+β(t)\mathbf{F}_{r}^{\mathrm{econ\text{-}EIM}}(t)\coloneqq\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}(t)+\beta(t) are given in ( 7.14 ). In this figure, r=70r=70 is considered.
rrGrecon\mathbf{G}_{r}^{\mathrm{econ}}Grecon-EIM\mathbf{G}_{r}^{\mathrm{econ\text{-}EIM}}GrEIM\mathbf{G}_{r}^{\mathrm{EIM}}
501.59×10−291.59\times 10^{-29}1.43×10−341.43\times 10^{-34}1.54×10−181.54\times 10^{-18}
602.72×10−362.72\times 10^{-36}3.70×10−413.70\times 10^{-41}1.24×10−211.24\times 10^{-21}
701.30×10−431.30\times 10^{-43}4.23×10−474.23\times 10^{-47}1.74×10−241.74\times 10^{-24}
804.81×10−574.81\times 10^{-57}6.98×10−536.98\times 10^{-53}2.82×10−272.82\times 10^{-27}
901.75×10−621.75\times 10^{-62}2.22×10−582.22\times 10^{-58}5.92×10−305.92\times 10^{-30}
1009.49×10−689.49\times 10^{-68}1.73×10−641.73\times 10^{-64}7.35×10−337.35\times 10^{-33}
Table 6: Maximum equilibrium function magnitude ∥[D^(γ(λ~0),…,γ(λ~r−1))]⊤∥∞\left\|\left[\widehat{\mathcal{D}}\left(\gamma\big(\widetilde{\lambda}_{0}\big),\ldots,\gamma\big(\widetilde{\lambda}_{r-1}\big)\right)\right]^{\top}\right\|_{\infty} in the companion reproducing kernel Hilbert space H2IH_{2}^{\mathbb{I}} for different reduced–order models. These results were computed using MATLAB’s variable precision arithmetic using 100100 digits.
Figure 7: Solution to the 4th–order Euler beam equation using the proposed adaptive kernel mesh based approach and FEM for r=5r=5 .
editflow\cellcolor red
received\cellcolor red
revised 
accepted\cellcolor red
communicated (by)\cellcolor red
 
Personal data (Author 1)
Table 14
More in this topic

Reduced-Order Modeling

Reduced-order modeling builds compact approximations of complex systems to make simulation, prediction, and control more efficient.

See all 97 papers

Explore similar work

CardsList

  1. KROM: Kernelized Reduced Order Modeling

    Feb 27, 2026Aras Bacho, Jonghyeon Lee, Houman OwhadiPDE Surrogate ModelingReduced-Order Modeling

    Department of Computing and Mathematical Sciences, Caltech, CA, USA.

  2. Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms

    Oct 22, 2026Caixing Wang, Yue WangGeneralization BoundsSpectral Methods

    School of Statistics and Data Science, Southeast University, Nanjing, 210000, Jiangsu, China · Faculty of Business for Science and Technology, School of Management, University of Science and Technology of China, Hefei, 230000, Anhui, China

  3. Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation

    Apr 27, 2026Dmitry B. Rokhlin, Georgiy A. KarapetyantsSequential Decision MakingReproducing Kernel Hilbert Spaces

    Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University