Neural Operators Can Discover Functional Clusters

Yicen Li; Jose Antonio Lara Benitez; Ruiyang Hong; Anastasis Kratsios; Paul David McNicholas; Maarten Valentijn de Hoop

cs.LGFeb 26, 2026

Neural Operators Can Discover Functional Clusters

Authors: Yicen Li, Jose Antonio Lara Benitez, Ruiyang Hong, Anastasis Kratsios, Paul David McNicholas, Maarten Valentijn de Hoop

Organizations: McMaster University and the Vector Institute, Main St., Hamilton, L8S 4L8, Ontario, Canada · Rice University, Main St., Houston, 77005, Texas, United States

Abstract

Operator learning is reshaping scientific computing by amortizing inference across infinite families of problems. While neural operators (NOs) are increasingly well understood for regression, far less is known for classification and its unsupervised analogue: clustering. We prove that sample-based neural operators can learn any finite collection of classes in an infinite-dimensional reproducing kernel Hilbert space, even when the classes are neither convex nor connected, under mild kernel sampling assumptions. Our universal clustering theorem shows that any $K$ closed classes can be approximated to arbitrary precision by NO-parameterized classes in the upper Kuratowski topology on closed sets, a notion that can be interpreted as disallowing false-positive misclassifications. Building on this, we develop an NO-powered clustering pipeline for functional data and apply it to unlabeled families of ordinary differential equation (ODE) trajectories. Discretized trajectories are lifted by a fixed pre-trained encoder into a continuous feature map and mapped to soft assignments by a lightweight trainable head. Experiments on diverse synthetic ODE benchmarks show that the resulting practical SNO recovers latent dynamical structure in regimes where classical methods fail, providing evidence consistent with our universal clustering theory.

Figures & tables

Panel of Figure 1 — Figure 1: ( Left: K-Means ) Standard finite-dimensional $K$ -means relies on mapping data to discrete cluster centers $\mu_{1},\mu_{2}$ in a reduced feature space, often discarding continuous dynamics. ( Right: Neural Operator ) In contrast, our proposed method leverages a single neural operator to generate continuous cluster indicator functions. The blue and green curves represent the two output components $\hat{f}_{1}$ and $\hat{f}_{2}$ of the neural operator (one for each output dimension), and the corresponding cluster regions are defined via thresholding at $\gamma$ .

Panel of Figure 2 — Figure 2: Geometric intuition of set-valued clustering via upper Kuratowski convergence. Unlike point-wise estimation, the learned cluster support $P_{\theta}$ (green) approximates the ground truth $P_{\text{data}}$ (black) strictly from the interior. Whether in the intermediate phase (a) or near convergence (b), the approximation is constrained to stay within the target boundaries, thereby maximizing cluster purity. This ensures no false positives, meaning the model prevents infeasible behaviors lying outside the target set.

Panel of Figure 3 — Figure 3: Architecture of the Sampling-Based Neural Operator (SNO). (a) The discretization process samples the function $f\in\mathcal{H}$ via inner products with reproducing kernels $\kappa(\cdot,x_{s})$ centered at points $x_{s}$ to form the input vector $v_{0}\in\mathbb{R}^{S}$ . (b) The deep neural network maps the discretized input $v_{0}$ through multiple layers of affine maps $A_{l}$ and nonlinearities (ReLU) to produce output logits, defining the clusters $C_{k}$ . Note that, only the first layers is a non-local kernel-based operator acting on the function space, and the remaining layers are lightweight standard deep learning models; in our analysis an MLP.

Method	ODE-6			ODE-4
Method	ACC	ARI	NMI	ACC	ARI	NMI
FPCA + K-Means	0.314	0.175	0.357	0.291	0.005	0.021
B-Spline + K-Means	0.422	0.185	0.380	0.293	0.006	0.022
DTW + K-Means	0.790	0.651	0.705	0.393	0.088	0.212
CLIP + K-Means	0.624	0.583	0.715	0.491	0.243	0.441
CLIP + Spectrogram + K-Means	0.684	0.528	0.647	0.500	0.284	0.436

Table 1: Comparison of clustering performance on test data. Best results are highlighted in bold. Both FPCA and B-spline struggle to capture shape distinctions. DTW achieves improved performance in the structured ODE-6 regime but fails to generalize to the high-variability ODE-4 setting. The proposed SNO outperforms baselines in the structured regime and in the high-variability regime, demonstrating the benefit of learning decision regions from sampled observations in both regimes.

Panel of Figure 4 — Figure 4: Visual comparison of trajectory structures across all systems in ODE-6. Each plot overlays 5 randomly selected trajectories. Due to path overlaps, fewer than 5 distinct lines may be visible in some panels. Rows represent different dynamical systems, and columns illustrate the effect of increasing resolution $S$ from 4 (left) to 224 (right).

Panel of Figure 5 — Figure 5: We evaluate the SNO model with 5 random seeds across varying sampling resolutions, ranging from a coarse $2\times 2$ to $224\times 224$ on the ODE-6 test set. The improvement of clustering metrics empirically demonstrates that the sequence of discretized operators converges toward a stable limit as the sampling density increases in this structured regime. This corroborates our claim that SNO approximates the true cluster partition.

Panel of Figure 6 — Figure 6: t-SNE visualization of the representation space for ODE-6. Both embeddings exhibit curve-like structures due to continuous dependence on initial conditions, but in (a) different dynamical systems remain entangled, leading to poor separability. (b) Our method successfully disentangles these manifolds, preserving the intrinsic continuous geometry within clusters while pushing distinct systems apart. Note that the clusters can consist of multiple non-convex components as highlighted in Figure 1 . This is a key feature of the NO-based clustering pipeline which cannot be performed by classical K-means-type clustering procedures.

Panel of Figure 7 — Figure 7: Qualitative comparison of clustering on the ODE-6 test set. (a) K-means (ACC=62.4%) fails to distinguish between structurally similar trajectories, resulting in significant overlap across different dynamical families. (b-c) In contrast, the proposed SNO (ACC=93.3%) and SNO+Spectrogram (ACC=94.5%) successfully disentangle these latent structures, yielding clean partitions that closely align with the (d) Ground-truth class labels. The color corresponds to the six distinct dynamical systems described in Section 5 .

Panel of Figure 8 — Figure 8: Qualitative comparison on the high-variability ODE-4 test set. (a) K-means (ACC=49.1%) struggles to handle the intra-class variability introduced by randomized neural vector fields, resulting in fragmented clusters. (b-c) In contrast, the SNO (ACC=61.3%) and SNO+Spectrogram (ACC=65.2%) recovers coherent dynamical groupings even in this stochastic regime. (d) Ground-truth clusters (magnified view) highlight the high-frequency fluctuations and complex overlaps that characterize these randomized neural ODEs.

$L_{\text{e}}$	$L_{\text{con}}$	$H(Y)$	ODE-6			ODE-4
$L_{\text{e}}$	$L_{\text{con}}$	$H(Y)$	ACC	ARI	NMI	ACC	ARI	NMI
✓			0.333	0.248	0.427	-	-	-
✓	✓		0.333	0.262	0.435	-	-	-
✓	✓	✓	0.935	0.871	0.913	0.636	0.366	0.400
	✓		0.333	0.143	0.402	-	-	-
	✓	✓	0.883	0.802	0.846	0.618	0.353	0.370

Table 2: Ablation study of loss components on the training sets of ODE-6 and ODE-4. A “–” indicates model collapse, where all samples are assigned to a single cluster. The combination of all three terms is required to satisfy the consistency, sharpness, and non-degeneracy conditions of the theoretical framework. Reported metrics reflect the best performance observed during optimization.

Figure 9: Comparison of different loss weights $\alpha$ . The solid line shows the mean ARI across five runs, and the shaded region indicates the standard deviation.

Clustering

Clustering groups data points by shared structure, developing methods to discover, validate, and interpret groups across diverse data and constraints.

See all 335 papers

Jul 25, 2026

Neural operator discovery from heterogeneous trajectories

Neural operators provide data-driven mappings for modeling dynamical systems. Extending them to families of systems typically requires explicit conditioning variables such as physical parameters, geometries, or boundary conditions. In many real-world settings, these quantities are unobserved. Here, we formulate neural operator discovery (NOD) as the problem of learning both shared solution operators and system-specific variation directly from heterogeneous trajectories without access to labeled governing factors. We introduce a factorized latent-conditioning formulation that jointly learns a neural operator and a low-dimensional latent representation through factorized prediction, trajectory-decoupled sampling, and dimension selection. Across diverse systems, the learned latent representation captures the intrinsic dimensionality of system variation and organizes system instances in a smooth and approximately invertible latent structure aligned with the underlying governing factors. This organization enables generalization to previously unseen system instances, including zero-shot extrapolation across regimes and stable long-horizon prediction. These results establish an interpretable paradigm for operator learning in the absence of explicit factor supervision.

Zituo Chen, Qiaofeng Li, Jiaxin Hu +1

Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, 02139, MA, USA. · State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University, 866 Yuhangtang Road, Hangzhou, 310058, Zhejiang Province, China.

Latent Dynamics Modeling Nonlinear System Identification Operator Learning Neural Operators

Ask PDF

May 8, 2026

Neural Operators as Efficient Function Interpolators

Neural operators (NOs) are designed to learn maps between infinite-dimensional function spaces. We propose a novel reframing of their use. By introducing an auxiliary base-space, any finite-dimensional function can be viewed as an operator acting by composition on functions of the base-space. Through a range of benchmarks on analytic functions of increasing complexity and dimensionality, we demonstrate that NOs can match or outperform standard multilayer perceptrons and Kolmogorov--Arnold Networks in accuracy while requiring significantly fewer parameters and training time. As a real-world application, we apply a two-dimensional Tensorized Fourier Neural Operator (TFNO) to the nuclear chart, learning a correction to state-of-the-art nuclear mass models as a partially observed residual field. A TFNO ensemble reaches a held-out root-mean-square error of 198.2 keV, placing it among the best recent neural-network approaches while retaining high parameter efficiency and short training times. More broadly, these results introduce NOs as a scalable framework for finite-dimensional function interpolation, from analytic benchmarks to structured scientific data.

Vasilis Niarchos, Angelos Sirbu, Sokratis Trifinopoulos

Department of Physics, CCTP and ITCP, University of Crete, 71303, Greece · Theoretical Physics Department, CERN, Geneva, Switzerland · Physik-Institut, Universität Zürich, 8057 Zürich, Switzerland

Neural Surrogate Modeling Fourier Neural Operator Neural Operators

Ask PDF

Aug 12, 2026

Kernel Methods for Learning Operators with Multiple Inputs and Outputs

Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation, representation, learning, and reconstruction. We develop this framework for multi-input, multi-output operator learning, where operators map between products of potentially distinct function spaces. Our approximation theory shows that, although the number of inputs and outputs can increase, the convergence rate is governed by the most challenging constituent approximation problem rather than the overall problem dimension. The framework leads to practical kernel methods with closed-form training and inference, combining mathematical tractability with computational efficiency. We further specialize the approach to multiple operator learning by introducing KernelMO, a family of kernel methods with complementary operator-valued and product-space formulations. Across five families of parametric partial differential equations, the proposed methods achieve competitive or state-of-the-art predictive accuracy while reducing training and inference costs relative to neural operator architectures and deep learning based models, offering an efficient and lightweight alternative.

Adrien Weihs, Chunyang Liao, Jingmin Sun +1

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA. · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701, USA. · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA.

Kernel Methods PDE Operator Learning Operator Learning

Ask PDF

Abstract

Clustering

Explore similar work

Neural operator discovery from heterogeneous trajectories

Neural Operators as Efficient Function Interpolators

Kernel Methods for Learning Operators with Multiple Inputs and Outputs