Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systems

Mark Kozdoba; Shie Mannor

cs.LGAug 19, 2026

Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systems

Authors: Mark Kozdoba, Shie Mannor

Organizations: Technion, IIT · Technion, IIT and NVIDIA

Abstract

Dimension reduction for dynamical systems is standard practice, and the standard route is spectral: model the transfer (Koopman) operator by its leading modes. We show that on systems assembled from several weakly interacting components --- a structure common in physical and biological settings --- this may either require an exponential number of modes, or drop an entire component: the component is absent from the model rather than modeled coarsely, and no function of it can be predicted at any accuracy. We call this linear masking. The cause is that a rank-based model pays one coordinate per mode. We propose to score instead the $σ$ -algebra the coordinates generate, so that products and powers come free and a component's cost is governed only by its generators rather than by all its interactions. The criterion is a $χ^2$ -divergence between the embedded present and future, and it carries a budget guarantee: twice the intrinsic dimension of the dynamics is enough coordinates for an embedding whose algebra carries the operator's entire spectrum, with its full infinite rank. In variational form the criterion admits off-the-shelf estimators, and restricting its critic to the bilinear class returns the VAMP score on the span, so rank-based methods are one end of the same family. We demonstrate the proposed objective on a composite of published benchmark systems. We exhibit examples where the rank-based methods completely miss the masked components at all ranks $k<100$ , while ten algebra coordinates recover all of them. In addition, the resulting algebra representation supports predicting the masked components from few labels, while direct regression from the high-dimensional observation or from the VAMP features fail.

Figures & tables

Panel of Figure 1 — Figure 1: The coupling at dial $\kappa$ : a symmetric torque between the two rings, and a phase-staggered tilt from cool 0 on each bit, with no feedback. At $\kappa=0$ every arrow vanishes.

Figure 2: The composite system’s spectrum. Top: the marginal of one cool, a ring walk, with the measured Markov state model overlaid, and the six bits as a degenerate block at

\sigma\approx 0.55

. Bottom: the joint spectrum at

\kappa=0

, the product of the marginals. The rank axis is broken because the crossover is near rank

100

but not pinned there, the closed form idealizing each cool as a ten-state ring walk.

bit recovery ( $R^{2}$ )	$\kappa=0$	$\kappa=0.5$	$\kappa=1$	$\kappa=2$
floor (cools only)	$0.00$	$0.02$	$0.12$	$0.39$
span, $k=30$	$0.00$	$0.02$	$0.12$	$0.39$
bilinear critic	$0.04$	—	$0.12$	—
algebra, $m=10$	$0.24$ – $0.27$	$0.24$ – $0.28$	$0.26$ – $0.32$	$0.47$
ceiling (raw state)	$0.26$	$0.27$	$0.31$	$0.48$

Table 1: Bit recovery along the coupling dial, mean over the six fast components. The floor is what the cools alone reveal about them. The bilinear-critic row is our own objective at

m=10

with the critic restricted to the span, run at two couplings. Algebra entries are min–max over three, two, three and two runs.

Figure 3: Masking along the coupling dial. Bit recovery of the rank-

k

span optimum stays on the floor benchmark until

k

approaches the product-mode count (

\approx 100

), at every coupling strength, while the algebra objective recovers the bits at

m=10

. The floor rises with

\kappa

as the cools come to carry more about the bits; the threshold does not move.

from $300$ labels	cools	bits
direct regression	$0.40$ – $0.44$	$0.03\pm 0.04$
span, any rank $12$ – $120$	—	$\leq 0$
autoencoder chart alone	$0.49$	$0.09\pm 0.02$
algebra, $m=10$	$0.77$ – $0.80$	$0.13\pm 0.02$
ceiling (full sample)	—	$0.26$

Table 2: Prediction from few labels on the warped observation, mean

\pm

sd over twenty paired label draws that resample the labels only, with nothing retrained. Spans are at or below zero on the bits at every rank we measured,

k=12

120

, including the ranks whose full-sample readout does recover them.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Convergence of the objective with the coordinate budget, on the composite system of Section 5 . Held-out

\hat{J}

against embedding dimension

m

at each coupling, seed spread shaded. The vertical scale is set by the critic architecture and is not comparable across architectures, so the curves identify shape and not height. Because the achievable value is non-decreasing in

m

, any decline past a plateau measures the estimator’s error rather than a loss of signal. Growth stabilizes on every curve, later at stronger coupling. This is an instrument diagnostic: the flattening reports that our estimator stops gaining, not that the coupling has a dimension.

Figure S2: Removal is the optimization. Initialized from the same unsupervised chart, the span objective trains away readout-verified bit information (orange, left axis) as its own held-out score improves (blue, right axis), while the algebra objective from that chart retains it. Two restarts.

Figure S3: The reachability wall, on both axes. Fraction of each mode built, as a linear readout of that mode from the trained features (

R^{2}/\sigma^{2}

). Left, against span rank at a

64^{2}

trunk: the bits are built by

k=80

even though

\sigma=0.55

makes them the lowest modes plotted, while the interactions above them stay unbuilt. Right, at fixed

k=80

against trunk width: the interactions arrive and the bits drain, the same slots changing hands.

c_{5}{\times}1

and

c_{3}{\times}c_{2}

are a matched pair at

\sigma\approx 0.84

, built and unbuilt respectively, so what fails is composition rather than spectral depth.

Dimensionality Reduction

Dimensionality reduction maps high-dimensional data to compact representations for visualization, prediction, compression, and scientific modeling.

See all 145 papers

Jun 3, 2026

Deep Embedded Multiplicative DMD for Algebra-Preserving Koopman Learning

Koopman theory turns nonlinear dynamics into a linear spectral problem. In computation, however, everything depends on a hard finite-dimensional choice: the observables must be expressive, nearly invariant under the dynamics, and, ideally, compatible with composition. Deep Koopman methods learn flexible coordinates, whereas structure-preserving methods enforce operator identities on fixed dictionaries. We combine these ideas by introducing Deep Embedded Multiplicative Dynamic Mode Decomposition (DeepMDMD), a method that learns a latent space and a partition of it, while enforcing the Koopman product rule as an exact algebraic constraint. Training alternates between an exact multiplicative operator update and a differentiable latent-clustering step that promotes Koopman closure. The result is a finite transition map on learned latent cells. Its nonzero spectrum lies on the unit circle, its dictionary is shaped by the dynamics rather than by ambient geometry, and forecasts are made in latent coordinates before being decoded to physical space. Across Hamiltonian, chaotic, and fluid examples, DeepMDMD learns dictionaries that are far more compact and dynamically coherent than those produced by geometric MDMD partitions. It reduces spectral pollution, reveals richer continuous-spectrum structure, and gives stable forecasts under severe noise. In high-dimensional flows, including a 158,624-dimensional cylinder wake and a noisy

Re=20,000

lid-driven cavity, it preserves coherent structures and long-time spectral statistics where state-space MDMD fails. These results suggest a practical rule for Koopman learning: learn the coordinates, constrain the algebra.

Kelan Gray, Finlay Brown, Nicolas Boullé +1

Department of Mathematics, Imperial College London, London, SW7 2AZ, UK. · Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, CB3 0WA, UK.

Koopman Operator Learning Latent Dynamics Modeling Dynamic Mode Decomposition

Ask PDF

Sep 24, 2026

Functional dynamic mode decomposition: Learning infinite-dimensional systems from data

Dynamic mode decomposition (DMD) is a data-driven method that computes the best linear approximation of the underlying dynamical system and decomposes the dynamics into a superposition of characteristic spatiotemporal patterns. Originally introduced by the fluid dynamics community, DMD and its extensions have found widespread use in many other research areas such as molecular dynamics, climate science, engineering, finance, and neuroscience. Applications include dimensionality reduction, forecasting, system identification, control, and spectral clustering. In order to apply DMD to partial differential equations, the spatial domain is typically first discretized using finite difference or finite element techniques, thus implicitly rendering the problem finite-dimensional. We extend projected and exact DMD to infinite-dimensional systems. Rather than estimating matrices from vector-valued observations, our DMD variants learn finite-rank operators from functional data such as observables, densities, or wavefunctions. We show that conventional DMD algorithms can be regarded as special cases of their functional DMD counterparts. All results will be illustrated with the aid of guiding examples. We focus in particular on Koopman, Perron-Frobenius, and Koopman-von Neumann operators associated with graphons, ordinary differential equations, and stochastic differential equations.

Stefan Klus, Eirini Ioannou

School of Mathematical & Computer Sciences, Heriot–Watt University, Edinburgh, UK · Maxwell Institute for Mathematical Sciences, University of Edinburgh and Heriot–Watt University, Edinburgh, UK

Koopman Operator Learning Dynamic Mode Decomposition

Ask PDF

Oct 8, 2026

Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability

In deep learning, approximation theory motivates increasing representation size. We ask whether this benefit extends to dynamics learning through autoregressive prediction. We analyze the learned time evolution through the eigenstructure of Koopman operators, using relative residuals to detect spurious eigenpairs arising even as one-step error falls. For bounded Koopman operators, we show that minimal residuals over learned dictionary spaces converge pointwise to their full-space counterparts as these spaces approximate the observable space in

L^2

. Our hypothesis is that Koopman spectral reliability helps explain how consistently rollout error decreases with increasing dimension. We compare two models of a shared Koopman autoencoder trained alternately for reconstruction and latent evolution, using latent-prediction loss (one-step prediction errors in latent coordinates) or spectral-residual loss (relative residuals of candidate eigenpairs). Across six chaotic systems, both models reduced median windowed rollout error from smallest to largest dimension. The spectral-residual model achieved lower medians than the latent-prediction model for all systems and dimensions, and its median fell by a larger factor in every system. Its median decreased monotonically with dimension in four systems, against one for latent prediction. Against four baseline families, its mean valid prediction times were nearly always longer. At the largest dimension under two-stage training, we compared eigenvalue positions with each learned dictionary's residual contours. Spectral-residual eigenvalues concentrated in low-residual regions, whereas latent-prediction eigenvalues also appeared in high-residual regions, consistent with the hypothesis.

Itsushi Sakata, Yuta Miyauchi, Yoshinobu Kawahara

RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. · Graduate School of Information Science and Technology, The University of Osaka, Suita, Osaka, Japan.

Latent Dynamics Modeling Dynamical Systems Koopman Operator Learning

Ask PDF

Abstract

Dimensionality Reduction

Explore similar work

Deep Embedded Multiplicative DMD for Algebra-Preserving Koopman Learning

Functional dynamic mode decomposition: Learning infinite-dimensional systems from data

Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability