Universal set of Observables for Forecasting Physical Systems through Causal Embedding

G Manjunath; A de Clercq; MJ Steynberg

math.DSMay 22, 2021

Universal set of Observables for Forecasting Physical Systems through Causal Embedding

Authors: G Manjunath, A de Clercq, MJ Steynberg

Organizations: *Department of Mathematics & Applied Mathematics, University of Pretoria, Pretoria 0028 · † Department of Physics, University of Pretoria, Pretoria 0028

Abstract

We show how a pair of points can uniquely represent a left-infinite sequence obtained from observations of an underlying dynamical system through a phenomenon called causal embedding. A driven dynamical system creates such pairs, and a function can be learnt on them that can reconstruct the underlying dynamics as in Takens delay embedding. The approach assures embedding stability unlike Takens delay embedding, and learnability which can be absent while stability can be present in the current reservoir computing framework. This accurately models underlying systems where recent methods like the next-generation reservoir computing fail. We demonstrate results and compare with the other methods including SINDY-PI.

Figures & tables

Figure 1: Schematic of employing the Takens embedding, and the effect of iterates of $\widetilde{F}_{S}$ slipping out of $\Phi_{\theta,2d}(W)$ .

Figure 2: A Double Pendulum Observation: In the top panel, a coordinate of the original time-series (in red) and the SINDY-PI predicted time-series (in blue) for 1000 time-steps are shown. All coordinates are used for training. In the bottom panel, the predicted values over 10000 time-steps is shown. The blow-up clearly indicates the predicted time-series veering off from the set of interest.

Figure 3: A Double Pendulum Observation: The map in $F_{S}$ in Takens Delay Embedding (Top to Bottom) is learnt by by a deep neural network. First 1000 values of the actual time-series (in the top panel) and the predicted values (in the bottom panel) are shown.

Figure 4: time-lag plot where three values $(y(t),y(t+8),y(t+16))$ are plotted for the two time-series in Fig. 3

Figure 5: A. Commutativity ladder: inner workings of a conjugate system when the Takens theorem holds. B. Commutativity ladder with input being an orbit from a non-invertible system.

Figure 6: Schematic of GD. Here $f=H_{2}\circ\varphi_{T}\circ P\circ\Phi_{2d,\theta}$ of $W$ in $X\times X$ is the embedding. The “cushion” around $Y_{T}$ is shown with the blue boundary.

Figure 7: Schematic of a RCN with three nodes in the RNN: (the print in red indicates the connections made to run in the autonomous mode)

Figure 8: Multi-lag recurrence plot of a forecasted time-series from an observation $\theta$ of a Double Pendulum System with a zero-mean normally distributed noise with standard deviation 0.01; the points $(\theta(w(t)),\theta(w(t+8)),\theta(w(t+16)))$ of the actual (in red) and forecasted (in blue) values are plotted.

Figure 9: A. Effect of GD: The underlying tuples $(x_{n-1},x_{n})$ of the data in (E) is perturbed by a high-amplitude Gaussian noise vector with standard deviation 2 once in every 100 time-steps in the autonomous mode of $g$ to simulate an error in $\Gamma$ ; despite this the readout in (blue) stabilizes towards the pattern of the actual readout conceived in the driven mode (red). B. Illustration of GD: three largest principal component analysis (PCA) values of $(x_{n-1},x_{n})$ corresponding to the time-series in the top panel is plotted for the driven (red) and autonomous (blue) mode; successive points are joined; thick dots show the high-amplitude noise instants. The blue trajectory drifts but is stabilized back.

Input, from	Dimension of $X$	$u_{n}$ vs $u_{n+1}$	$\begin{bmatrix}x_{n-1}\\ x_{n}\end{bmatrix}$ vs $\begin{bmatrix}x_{n}\\ x_{n+1}\end{bmatrix}$
Double Pendulum:
	10	0.9710	0.9822
	100		0.9803
	1000		0.9854
Lorenz map
	10	0.8568	0.9444

Table 1: Multidimensional Correlation Coefficients

\rho

to indicate a general reduction in the functional complexity of the map

G_{T}

that arises from a RNN. Each rows corresponds to a different dynamical system. The second column corresponds to the dimension of

X

(here, number of artificial neurons) in the RNN used. The numerical estimate of

\rho

for the relevant vectors is plotted in the last two columns. The reader can compare the linear relationship

u_{n}

T(u_{n})

and the linear relationship

\begin{bmatrix}x_{n-1}\\ x_{n}\end{bmatrix}

G_{T}\left(\begin{bmatrix}x_{n-1}\\ x_{n}\end{bmatrix}\right)

. The dimension of the RNN is also given, showcasing that increasing the dimension of the driven system typically indicates a map

G_{T}

of lower functional complexity.

Dynamical Systems

Dynamical systems studies how states evolve over time, including stability, feedback, chaos, and transitions in physical, biological, and engineered systems.

See all 474 papers

Generalizing Dynamics Modeling More Easily from Representation Perspective
Mar 24, 2026Yiming Wang, Zhengnan Zhang, Genghe Zhang +7Dynamical Systems Latent Dynamics Modeling
College of Computer Science and Technology, and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China · College of Software and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China · School of Computing, Belfast, Northern Ireland, UK +1
Causal Local States: Scalable Simultaneous Causal Network Inference and Forecasting for Dynamical Systems
Aug 18, 2026Jonas Braun, Fabian Fischbach, Daniel Köglmayr +2Time-Series Causal Discovery Explainable Time Series Forecasting
Ludwig-Maximilians-Universität München, Faculty of Physics, Geschwister-Scholl-Platz 1, 80539 Munich, Germany · Institut für Theoretische Physik, Universität zu Köln, Zülpicher Strasse 77, 50937 Cologne, Germany · Institut für KI-Sicherheit, Deutsches Zentrum für Luft- und Raumfahrt (DLR), St. Augustin & Ulm, Germany +2
Ultra-Early Prediction of Tipping Points: Integrating Dynamical Measures with Reservoir Computing
Mar 16, 2026Xin Li, Qunxi Zhu, Chengli Zhao +4Early Warning Detection Dynamical Systems
College of Science, National University of Defense Technology, Changsha, Hunan 410073, China · Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China · School of Mathematical Sciences, SCMS, SCAM, and CCSB, Fudan University, Shanghai 200433, China +1

Mar 24, 2026J/K move · Enter open · S save

Generalizing Dynamics Modeling More Easily from Representation Perspective

Yiming Wang, Zhengnan Zhang, Genghe Zhang, Jiawen Dan+6

College of Computer Science and Technology, and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China · College of Software and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China · School of Computing, Belfast, Northern Ireland, UK+1

Dynamical Systems Latent Dynamics Modeling State Representation Learning Representation Learning

Open paper AskPDF arXiv

Learning system dynamics from observations is a critical problem in many applications over various real-world complex systems, e.g., climate, ecology, and fluid systems. Recently, neural dynamics modeling method have become a prevalent solution that embeds the object's observations into a latent space before learning dynamics using neural methods such as neural Ordinary Differential Equations (ODE). Existing dynamics modeling methods induce a specific model for each observation of different complex systems, resulting in poor generalization across systems. Inspired by the great success of pre-trained models, we conduct a generalized Pre-trained Dynamics EncoDER (PDEDER) which can embed the original state observations into a latent space where the dynamics can be captured more easily. To conduct the generalized PDEDER, we pre-train any Pre-trained Language Model (PLM) by minimizing the Lyapunov exponent objective, which constrains the chaotic behavior of governing dynamics learned in the latent space. By penalizing the divergence of embedded observations, our PDEDER promotes locally stable and well-structured latent dynamics, thereby facilitating more effective dynamics modeling than in the original observation space. In addition, we incorporate reconstruction and forecasting objectives to mitigate the risk of obtaining an over-smoothed latent space. Specifically, we collect 152 sets of real-world and synthetic observations from 23 complex systems as pre-training corpora and employ them to pre-train PDEDER. Given any future dynamic observation, we can fine-tune PDEDER with any specific dynamics modeling method. We evaluate PDEDER on 12 dynamic systems by short/long-term forecasting under both in-domain and cross-domain settings, and the empirical results indicate the effectiveness and generalizability of PDEDER.

Abstract

Dynamical Systems

Explore similar work

Generalizing Dynamics Modeling More Easily from Representation Perspective

Causal Local States: Scalable Simultaneous Causal Network Inference and Forecasting for Dynamical Systems

Ultra-Early Prediction of Tipping Points: Integrating Dynamical Measures with Reservoir Computing