Adaptive Sensing of Continuous Physical Systems for Machine Learning

Felix Köster; Atsushi Uchida

cs.LGMar 4, 2026

Adaptive Sensing of Continuous Physical Systems for Machine Learning

Authors: Felix Köster, Atsushi Uchida

Organizations: Department of Information and Computer Sciences, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama City, Saitama, 338–8570, Japan

Abstract

Physical dynamical systems can be viewed as natural information processors: their systems preserve, transform, and disperse input information. This perspective motivates learning not only from data generated by such systems, but also how to measure them in a way that extracts the most useful information for a given task. We propose a general computing framework for adaptive information extraction from dynamical systems, in which a trainable attention module learns both where to probe the system state and how to combine these measurements to optimize prediction performance. As a concrete instantiation, we implement this idea using a spatiotemporal field governed by a partial differential equation as the underlying dynamics, though the framework applies equally to any system whose state can be sampled. Our results show that adaptive spatial sensing significantly improves prediction accuracy on canonical chaotic benchmarks. This work provides a perspective on attention-enhanced reservoir computing as a special case of a broader paradigm: neural networks as trainable measurement devices for extracting information from physical dynamical systems.

Figures & tables

Panel of Figure 1 — Figure 1: Flow chart of the Adaptive-Sensing-Attention–Enhanced-Reservoir-Computer (ASAERC). Inputs drive a fixed continuous reservoir. The reservoir is measured at fixed locations given via the measurement kernels $\psi_{t}^{(j)}(\mathbf{x})$ and the resulting $N_{\mathrm{fix}}$ states $\tilde{r}_{t}$ are fed into the attention module. The trainable attention module outputs parameterized kernels $\phi_{t+T}^{(i)}(\mathbf{x})$ to guide sampling and attention weights $\mathbf{W}_{\mathrm{att},t+T}$ to combine sampled features. The weighted combination produces $\bar{\mathbf{y}}_{n}$ , and gradient descent (dashed path) updates the attention module. No gradients pass through the PDE reservoir.

Figure 2: Illustration of ASAERC behavior. (a) Example predicted time series (orange-dashed line) compared with ground truth (blue-solid line) on the Lorenz attractor. (b) Snapshot of the PDE reservoir field $u(\mathbf{x},t^{\star})$ with fixed injection points (light-blue triangles), fixed measurement points (attention input, crosses) $\tilde{r}_{t}^{(j)}=\int_{\Omega}\psi_{t}^{(j)}(\mathbf{x})\,u(\mathbf{x},t)\,d\mathbf{x},$ and dynamic measurement locations (attention output) overlaid $r_{t+T}^{(j)}=\int_{\Omega}\phi_{t+T}^{(i)}(\mathbf{x})\,u(\mathbf{x},t+T)\,d\mathbf{x},$ . The adaptive sensors (blue circles) cluster around dynamically active regions, demonstrating the network’s ability to focus its attention on informative spatial locations. (c) A histogram of the attention output locations, giving an intricate pattern that is a nonlinear projection of the Lorenz attractor on the measurement process.

Figure 3: Prediction error as a function of number of measurement points. All methods use the same PDE reservoir. (a) Classical reservoir computing with linear readout; (b) AERC with fixed measurement positions; (c) ASAERC with adaptive sensing. Color indicates the number of measurement points at fixd locations.

Figure 4: Number of trainable parameters versus number of measurement points. The dotted line with triangles denotes the linear readout, the dashed line with squares denotes AERC, and the solid lines with dots denote ASAERC. For both AERC and ASAERC, we show three cases: one with 16, 64, and 256 adaptive queries. All three methods use the same PDE reservoir. ASAERC has the largest number of parameters, however only slightly more than AERC. Linear readout has far fewer parameters.

Figure 5: Correlation analysis of readout contributions. Distribution of pairwise Pearson correlation coefficients $\rho_{ij}$ between readout nodes $i$ and $j$ for (left column) raw PDE values, (middle) attention weights, and (right) their product. Rows correspond to (top row) linear readout, (middle row) AERC, and (bottom row) ASAERC ASAERC exhibits substantially lower correlation than linear readout and AERC, indicating that its measurements are less redundant and more complementary, which likely aids generalization and lowers prediction error.

Aspect	Classical RC	AERC	ASAERC
Sensing / readout	Static linear map $\bar{\mathbf{y}}_{t}=\mathbf{W}_{\text{out}}\mathbf{r}_{t}$ .	State–conditional weights $\mathbf{W}_{\text{att},t}=F_{\text{att}}(\mathbf{r}_{t})$ (small MLP) used in prediction, $\bar{\mathbf{y}}_{t}=\mathbf{W}_{\text{att},t}\mathbf{r}_{t}$ .	Dual mechanism: (i) adaptive kernels $\{\phi^{(i)}_{t+T}\}$ select where to measure ; (ii) attention weights $\mathbf{W}_{\text{att},t+T}$ decide how to combine .
Trainable parameters	Only $\mathbf{W}_{\text{out}}$ trained by ridge regression or gradient descent.	Parameters of $F_{\text{att}}$ trained by gradient descent.	Parameters of $F_{\theta}$ and $F_{\text{att}}$ (often shared backbone) trained by gradient descent.

Table 1: Comparison of classical Reservoir Computing (RC), Attention–Enhanced RC (AERC), and Adaptive-Sensing AERC (ASAERC).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Delay-embedding MLP baseline. Full (all eight systems) and individual normalized MSE versus number of delay steps

k

and parameter count. Double Pendulum (blue), Duffing Oscillator (orange), Henon Map (green), Logistic Map (red), Lorenz System (purple), Mackey-Glass Equation (brown), Roessler System (pink), Van der Pool Oscillator (grey). Performance improves for small

k

however worsens beyond an optimal value, likely due to increasing input dimensionality and insufficient training or overfitting.

Figure 7: LSTM baseline. Full (all eight systems) and individual normalized MSE versus total parameter count and parameter count over hidden size. Performance improves monotonically for higher parameter counts, showing a more stable behaviour than the delay-embedding approach. The performance is on par with the best ASAERC models while also having roughly the same parameter count. Double Pendulum (blue), Duffing Oscillator (orange), Henon Map (green), Logistic Map (red), Lorenz System (purple), Mackey-Glass Equation (brown), Roessler System (pink), Van der Pool Oscillator (grey).

Figure 8: Fig. 3 MSE for the individual dynamical systems.

Adaptive Sensing

See all 19 papers

Representation Learning for Spatiotemporal Physical Systems
Mar 13, 2026Helen Qu, Rudy Morel, Michael McCabe +4Representation Learning Scientific ML
Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM +1
Information-theoretic receding-horizon active learning of nonlinear dynamical systems
Sep 29, 2026Juncal Arbelaiz, Anushri Arora, Jonathan W. PillowActive Learning Nonlinear System Identification
Princeton Neuroscience Institute and Center for Statistics and Machine Learning, Princeton University · Computer Science Department, Princeton University
Adaptive Reservoir Computing for Multi-Scenario Chaotic System Forecasting
May 27, 2026Shadmehr Zaregarizi, Khashayar YavariDynamic Neural Networks Forecasting Benchmarks
Politecnico di Torino Turin, Italy

Mar 13, 2026J/K move · Enter open · S save

Representation Learning for Spatiotemporal Physical Systems

Helen Qu, Rudy Morel, Michael McCabe, Alberto Bietti+3

Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM+1

Representation Learning Scientific ML Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture

Open paper AskPDF arXiv

Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.

Abstract

Adaptive Sensing

Explore similar work

Representation Learning for Spatiotemporal Physical Systems

Information-theoretic receding-horizon active learning of nonlinear dynamical systems

Adaptive Reservoir Computing for Multi-Scenario Chaotic System Forecasting