Dynamical Systems

Ask about this topic

Momentum

44 papers in the last four weeks, up 159% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 319

All topics
CardsList
Dec 5, 2025stat.ML

Symmetric Linear Dynamical Systems are Learnable from Few Observations

We consider the problem of learning the parameters of a NN-dimensional stochastic linear dynamics under both full and partial observations from a single trajectory of time TT. We introduce and analyze a new estimator that achieves a small maximum element-wise error on the recovery of symmetric dynamic matrices using only T=O(log⁡N)T=\mathcal{O}(\log N) observations, irrespective of whether the matrix is sparse or dense. This estimator is based on the method of moments and does not rely on problem-specific regularization. This is especially important for applications such as structure discovery.

Minh Vu, Andrey Y. Lokhov, Marc Vuffray

Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA

Dynamical SystemsParameter EstimationStochastic Linear Dynamical SystemsSystem Identification
AskPDF
Nov 12, 2025math.DS

When is a System Discoverable from Data? Discovery Requires Chaos

The deep learning revolution has spurred a rise in advances of using AI in sciences. Within physical sciences the main focus has been on discovery of dynamical systems from observational data. Yet the reliability of learned surrogates and symbolic models is often undermined by the fundamental problem of non-uniqueness. The resulting models may fit the available data perfectly, but lack genuine predictive power. This raises the question: under what conditions can the systems governing equations be uniquely identified from a finite set of observations? We show, counter-intuitively, that chaos, typically associated with unpredictability, is crucial for ensuring a system is discoverable in the space of continuous or analytic functions. The prevalence of chaotic systems in benchmark datasets may have inadvertently obscured this fundamental limitation. More concretely, we show that systems chaotic on their entire domain are discoverable from a single trajectory within the space of continuous functions, and systems chaotic on a strange attractor are analytically discoverable under a geometric condition on the attractor. As a consequence, we demonstrate for the first time that the classical Lorenz system is analytically discoverable. Moreover, we establish that analytic discoverability is impossible in the presence of first integrals, common in real-world systems. These findings help explain the success of data-driven methods in inherently chaotic domains like weather forecasting, while revealing a significant challenge for engineering applications like digital twins, where stable, predictable behavior is desired. For these non-chaotic systems, we find that while trajectory data alone is insufficient, certain prior physical knowledge can help ensure discoverability. These findings warrant a critical re-evaluation of the fundamental assumptions underpinning purely data-driven discovery.

Zakhar Shumaylov, Peter Zaika, Philipp Scholl +3

University of Cambridge · LMU Munich and Munich Center for Machine Learning (MCML) · IBM Research

Dynamical SystemsNonlinear System IdentificationSystem IdentificationChaotic Dynamical Systems
AskPDF
Oct 29, 2025math.OC

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU

In this work, we study the nonlinear dynamics of a shallow neural network trained with mean-squared loss and leaky ReLU activation. Under Gaussian inputs and equal layer width k, (1) we establish, based on the equivariant gradient degree, a theoretical framework, applicable to any number of neurons k>= 4, to detect bifurcation of critical points with associated symmetries from global minimum as leaky parameter αα varies. Typically, our analysis reveals that a multi-mode degeneracy consistently occurs at the critical number 0, independent of k. (2) As a by-product, we further show that such bifurcations are width-independent, arise only for nonnegative αα and that the global minimum undergoes no further symmetry-breaking instability throughout the engineering regime αα in range (0,1). An explicit example with k=5 is presented to illustrate the framework and exhibit the resulting bifurcation together with their symmetries.

Jingzhou Liu

Shallow Neural NetworksDynamical SystemsNeural Network OptimizationNeural Network Activation FunctionsNeural Network Training Dynamics
AskPDF
Oct 18, 2025cs.LG

Simulation-free Structure Learning for Stochastic Population Dynamics

Modeling dynamical systems and unraveling their underlying structural dependencies is central to many domains in the natural sciences. Various physical systems, such as those arising in cell biology, are inherently high-dimensional and stochastic in nature, and admit only partial, noisy state measurements. Our primary motivating setting is single-cell biology, where destructive measurements yield unpaired population snapshots rather than longitudinal trajectories of the same cells. This poses a significant challenge for addressing the problems of modeling the underlying dynamics and inferring the network structure of these systems. Existing methods are typically tailored either for structure learning or modeling dynamics at the population level, but are limited in their ability to address both problems together. In this work, we address both problems simultaneously: we present StructureFlow, a novel and principled simulation-free training approach for jointly learning the structure and stochastic population dynamics of physical systems. We showcase the utility of StructureFlow for the tasks of structure learning from interventions and dynamical (trajectory) inference of conditional population dynamics. We empirically evaluate our approach on high-dimensional synthetic systems, a set of biologically plausible simulated systems, and an experimental single-cell dataset. We show that StructureFlow can learn the structure of underlying systems while simultaneously modeling their conditional population dynamics --- a key step toward model-based mechanistic understanding of systems behavior.

Noah El Rimawi-Fine, Adam Stecklov, Lucas Nelson +4

McGill University · Mila – Quebec AI Institute · AITHYRA +5

Dynamical SystemsCausal Structure LearningConditional Generative Modeling
AskPDF
Oct 2, 2025cs.LG

Neural non-canonical Hamiltonian dynamics for long-time simulations

This work focuses on learning non-canonical Hamiltonian dynamics from data, where long-term predictions require the preservation of structure both in the learned model and in numerical schemes. Previous research focused on either facet, respectively with a potential-based architecture and with degenerate variational integrators, but new issues arise when combining both. In experiments, the learnt model is sometimes numerically unstable due to the gauge dependency of the scheme, rendering long-time simulations impossible. In this paper, we identify this problem and propose two different training strategies to address it, either by directly learning the vector field or by learning a time-discrete dynamics through the scheme. Several numerical test cases assess the ability of the methods to learn complex physical dynamics, like the guiding center from gyrokinetic plasma physics.

Clémentine Courtès, Emmanuel Franck, Michael Kraus +2

IRMA, MACARON · Institut de Recherche Mathématique Avancée, UMR 7501 Université de Strasbourg et CNRS, 7 rue René Descartes 672000 Strasbourg, France. · INRIA Nancy-Grand Est, MACARON Project, Strasbourg, France. +5

Dynamical SystemsNonlinear System IdentificationHamiltonian Neural NetworksPhysics-Informed MLHamiltonian Dynamics
AskPDF
Sep 21, 2025math.NA

Data-efficient Kernel Methods for Learning Hamiltonian Systems

Hamiltonian dynamics describe a wide range of physical systems. As such, data-driven simulations of Hamiltonian systems are important for many scientific and engineering problems. In this work, we propose kernel-based methods for identifying and forecasting Hamiltonian systems directly from trajectory data. We present two approaches: a 2-step method that reconstructs trajectories before learning the Hamiltonian, and a 1-step method that jointly infers both. Across several benchmark systems, including mass-spring dynamics, a nonlinear pendulum, and the Henon-Heiles system, we demonstrate that our framework achieves accurate, data-efficient predictions and outperforms 2-step kernel-based baselines, particularly in scarce-data regimes, while preserving the Hamiltonian structure. Moreover, we prove a priori error estimates, ensuring reliability of the learned models. We also provide a more general, problem-agnostic numerical framework that goes beyond Hamiltonian systems and can be used for data-driven learning of arbitrary dynamical systems.

Yasamin Jalalian, Mostafa Samir, Boumediene Hamzi +2

Department of Computing and Mathematical Sciences, Caltech, CA, USA. · Beyond Limits, CA, USA. · The Alan Turing Institute, London, UK. +1

Dynamical SystemsNonlinear System IdentificationSystem IdentificationKernel MethodsHamiltonian Dynamics
AskPDF
Jul 30, 2025cs.LG

Diagrams-to-Dynamics (D2D): Exploring Causal Loop Diagram Leverage Points under Uncertainty

Background: Causal loop diagrams (CLDs) are widely used in health and environmental research to represent hypothesized causal structures underlying complex problems. However, as qualitative and static representations, CLDs are limited in their ability to support dynamic analysis and inform intervention strategies. We propose Diagrams-to-Dynamics (D2D), a method for converting CLDs into exploratory system dynamics models in the absence of empirical data. With minimal user input - following a protocol to label variables as stocks, flows or auxiliaries, and constants - D2D utilizes the structural information already encoded in CLDs, namely the existence and polarity of causal connections, to simulate hypothetical interventions and explore potentially influential places to intervene, known as 'leverage points,' under uncertainty. Results: D2D helps distinguish between high- and low-ranked leverage points. We compare D2D to a calibrated system dynamics model constructed from the same CLD and variable labels. D2D showed greater consistency with the calibrated model than did static network centrality analysis, while also providing uncertainty estimates and guidance for future data collection. Conclusions: The D2D method is implemented in an open-source Python package and a web-based application to support further testing and to lower the barrier to dynamic modeling for researchers working with CLDs. Future studies could help establish the approach's utility across a broad range of cases and domains.

Jeroen F. Uleman, Loes Crielaard, Leonie K. Elsenburg +4

Copenhagen Health Complexity Center, University of Copenhagen, Copenhagen, Denmark. · Department of Public and Occupational Health, Amsterdam UMC University of Amsterdam, Amsterdam, The Netherlands. · Center for Urban Mental Health, University of Amsterdam, Amsterdam, The Netherlands. +4

Dynamical SystemsCausal Intervention
AskPDF
Jul 14, 2025cond-mat.dis-nn

Dynamical stability for dense patterns in attractor neural networks

Recurrent neural networks are canonical models of biological memory. In these models, memories are represented by distributed patterns of neural activity that are stored in the recurrent connections between neurons, such that they become attractors of the network's dynamics. During memory recall, network dynamics thus converge toward one of these memory patterns when started from a noisy or partial cue. Therefore, memory performance critically hinges on the dynamical stability of the stored patterns. However, previous theoretical approaches only studied dynamical stability under highly restrictive conditions that do not readily apply to biological neural circuits. Here, we develop a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise. Using methods from random matrix theory, we analyze the bulk and outliers of the eigenvalue spectra of the Jacobians that characterize network dynamics around fixed points. We show that either all fixed points are stable or all of them are unstable, depending on whether their number is below a ``critical load for stability'', which is distinct from the classical critical capacity that measures the maximal number of achievable fixed points regardless of their stability. We further analyze the dependence of this critical load for stability on experimentally measurable quantities characterizing the statistics of memory patterns and the activation functions of neurons. Our analysis highlights the computational benefits of sparse-like patterns and threshold-linear activation functions and offers testable predictions for neural circuits supporting memory.

Uri Cohen, Máté Lengyel

Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK · Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary

Dynamical SystemsNeural ProcessesNeural Network MemorizationHopfield NetworksRecurrent Neural NetworksAssociative Memory
AskPDF
Jun 25, 2025cs.LG

Stochastic and Non-local Closure Modeling for Nonlinear Dynamical Systems via Latent Score-based Generative Models

We propose a latent score-based generative AI framework for learning stochastic, non-local closure models and constitutive laws in nonlinear dynamical systems of computational mechanics. This work addresses a key challenge of modeling complex multiscale dynamical systems without a clear scale separation, for which numerically resolving all scales is prohibitively expensive, e.g., for engineering turbulent flows. While classical closure modeling methods leverage domain knowledge to approximate subgrid-scale phenomena, their deterministic and local assumptions can be too restrictive in regimes lacking a clear scale separation. Recent developments of diffusion-based stochastic models have shown promise in the context of closure modeling, but their prohibitive computational inference cost limits practical applications in many real-world settings. This work addresses this limitation by jointly training convolutional autoencoders with conditional diffusion models in latent space, significantly reducing the dimensionality of the sampling process while preserving essential physical characteristics. Numerical results demonstrate that the joint training approach helps discover a proper latent space that not only guarantees small reconstruction errors but also ensures good performance of the diffusion model in the latent space. When integrated into numerical simulations, the proposed stochastic modeling framework via latent conditional diffusion models achieves significant computational acceleration while maintaining comparable predictive accuracy to standard diffusion models in physical space.

Xinghao Dong, Huchen Yang, Jin-Long Wu

Department of Mechanical Engineering, University of Wisconsin-Madison, Madison, WI, 53706, USA

Dynamical SystemsLatent Diffusion ModelsConditional Diffusion ModelsDiffusion Model Inference Acceleration
AskPDF
May 16, 2025cs.LG

Regularity and Stability Properties of Selective SSMs with Discontinuous Gating

Selective State-Space Models (SSMs) such as Mamba have become central to long-sequence modeling. Still, their stability is poorly understood: their state-space coefficients are modulated online by a token-dependent gating signal, making the recurrence neither linear time-invariant nor classically nonlinear. We study continuous-time selective SSMs through passivity, dissipativity, and Input-to-State Stability (ISS), explicitly separating the selection signal x(⋅)x(\cdot) from the driving input u(⋅)u(\cdot). We obtain four results: exponential forgetting under strict dissipativity; a canonical AUCloc\mathrm{AUC}_{\mathrm{loc}} quadratic storage for the frozen-selection subsystem that accommodates discontinuous gating; a parametric LMI together with universal kernel constraints and "irreversible forgetting" under universal quadratic storage; and sufficient conditions for global ISS uniformly over admissible selection schedules. We then bridge to practice by deriving a sampled block LMI for the Mamba selective-scan core, which is used as a differentiable training-time regularizer. Across seven standard time-series datasets and four prediction horizons, the regularizer reduces sampled Mamba-core LMI violations by roughly 92%92\% in 28/2828/28 pairs at a clean-MSE cost of less than 0.018%0.018\%. It improves internal Mamba passivity and state-norm diagnostics under injected perturbations. Our results turn classical control-theoretic tools into verifiable structural and training criteria for selective SSMs, while honestly scoping which guarantees transfer to a deep selective-scan architecture.

Nikola Zubić, Davide Scaramuzza

Robotics and Perception Group University of Zurich

Dynamical SystemsSelective SSMs
AskPDF
Apr 11, 2025cs.MA

A Hybrid ABM-PDE Framework for Real-World Infectious Disease Simulations

This paper presents a hybrid modeling approach that couples an Agent-Based Model (ABM) with a partial differential equation (PDE) model in an epidemic setting to simulate the spatial spread of infectious diseases using a compartmental structure with seven health states. The goal is to reduce the computational complexity of a full-ABM by introducing a coupled ABM-PDE model that offers significantly faster simulations while maintaining comparable accuracy. Our results demonstrate that the hybrid model not only reduces the overall simulation runtime (defined as the number of runs required for stable results multiplied by the duration of a single run) but also achieves smaller errors across both 25% and 100% population samples. The coupling mechanism ensures consistency at the model interface: agents crossing from the ABM into the PDE domain are removed and represented as density contributions, while surplus density in the PDE domain is used to generate agents with plausible trajectories derived from mobile phone data. We evaluate the hybrid model using real-world mobility and infection data for the Berlin-Brandenburg region in Germany, showing that it captures the core epidemiological dynamics while enabling efficient large-scale simulations. These results demonstrate that the proposed ABM-PDE framework provides a robust and computationally efficient alternative to full-scale agent-based simulations, making it suitable for realistic epidemic modeling and scenario analysis.

Kristina Kehrer, Tim O. F. Conrad

Zuse Institute Berlin Berlin, 14195

Dynamical SystemsAgent-Based Modeling
AskPDF
Feb 1, 2025cs.LG

Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs

Surrogate models of parametric dynamical systems are essential for many-query and real-time predictions in engineering applications such as design optimization and digital twins. However, generating high-fidelity (HF) training data over a broad range of parameters and operating conditions remains computationally expensive. To address this challenge, we propose a Bayesian multi-fidelity Laplace neural operator (MF-LNO) for uncertainty-aware active learning of oscillatory parametric PDEs. Specifically, the proposed Bayesian MF-LNO iteratively calibrates the discrepancy between low- and high-fidelity data, where predictive uncertainty guides the adaptive acquisition of informative HF trajectories. Such predictive uncertainty is quantified via replica-exchange stochastic gradient Langevin dynamics (reSGLD), whose broad posterior exploration enables uncertainty to serve as an error indicator for adaptive HF sample acquisition. Numerical experiments on the Lorenz system, Duffing oscillator, and beam dynamics demonstrate that uncertainty-guided HF sample acquisition consistently outperforms random sampling, while the proposed Bayesian MF-LNO achieves higher prediction accuracy than MF-DeepONet with predictive uncertainty quantification. These results demonstrate that Bayesian multi-fidelity LNOs, combined with uncertainty-guided active learning, provide a data-efficient framework for operator learning in engineering dynamical systems.

Bongseok Kim, Haoyang Zheng, Michael Penwarden +1

School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA. · Department of Mathematics, Purdue University, 150 North University Street, West Lafayette, 47907, IN, USA.

Dynamical SystemsSurrogate ModelingUncertainty QuantificationActive LearningPDE Operator LearningNeural Operators
AskPDF
Oct 31, 2024cs.LG

Projected Neural Differential Equations for Learning Constrained Dynamics

Neural differential equations offer a powerful approach for learning dynamical systems from data. However, they do not inherently respect known constraints, such as conservation laws, that should be obeyed by the learned dynamics. It is well known that enforcing constraints in data-driven models can enhance their generalizability and numerical stability. In this paper, we introduce projected neural differential equations (PNDEs), a method for constraining neural differential equations based on projection of the predicted velocities onto the tangent space of the manifold that fulfills the constraint. In tests on several examples from different fields, including chaotic dynamical systems and power grid models, PNDEs outperform existing methods for constraining learned dynamics, require fewer hyperparameters, and are computationally more efficient. The proposed approach demonstrates potential for enhancing the modeling of constrained dynamical systems, particularly in domains where accuracy and reliability are essential.

Alistair White, Anna Büttner, Maximilian Gelbrecht +4

Munich Climate Center and Earth System Modelling Group, Department of Aerospace and Geodesy, School of Engineering and Design, Technical University of Munich, Munich, Germany · Potsdam Institute for Climate Impact Research, Potsdam, Germany · California Institute of Technology, Pasadena, California, USA +4

Dynamical SystemsNeural ODEs
AskPDF
Oct 14, 2024eess.SY

Automated Discovery of Operable Dynamics from Videos

Dynamical systems form the foundation of scientific discovery, traditionally modeled with predefined state variables such as the angle and angular velocity, and differential equations such as the equation of motion for a single pendulum. We introduce a framework that automatically discovers a low-dimensional and operable representation of system dynamics, including a set of compact state variables that preserve the smoothness of the system dynamics and a differentiable vector field, directly from video without requiring prior domain-specific knowledge. The prominence and effectiveness of the proposed approach are demonstrated through both quantitative and qualitative analyses of a range of dynamical systems, including the identification of stable equilibria, the prediction of natural frequencies, and the detection of chaotic and limit cycle behaviors. The results highlight the potential of our data-driven approach to advance automated scientific discovery.

Kuang Huang, Dong Heon Cho, Boyuan Chen

Department of Applied Physics and Applied Mathematics, Columbia University, 500 W. 120th Street, New York, 10027, NY, USA. · Department of Computer Science, Duke University, 308 Research Dr., Durham, 27705, NC, USA. · Department of Mechanical Engineering and Material Science, Duke University, 121 Hudson Hall, Durham, 27708, NC, USA. +1

Dynamical SystemsLatent Dynamics ModelingSystem IdentificationAutonomous Scientific Discovery
AskPDF
Jun 7, 2024cs.LG

Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes

We study gradient descent (GD) dynamics on logistic regression problems with large, constant step sizes. For linearly-separable data, it is known that GD converges to the minimizer with arbitrarily large step sizes, a property which no longer holds when the problem is not separable. In fact, the behaviour can be much more complex -- a sequence of period-doubling bifurcations begins at the critical step size 2/λ2/λ, where λλ is the largest eigenvalue of the Hessian at the solution. Using a smaller-than-critical step size guarantees convergence if initialized nearby the solution: but does this suffice globally? In one dimension, we show that a step size less than 1/λ1/λ suffices for global convergence. However, for all step sizes between 1/λ1/λ and the critical step size 2/λ2/λ, one can construct a dataset such that GD converges to a stable cycle. In higher dimensions, this is actually possible even for step sizes less than 1/λ1/λ. Our results show that although local convergence is guaranteed for all step sizes less than the critical step size, global convergence is not, and GD may instead converge to a cycle depending on the initialization.

Si Yi Meng, Antonio Orvieto, Daniel Yiming Cao +1

Department of Computer Science Cornell University Ithaca, NY, USA · ELLIS Institute Tübingen MPI for Intelligent Systems Tübingen AI Center, Germany

Dynamical SystemsLogistic RegressionOptimization Convergence AnalysisGradient DescentGradient Descent Dynamics
AskPDF
Oct 28, 2021cs.LG

Roto-translated Local Coordinate Frames For Interacting Dynamical Systems

Modelling interactions is critical in learning complex dynamical systems, namely systems of interacting objects with highly non-linear and time-dependent behaviour. A large class of such systems can be formalized as geometric graphs\textit{geometric graphs}, i.e.\textit{i.e.}, graphs with nodes positioned in the Euclidean space given an arbitrarily\textit{arbitrarily} chosen global coordinate system, for instance vehicles in a traffic scene. Notwithstanding the arbitrary global coordinate system, the governing dynamics of the respective dynamical systems are invariant to rotations and translations, also known as Galilean invariance\textit{Galilean invariance}. As ignoring these invariances leads to worse generalization, in this work we propose local coordinate frames per node-object to induce roto-translation invariance to the geometric graph of the interacting dynamical system. Further, the local coordinate frames allow for a natural definition of anisotropic filtering in graph neural networks. Experiments in traffic scenes, 3D motion capture, and colliding particles demonstrate that the proposed approach comfortably outperforms the recent state-of-the-art.

Miltiadis Kofinas, Naveen Shankar Nagaraja, Efstratios Gavves

University of Amsterdam · BMW Group

Dynamical SystemsEquivariant GNNsSpatiotemporal GNNs
AskPDF
Date pendingcs.RO

A Dynamic Toolkit for Transmission Characteristics of Precision Reducers with Explicit Contact Geometry

Precision reducers couple contact geometry, bearing support, structural deformation, and loading history. This paper presents a dynamic toolkit connecting distributed local contacts to the complete mechanical reaction path. Work-conjugate maps transfer displacement, reaction, and tangent contributions between contacts and rigid or reduced coordinates, with explicit allocation of contact and body elasticity. An implicit generalized-alpha solution distinguishes trial evaluations from accepted history. Contact records then support performance protocols and configured geometric or constitutive feedback. The numerical studies focus on mechanical coupling and pressure recovery. An idealized annular housing retains physical interfaces while its structural coordinates are reduced. Two reductions with similar static errors have cross-port response errors of 24.975 and 0.312 percent over the same frequency band relative to a common parent model. In a shared-pin example, a 20 micrometer radial displacement of one wheel changes the load on a second, fixed wheel by approximately 75 N. Removing cross-station compliance removes this incremental transfer on the tested sleeve-seating branch. A double-wheel cycloidal assembly relates torsional branch response to aggregate contact-load variation and normalized pressure fields. The pressure maxima remain sensitive to resolution despite small discrete force residuals. Further formulations specify how motion and contact records support precision, vibration, heat, wear, and durability models with their required inputs. The framework separates model representation and numerical resolution while retaining common definitions of motion, force, and observation.

Jiacheng Miao, Chao Liu, Qiliang Wang +2

Intelligent Equipment Division, CRRC Qishuyan Institute Co., Ltd, Changzhou, 213011, China · School of Mechanical Engineering, Dalian Jiaotong University, Dalian, 116028, China · Research Institute, CRRC Qishuyan Institute Co., Ltd, Changzhou, 213011, China

Dynamical SystemsMultibody DynamicsReduced-Order Modeling
AskPDF
Date pendingcs.LG

Attention by Synchronization in Coupled Oscillator Networks

We address transformer attention on energy-constrained physical substrates. Softmax attention requires exponentiation and global reduction, operations with high energy cost on von Neumann hardware and no natural physical analog. We show that Kuramoto synchronization dynamics (which arise in electrical, mechanical, superconducting, and charge-density-wave oscillator arrays, among other physical systems) implement a well-defined attention operation. The resulting mechanism, \emph{fixed-query oscillator attention}, replaces softmax's arithmetic with the equilibration of a gradient flow on the sphere: queries are learned anchors fixed on the sphere, and free oscillators evolve under Kuramoto--Lohe dynamics until they settle at positions encoding attention weights via cosine similarity. Because the computation is equilibration, no global exponential normalization is needed. The fixed point is provably unique and globally attractive from almost every initial condition, a guarantee that holds across every physical realization. Empirically, at the minimal hardware configuration (oscillator dimension dosc=2d_{\mathrm{osc}} = 2), oscillator attention matches softmax on keyword spotting, and on subject-verb agreement it trains more reliably while reaching softmax's accuracy. Softmax retains an advantage on causal language modeling, but the gap decays as a power law in doscd_{\mathrm{osc}}. The main objective of this work is not to replace softmax in software but to provide a mathematically grounded blueprint for accurate attention on physical substrates.

Fabio Pasqualetti, Taosha Guo

Electrical Engineering and Computer Science University of California, Irvine

Softmax AttentionDynamical SystemsTransformer AttentionAttention Mechanisms
AskPDF
Date pendingq-bio.QM

Diffusion learning reveals viable parameter manifolds and compensation geometry in biological dynamical systems

Models of complex systems often have many parameters, yet are constrained by far fewer experimentally accessible observables; consequently, similar activity can emerge from coordinated parameter changes. We formalize these compatible parameter sets as \emph{viable parameter manifolds}: the inverse images of target dynamical features under a parameter-to-feature map. The relevant codimension is not the number of reported features, but the effective rank of that map at the target scale. Locally redundant features lower the effective codimension, while poor conditioning, high curvature, or regime mixing degrade learnability. We train conditional score-based diffusion models on simulated parameter--feature pairs and use them as amortized samplers of prior-weighted viable sets. In the Lorenz system, scalar trajectory statistics generate thin viable sheets, and a finite-tolerance conditioning localizes a transition-adjacent corridor. In the Izhikevich neuron model, four firing descriptors lie close to a nearly two-dimensional family of features, and the learned inverse images reveal distinct regular and irregular compensation geometries. In a deterministic ODE reduction of finite spiking networks, the same framework reveals excitatory--inhibitory compensation, timescale--coupling tradeoffs, and viable manifolds across 4--12 parameter dimensions. In this view, robustness, compensation, and hidden parameter dependencies are organized as inverse geometry, with diffusion models providing practical tools for sampling, visualizing, and interrogating that geometry.

Ruilin Zhang, Louis Tao, Zhuo-Cheng Xiao

Peking University–Tsinghua University–National Institute of Biological Sciences Joint Graduate Program, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China · Center for Bioinformatics, School of Life Sciences, Peking University, Beijing 100871, China · Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China +2

Dynamical SystemsParameter Identifiability
AskPDF