Authors: Ellis R. Crabtree, Dimitris G. Giovanis, Anastasia Georgiou, George Datseris, Ioannis G. Kevrekidis
Organizations: Dept. of Predictive Analytics Johns Hopkins All Children’s Hospital St. Petersburg, FL 33701 · Dept. of Civil and Systems Engineering Johns Hopkins University Baltimore, MD 21218 · Dept. of Chemical and Biomolecular Engineering Johns Hopkins University Baltimore, MD 21218 · Dept. of Mathematics and Statistics University of Exeter Exeter, EX4 4QJ
The identification and reconstruction of the boundaries separating basins of attraction in multistable, multidimensional dynamical systems presents a fundamental challenge in computational dynamics. These structures govern transition pathways and other important large timescale behavior, yet they remain typically under-sampled since their neighborhood does not get routinely visited during direct simulations. Traditional computational approaches face computational limitations in high-dimensional systems and require a priori knowledge of the dynamical system and its equations. Simplistic sampling methods such as random or uniform sampling of the phase space typically fail to quantitatively approximate separatrices and their structure altogether. We introduce and implement a framework that combines supervised classification with generative modeling to address this challenge. Our approach first trains neural network classifiers on uniformly or randomly sampled initial conditions labeled by their corresponding basins of attraction in the system of interest. Using uncertainty metrics of the trained classifier to quantify decision boundaries, the method then identifies these high uncertainty regions and boundaries of the classifier as preliminary approximate separatrices. Subsequently, score-based generative models are trained specifically on samples from high-uncertainty regions, ultimately generating densities of samples consistent with the empirical density of samples on or close to the manifold that constitutes the separatrix between basins in the sampled region. This approach leverages the complementary strengths of (a) discriminative models for global phase space partitioning and (b) generative models for detailed geometric sampling, resulting in a systematic, iterative, data-driven framework that produces empirically consistent reconstructions of (approximate) separatrix manifolds.
Figures & tables
Figure 1: Newton fractal basin classification for the complex polynomial f(z)=z3−1 , illustrating the uncertainty quantification and generative sampling framework across four subfigures. Top-left: uniformly sampled initial conditions in the complex plane, labeled and colored by their basin of attraction under Newton’s method iteration zn+1=zn−f(zn)/f′(zn) . Top-right: high-entropy points identified by the UQ metric derived from the classifier neural network, highlighting regions of fractal boundary ambiguity where basin membership is uncertain. Bottom-left: generated samples produced by a score-based generative model (SGM) trained exclusively on the high-entropy initial conditions, learning to reproduce the complex boundary structure. Bottom-right: overlay comparison in the complex plane contrasting the true high-entropy data against the SGM-generated samples, demonstrating the model’s ability to capture the fractal geometry near basin boundaries.
Figure 2: For the Newton fractal, marginal distributions in histogram form of (a) the high entropy samples corresponding to the decision boundary of the classifier plotted along with (b) the marginal distributions of the SGM generated samples.
Metric
Value
Chamfer Distance
0.0115
Sliced Wasserstein Distance
0.0493
Table 1: Marginal Distribution Comparison Metrics for the Newton Fractal
Figure 3: Uniformly sampled initial conditions of the CSTR model equations defined by equations 2 and 3 colored by binary label of the corresponding attractor identified by time to convergence (purple for the stable steady state and yellow for the stable limit cycle).
Figure 4: Sensitivity analysis of the entropy threshold selection for the CSTR system. Top row: High-entropy regions identified at the 85th, 90th, and 95th percentile thresholds of the instability score defined by equation 4 colored as purple, cyan, and light green, respectively, plotted with the entropy field computed by the MC Dropout classifier shaded from dark red to white. Bottom: Instability score as a function of the percentile threshold; lower values indicate greater stability with respect to threshold perturbations. The green dashed line indicates the most stable threshold, while colored dotted lines correspond to the thresholds visualized in the top row.
Figure 5: Comparison of high-entropy points and the SGM-generated samples for the CSTR system. Left: Scatter plot overlay of the high-entropy training points (blue) extracted from the MC Dropout classifier at the 90th percentile threshold and the samples (red) generated by the trained score-based generative model. The spatial agreement between the two distributions indicates that the SGM has successfully learned to sample from the separatrix. Middle: Marginal density distributions along the x1 coordinate (dimensionless concentration), showing close agreement between the high-entropy and the generated samples. Right: Marginal density distributions along the x2 coordinate. The strong overlap in both marginal distributions demonstrates that the generative model accurately captures the statistical structure of the classifier’s decision boundary corresponding to the dynamical separatrix. The “peaks" in the high entropy marginals are a product of their uniform sampling.
Figure 6: For the Lorenz System, Comparison of high entropy points from three different methods (MC dropout, deep ensemble averaging, and Laplace approximation) versus a sampled manifold obtained via SCIGMA. The three metrics used are Chamfer distance, coverage, and Hausdorff distance.
Figure 7: Three-dimensional plots of the approximated 2D Lorenz Manifold. Left: An overlaid comparison of the generated approximation of the manifold using the generative method plotted as red points and the sampled manifold found via SCIGMA in blue points. Right: the SCIGMA calculated manifold points, colored by distance to the approximated manifold (effectively the Chamfer distance).
Figure 8: Sets of two dimensional projections from various sections of the separatrix calculated using traditional methods (plotted in blue), versus the generated points that approximate the separatrix (maked in red). Inset metrics describing the Chamfer distance, coverage, and sliced Wasserstein distance of each projection are included in each plot.
Dynamical systems studies how states evolve over time, including stability, feedback, chaos, and transitions in physical, biological, and engineered systems.
We study the problem of identifying dynamically distinct basins of attraction in high dimensional time-homogeneous Markov processes using only trajectory sampling. This problem is fundamental in the analysis of metastable dynamical systems, where the process rapidly mixes within basins while transitions between basins occur rarely on the timescale of interest, or even when the state space is reducible. Existing approaches typically rely on spatial discretization or spectral analysis of estimated transition operators, which can become unreliable in high dimensional settings or when the underlying basin geometry is highly nonlinear. We propose a discriminative approach to basin identification based on marginal trajectory distribution comparison. We prove a simple risk separation result: if two initial states belong to the same basin, the Bayes-optimal classifier distinguishing their marginal trajectory distributions achieves risk close to 1/2, whereas if they lie in distinct basins, the optimal risk is close to zero. This observation reduces basin detection to a two-sample discrimination problem between marginal trajectory distributions. Motivated by this principle, we develop a neural algorithm that receives a set of candidate basin representatives and iteratively merges them by estimating classification risk with a neural network that approximates the Bayes classifier. We evaluate the method on various metastable systems. These include synthetic systems constructed by embedding low-dimensional dynamics into high dimensional noisy ambient spaces. In these settings, standard spectral and clustering-based methods often fail, while our approach accurately recovers the underlying basin structure. These results display a shortcoming of existing methods and highlight trajectory discrimination as an effective tool for identifying dynamical basins in high dimensional stochastic systems.
Bifurcations are ubiquitous in physical systems, from structural buckling to fluid and climate dynamics, yet they remain largely unexplored in deep learning. At a symmetry-breaking bifurcation, a single input admits multiple equally valid solutions, violating the one-to-one assumption underlying most learned physical surrogates. We introduce Bi-FORK, a generative framework for learning these one-to-many solution maps in high-dimensional systems. Bi-FORK generates complete trajectories through latent flow matching, preserving space and time coherence, and uses repulsion-guided sampling to recover distinct solution branches in a single amortized pass. We evaluate Bi-FORK on buckling beams, mechanical metamaterials, and Allen-Cahn phase separation, spanning continuous, discrete, and field-valued bifurcations with discretizations up to 260,000 points. Bi-FORK recovers the multimodal solution structure while scaling several orders of magnitude beyond prior approaches, opening generative modeling to high-dimensional bifurcating physical systems.
Anna Zimmel, Fleur Hendriks, Markus Holzleitner +4
ELLIS Unit, LIT AI Lab, Institute for Machine Learning, JKU Linz · Department of Mechanical Engineering, Eindhoven University of Technology · DIFFER – Dutch Institute for Fundamental Energy Research +2
Koopman autoencoders (KAEs) seek a higher-dimensional latent representation in which nonlinear dynamics evolve linearly. However, many interesting systems have multiple basins of attraction, and both theoretical and empirical work has shown these multibasin systems cannot generally admit a single finite-dimensional global Koopman embedding under standard assumptions. We posit that encoders with a sparsity-inducing objective encouraging few active latent coefficients will provide latent supports as an inspectable basin-modeling principle for Koopman autoencoders. We use these encoders producing sparse latents in training Sparse Koopman Autoencoders (SKAEs) without basin labels or other regime annotations, and treat the learned latent supports as model-produced regime variables after training. Across a range of procedurally generated multibasin systems and chaotic flows, we show that SKAEs have superior forecasting performance compared to dense-latent KAEs. We also perform a mechanistic study that shows latent supports produced by SKAEs are both essential for the quality of the representation and useful for identifying basins on basin-interior states, whereas dense-latent KAEs collapse to an uninformative single support family. These results identify sparse latents and their corresponding supports as label-free, interpretable regime variables for Koopman learning in nonlinear systems with multiple local dynamical laws.