Bifurcations from Weight Space: JEPAWG on Parametric Dynamical Systems

Tobias Göbel; Miranda C. N. Cheng

cs.LGOct 22, 2026

Bifurcations from Weight Space: JEPAWG on Parametric Dynamical Systems

Authors: Tobias Göbel, Miranda C. N. Cheng

Organizations: Institute for Mathematics, Academia Sinica, Taipei, Taiwan · Department of Physics, National Taiwan University, Taipei, Taiwan · Institute of Physics, University of Amsterdam, Amsterdam, Netherlands

Abstract

Trained network weights are increasingly treated as data, but there is rarely ground truth about what a population of models should encode, which hampers systematic validation of weight-space methods. This makes parametric dynamical systems a natural test-bed: the dynamics change qualitatively at submanifolds of parameter space that are, in well-studied families, known analytically or numerically. Fine-tuning a forecasting network at each point of a parameter grid yields a model zoo indexed by a physical parameter, against which weight-space diagnostics can be checked. We build such model zoos for four dynamical systems and analyse them with JEPAWG, a recently proposed joint-embedding hypernetwork built around a learned parameter--weight representation. It was first introduced for neural samplers for lattice field theories, where its latent geometry located the phase transition and estimated the 2D Ising critical exponent. Applied to dynamical systems, JEPAWG is competitive on forecasting with a hypernetwork trained directly for that task, despite being supervised on the parameter--weight correspondence rather than on forecasts. Beyond forecasting, the norm of the parameter-derivative of the decoded weights locates Hopf points and curves, tracks a period-doubling cascade closely enough to give a qualitative estimate of the Feigenbaum constant, and detects a global bifurcation, which has no eigenvalue signature and is therefore invisible to linearisation. As such, JEPAWG provides interpretable insight into the dynamics that is not available from other hypernetworks. Together, these results indicate that JEPAWG is a workable instrument for reading physical structure off a population of trained networks.

Figures & tables

Figure 1: JEPAWG pipeline. (1) A conditional forecaster is trained on trajectories from the parametric family and fine-tuned at each of

M

parameter values, yielding the weight grid

\{(p_{i},\theta_{i})\}_{i=1}^{M}

(Section 4.1 ). (2) Encoders, predictor and decoder are trained in two stages (Eqs. 4 , 5 ), giving the inference path

\hat{\theta}(p)=D(P(E_{p}(p)))

. (3) The decoded weights are evaluated as a hypernetwork and probed via

G(p)=\|\nabla_{\!p}\hat{\theta}(p)\|

, verified against known bifurcations.

system	split	PHLieNet	JEPAWG	VAE	AE
Lorenz, $M{=}800$	interp $\rho\in[10,35]$	$\mathbf{20.23\pm 0.89}$	$\mathbf{20.19\pm 0.15}$	$18.69\pm 0.27$	$18.31\pm 0.64$
	below $\rho\in[5,9]$	$\mathbf{15.53\pm 4.18}$	$3.61\pm 0.25$	$5.58\pm 0.81$	$4.31\pm 1.82$
	above $\rho\in[36,40]$	$2.04\pm 0.22$	$2.03\pm 0.09$	$\mathbf{2.48\pm 0.11}$	$\mathbf{2.21\pm 0.22}$
Rössler, $M{=}1000$	interp $c\in[3,9]$	$\mathbf{50.16\pm 1.93}$	$\mathbf{47.42\pm 0.90}$	$35.84\pm 2.58$	$30.10\pm 2.49$
	below $c\in[1.8,2.8]$	$6.98\pm 2.25$	$\mathbf{42.66\pm 12.53}$	$\mathbf{27.64\pm 12.20}$	$10.44\pm 8.30$
	above $c\in[9.2,10.2]$	$\mathbf{15.72\pm 1.71}$	$9.06\pm 0.92$	$7.86\pm 0.51$	$6.09\pm 0.54$

Table 1: Forecasting performance:

\mathrm{TtT}_{0.2}

on held-out interpolation parameters and on each side of the training range, mean

\pm

std over 5 seeds. Bold: best mean and all entries whose interval overlaps the best entry’s. Rows in which all entries overlap carry no bold. Continuous-time systems in time units, the discrete Hénon map in iterations; Hénon extrapolation is omitted: for discrete maps one inaccurate iteration exits the training region, so

\mathrm{TtT}

collapses to

\approx 0

for every method. Per-experiment transient handling is given in Table 6 and the full test sets in Table 7 .

Figure 2: Extrapolation

\mathrm{TtT}_{0.2}

of JEPAWG and PHLieNet, by regime beyond the training range. Left: limit cycles that keep changing; for Rössler the cycle shrinks as

c

decreases below the training range (

c<3

), for the Brusselator it grows and moves as

b

increases above it (

b>4

). Middle: a stable fixed point, with initial transients included for Lorenz and excluded for the Brusselator. Right: chaotic dynamics.

			JEPAWG		PHLieNet
side	regime	$\lambda_{\max}$	$\mathrm{TtT}_{0.2}$	bounded (%)	$\mathrm{TtT}_{0.2}$	bounded (%)
Rössler, $c\in[1.8,2.8]$	limit cycle	$0$	$\mathbf{42.7\pm 12.5}$	$66\pm 8$	$7.0\pm 2.3$	$49\pm 11$
Brusselator, $b\in[4.2,4.8]$	limit cycle	$0$	$5.7\pm 3.3$	$\mathbf{100\pm 0}$	$3.2\pm 1.1$	$70\pm 27$
Lorenz, $\rho\in[5,9]$	stable fixed pt.	$-0.65$	$3.6\pm 0.3$	$54\pm 7$	$\mathbf{15.5\pm 4.2}$	$\mathbf{97\pm 6}$
Brusselator, $b\in[1.1,1.4]$	stable fixed pt.	$-0.58$	$0.8\pm 0.2$	$0\pm 0$	$5.4\pm 6.0$	$0\pm 0$
Rössler, $c\in[9.2,10.2]$	chaotic	$0.11$	$9.1\pm 0.9$	$100\pm 0$	$\mathbf{15.7\pm 1.7}$	$93\pm 7$

Table 2: Extrapolation by regime of the true dynamics beyond the training range:

\mathrm{TtT}_{0.2}

(short-horizon tracking) and bounded fraction (share of rollouts that stay within

1.5\times

the extent of the true test trajectories over the full horizon), mean

\pm

std over 5 seeds of per-side means; bold as in Table 1 . Intervals as in Table 7 ;

\lambda_{\max}

is the largest Lyapunov exponent over the range,

0

on limit cycles and the leading Jacobian eigenvalue at fixed points. JEPAWG leads on both measures where the limit cycle keeps changing; PHLieNet holds a fixed point that JEPAWG loses on Lorenz; in chaos the measures split: PHLieNet tracks longer for Rössler with

c\in[9.2,10.2]

, but JEPAWG’s rollouts always stay bounded.

Panel of Figure 3 — Figure 3: Lorenz. (a) Raw-weights derivative $\|d\theta/d\rho\|$ across the $\rho$ -sweep, with a Gaussian-smoothed version as the thick line (top); derivative $G(\rho)=\|d\hat{\theta}/d\rho\|$ of the JEPAWG-decoded ( $M{=}800$ ) and PHLieNet-generated weights, 5 seeds each, with their mean as the thick curves (middle); the ground-truth bifurcation diagram (bottom). The Hopf point $\rho_{H}$ , at which the fixed points lose their stability, is shown as a red dashed line in all panels. (b) Weight-space response $G(\sigma,\rho)=\|\nabla_{\sigma,\rho}\hat{\theta}\|$ for a zoo trained on $(\sigma,\rho)\in[5,20]\times[10,40]$ ( $M{=}1024$ , $\beta{=}8/3$ ), shown for $\rho\leq 35$ because the finite-difference derivative is unreliable at the edge of the training range; the analytic Hopf curve is in red, and the blue curve marks the level set $\lambda_{\max}=0$ of the largest Lyapunov exponent. (c) Zoom above the Hopf curve with parameter ranges $(\sigma,\rho)\in[8.5,12]\times[28.5,33.5]$ ( $M{=}1024$ , $\beta{=}8/3$ ): a ring of elevated response surrounds the marked Bykov T-point at $(\sigma,\rho)\approx(10.17,\,30.87)$ . White lines: the principal homoclinic curves of the origin, obtained independently of the model by integrating its unstable manifold $W^{u}(0)$ on a $600^{2}$ grid and locating the parameter values at which its winding number around the nearest equilibrium before escaping to the other wing jumps by one turn. They form a single spiral winding onto the T-point [ 20 , 21 ] ; the ring of $G$ lies on its innermost turns, where successive curves are closest, while no fixed point changes stability anywhere in the window.

Panel of Figure 4 — Figure 4: Brusselator. (a) Weight-space response $G(a,b)=\|\nabla_{a,b}\hat{\theta}\|$ with the analytic Hopf curve $b=1+a^{2}$ in red; white dots mark the $M{=}1024$ Sobol sampled training parameters. (b) Latent space (PCA-2D with $86\%$ explained variance), coloured by the signed distance $s=b-(1+a^{2})$ to the curve; circled points ( $|s|<0.12$ ) correspond to the parameters nearest to the curve and trace a coherent boundary in every seed. Parameter ranges are $(a,b)\in[0.5,2]\times[1,5]$ ( $M{=}1024$ ).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

grid	method	TwoNN	$k{=}5$	$k{=}10$	$k{=}20$
Lorenz	raw $\theta$	19.9	17.9	15.1	12.3
( $\dim p{=}1$ )	PCA-16	4.7	4.1	3.5	3.3
	AE	7.9	7.3	6.5	5.7
	VAE	11.4	10.4	9.7	8.9
	JEPAWG	2.9	2.3	1.7	1.4
Rössler	raw $\theta$	2.8	8.9	14.2	17.1

Appendix

Table 4: Intrinsic-dimension estimates across neighbourhood scale (TwoNN, then Levina–Bickel with

k{=}5,10,20

) for the weight grids of four families and their embeddings (all bottleneck

d_{z}{=}16

; PCA: rank 16). The JEPAWG latent dimension is closest to

\dim p

in 15 of 16 columns (exception: TwoNN on Rössler); PCA, AE and VAE are fitted without access to

p

	$\mathrm{TtT}_{0.2}$			one-step MSE
system	base $\theta_{0}$	raw $\theta_{i}$	decoded $\hat{\theta}$	base $\theta_{0}$	raw $\theta_{i}$	decoded $\hat{\theta}$
Lorenz	$\mathbf{20.64}$	$15.79$	$20.19\pm 0.15$	$1.8\text{e-}3$	$2.5\text{e-}3$	$\mathbf{6.7\text{e-}4}$
Rössler	$42.08$	$15.33$	$\mathbf{47.42\pm 0.90}$	$2.1\text{e-}3$	$6.6\text{e-}4$	$\mathbf{1.1\text{e-}4}$
Brusselator	$17.44$	$24.25$	$\mathbf{25.40\pm 0.51}$	$9.7\text{e-}4$	$6.8\text{e-}4$	$\mathbf{8.6\text{e-}5}$
Hénon	$\mathbf{856}$	$733$	$\mathbf{772\pm 85}$	$\mathbf{9.6\text{e-}6}$	$1.3\text{e-}5$	$1.5\text{e-}5$

Appendix

Table 5: Shared

p

-conditional base network, raw fine-tuned grid weights and JEPAWG-decoded weights on the held-out interpolation test sets. The base

\theta_{0}

is the single conditional TCNN-CD every zoo member was fine-tuned from; raw weights use the nearest grid point’s

\theta_{i}

; both are deterministic given the zoo. Decoded

\mathrm{TtT}

entries are mean

\pm

std over 5 hypernetwork seeds; decoded MSE entries are seed means.

\mathrm{TtT}_{0.2}

in time units (Hénon: iterations); one-step MSE in the standardised training space. The decoder has the lowest one-step error on the three continuous systems and recovers the Rössler horizon the per-parameter fine-tunes lose; on Hénon fine-tuning gives no improvement over the base.

Panel of Figure 8 — Figure 8: Random parameters. $k$ random coordinates $\xi\sim\mathcal{U}(0,1)^{k}$ , with no effect on the dynamics, are appended to $p=(\rho,\xi)$ on the Lorenz zoo ( $M{=}800$ , weights unchanged), and both models are retrained. Shown is $G(\rho)$ evaluated at a fixed random draw of $\xi$ , averaged over 5 seeds; the red line marks $\rho_{H}\approx 24.74$ . (a) JEPAWG; the curves for $k=0,2,5,10$ coincide. (b) the random-feature control, with the same architecture but encoders frozen at initialisation. As $k$ grows, its baseline rises, the Hopf peak broadens and loses prominence.

Figure 9: Predicted and true trajectories at the furthest extrapolation parameter in each regime (one seed). In the limit-cycle regimes, JEPAWG-generated forecasters follow the true cycle more closely, while PHLieNet-generated trajectories tend to diverge after a number of steps, as quantified by the bounded fraction in Table 2 .

Figure 10: Forecast trajectories of the Lorenz system at held-out

\rho

(solid: truth; dashed: JEPAWG) at

\sigma=10

. From left to right: spiral trajectory into a stable fixed point, slowly decaying spiral just below the subcritical Hopf point, and the chaotic attractor. At these

\rho

inside the training range, the forecasts follow the true trajectory with reasonable precision.

Figure 11: Rössler weight-space response at $M{=}1000$ ( $a{=}b{=}0.2$ , $c\in[3,9]$ ). Top: the response $G(c)$ of JEPAWG-generated weights (Gaussian smoothed), per-seed curves and seed mean. Bottom: ground-truth diagram (successive local maxima of $x(t)$ per parameter). The peaks of $G$ correspond to features of the bifurcation diagram in the plot below: the $2\!\to\!4$ doubling at $c{=}3.83$ and the entry/exit boundaries of the periodic windows show clear signal; the finite parameter resolution limits the visibility of the doublings closer to chaos.

Figure 12: Hénon $u$ -cascade. Top: weight-space response $G(u)$ of the cascade zoo, 5 seeds in colour and their mean in black, over the full trained range $u\in[-4.5,-0.12]$ (Fig. 5 shows the part $u\geq-3.2$ ). Bottom: bifurcation diagram of the Hénon map ( $b{=}0.3$ ) in the same coordinate $u=\log_{10}(a_{\infty}-a)$ , with $a_{\infty}=1.05805$ [ 33 ] . For each of $6000$ parameter values, $512$ iterates of the attractor are shown after a $2{\times}10^{5}$ -iterate transient. Red dashed lines mark the period-doubling bifurcations $a_{1},\dots,a_{7}$ (labels on the top axis); $a_{1}$ is exact, $a_{2},\dots,a_{7}$ were located by bisection on the attractor period. The last two splittings of the orbit points are narrower than $10^{-3}$ in $x$ and are not resolved at this scale. $G(u)$ peaks at each of the first five doublings, shows a step at $a_{6}$ and no feature at $a_{7}$ , where the seeds begin to disagree.

Figure 13: Hénon. Left: ground-truth orbit period over $(a,b)$ ; the boundaries between coloured bands are the bifurcation curves. Right: JEPAWG weight-space response $G(a,b)=\|\nabla_{a,b}\hat{\theta}\|$ on the same rectangle, mean over seeds. The ridge of $G$ tracks the interior doubling curves and the chaotic band.

exp.	$M$	trans.	$\Delta t$	len (tr/te)	ICs	$h$	$\Delta p$	$\lambda_{\max}^{\sup}$
Forecasting
Lorenz, $\rho$	800	0	0.02	20/30	5	$3.1{\times}10^{-2}$	$3.1{\times}10^{-2}$	1.06
Rössler, $c$	1000	300	0.1	20/100	10	$7.5{\times}10^{-3}$	$6.0{\times}10^{-3}$	0.09
Brusselator, $b$	1024	20	0.05	30/30	5	$3.1{\times}10^{-3}$	$8{\times}10^{-4}$	$\leq 0$
Hénon, $a$	1024	200 it.	1	800/1500	5	$3.1{\times}10^{-4}$	$2.4{\times}10^{-4}$	0.18
Interpretability

Appendix

Table 6: Per-experiment protocol. The

M

parameter values are drawn from scrambled Sobol sequences (seed 0; stratified per-band draws for the Brusselator

b

sweep and the Hénon

u

-cascade, joint draws for the two-parameter families). Transients are integrated in addition to the recorded lengths; a zero entry means transients are retained in the data.

\Delta t

: observation interval (maps: 1 iteration); recorded lengths as training/test. ICs: initial conditions per training parameter (test sets use 10; Lorenz 20).

h

: evaluation step of the

G

sweep (two-parameter rows list the per-axis pair

(h_{1},h_{2})

in the order of the experiment label), alongside the mean spacing

\Delta p

of the fine-tuned parameters (Brusselator

b

: spacing in the dense band

b\in[2.35,2.85]

around

b_{c}

; overall mean

2.4{\times}10^{-3}

) (

u

-cascade: both in

u=\log_{10}(a_{\infty}-a)

, on which it is trained).

\lambda_{\max}^{\sup}

: largest Lyapunov exponent over the training range (per time unit; maps: per iteration);

\leq 0

marks ranges without chaotic parameters. Input noise

5\%

of the standardised feature scale. The Lorenz-

\rho

grid also underlies Fig. 3(a) .

experiment	train range	interp. test	extrapolation
Lorenz, $\rho$	$[10,35]$	10 fixed	$36$ – $40$ ; low $5$ – $9$
Rössler, $c$	$[3,9]$	10 Sobol	$1.8$ – $2.8$ ; $9.2$ – $10.2$
Brusselator, $b$	$[1.5,4]$	32 Sobol	$1.1$ – $1.4$ ; $4.2$ – $4.8$
Hénon, $a$	$[0.85,1.10]$	32 Sobol	—

Appendix

Table 7: Interpolation and extrapolation test sets for the forecasting experiments. Interpolation test parameters are Sobol draws from the training range, except Lorenz-

\rho

, which uses 10 fixed values. Fixed parameters: Lorenz

\sigma{=}10

\beta{=}8/3

; Rössler

a{=}b{=}0.2

; Brusselator

a{=}1.25

; Hénon

b{=}0.3

Weight-Space Analysis

Weight-space analysis studies neural network parameters as structured objects to understand, compare, edit, and reuse learned models.

See all 41 papers

Weight-Space Physics: Interpretable Hypernetworks for Lattice Quantum Field Theories
Jul 8, 2026Tobias Göbel, Julian R. Ebelt, Zier Mensch +2Phase Transitions Neural Network Interpretability
Institute of Physics, University of Amsterdam, Amsterdam, Netherlands · Department of Physics, National Taiwan University, Taipei, Taiwan · Department of Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA +2
Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems
Aug 23, 2026Zhou Fang, Gianmarco MengaldoChaotic Time-Series Prediction Adaptive Loss Weighting
Department of Mechanical Engineering National University of Singapore Singapore
EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models
Jun 11, 2026Vedant PandyaTest-Time Adaptation World Model Learning
School of Artificial Intelligence and Data Engineering (SAIDE) Indian Institute of Technology Jodhpur

Jul 8, 2026J/K move · Enter open · S save

Weight-Space Physics: Interpretable Hypernetworks for Lattice Quantum Field Theories

Tobias Göbel, Julian R. Ebelt, Zier Mensch, Mathis Gerdes+1

Institute of Physics, University of Amsterdam, Amsterdam, Netherlands · Department of Physics, National Taiwan University, Taipei, Taiwan · Department of Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA+2

Phase Transitions Neural Network Interpretability Hypernetworks Weight-Space Analysis Lattice Field Theory

Open paper AskPDF arXiv

Lattice field theory is the workhorse of non-perturbative physics, used to simulate phenomena from the strong nuclear force to critical phenomena in materials. Its Boltzmann distributions are parametrized analytically by coupling constants, but these bare parameters are weak predictors of observables -- extracting physics typically requires extensive simulation. While normalizing flows have emerged as effective samplers at fixed couplings, it remains difficult to interpret what these networks have learned. This raises a natural question: can the physics be read off directly from the flow network parameters themselves, and can those parameters be generated for unseen theories? We propose lattice field theory as a testbed for neural network interpretability: because the target physics is qualitatively well-understood and smoothly varying, it provides ideal synthetic data with known ground truth. To this end, we introduce JEPAWG, a Joint-Embedding Predictive Architecture-based Weight Generator that maps couplings directly to flow weights via a learned latent space. On a scalar theory at lattices of size

6^2

11^2

, the JEPAWG latent space recovers the correct intrinsic dimension of the underlying manifold, locates the phase transition, and encodes a finite-size shift aligned with the 2D Ising exponent

ν\approx 1

, allowing us to uncover physical structure by studying the network weights alone. This suggests the fascinating idea of treating the network weights as a new type of physical observable. As a generator, JEPAWG also interpolates and extrapolates to unseen couplings effectively and remains robust to weight-space discontinuities introduced by multi-seed training data, outperforming PCA, AE, and VAE baselines.

Abstract

Weight-Space Analysis

Explore similar work

Weight-Space Physics: Interpretable Hypernetworks for Lattice Quantum Field Theories

Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems

EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models