In conditioned generative models of physical systems, the densities at different values of the conditioning parameters are often related through symplectic transformations. For stable linear Hamiltonian systems, there exists a symplectic transformation to "normal coordinates" such that the dynamics is reduced to rotations in each phase-space plane. This picture extends to the nonlinear case where, away from resonances, the Birkhoff normal form again consists of rotations in phase space, now with amplitude dependence. In this work, we introduce an invertible symplectic flow layer consisting of amplitude-dependent rotations in learned normal coordinates. We derive the condition under which the nonlinear rotation layer is symplectic and detail an architecture that guarantees it. Experiments are carried out on three conditional generative modeling problems: a synthetic chain of coupled nonlinear oscillators and two problems from accelerator physics. We show empirically that normalizing flows with the new layers achieve competitive or superior negative log-likelihoods compared to baseline flows, with gains that grow with the strength of the nonlinearity on the synthetic system.
Figures & tables
Figure 1: Architecture of the conditioned normalizing flow showing evolution to an initial state, then transformation through the normal coordinates flow layers.
Figure 2: Held out negative log-likelihood on the Duffing chain example as the nonlinearity parameter ( β ) is swept. Points show mean and standard error.
Figure 3: True and predicted phase space evolution of the ensemble of initial conditions for the first oscillator in the Duffing chain experiment with β=0.1 .
Figure 4: Models trained on the uniform focusing channel dataset. Histograms of samples drawn from the models at s positions of held out test data compared with the ground-truth distribution.
Figure 5: t - pz beam distributions from the held out test data in the chicane problem. (top) ground-truth distribution; (top-middle) distribution from the best Real NVP flow; (bottom-middle) distribution from the best SympNet flow; (bottom) distribution from the normal coordinates flow.
Figure 6: Evolution of the beam’s second moments along the chicane lattice. Beam size ( σx/y ), bunch length ( σt ), dispersion ( ηx ), and normalized emittance ( εn,x/y ) are all shown. The cosine similarity of each model to the ground truth is given in Tab. 1 .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Layout of the chicane used in the 6D generative modeling experiment. (top) beta functions showing the beam envelope throughout the lattice; (middle) dispersion, showing the x - pz correlation of the beam; (bottom) the layout with dipole (bending) magnets in black and quadrupole (focusing) magnets in magenta.
Figure 8: Held out examples from the chicane dataset showing the x - px phase space projection. Same model labels as in Fig. 5 .
Figure 9: Held out examples from the chicane dataset showing the y - py phase space projection. Same model labels as in Fig. 5 .
Model
σx
σy
σt
ηx
εn,x
Average
Real NVP
0.974
0.988
0.960
0.961
0.987
0.974
SympNet
0.913
0.924
0.589
0.610
0.915
0.790
Normal coordinates
0.969
0.997
0.994
0.960
0.986
0.981
Appendix
Table 1: Mean-subtracted cosine similarity between model and ground-truth traces from Fig. 6 over all 128 lattice positions. The ground truth vertical emittance ( εn,y ) does not change across s , causing cosine similarity to only show the residual noise in the models. Because of this, we don’t include it in this table.
Experiment
NLL (nats/sample)
Baseline
11.6 (0.9)
Remove amplitude dependence of rotation layer
17.0 (0.4)
Remove rotation layer
16.7 (0.1)
Use MLP instead of GradNet in rotation layer
13.0 (0.5)
Appendix
Table 2: Ablation studies of the normal coordinates flow on the Duffing chain with β=0.1 . Mean and standard deviation (in parentheses) of the validation NLL are shown.
Conditioning Network Layer Width
NLL (nats/sample)
16
11.2 (0.3)
32
11.6 (0.8)
64 (baseline)
11.6 (0.9)
128
11.2 (0.2)
Appendix
Table 3: Sensitivity to the conditioning network layer width on the Duffing chain with β=0.1 . Mean and standard deviation (in parentheses) of the validation NLL are shown.
Hamiltonian normalizing flows are attractive generative models because their phase-space maps are invertible and volume preserving, but most neural constructions are formulated in Euclidean space. We introduce Riemannian Neural Hamiltonian Flows, which combine the fixed kinetic energy of a Riemannian manifold, a learned scalar potential, and an explicit geodesic integrator. We then study the interpretability of the learned potential through a thermodynamic reading in which the momentum variance plays the role of a temperature. The matched potential, whose Boltzmann distribution is the data, is the potential whose force balances the pressure of the data. The output of one pass of the flow is the Boltzmann distribution of the learned potential multiplied by a memory factor, so that the learned potential is the matched potential whenever the pass forgets its starting point. Numerical experiments on Euclidean, hyperbolic, and spherical spaces show competitive sample quality and numerical cost against a Riemannian continuous normalizing flow, and confirm the interpretability of the learned potential. We then show that this reading has a computational use. When the energy of the data is known but expensive, the landscape learned from existing samples is a surrogate energy for Hamiltonian Monte Carlo, allowing exact sampling with significant acceleration. We test this approach on the localization of a neutrino source from public IceCube data.
Scalable approaches to learning neural dynamical systems have placed diffusion, flow matching, and few-step generators in flow maps at the forefront of generative modeling. Such state-of-the-art models have limited access to explicit model densities despite evolving from classical generative modeling approaches that leveraged tractable likelihoods, e.g., Normalizing Flows and Autoregressive models. In this paper, we combine these threads of few-step generation and tractable sample likelihoods and introduce Normalizing Flow Maps (NFM), a class of exactly invertible pushforward maps that generalizes normalizing flows and flow maps. More precisely, NFMs learn the integrator of an underlying dynamical system, but crucially, they provide efficient exact likelihood evaluation by construction. To enable scalable training of NFMs, we derive two complementary axes of consistency inherent to NFMs that yield (self) distillation objectives: (1) pointwise matching, which unlocks ℓ2-based flow-map style regression losses, and (2) distributional matching, which matches pushforward distributions to the marginals of a prescribed forward process. On conditional image generation on CIFAR-10 and ImageNet-64, NFMs achieve state-of-the-art sample quality among parameter-matched normalizing flows, while supporting likelihood-based RLHF. For equilibrium sampling in Boltzmann Generators, NFMs outperform prior state-of-the-art normalizing flows on both single-system and transferable peptide benchmarks.
Francesco M. Ruscio, Timothy James Hitge, Fred Zhangzhi Peng +6
Learning solution operators for differential equations is a central problem in scientific machine learning. However, many neural operator methods optimize prediction accuracy without explicitly enforcing the geometric structure of the dynamics. Structure-preserving models such as SympNets and Symplectic Neural Flows address this issue for conservative Hamiltonian systems by preserving the symplectic form. In dissipative Hamiltonian systems with conformal symplectic structure, however, the symplectic form evolves according to a conformal factor determined by the dissipation. We propose CoSynFlow, a conformal symplectic neural flow for learning continuous-time solution maps of dissipative Hamiltonian dynamics. CoSynFlow composes symplectic shear maps with explicit conformal scaling, preserving the conformal symplectic structure by construction. By conditioning it on a finite-dimensional Hamiltonian descriptor and the dissipation parameter, a single trained model predicts solution maps for unseen systems without retraining. CoSynFlow keeps the structure error at machine precision, attains the lowest long-horizon error, and admits physics-informed training.
Baige Xu, Takaharu Yaguchi
RIKEN AIP · Motooka Nishi-ku, Fukuoka, Japan · IMI, Kyushu University