Hamiltonian Metric Learning and Energy-Based Training: A Dissipative Geometric Framework for Optimization

Sparsho Chakraborty; Mohammad Alamgir; Nishanth M; Akshit Nanda; Ram Prasad Padhy; Sayan Mukherjee

cs.LGOct 4, 2026

Hamiltonian Metric Learning and Energy-Based Training: A Dissipative Geometric Framework for Optimization

Authors: Sparsho Chakraborty, Mohammad Alamgir, Nishanth M, Akshit Nanda, Ram Prasad Padhy, Sayan Mukherjee

Organizations: Department of Physics, School of Basic Sciences, Indian Institute of Technology Bhubaneswar, Odisha, India · Department of Electronics and Computer Science, Narula Institute of Technology, West Bengal, India · Department of Computer Science and Engineering, School of Electrical and Computer Sciences, Indian Institute of Technology Bhubaneswar, Odisha, India · Department of Physics, The University of Tokyo, Hongo, Bunkyo-ku, Tokyo, Japan

Abstract

Optimization in machine learning is usually expressed through iterative rules that update model parameters using information from the loss landscape. In this work, we study an alternative viewpoint in which parameter optimization is treated as the evolution of a dissipative dynamical system. The model parameters are regarded as generalized coordinates, the loss acts as a potential energy, and a positive-definite metric defines the local kinetic geometry of parameter space. Starting from a variational formulation, we derive the corresponding Hamiltonian dynamics, the geometric force generated by a position-dependent metric, and a metric-compatible Rayleigh dissipation law. The resulting continuous system satisfies a monotonic energy-dissipation relation, while its discrete form allows the influence of curvature on the optimization trajectory to be studied directly. We illustrate the framework using a controlled CIFAR-10 image-reconstruction problem for which the optimum is known analytically. With the image Hessian used as the metric, the curvature dependence of the quadratic modal dynamics is removed. We then reparameterize the same image as a matrix product state, producing a genuinely position-dependent metric and a nonzero geometric force. Poincaré return maps provide a complementary phase-space view of the resulting contraction dynamics. These examples establish HAMLET as a geometric, energy-based framework for studying optimization as dissipative motion in parameter space. On a five-seed MNIST MLP benchmark, HAMLET attains the highest mean test accuracy (97.95%) and the lowest mean test negative log-likelihood (0.0696) among the three evaluated optimizers.

Figures & tables

Symbol	Meaning
$\bm{\theta},\bm{v},\bm{p}$	Parameters, velocity, canonical momentum
$L,B_{L}$	Loss and its Hessian
$g,g^{-1}$	Kinetic metric and inverse metric
$\mathcal{L},\mathcal{A}$	Lagrangian and action
$H_{0}$	HAMLET Hamiltonian
$\mathcal{R},\gamma$	Rayleigh function and damping rate

TABLE I: Notation Used in the Derivation

Method	$n$	Test accuracy (%)	Test NLL	Best val. NLL	Final val. NLL
SGD+momentum	5	$97.87\pm 0.21$	$0.0748\pm 0.0070$	$\mathbf{0.0670\pm 0.0045}$	$\mathbf{0.0742\pm 0.0104}$
AdamW	5	$97.28\pm 0.24$	$0.1252\pm 0.0169$	$0.0851\pm 0.0025$	$0.1241\pm 0.0133$
HAMLET	5	$\mathbf{97.95\pm 0.09}$	$\mathbf{0.0696\pm 0.0028}$	$0.0677\pm 0.0010$	$0.0743\pm 0.0029$

TABLE II: MNIST MLP results over five paired evaluation seeds. Entries are mean

\pm

sample standard deviation; accuracy is in percent. Lower NLL is better; higher accuracy is better. Bold marks the best mean in each column.

Fig. 1: MNIST MLP comparison over five paired seeds using one HAMLET formulation and two standard baselines. Bars show the mean final test metrics, error bars the sample standard deviation, and open circles the individual seeds; the numerical values are reported in Table II .

Fig. 2: Reconstruction of the selected CIFAR-10 image. From left to right: original image, blurred and noisy observation, zero initialization, two intermediate HAMLET iterates, and the final reconstruction.

Fig. 3: Convergence of Euclidean momentum and HAMLET using the same numerical settings. The panels show the objective gap, gradient norm, and reconstruction quality during optimization.

Fig. 4: Potential, kinetic, and total energy during HAMLET optimization. The logarithmic representation emphasizes the decay toward the minimum.

Fig. 5: Spectrum of the image Hessian before and after curvature normalization. Choosing $g=H$ maps the transformed spectrum to unity.

Fig. 6: Evolution of representative low- and high-curvature modes. Euclidean momentum relaxes the two modes at different rates, whereas the curvature-matched HAMLET dynamics collapse them onto the same normalized trajectory.

Panel of Figure 9 — Fig. 7: The $32\times 32\times 3$ cat image as an $11$ -site MPS. Circles are cores $A^{(k)}$ , lower legs are the color and binary row/column indices of ( 73 ), and edge labels are bond dimensions for $\chi=8$ .

Fig. 8: Representation of the image at increasing MPS bond dimension. Larger bond dimension increases the expressive capacity of the tensor network and allows progressively finer image structure to be retained.

Fig. 9: MPS reconstruction ( $\chi=8$ ). Top row: MPS-HAMLET iterates at increasing $k$ , evolving toward a low-loss image while the underlying metric changes with the MPS cores, next to the exact pixel-space optimum $x_{\mathrm{opt}}$ . Bottom row: SGD, SGD with momentum, Adam, ALS, and HAMLET on the same MPS after $300$ iterations, with PSNR, and the observation $b$ .

Fig. 10: Energy and geometric dynamics of HAMLET on the MPS parameter manifold. The left panel shows the potential, kinetic, and total energy, while the right panel shows the gradient and geometric-force norms during the optimization.

Fig. 11: Modal Poincaré analysis of the pixel-space image problem. Euclidean momentum produces curvature-dependent phase-space trajectories and return contractions, whereas the curvature-matched HAMLET metric collapses all modes onto the same normalized return dynamics.

Fig. 12: Aggregate Poincaré return dynamics for the pixel-space and MPS reconstructions. The curvature-matched pixel problem exhibits a regular contracting return map, while the MPS trajectory shows irregular early returns that disappear as overdamped directions dominate the late-time motion.

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 327 papers

Optimization Geometrodynamics: Variational Reduction and Interaction Curvature
Jul 7, 2026Zavier LiRiemannian Optimization Second-Order Optimization
Xidian University Xi’an, China
Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization
Jun 29, 2026Marcelina Marjankowska, Valerio Modugno, Paolo BaruccaSpectral Methods Adaptive Gradient Methods
Financial Computing and Analytics Group, University College London · Robot Perception and Learning Lab, University College London
Optimization over covariance matrices with a parameterized metric
Sep 15, 2026Yibang Li, Bamdev Mishra, Pratik Jawanpuria +1Riemannian Optimization Second-Order Optimization
Nanyang Technological University, Singapore · Microsoft India · Indian Institute of Technology Bombay, India

Jul 7, 2026J/K move · Enter open · S save

Optimization Geometrodynamics: Variational Reduction and Interaction Curvature

Zavier Li

Xidian University Xi’an, China

Riemannian Optimization Second-Order Optimization Quasi-Newton Methods Adaptive Gradient Methods

Open paper AskPDF arXiv

Adaptive optimizers carry hidden states that change how visible gradients become parameter motion. We develop optimization geometrodynamics as a variational theory of this hidden geometry. Infimal pushforward eliminates all hidden states realizing the same action and composes across optimizer hierarchies. Under smooth nondegeneracy, it yields hidden susceptibility and the Schur-complement curvature seen after relaxation. For affine pre-reduction perturbations, the induced interaction curvature is the negative-semidefinite operator

-G^*H^{-1}G

, whose mixed entries integrate to finite mechanism contrasts. Our main realization is the determinant-one affine-invariant SPD action map

P\mapsto PA

. We prove a global analytic bundle with closed totally geodesic fibers and a unique analytic nearest-controller section. A strongly convex fiber theorem and an explicit logarithmic action residual give a globally linearly convergent solver from every feasible initializer, together with nonasymptotic value, controller-distance, and residual bounds and observable posterior stopping certificates. A conditional inexact result propagates supplied rigorous residual-error and radius majorants. The dense spectral kernel is confined to an active subspace of dimension

r\le 2m

, yielding an explicit spectral-arithmetic operation bound and a strict dimensional reduction when

r<d

. For nested shape-normalized quadratic actions, canonical multi-secant projections satisfy an exact CAT(0) Pythagorean decrease and recover the determinant-one inverse Hessian shape at the sharp rank threshold

d-1

, provided the scalar gauge

c_H=(\det H)^{1/d}

is known. These results turn the action bundle into an exact iterative computation with posterior certificates and a finite-identification theory.

Abstract

Neural Network Optimization

Explore similar work

Optimization Geometrodynamics: Variational Reduction and Interaction Curvature

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

Optimization over covariance matrices with a parameterized metric