HamVision: Hamiltonian Dynamics as Inductive Bias for Medical Image Analysis

Mohamed A Mabrok

cs.CVMar 22, 2026

HamVision: Hamiltonian Dynamics as Inductive Bias for Medical Image Analysis

Authors: Mohamed A Mabrok

Organizations: Department of Mathematics and Statistics, Qatar University, Doha, Qatar

Abstract

We present HamVision, a framework for medical image analysis that uses the damped harmonic oscillator, a fundamental building block of signal processing, as a structured inductive bias for both segmentation and classification tasks. The oscillator's phase-space decomposition yields three functionally distinct representations: position~ $q$ (feature content), momentum~ $p$ (spatial gradients that encode boundary and texture information), and energy $H = \tfrac{1}{2}|z|^2$ (a parameter-free saliency map). These representations emerge from the dynamics, not from supervision, and can be exploited by different task-specific heads without any modification to the oscillator itself. For segmentation, energy gates the skip connections while momentum injects boundary information at every decoder level (HamSeg). For classification, the three representations are globally pooled and concatenated into a phase-space feature vector (HamCls). We evaluate HamVision across ten medical imaging benchmarks spanning five imaging modalities. On segmentation, HamSeg achieves state-of-the-art Dice scores on ISIC,2018 (89.38%), ISIC,2017 (88.40%), TN3K (87.05%), and ACDC (92.40%), outperforming most baselines with only 8.57M parameters. On classification, HamCls achieves state-of-the-art accuracy on BloodMNIST (98.85%) and PathMNIST (96.65%), and competitive results on the remaining MedMNIST datasets against MedMamba and MedViT. Diagnostic analysis confirms that the oscillator's momentum consistently encodes an interior $\,{>}\,$ boundary $\,{>}\,$ exterior gradient for segmentation and that the energy map correlates with discriminative regions for classification, properties that emerge entirely from the Hamiltonian dynamics. Code is available at https://github.com/Minds-R-Lab/hamvision.

Figures & tables

Property	Generic SSM	Hamiltonian (ours)
Transition $\bar{A}$	Learned diagonal	$e^{(-\nu+i\omega)\Delta}$
State space	Real $\mathbb{R}^{N}$	Complex $\mathbb{C}^{N}$
$\|\bar{A}\|$ bounded?	Not guaranteed	$<1$ by construction
Frequency structure	None	$\omega_{c}$ per channel
Quality factor	None	$Q_{c}=\omega_{c}/\nu_{c}$
Outputs	Features only	$q$ , $p$ , $\mathcal{H}$

Table 1: Generic state-space models versus the Hamiltonian oscillator. The physics constraint reduces the parameter space but enriches the representation with structured outputs.

Figure 1: Detailed architecture of the Hamiltonian Bottleneck. The input is processed through two parallel paths: a ConvNeXt block (left) providing reliable gradient flow, and an oscillator bank (right) performing physics-structured processing via four-directional scanning. The oscillator’s complex state

z

is decomposed into position

q

, momentum

p

, and energy

H=\tfrac{1}{2}|z|^{2}

. A learned gate fuses the ConvNeXt and oscillator outputs into features

f

, while a squeeze-and-excitation channel attention module collapses the per-channel energy tensor into a single-channel saliency map

H_{\text{map}}

. All three outputs propagate to downstream task-specific heads.

Figure 2: Overview of the HamSeg segmentation architecture. A shared ConvNeXt encoder with a Hamiltonian oscillator bottleneck produces position

q

, momentum

p

, and energy

H

representations, which are injected into a U-Net decoder via energy-gated skip connections and multi-scale momentum concatenation.

Figure 3: Architecture of HamCls, the classification head of HamVision. The shared encoder (Stem

\to

ConvNeXt stages

\to

Hamiltonian Bottleneck) produces three phase-space quantities: features

q

, momentum magnitude

|p|

, and scalar energy

H

. Each is globally pooled and concatenated into a 784-dimensional phase-space vector, which is classified by a LayerNorm–MLP head.

ISIC 2018 (dermoscopy, binary)
Method	Params	DSC	mIoU
U-Net	–	87.06	79.08
TransUNet	${\sim}$ 105M	87.47	79.50
SwinUNet	${\sim}$ 27M	87.16	79.54
VM-UNet	${\sim}$ 36M	88.10	78.89
FreqConvMamba	${\sim}$ 16M	89.19	80.64

Table 2: Segmentation results. Best in bold , second underlined . HamSeg uses 8.57M parameters throughout.

Figure 4: Qualitative segmentation for different datasets (TN3K thyroid ultrasound, ACDC cardiac MRI, and ISIC 2018 dermoscopy). Left to right: input, ground truth, predicted segmentation, and overlay.

Dataset	Method	AUC	ACC
BloodMNIST	ResNet-50	99.7	95.6
	MedMamba-S	99.9	98.4
	HamCls	99.93	98.85
PathMNIST	MedViT-S	99.3	94.2
	MedMamba-B	99.9	96.4
	HamCls	99.36	96.65

Table 3: Classification accuracy (%) on six MedMNIST benchmarks (

224{\times}224

). Best in bold , second underlined . Baselines from [ 28 ] .

Diagnostic measure	Value
Bottleneck gate (oscillator usage)
Gate $g$ mean / spatial std	0.54 / 0.043
Oscillator contribution (block 0 / 1)	79.6% / 58.8%
Phase-space attention (energy modulation)
Attention $\alpha$ range / std	[0.16, 0.78] / 0.136
Energy-gated skip connections

Table 4: Quantitative signal diagnostics on ISIC 2018 test samples. Every oscillator-derived signal is functionally active and produces physically interpretable measurements.

Figure 5: Multiscale energy-gated skip connections across four datasets (rows: ACDC, ISIC 2018, ISIC 2017, TN3K). Columns show the gate activation $\sigma(\gamma_{l}(H_{l}-\bar{H}_{l}))$ at decoder levels $d_{3}$ (coarsest), $d_{2}$ , and $d_{1}$ (finest). The progressive sharpening from diffuse spatial selection at $d_{3}$ to precise boundary delineation at $d_{1}$ demonstrates that the energy map, computed once at the bottleneck, provides complementary information at each decoder resolution. This coarse-to-fine pattern is consistent across all imaging modalities.

Figure 6: Oscillator response across four datasets (rows: ACDC cardiac MRI, ISIC 2018 dermoscopy, ISIC 2017 dermoscopy, TN3K thyroid ultrasound). Left to right: input with ground truth overlay; energy-gated skip connections at decoder levels $d_{3}$ , $d_{2}$ , $d_{1}$ (note progressive spatial sharpening); momentum magnitude at $d_{3}$ ; momentum overlay. The skip gates highlight salient structures while suppressing background, and the momentum maps align with anatomical boundaries, all emerging from the Hamiltonian dynamics without boundary-specific supervision.

Explore similar work

Mar 1, 2026

LUMOS: Latent Universal Medical Priors for Segmentation

General vision foundation models (VFMs) have been primarily developed on natural images, and their utility for medical image segmentation is therefore often considered to depend on costly adaptation or domain-specific fine-tuning. In this paper, we revisit this assumption from a different perspective: rather than requiring VFM segmentors to relearn visual regularities, we investigate whether the low-level visual priors necessary for anatomical delineation already lie dormant within general VFMs. We observe that frozen VFMs, despite lacking medical supervision, encode transferable visual regularities. These properties are not exclusive to natural images but are also fundamental to medical image understanding. Motivated by this observation, we propose Latent Universal Medical PriOrs for Segmentation (LUMOS), a novel framework that amplifies general VFM priors to conventional medical segmentors. LUMOS consists of two key components: (1) Pathfinder that distills visual cues from a frozen vision foundation model, and (2) Inspiror that sparks the conventional medical networks with spatial guidance from distilled visual regularities. In this way, the segmentor is relieved from learning complex visual regularities entirely from limited medical annotations and can instead focus on task-specific anatomical delineation. Across diverse medical datasets and token-based VFMs, LUMOS shows that general VFMs can serve as spatial prior generators when their frozen token spaces preserve patch-level pattern relevance. DINO provides stable matched-backbone gains, while SigLIP exposes VFM-specific sensitivity caused by its different token granularity and representation objective.

Zhuonan Liang, Wei Guo, Jie Gan +4

The University of Sydney, Sydney, NSW 2006, Australia. · Biological System & Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA. · Berkeley Biomedical Data Science Center, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA.

Medical Image Segmentation Transfer Learning Vision Foundation Model Adaptation

Ask PDF

Jul 12, 2026

Learning To Focus: Anatomy-Guided Attention Regularization for Medical Image Classification

Medical image classification models are ideally expected to identify diagnostically relevant regions while making predictions, yet standard classification losses rarely provide spatial supervision. Explicit supervision via anatomical shape information, such as segmentation masks of task-relevant anatomy, has been shown to guide the network toward regions relevant to the target prediction. However, obtaining such masks incurs substantial manual annotation effort and computational overhead. With the advent of segmentation foundation models that exhibit strong localization of anatomical structures across diverse imaging modalities, we leverage this capability to extract anatomical shape priors without the burden of training a dedicated segmentation model. In this paper, we propose a new framework, Locus, an anatomical attention regularization framework that leverages pretrained segmentation foundation models to guide a classifier's attention toward diagnostically meaningful anatomical structures across diverse imaging modalities. Instead of enforcing pixel-wise alignment with the foundation-model-derived mask, we introduce a regularization term that adaptively balances attention between anatomical (foreground) and background regions, penalizing the classifier when background attention dominates. We validate Locus on eight diverse medical imaging datasets spanning dermoscopy, X-ray, histopathology, and cardiac MRI, showing consistent gains in classification performance alongside improved anatomically grounded attention.

Tonmoy Hossain, Atiqur Rahman, Farhana Hossain Swarnali +1

Department of Computer Science, University of Virginia, Virginia, USA · Ahsanullah University of Science and Technology, Dhaka, Bangladesh · Kahlert School of Computing, University of Utah, Utah, USA +1

Medical Image Classification Attention Mechanisms Explainable Medical Image Analysis

Ask PDF

Sep 1, 2026

GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation

Medical image segmentation remains difficult to scale because high-performing methods typically rely on dense expert annotations and task-specific training. We introduce GazeRefine, a training-free framework that uses gaze as an inference-time prompt for zero-shot medical image segmentation. Sparse, duration-weighted fixations are converted into foreground and background priors that initialize semantic prototypes in frozen DINOv3 feature space. These prototypes are iteratively refined through foreground-background discrimination, feature-space affinity propagation, and anchoring to the initial gaze guidance, allowing segmentation to extend beyond directly fixated regions while limiting semantic drift. GazeRefine requires no segmentation masks, fine-tuning, adapters, prompt encoders, or gradient updates. We evaluate the method on gaze-annotated polyp segmentation and prostate MRI segmentation. The results show strong performance on colonoscopy images and competitive performance on prostate MRI, supporting gaze-guided prototype refinement as a promising approach for segmentation-label-efficient, human-in-the-loop medical image segmentation. Our tools and code can be found in the following repository: https://github.com/MohammedOussamaBEN/GazeRefine.git

Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri +10

Université Sorbonne Paris Nord · Northwestern University · VSB-Technical University of Ostrava

Interactive Medical Image Segmentation Image Segmentation Promptable Image Segmentation Medical Image Segmentation Zero-Shot Medical Image Segmentation

Ask PDF