Low-Rank-Modulated Functa: Exploring the Latent Space of Implicit Neural Representations for Interpretable Ultrasound Video Analysis
Authors: Julia Wolleb, Cristiana Baloescu, Alicia Durrer, Hemant D. Tagare, Xenophon Papademetris
Organizations: Dept. of Biomedical Informatics & Data Science, Yale University, New Haven, USA · Yale Biomedical Imaging Institute, Yale University, New Haven, USA · Department of Emergency Medicine, Yale School of Medicine, New Haven, USA · Dept. of Biomedical Engineering, University of Basel, Allschwil, Switzerland · Dept. of Radiology & Biomedical Imaging, Yale University, New Haven, USA
Implicit neural representations (INRs) have emerged as a powerful framework for continuous image representation learning. In Functa-based approaches, each image is encoded as a latent modulation vector that conditions a shared INR, enabling strong reconstruction performance. However, the structure and interpretability of the corresponding latent spaces remain largely unexplored. In this work, we investigate the latent space of Functa-based models for ultrasound videos and propose Low-Rank-Modulated Functa (LRM-Functa), a novel architecture that enforces a low-rank adaptation of modulation vectors in the time-resolved latent space. When applied to cardiac ultrasound, the resulting latent space exhibits clearly structured periodic trajectories, facilitating visualization and interpretability of temporal patterns. The latent space can be traversed to sample novel frames, revealing smooth transitions along the cardiac cycle, and enabling direct readout of end-diastolic (ED) and end-systolic (ES) frames without additional model training. We show that LRM-Functa outperforms prior methods in unsupervised ED and ES frame detection, while compressing each video frame to as low as rank k=2 without sacrificing competitive downstream performance on ejection fraction prediction. Evaluations on out-of-distribution frame selection in a cardiac point-of-care dataset, as well as on lung ultrasound for B-line classification, demonstrate the generalizability of our approach. Overall, LRM-Functa provides a compact, interpretable, and generalizable framework for ultrasound video analysis. The code is available at https://github.com/JuliaWolleb/LRM_Functa.
Figures & tables
Figure 1: Overview of our LRM-Functa framework. A) Our approach builds on VidFuncta , where each ultrasound video is represented by a video-level modulation vector v and time-resolved modulation vectors ϕ={ϕt}t=1T . (B) Pairwise cosine similarity of ϕ over time reveals a clear low-rank pattern. (C) Motivated by this observation, we constrain the latent modulation space to a low-rank subspace Bβ . (D/E) This structured latent space enables efficient compression-reconstruction and cardiac phase analysis.
Figure 2: Proposed architecture for low-rank adaptation of the modulation vectors. Each video V is encoded into a video-level representation v and time-resolved frame representations ϕt , which are projected onto a rank- k subspace Bβ to produce modulation vectors mt=v+Bβϕt . These vectors shift the activations of a shared MLP Mθ , trained to reconstruct pixel intensities z^ at each coordinate (x,y) , shown as a red cross.
Figure 3: For k=2 , we visualize the raw trajectories ϕt over time. LRM-Functa o exhibits spiral-like trajectories, whereas LRM-Functa b collapses the trajectory to a line. Projecting ϕt onto the principal motion direction p and filtering yields the 1D signal sfilt , which enables direct interpretable identification of ED and ES frames.
Figure 5: Mean SSIM and PSNR scores on the EchoNet-Dynamic test set for all comparing methods across varying latent dimensions k (x-axis shown on a log2 scale). For small k , our LRM-Functa variants are the only methods to produce stable reconstructions, while for higher ranks, LRM-Functa o achieves among the highest performance.
EchoNet, k=2
EchoNet, k=512
OOD POCUS, k=512
Method
ED MAE
ES MAE
ED MAE
ES MAE
ED MAE
ES MAE
Coin++
8.29±12.3
5.82±10.8
4.46±6.9
3.26±5.5
1.83±1.1
2.32±1.4
MedFuncta
3.72±3.8
3.48±3.4
2.90±3.3
2.17±2.4
1.61±1.3
1.27±1.5
VidFuncta
3.06±2.6
2.90±2.8
3.08±3.3
2.37±2.4
1.33±1.2
1.89±3.8
Autoencoder
2.81±3.5
2.26±2.7
3.82±4.7
2.44±2.6
3.06±3.5
4.24±2.6
LRM-Functa b
2.26±2.5
1.98±2.1
2.48±2.9
2.21±2.2
1.40±1.5
1.13±1.4
Table 1: Mean ± standard deviation for the ED and ES MAE on the EchoNet-Dynamic and the OOD POCUS test sets. All methods are unsupervised.
Ejection fraction, k=2
B-line classifier [ ×10−2 ], k=512
Method
MAE
RMSE
R 2 score
Accuracy
F1 score
AUROC
Coin++
9.15±0.1
12.17±0.2
0.03±0.03
77.5 ± 5.0
67.8 ± 8.7
80.9 ± 9.0
MedFuncta
9.42±0.1
12.94±0.1
-0.02±0.02
81.0 ± 6.3
73.1 ± 7.7
90.1 ± 1.9
VidFuncta
9.14±0.1
13.09±0.2
-0.02±0.03
83.5 ± 5.2
76.6 ± 7.3
91.9 ± 3.6
Autoencoder
7.16±0.3
9.79±0.5
0.47±0.06
68.0 ± 2.1
50.2 ± 6.5
66.6 ± 5.7
LRM-Functa b
5.70±0.2
7.63±0.3
0.67±0.03
77.5 ± 6.4
70.9 ± 3.9
89.5 ± 2.2
Table 2: Mean ± std of downstream performance on V^ , evaluated on the reconstructed EchoNet-Dynamic and LUS test sets using 5-fold cross-validation.
Ultrasound is the most widely used real-time imaging modality in clinical practice, yet per-frame video annotation remains a major bottleneck: expert labels are scarce and costly, and image appearance varies with speckle, shadowing, attenuation, and operator-dependent probe pose. This is especially limiting because clinically relevant information is often dynamic, from left-ventricular motion in echocardiography to muscle and bone kinematics in musculoskeletal imaging. Population atlases can amortize annotation cost by registering observations to a shared canonical coordinate system, but existing neural atlas methods mainly target single videos, small test-time image sets, or object-centric image collections. We introduce a cohort-scale neural atlas for ultrasound video: a single canonical chart with per-video Generative Latent Optimization embeddings, trained jointly over thousands of frames in DINOv3 feature space. Across five cardiac and musculoskeletal datasets with point landmarks and segmentation masks, our method learns coherent canonical templates and enables accurate atlas-space annotation transfer. On EchoNet-Dynamic and MSK-Bone, it supports single- and few-shot transfer with accuracy competitive with strong dense-correspondence baselines, while training in minutes on a single consumer GPU. The learned embeddings are interpretable: linear projections reveal structured cohort variation, image-decoder interpolation produces anatomically plausible intermediate frames, and test-time latent inversion reconstructs held-out frames through the atlas. These results suggest that cohort-scale neural atlases offer a practical, interpretable representation for reducing expert annotation burden in ultrasound video analysis.
Zhuorui Zhang, Roger Pallarès-López, Xuan Wu +2
Department of Mechanical Engineering, MIT · Institute for Medical Engineering and Science, MIT · 3MIT.nano Immersion Lab, MIT
Self-supervised pre-training paradigm has gained increasing prominence for learning transferable representations in medical imaging, yet existing methods for ultrasound (US) images operate at the image or frame level, overlooking the anatomical context for clinical-aligned representation learning. In this work, we propose an anatomy-anchored ultrasound self-supervision framework ANAUS that shifts representation learning from generic visual regions to clinically meaningful anatomical structures. Utilizing a learnable latent prompt engine alongside a one-time domain adaptation on existing public image-mask pairs, we empower the LP-SAM module to achieve annotation-free anatomy delineation at scale. Building upon this anatomical grounding, we propose a dual-policy self-supervised learning paradigm consisting of inter-view semantics-aware anatomy-separating alignment and contextual core-region prediction to enhance representation learning. Specifically, the former enforces feature invariance within identical anatomical regions while promoting discriminability across distinct structures; the latter compels the model to reconstruct corrupted regions, thereby capturing fine-grained structural details. Extensive evaluations on six public datasets demonstrate that ANAUS consistently outstrips current state-of-the-art methods while maintaining the computational efficiency essential for clinical deployment. Code is available at https://github.com/zhcz328/ANAUS.
Chunzheng Zhu, Yijun Wang, Jianxin Lin +5
Hunan University, Changsha, China · Shenzhen Maternity and Child Healthcare Hospital, Shenzhen, China
Ultrasound images vary widely across scanners, operators, and anatomical targets, which often causes models trained in one setting to generalize poorly to new hospitals and clinical conditions. The Foundation Model Challenge for Ultrasound Image Analysis (FMC-UIA) reflects this difficulty by requiring a single model to handle multiple tasks, including segmentation, detection, classification, and landmark regression across diverse organs and datasets. We propose a unified multi-task framework based on a transformer visual encoder from the Qwen3-VL family. Intermediate token features are projected into spatial feature maps and fused using a lightweight multi-scale feature pyramid, enabling both pixel-level predictions and global reasoning within a shared representation. Each task is handled by a small task-specific prediction head, while training uses task-aware sampling and selective loss balancing to manage heterogeneous supervision and reduce task imbalance. Our method is designed to be simple to optimize and adaptable across a wide range of ultrasound analysis tasks. The performance improved from 67% to 85% on the validation set and achieved an average score of 81.84% on the official test set across all tasks. The code is publicly available at: https://github.com/saitejalekkala33/FMCUIA-ISBI.git
Ufaq Khan, L. D. M. S. Sai Teja, Ayuba Shakiru +4
MBZUAI, Abu Dhabi, UAE · NIT Silchar, India · Birmingham City University, UK