OSA: Echocardiography Video Segmentation via Orthogonalized State Update and Anatomical Prior-aware Feature Enhancement

Rui Wang; Huisi Wu; Jing Qin

cs.CVMar 27, 2026

OSA: Echocardiography Video Segmentation via Orthogonalized State Update and Anatomical Prior-aware Feature Enhancement

Authors: Rui Wang, Huisi Wu, Jing Qin

Organizations: College of Computer Science and Software Engineering, Shenzhen University · Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University

Abstract

Accurate and temporally consistent segmentation of the left ventricle from echocardiography videos is essential for estimating the ejection fraction and assessing cardiac function. However, modeling spatiotemporal dynamics remains difficult due to severe speckle noise and rapid non-rigid deformations. Existing linear recurrent models offer efficient in-context associative recall for temporal tracking, but rely on unconstrained state updates, which cause progressive singular value decay in the state matrix, a phenomenon known as rank collapse, resulting in anatomical details being overwhelmed by noise. To address this, we propose OSA, a framework that constrains the state evolution on the Stiefel manifold. We introduce the Orthogonalized State Update (OSU) mechanism, which formulates the memory evolution as Euclidean projected gradient descent on the Stiefel manifold to prevent rank collapse and maintain stable temporal transitions. Furthermore, an Anatomical Prior-aware Feature Enhancement module explicitly separates anatomical structures from speckle noise through a physics-driven process, providing the temporal tracker with noise-resilient structural cues. Comprehensive experiments on the CAMUS and EchoNet-Dynamic datasets show that OSA achieves state-of-the-art segmentation accuracy and temporal stability, while maintaining real-time inference efficiency for clinical deployment. Codes are available at https://github.com/wangrui2025/OSA.

Figures & tables

Figure 1: Illustration of different spatiotemporal memory update paradigms for echocardiography video segmentation. Top: Memory bank methods rely on sparse key-frame retrieval to capture temporal dynamics. Middle: Linear recurrent models perform unconstrained element-wise updates for state propagation. Bottom: Our OSA enforces a Stiefel manifold constraint, yielding stable and drift-free tracking across the cardiac cycle.

Figure 2: Challenges in echocardiography video segmentation: (a–b) red boxes of speckle noise and blue boxes indicate indistinct or blurred contours; (c–f) large shape and scale variations across the cardiac cycle.

Figure 3: Overview of the OSA architecture. The Anatomical Prior-aware Feature Enhancement (APFE) encoder extracts contrast-decomposed features from echocardiography video frames, which are recurrently updated through the Orthogonalized State Update (OSU) mechanism to maintain stable and geometry-aware spatiotemporal representations. The decoder reconstructs segmentation maps, and the entire model is optimized with loss.

Figure 4: Illustration of OSU. Standard Euclidean recurrences suffer from rank collapse over long sequences (manifested as singular value decay and increasing condition number). OSU constrains state evolution on the Stiefel manifold via orthogonalized update. The unconstrained intermediate state $\mathbf{S}_{t}^{\text{Euc}}$ is projected back to the manifold as $\mathbf{S}_{t}$ , ensuring stable temporal transitions.

Figure 5: Behavioral comparison of optimization vector fields. Unconstrained Euclidean updates cause the state to drift away from orthogonal constraints (climbing valley walls). OSU generates a corrective flow that pulls iterates back onto the Stiefel manifold (valley floor).

Method	CAMUS					EchoNet-Dynamic
	mDice $\uparrow$	mHD95 $\downarrow$	corr $\uparrow$	bias $\pm$ std (%)		mDice $\uparrow$	mHD95 $\downarrow$	corr $\uparrow$	bias $\pm$ std (%)
PolaFormer [ 32 ] ICLR’25	92.47	3.66	0.792	-0.28 ${}\pm{}$	7.5	91.51	3.89	0.678	-0.01 ${}\pm{}$	8.1
Vision LSTM [ 2 ] ICLR’25	92.71	3.47	0.745	-2.36 ${}\pm{}$	7.6	91.71	3.85	0.691	-0.37 ${}\pm{}$	7.9
Cutie [ 9 ] CVPR’24	91.43	3.97	0.629	0.44 ${}\pm{}$	9.8	90.44	4.05	0.600	0.40 ${}\pm{}$	9.6
LiVOS [ 29 ] CVPR’25	92.89	3.56	0.718	-1.69 ${}\pm{}$	7.8	91.92	3.82	0.702	-0.04 ${}\pm{}$	7.5
SAMed-2 [ 50 ] MICCAI’25	93.55	3.29	0.815	-2.16 ${}\pm{}$	6.8	92.67	3.60	0.731	-0.02 ${}\pm{}$	7.3

Table 1: Comparison of OSA with state-of-the-art methods for LV segmentation on the CAMUS [ 25 ] and EchoNet-Dynamic [ 34 ] datasets. Metrics include mean Dice coefficient (mDice) and mean 95% Hausdorff distance (mHD95) for segmentation evaluation, as well as correlation coefficient (corr) and bias

\pm

std (%) for

\text{LV}_{\text{EF}}

estimation.

Figure 6: Illustration of the APFE module. APFE decomposes the input into high- and low-intensity components, processes them via dual convolutional branches, and fuses them through a gated mechanism to produce enhanced features $\mathbf{Z}_{t}$ .

Method	Paradigm	Para	Mem	Time
PolaFormer	Linear	29.0	13.2	3.5
Vision LSTM	Linear	5.9	4.5	2.5
Cutie	Retrieval	39.1	8.1	6.0
LiVOS	LKVA	35.1	7.4	5.0
SAMed-2	Retrieval	110.1	27.5	7.0
EchoVim	SSM	42.0	34.9	9.0

Table 2: Comparison of different models on CAMUS. Paradigm denotes the core temporal memory and update mechanism; Para is the number of parameters (M); Mem is the GPU memory usage during training (GB); and Time is the approximate training time (hours) to reach the best mDice.

Figure 7: Visual comparison with state-of-the-art methods. The top two rows are from CAMUS, and the bottom two rows are from EchoNet-Dynamic. Green, red, and yellow regions represent the ground truth, prediction, and overlapping regions, respectively.

Method	mDice $\uparrow$	mHD95 $\downarrow$	Mem $\downarrow$	Time $\downarrow$
Baseline	92.94	3.56	7.5	4.5
w/o APFE	93.61	3.29	7.6	3.0
w/o OSU	94.12	3.21	7.5	4.7
Full	94.82	3.25	7.6	3.0

Table 3: Ablation study of the components on CAMUS.

Figure 8: Heatmap ablation on EchoNet-Dynamic. Warmer colors indicate higher prediction confidence.

Method	MSV $\downarrow$	SVVar $\downarrow$	OrthE $\downarrow$	ColR $\downarrow$
Baseline	$6.82$	$1.47$	$21.30$	$91.40$
w/o Ortho.	$2.91$	$0.63$	$15.80$	$88.10$
w/o Spectral	$3.77$	$1.12$	$9.74$	$41.60$
Full	2.00	0.00	8.48	77.78

Table 4: Geometric stability. MSV/SVVar measure spectral behaviour; OrthE quantifies orthogonality deviation; ColR (%) is the percentage of steps with

\sigma_{\min}<10^{-3}

Method	GVar $\downarrow$	UpdVar $\downarrow$	MSV $\downarrow$	Drift $\downarrow$
Baseline	4.7e -3	1.82	6.82	91.4
w/o WD/RMS	1.9e -3	0.97	2.91	57.0
w/o Spectral	8.3e -4	0.41	3.77	42.5
Full	2.5e -4	0.12	2.00	36.0

Table 5: Numerical stability. GradVar measures gradient variance, UpdateNorm Var quantifies update-scale fluctuations, MeanSV reflects state magnitude, and Drift measures long-horizon deviation.

Figure 9: Failure cases on CAMUS (a-b) and EchoNet-Dynamic (c-d).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1 : 3D geometric intuition of update directions on the loss landscape. The Euclidean Direction updates the state element-wise along the steepest descent, often ignoring local curvature. OSU evolves the state by projecting gradients via the matrix sign function, thereby maintaining the consistency of the underlying subspace structure. The

x

and

y

axes represent the two dimensions of the state

\mathbf{S}

. The movement toward the lower regions of the surface represents the direction of loss optimization.

Panel of Figure 16 — Figure A.2 : Spatiotemporal training dynamics visualization. Qualitative results on CAMUS patient 0225 (4-chamber view) at four training stages. For each stage (a-d), the top row displays input frames with segmentation masks, and the bottom row shows the corresponding feature activation maps. The progression from (a) to (d) illustrates the transition from diffused spatial activations and coarse masks to localized feature representations and temporally consistent segmentations throughout the cardiac cycle.

Explore similar work

Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction
Jul 8, 2026Xinran Chen, Xiyuan Wang, Guangquan Zhou +1Ultrasound Image Segmentation Self-Supervised Pre-Training
School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
Spatio-Temporal Fusion Model for Standard View Classification of Echocardiographic Videos
Jun 16, 2026Bo Gou, Jicheng Zhang, Jianlong Xiong +9Echocardiography Temporal Video Understanding
Department of Ultrasound, The First Affiliated Hospital of Chengdu Medical College, School of Clinical Medicine, Chengdu Medical College, Chengdu, China · College of Computer Science, Sichuan University, Chengdu, China · Department of Medical Ultrasound, West China Hospital of Sichuan University, Chengdu, China +1
The segmentation ceiling: why explicit left-ventricular masks do not improve learned ejection-fraction regression
Sep 17, 2026Farshid Farhadi Khouzani, Paul La Plante, Bryar Mustafa Shareef +1Left Ventricular Ejection Fraction Estimation Echocardiography
Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, NV, USA

Jul 8, 2026J/K move · Enter open · S save

Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction

Xinran Chen, Xiyuan Wang, Guangquan Zhou, Chuan Chen

School of Biological Science and Medical Engineering, Southeast University, Nanjing, China

Ultrasound Image Segmentation Self-Supervised Pre-Training Echocardiography Medical Image Segmentation Semantic Segmentation

Open paper AskPDF arXiv

While echocardiography is essential for cardiovascular diagnosis, inherent speckle noise and low signal-to-noise ratio often lead to ambiguous semantic features and fragmented boundaries. These limitations significantly hinder the segmentation accuracy of deep learning models in complex clinical cases. Moreover, temporal motion of the heart plays a critical role in recognizing anatomical structures. To address these challenges, we designed a STLSF module which comprises a window-matching-based semantic correction component and a semantics-guided texture enhancement component. By leveraging local transition probability correlations to correct semantics and employing semantics-guided texture enhancement, the STLSF module effectively mitigates texture instability and ambiguous semantic interpretations caused by disadvantaged echocardiography quality. Additionally, to facilitate the encoder's adaptation to the intrinsic priors of ultrasound-specific imaging patterns, we propose a frequency-aware denoising pre-training method. The entire work builds a convolution-based network with locality inductive bias and long-range dependencies. Extensive experiments confirm our SOTA performance, achieving 93.87% Dice on CAMUS and 92.62% on EchoNet-Dynamic, with respective HD95 values of 3.29mm and 2.73mm.

Abstract

Explore similar work

Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction

Spatio-Temporal Fusion Model for Standard View Classification of Echocardiographic Videos

The segmentation ceiling: why explicit left-ventricular masks do not improve learned ejection-fraction regression