Anatomical Prior-Driven Framework for Autonomous Robotic Cardiac Ultrasound Standard View Acquisition
Authors: Zhiyan Cao, Zhengxi Wu, Yiwei Wang, Pei-Hsuan Lin, Li Zhang, Zhen Xie, Huan Zhao, Han Ding
Organizations: The authors are with the State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China. · Zhengxi Wu is with the School of Biomedical Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, China. · Yiwei Wang is also with the Institute of Medical Equipment Science and Engineering, Huazhong University of Science and Technology, Luoyu Road 1037, Wuhan, Hubei 430074, China. · Pei-Hsuan Lin is with the Information Intelligence Lab, Department of Electrical Engineering, National Chung Hsing University, Taichung 402, Taiwan. · Li Zhang is with the Department of Ultrasound Medicine, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Jiefang Avenue 1277, Wuhan, Hubei 430022, China. · Zhen Xie is with the Institute of Systems Science (ISS), National University of Singapore (NUS), Singapore 119615, Singapore.
Cardiac ultrasound diagnosis is critical for cardiovascular disease assessment, but acquiring standard views remains highly operator-dependent. Existing medical segmentation models often yield anatomically inconsistent results in images with poor textural differentiation between distinct feature classes, while autonomous probe adjustment methods either rely on simplistic heuristic rules or black-box learning. To address these issues, our study proposed an anatomical prior (AP)-driven framework integrating cardiac structure segmentation and autonomous probe adjustment for standard view acquisition. A YOLO-based multi-class segmentation model augmented by a spatial-relation graph (SRG) module is designed to embed AP into the feature pyramid. Quantifiable anatomical features of standard views are extracted. Their priors are fitted to Gaussian distributions to construct probabilistic APs. The probe adjustment process of robotic ultrasound scanning is formalized as a reinforcement learning (RL) problem, with the RL state built from real-time anatomical features and the reward reflecting the AP matching. Experiments validate the efficacy of the framework. The SRG-YOLOv11s improves mAP50 by 11.3% and mIoU by 6.8% on the Special Case dataset, while the RL agent achieves a 92.5% success rate in simulation and 86.7% in phantom experiments.
Figures & tables
Fig. 1: Framework of anatomical prior (AP)–driven robotic cardiac standard view acquisition system. The AP standard takes the apical four-chamber (A4C) standard view as an example.
Fig. 2: Typical segmentation challenges in cardiac US imaging. (a) represents missed detection of the RV when obscured by lung artifacts, (b) denotes mislabeling between the RV and LV, and (c) describes duplicate prediction of the LV.
Fig. 3: Framework of the YOLO-based multi-class cardiac segmentation model with the spatial-relation graph (SRG) module. Specifically, a backbone extracts semantic features from the input image. A multi-scale neck then fuses multi-resolution features to enhance the representation of cardiac structures. The SRG module subsequently embeds AP-guided constraints into the neck feature maps to promote anatomical consistency in the predictions. Finally, a segmentation head decodes the enhanced features into bounding boxes and instance segmentation masks.
Fig. 4: Visualized examples present scenarios involving the three critical errors in cardiac US segmentation: missed detection, mislabeling between classes, and duplicate predictions. Each row corresponds to a distinct segmentation scenario. The first column shows the ground truth, and each subsequent column displays the output of one evaluated model. As a result, the segmentation results in 3-b, 3-e, 3-f, and 3-g exhibit missed detection, the results in 1-b, 2-b, 2-g, 3-c, 3-d, and 3-e show mislabeling between classes, and the results in 1-b, 1-g, 2-b, 2-g, 3-c, 3-d, and 3-f have duplicate predictions. Notably, the SRG-YOLOv11s does not exhibit any of these three critical errors across the scenarios.
Models
mAP 50
mAP 50–95
mIoU
mDice
YOLOv5s
0.397
0.146
0.411
0.490
YOLOv8s
0.352
0.128
0.397
0.478
YOLOv11s
0.423
0.169
0.426
0.506
FastSAM-S
0.418
0.144
0.373
0.456
FastSAM-X
0.407
0.165
0.357
0.427
H-SAM
–
–
0.261
0.334
TABLE I: Comparison experiments on the Special Case dataset.
Variant
Global
Local
Precision
Recall
mAP 50
mAP 50–95
mIoU
mDice
SRG-YOLOv11s
✓
Full
0.471
0.480
0.471
0.194
0.455
0.530
SRG-YOLOv11s
✗
Full
0.461
0.496
0.468
0.185
0.444
0.523
SRG-YOLOv11s
✓
Identity
0.392
0.368
0.359
0.141
0.407
0.485
SRG-YOLOv11s
✗
Identity
0.402
0.412
0.362
0.126
0.409
0.490
YOLOv11s-only
None
None
0.485
0.433
0.423
0.169
0.426
0.506
TABLE II: Ablation on SRG-YOLOv11s: effect of the global encoding and local relation scorer.
Param.
Value
Param.
Value
Param.
Value
δ
1∘
w1
0.7
w2
0.14
w3
3
w4
0.1
∀(i,j)∈Ptar:wi,j
1
wRV
0.2
wLA
0.5
wRA
0.5
TABLE III: Parameters of the MDP Simulation for RL Training.
Fig. 5: Visualization of two representative experimental results. (a) represents a moderate deviation example, and (b) represents a severe deviation example. In both examples, the content is organized in three rows. The top row illustrates the action Q-value of different action indices, where the red bar represents the selected action. The middle row presents segmented cardiac images, with annotations of cardiac chambers, RV (green), LV (blue), RA (orange), and LA (light blue), overlaid on them. The bottom row displays the corresponding snapshot utilized in the experiment.
Robotic ultrasound has advanced local image-driven control, contact regulation, and view optimization, yet current systems lack the anatomical understanding needed to determine what to scan, where to begin, and how to adapt to individual patient anatomy. These gaps make systems still reliant on expert intervention to initiate scanning. Here we present SAMe, a semantic anatomy mapping engine that provides robotic ultrasound with an explicit anatomical prior layer. SAMe addresses scan initiation as a target-to-anatomy-to-action process: it grounds under-specified clinical complaints into structured target organs, instantiates a patient-specific anatomical representation for the grounded targets from a single external body image, and translates this representation into control-facing 6-DoF probe initialization states without any additional registration using preoperative CT or MRI. The anatomical representation maintained by SAMe is explicit, lightweight (single-organ inference in 0.08s), and compatible with downstream control by design. Across semantic grounding, anatomical instantiation, and real-robot evaluation, SAMe shows strong performance across the full initialization pipeline. In real-robot experiments, centroid-based SAMe initialization outperformed the body-keypoint-based heuristic baseline under a budget-matched single-target setting for both liver (86.7% versus 46.7%) and kidney (80.0% versus 73.3%) initialization. Furthermore, The trial-level organ-hit rate reached 97.3% for liver and 83.3% for kidney when multiple candidate targets were available. These results establish an explicit anatomical prior layer that addresses scan initialization and is designed to support broader downstream autonomous scanning pipelines, providing the anatomical foundation for complaint-driven, anatomically informed robotic ultrasonography.
Jing Zhang, Duojie Chen, Wentao Jiang +7
School of Computer Science, Wuhan University, China · Hubei Center for Applied Mathematics, Wuhan University, China · Department of Ultrasound, The Central Hospital of Wuhan, China +4
Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.
Cheng Zhang, Xingzheng Wu, Guihao Yan +4
Innovation School of Artificial Intelligence, Hefei University of Technology Hefei, China
Self-supervised pre-training paradigm has gained increasing prominence for learning transferable representations in medical imaging, yet existing methods for ultrasound (US) images operate at the image or frame level, overlooking the anatomical context for clinical-aligned representation learning. In this work, we propose an anatomy-anchored ultrasound self-supervision framework ANAUS that shifts representation learning from generic visual regions to clinically meaningful anatomical structures. Utilizing a learnable latent prompt engine alongside a one-time domain adaptation on existing public image-mask pairs, we empower the LP-SAM module to achieve annotation-free anatomy delineation at scale. Building upon this anatomical grounding, we propose a dual-policy self-supervised learning paradigm consisting of inter-view semantics-aware anatomy-separating alignment and contextual core-region prediction to enhance representation learning. Specifically, the former enforces feature invariance within identical anatomical regions while promoting discriminability across distinct structures; the latter compels the model to reconstruct corrupted regions, thereby capturing fine-grained structural details. Extensive evaluations on six public datasets demonstrate that ANAUS consistently outstrips current state-of-the-art methods while maintaining the computational efficiency essential for clinical deployment. Code is available at https://github.com/zhcz328/ANAUS.
Chunzheng Zhu, Yijun Wang, Jianxin Lin +5
Hunan University, Changsha, China · Shenzhen Maternity and Child Healthcare Hospital, Shenzhen, China