Organizations: Faculty of Computer Science and Technology, Ocean University of China Qingdao, China · School of Information Science and Engineering, Shandong University Qingdao, China · Innovation School of Artificial Intelligence, Hefei University of Technology Hefei, China
Embodied intelligent ultrasound scanning enables the automation and standardization of the ultrasound examination process by integrating perception, decision-making, and execution capabilities. However, existing methods suffer from loosely coupled modeling between force and ultrasound modalities and lack awareness of scanning stages, which limits their ability to capture dynamic probe-tissue interactions. To address these issues, we propose ForceU-VLA, a force-aware Vision-Language-Action model for autonomous embodied ultrasound scanning, which leverages force signals and ultrasound image feedback throughout the scanning process to enable accurate and high-quality ultrasound acquisition. Firstly, we propose a Force-Ultrasound Synergistic Fusion Module (FUSFM) that synergistically fuses ultrasound visual and force-feedback information to provide stable, reliable guidance for probe motion. Secondly, a Stage-Adaptive Modulation Mechanism (SAMM) is proposed to accommodate the task requirements across different scanning stages by adaptively modulating multimodal features to enhance their representation quality. Additionally, we introduce ForceU-VLA-Data, a real-world, force-aware embodied ultrasound dataset that integrates visual, force, and action signals, including data from two organs across five representative clinical scanning views, and comprising 450 expert-collected trajectories with approximately 100,000 synchronized multimodal frames. Extensive experimental results demonstrate that ForceU-VLA significantly improves contact stability and probe pressure regulation in embodied ultrasound scanning, thereby effectively enhancing task execution quality and overall system reliability. The source code is available at https://github.com/VMVLab/ForceU-VLA.
Figures & tables
Figure 1. Comparison between classical VLA methods and the proposed ForceU-VLA. By explicitly modeling both ultrasound and force information, ForceU-VLA improves contact stability and probe pressure regulation, thereby enhancing both the efficiency and quality of ultrasound scanning.
Figure 2. Overview of ForceU-VLA. First, multimodal inputs are encoded into high-dimensional representations. Then, the Force–Ultrasound Synergistic Fusion Module performs collaborative fusion. Based on the fused features, the Stage-Adaptive Modulation Mechanism applies stage-aware adaptive weighting. Finally, the model generates the action sequence.
Figure 3. Overview of ForceU-VLA-Data, illustrating the dataset contents, acquisition system, collection trajectories, and representative standard planes.
Figure 4. Comparison of success rates between the proposed method and baseline approaches across five tasks. The proposed method significantly outperforms all baselines.
Figure 5. Visualization comparison of the scanning processes of π0.5 and ForceU-VLA on Task 1.
Method
Task1
Task2
Task3
Task4
Task5
Average
π0
593
831
611
557
399
598.2
π0 w/ US
461
735
601
457
351
521.0
π0 w/ US, F
420
702
586
410
342
492.0
π0 w/ US, F, S
414
695
536
435
328
481.6
ForceVLA
450
742
669
466
365
538.4
π0.5
580
645
532
440
301
499.6
Table 1. Scanning timesteps on five ultrasound tasks. Bold values denote the best performance.
Figure 6. Visualization of force comparison during the scanning processes of π0.5 and ForceU-VLA on Task 2.
Figure 7. Comparison of scanning processes under different lying positions
US
FUSFM
SAMM
Success rates ↑
Scanning timesteps ↓
Task1
Task2
Task3
Task4
Task5
Average
Task1
Task2
Task3
Task4
Task5
Average
80.0
66.7
62.2
68.9
84.4
72.4
580
645
532
440
301
499.6
✓
84.4
71.1
71.1
84.4
91.1
80.4
423
576
476
351
287
422.6
✓
✓
91.1
77.8
80.0
91.1
97.8
87.6
351
563
447
274
260
379.0
✓
✓
✓
97.8
86.7
84.4
97.8
100.0
93.3
329
528
419
237
243
351.2
Table 2. Ablation study of core Modules in ForceU-VLA.
Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.
Cheng Zhang, Xingzheng Wu, Guihao Yan +4
Innovation School of Artificial Intelligence, Hefei University of Technology Hefei, China
In this paper, we present a robotic ultrasound acquisition system that integrates stereo vision, touch-based feedback, and expert-informed strategies to perform autonomous and adaptive abdominal scans. The system records freehand motion and force data from expert radiologists, creating a framework to capture transducer motion, applied forces, and anatomical scanning strategies. This expert data is replayed to replicate characteristic scans with the robot, forming a foundation for further autonomous capabilities. Using stereo vision, the system generates three-dimensional topography maps of the patient's abdomen, which are refined through stiffness measurements at key points to delineate the rib cage boundary. These combined techniques enable the robot to execute two distinct scanning paths: an upward-angled sweep beneath the rib cage to visualize structures near the upper abdomen and a perpendicular sweep across soft tissue regions. A compliant, torque-controlled seven degree-of-freedom robotic manipulator is controlled to maintain consistent probe contact through closed-loop force control over the varied anatomical surfaces. Physical experiments demonstrate that the system achieves high-quality imaging comparable to expert scans while dynamically adapting to patient-specific topographies. Furthermore, the robotic system surpasses expert capabilities by enabling three-dimensional volume acquisition, which enhances diagnostic potential and provides volumetric data for advanced analyses. This work highlights the integration of expert knowledge into autonomous robotic systems and underscores the potential of combining perception-based autonomy with physical reasoning for enhanced diagnostic performance.
Adrian Piedra, R Brooke Jeffrey, Oussama Khatib
Stanford Robotics Laboratory, Computer Science Department, Stanford University, Stanford, CA 94305 USA · Department of Radiology, School of Medicine, Stanford University, Stanford, CA 94305 USA
We present an action-conditioned world model framework for goal plane probe guidance in robotic ultrasound, with a focus on neck ultrasound scanning. Autonomous ultrasound tasks often require large numbers of probe-motion trajectories for training, but collecting high-quality demonstrations is labor-intensive and explicit simulators are difficult to build because ultrasound appearance depends on contact, tissue deformation, and view-dependent acoustic artifacts. We address this problem with a two-stage model-based learning pipeline. First, a latent conditional diffusion world model predicts future ultrasound observations from recent context frames, probe motions and temporal offset. Second, a goal-conditioned temporal transformer predicts ordered probe motions and is fine-tuned using rewards from the frozen world model. Experiments on the self-collected dataset show that the world model preserves action-dependent anatomical structure on target-directed scans. In real-world closed loop experiments, the framework achieves success rates of 70.0% for carotid guidance and 65.0% for thyroid guidance. These results demonstrate the potential of learned ultrasound dynamics for training goal-directed robotic probe navigation.
Siqi Fan, Mingcong Chen, Ran Liu +5
Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Hong Kong SAR · Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong SAR · Department of Biomedical Engineering, City University of Hong Kong, Hong Kong SAR +3