Robotic Ultrasound

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 17

Oct 1, 2026cs.CL

Child-Adapted Structured Phonological Representations for Interpretable Speech Sound Analysis

Structured phonological representations provide an interpretable alternative to generic speech embeddings, but existing models are largely trained on adult speech. We adapt PhonoQ-2.0 to child speech using CHILDES-Aligned data and compare three alignment-supervision conditions (Adult, Adult+Child, and Child-only) across two initialization strategies (Adult PhonoQ and scratch). Generalization is evaluated against manual child-speech annotations. On 1,352 consonant targets from 58 typically developing children, child-speech adaptation improves voicing recognition across all supervision conditions, from 0.922 macro-F1 for Adult PhonoQ to 0.972--0.987 after adaptation. Manner is more sensitive to alignment supervision: Adult+Child MFA reaches 0.804 and 0.796, compared to approximately 0.70 under Adult MFA supervision. Place remains comparatively strong across systems (0.871--0.902), although per-class performance varies substantially. The velar-fronting contrast is preserved across all seven model variants. Longitudinal UltraPhonix analysis further reveals speaker-specific velar and post-alveolar changes that are largely preserved across models and broadly consistent with reported clinical progress.

Abner Hernandez, Tomás Arias Vergara, Andreas Maier +1

Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · GITA Lab. Facultad de Ingenier´ıa. Universidad de Antioquia UdeA, Medell´ın, Colombia · Chair for AI in Healthcare and Medicine, Technical University of Munich, Munich, Germany

Grapheme-To-Phoneme Speaker Robotic Ultrasound

Sep 28, 2026cs.AR

Analog Computing revisited: A fully analog and minimalistic Damage Detector for Ultrasonic Testing enabling Material-Integrated Structural Health Monitoring

Ultrasonic Testing (UT) is commonly used to detect damage in structures, e.g., metal plates. A sensor acquires Ultrasonic waves, e.g., by using PZT transducers. The time-resolved sensor signal must be processed with analog electronics, e.g., amplified and filtered. Commonly a digitalization follows using an Analog-to-Digital converter, finally processing the digital sensor signal, applying digital signal processing, feature extraction, and Machine Learning by using powerful microprocessor systems. The disadvantages of digital processing systems are their high number of transistors (microchip area), energy consumption, state-dependent processing and therefore sensitivity to energy supply interruption. Beyond silicon electronics, printed organic electronics gains interest. But printed electronics is still limited to low transistor and electronic component counts (typically 100). We will investigate and demonstrate a fully analog signal processing and feature extraction system consisting of an analog Hilbert transform deriving the signal envelope, simple analog arithmetic calculations for feature extraction, and finally damage classification and regression using an analog Artificial Neural Network. We expect a full damage detection system with less than 100 transistors. We will test our damage detection system with PZT transducer signals from Steel plates with circular defects. The focus of this work is the analog computation of the signal envelope (using all-pass filter networks for approximation of the Hilbert transform) and the analog feature extraction as well as the prediction of damage, forming an analog computer which can perform in-sensor computation, computing without a digital computer.

Stefan Bosse

University of Koblenz, Practical Computer Science, Koblenz, Germany

Piezoelectric Transducers Robotic Ultrasound Structural Damage Fourier

Sep 28, 2026cs.RO

Contact-Aware Impedance Controller for Robot-Assisted Ultrasound Imaging

Safe robot-assisted ultrasound imaging requires a reliable controller able to detect and localize probe--tissue interaction. In this paper, we present a B-mode ultrasound image-based contact perception method and a contact-aware impedance controller for robotic ultrasound imaging. The proposed method detects acoustic contact independently of force measurements, enabling contact-conditioned force/torque taring to reduce residual wrench bias. During contact, the method continuously estimates the effective contact location along the curved probe surface and uses it to update the controller interaction frame, enabling visual servoing of the physical probe--tissue contact point during imaging. Experiments on an agar phantom demonstrated a contact-localization RMSE of

\mathbf{1.46 \pm 0.14}

~mm over probe roll angles from

\mathbf{-15^\circ}

to

\mathbf{15^\circ}

. During static rolling, the proposed controller maintained task-space tracking accuracy comparable to a conventional fixed-frame impedance controller while reducing the maximum compressive interaction force from

\mathbf{31.56}

~N to

\mathbf{20.09}

~N, corresponding to a

\mathbf{36.3\%}

reduction. These results demonstrate the potential of ultrasound images as direct contact feedback for safe and accurate robot-assisted ultrasound imaging.

MD Miraj Arefin, M Efe Tiryaki

Center for Robotics and AI (ROMER), Middle East Technical University, Ankara, Turkey

Robotic Ultrasound Impedance Control Ground Reaction Forces Soft Robotics Acoustic

Aug 11, 2026cs.RO

From Self-Normal-Positioning to Omni-Directional Tracking: Real-Time Surface Modeling Enabled Probe Tilt Control for Robotic Ultrasound Imaging

Ultrasound (US) provides real-time, radiation-free imaging, but the image quality depends strongly on how the probe is oriented against the patient body. Robotic US can reduce operator workload and improve acquisition consistency; however, most existing systems focus on normal positioning, where the probe is maintained perpendicular to the local surface. This constraint is inadequate for examinations like echocardiography, where obtaining a diagnostic view requires a non-normal probe angle. Consequently, a clinically useful robotic system must sense the local surface in real-time and preserve the desired probe orientation. Here, we propose an omni-directional probe-orientation control framework that integrates RGB-D perception, local-surface modeling, and task-space orientation control. The surface model fuses multi-view point clouds and provides a quadratic estimate of the local surface. A desired imaging direction is then encoded relative to the normal, enabling the probe to track arbitrary angles. The framework was evaluated through flat-surface tracking, phantom target-angle recovery, and in-vivo tracking of an expert selected view. Results show that the mean angular tracking error was 1.06 +- 0.66 deg. The system recovered a non-normal tilt angle of up to 44.39 +- 2.59 deg relative to the surface normal, and acquired the desired heart chamber view in the phantom and in-vivo experiments.

Xihan Ma, Haichong Zhang

Department of Robotics Engineering, Worcester Polytechnic Institute, 100 Institute Road, Worcester, MA 01609 USA

Robotic Ultrasound Echocardiography Real-Time Systems

Aug 8, 2026cs.LG

Machine-Learning-Based Diagnostic Framework for Passive Ultrasonic Detection of Railway Wheel Defects

Reliable identification of railway wheel defects is important for safety and maintenance. This study develops a machine-learning-based diagnostic framework for multi-class defect identification using passive air-coupled ultrasonic acoustic emission signals. Data were collected from eleven full-scale railway wheelsets representing nine health states. Time- and frequency-domain features were evaluated using Kruskal-Wallis statistical testing and mutual-information analysis to identify the most discriminative indicators. A Random Forest classifier was then trained using the selected features with stratified 5-fold cross-validation. The model achieved a balanced accuracy of approximately 0.66 and a Macro-F1 score of 0.65 across the nine classes. Decay rate, kurtosis, skewness, and envelope low-frequency power emerged as the most influential features, while a compact subset of features retained most of the classification performance. The results demonstrate the feasibility of combining passive ultrasonic sensing, statistical feature selection, and supervised machine learning for non-contact railway wheel defect classification and provide a foundation for future field-deployable inspection systems.

Aashish Shaju, Steve Southward, Mehdi Ahmadian

Department of Mechanical Engineering, Virginia Tech Blacksburg, VA 24061, USA

Defect Detection Robotic Ultrasound Vibration Inspection Diagnostic Framework Diagnosis

Aug 6, 2026cs.RO

Acoustic-driven millimetric helical robot: ultrasonic synergistic manipulation in confined fluidic environment

Acoustic field-driven manipulation provides a non-contact and non-invasive strategy for controlling microscale and nanoscale objects, yet its extension to millimeter-scale robots was limited by insufficient propulsion efficiency in confined biological environments. Here, a coordinated multi-acoustic-field approach is introduced, which harnesses the synergistic action of acoustic radiation forces and acoustic streaming flows to enable controlled locomotion of millimeter-scale helical robots and enhance propulsion. Multiphysics simulations captured the dynamics of millimeter-scale helical robots under combined acoustic fields, and experimental validation demonstrated their locomotion capabilities, including planar navigation, inclined climbing, and vertical motion. Semi-autonomous navigation experiments further confirmed that ultrasonic synergy substantially improved maneuverability. In vitro tests in porcine venous vessels demonstrated that coordinated acoustic fields supported both unidirectional and reciprocating motion under biologically relevant confinement. These findings provide mechanistic insight into scaling acoustic micromanipulation to the millimetre regime and support biomedical applications requiring versatile and controllable robotic mobility.

Hanlin Wang, Xin Wang, Xinwei Wei +4

State Key Laboratory of Industrial Control Technology and the Institute of Cyber-Systems & Control, Zhejiang University, Hangzhou 310027, Zhejiang, China · Huzhou Institute of Zhejiang University, Huzhou Key Laboratory of Robot System Integration and Intelligent Equipment, Huzhou 313099, Zhejiang, China · State Key Laboratory of Advanced Drug Delivery and Release Systems, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou 310058, Zhejiang, China +4

Robotic Ultrasound Fluid Controls Soft Robotics Robot Systems

Jul 24, 2026eess.IV

Learning-based Hierarchical Tracheal Anatomy Understanding from Sparse Surgical Demonstration Annotations for Ultrasound Robots

Tracheostomy requires precise localization of the tracheal incision site; however, conventional manual palpation is subjective and often unreliable, while ultrasound utility remains operator-dependent. This work presents a learning-based framework for hierarchical tracheal anatomy understanding, designed specifically for ultrasound-guided robotic systems. We propose a two-stage perception pipeline integrating a YOLOv8n localization backbone with a sparse, prompt-optimized SAM2 decoder to achieve high-fidelity segmentation from sparse surgical annotations. Our hybrid training strategy, bridging curated laboratory data with unconstrained sequences, ensures clinical robustness. Experimental benchmarks demonstrate that this decoupled architecture effectively balances generalization, precision, and efficiency. The YOLOv8n and SAM2 framework achieves a consistent Mean Dice Similarity Coefficient (DSC) of 0.777 across both controlled and generalized domains. This significantly outperforms U-Net baselines, which often suffer from anatomical fragmentation and performance degradation (Generalization DSC

\le

0.494). By constraining mask decoding to targeted, sparse regions of interest, our model achieves a throughput of 6.92 FPS, which is vital for closed-loop robotic teleoperation. This study confirms that a robust hierarchical understanding of tracheal anatomy can be derived by coupling lightweight localization with foundation-scale visual models. Our framework establishes a scalable foundation for standardized, autonomous surgical assistance, effectively navigating the variability of real-world ultrasound to enhance the safety and precision of robotic-assisted tracheostomy.

Hiu Ching Cheung, Wenchao Yue, Zhengran Han +4

Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom · Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences, Hong Kong, China +1

Robotic Ultrasound Anatomy Yolo26

Jul 24, 2026cs.RO

Action-Conditioned World Model for Goal Plane Probe Guidance in Robotic Ultrasound

We present an action-conditioned world model framework for goal plane probe guidance in robotic ultrasound, with a focus on neck ultrasound scanning. Autonomous ultrasound tasks often require large numbers of probe-motion trajectories for training, but collecting high-quality demonstrations is labor-intensive and explicit simulators are difficult to build because ultrasound appearance depends on contact, tissue deformation, and view-dependent acoustic artifacts. We address this problem with a two-stage model-based learning pipeline. First, a latent conditional diffusion world model predicts future ultrasound observations from recent context frames, probe motions and temporal offset. Second, a goal-conditioned temporal transformer predicts ordered probe motions and is fine-tuned using rewards from the frozen world model. Experiments on the self-collected dataset show that the world model preserves action-dependent anatomical structure on target-directed scans. In real-world closed loop experiments, the framework achieves success rates of 70.0% for carotid guidance and 65.0% for thyroid guidance. These results demonstrate the potential of learned ultrasound dynamics for training goal-directed robotic probe navigation.

Siqi Fan, Mingcong Chen, Ran Liu +5

Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Hong Kong SAR · Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong SAR · Department of Biomedical Engineering, City University of Hong Kong, Hong Kong SAR +3

Robotic Ultrasound Efficient World-Action Model Scalable Robot Learning Inverse Dynamics Geometric Guidance World

Jul 22, 2026cs.RO

Distributed Acoustic Localization Array Deployed Using a Soft Everting Vine Robot

Soft robot exteroception is increasingly being explored for a variety of field applications. In this work, we present a sound-based system for localizing disaster victims in confined and unstructured environments, based on a distributed acoustic sensing architecture embedded along the body of a soft everting vine robot. We propose a dynamic Steered Response Power with Phase Transform framework that supports both far-field direction-of-arrival estimation and near-field three-dimensional source localization as the robot approaches the sound source. To better understand the design and control space related to localizing sound using a soft, shape-morphing robot body, we conduct experiments measuring the accuracy of these methods for a five-microphone array attached to the robot body using three placements relative to the outer membrane of the robot (inside the pressurized body, inside the inner tail, and outside the outer wall) and in four robot configurations (linear, double linear, circular, and sinusoidal). We measure the change in accuracy as the signal-to-noise ratio, the direction of approach, and the distance of the sound source from the center of the array change. Finally, we demonstrate a vine robot growing into an arbitrary shape while carrying microphones along its outer wall, and show that a sound source located with the array's near field can be localized with high accuracy after only three microphones have everted from the robot body. These results highlight the potential of distributed acoustic sensing for reliable victim localization using soft growing robots.

Sebastian Lorca Godoy, Ciera McFarland, Michael Val +3

Sound Source Localization Soft Robotics Robotic Ultrasound Mobile Robots Robot Systems

Jul 8, 2026cs.RO

SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences

Powered prosthetic hands are frequently abandoned, largely due to the limited functionality of current devices that rely on surface electromyography (sEMG). Sonomyography (ultrasound) has emerged as a promising alternative, owing to its ability to observe muscle activity in real time and control a greater number of degrees of freedom. Yet, existing ultrasound-based methods require per-user fine-tuning, limiting their commercialization. We propose SonoRank, an important step towards calibration-free finger flexion detection from forearm ultrasound video. SonoRank first learns to rank pairs of ultrasound sequences by their relative motion magnitude for each of the five fingers. The learned representations are then fine-tuned to classify whether each finger is actively flexing, using a rest reference that is captured at the beginning of the operation. Under 12-fold leave-one-subject-out cross-validation on a dataset of twelve subjects with synchronized kinematics, SonoRank achieves a 28% improvement in F1 score over direct classification baselines that skip the ranking stage. These results establish pairwise ranking as an effective pretraining signal for subject-independent control, bringing ultrasound-based prosthetics closer to practical, calibration-free deployment.

Dean Zadok, Alon Wolf, Alex M. Bronstein +1

Department of Computer Science, Technion, Haifa, Israel · Department of Mechanical Engineering, Technion, Haifa, Israel

Abox Abduction Problem Robotic Ultrasound Finger Ultrasound Pretraining

Jun 24, 2026eess.SP

Transformer-based Multisensor Data Fusion of Ultrasonic Guided Wave and FBG-based Strain Measurements for Multitask Aerospace Structural Health Monitoring

Structural health monitoring (SHM) has emerged as an essential tool for ensuring the integrity and reliability of critical engineering structures, particularly in aerospace applications. Since each sensing technology has its limitations, the fusion of different modalities enables capturing a more complete picture of inhomogeneous materials, like composites. However, effective multisensor data fusion in SHM is often hindered by heterogeneous sensing modalities that operate at disparate sampling frequencies and acquisition intervals. To address these challenges, this paper proposes a Transformer-based data fusion framework that integrates multisensor data streams from piezoelectric transducer (PZT) capturing ultrasonic guided wave signals and fiber Bragg grating (FBG) sensors for strain measurements. By incorporating an attention-mechanism visualization, the proposed framework enables transparent, multitask learning for both health indicator (HI) prediction and damage localization. The framework was experimentally validated using aircraft composite structures subjected to compression-compression fatigue cyclic loading. For HI prediction, the framework consistently achieved a mean absolute error (MAE) and root mean squared error (RMSE) below 0.1, representing a nearly 60% performance improvement over single-sensor approaches (PZT or FBG alone) and baseline deep learning models. For damage localization, the model demonstrated the highest accuracy, maintaining an MAE and RMSE below 0.0465 and 0.1571, respectively. These results demonstrate that the proposed Transformer-based data fusion framework significantly outperforms single-source models and state-of-the-art deep learning models in both HI prediction and damage localization accuracy.

Xin Yang, Morteza Moradi, Tongtong Yan +4

Department of Mechanical Engineering & Division of Mechatronic System Dynamics (LMSD), KU Leuven, Belgium · Center of Excellence in Artificial Intelligence for Structures, Prognostics & Health Management, Aerospace Engineering Faculty, Delft University of Technology, Kluyverweg 1, Delft, 2629 HS, the Netherlands · Department of Civil Engineering, The University of British Columbia, Vancouver, Canada +2

Multi-Sensor Fusion Piezoelectric Transducers Robotic Ultrasound Structural Damage Measures

Jun 21, 2026cs.RO

EmbodiedUS-FS: Fast Slow Intelligence for Ultrasound Robotics

Robotic ultrasound scanning in real clinical environments requires both high-level clinical workflow reasoning and low-level closed-loop execution. Physicians natural-language instructions often contain implicit anatomical targets, procedural logic, image-quality requirements, and safety constraints, while execution is affected by patient motion, contact variations, and target drift. We propose a fast and slow hierarchical embodied ultrasound system for safe and interpretable robotic ultrasound assistance. The Slow Brain performs intent parsing and stage-wise task planning with knowledge augmentation from an API and handbook corpus, and generates executable plans through task-graph construction and structured plan verification. The Fast Brain fuses multimodal feedback, including ultrasound images, robot pose and force states, and patient-motion information, to refine local actions and perform image-quality-guided recovery behaviors. The system further integrates a Safety Shield and a hierarchical escalation policy to constrain risky actions and trigger replanning or human confirmation under persistent failures or safety-bound violations. Experiments on planning evaluation, closed-loop execution under dynamic perturbations, and safety-mechanism validation demonstrate that the proposed hierarchical design improves task success rates while reducing safety violations.

Fangzhuo Zhang, Xinyu Wang, Xiao Yang +1

independent researcher · University of Wyoming · Shanghai World Foreign Language Academy +1

Robotic Ultrasound Robotic Systems Safety Claims Anatomy Intelligence

Jun 12, 2026cs.CV

Giving AI a Headache: Acoustic Adversarial Attacks to Computer Vision Applications

Artificial Intelligence (AI) is increasingly used to automate a variety of real-world computer vision (CV) applications, such as autonomous vehicle control, facial recognition, and security cameras. Recent research has shown that acoustic vibration can induce real physical motion in cameras, interfering with their internal stabilization mechanisms. Because the motion falls outside the conditions the stabilization system was designed to handle, the system introduces artifacts into the frame, causing AI-based CV models to misclassify, miss targets, or hallucinate objects. Previous work used ultrasonic frequencies (>20 kHz) to perform short-range attacks, which limits them to short distances due to the attenuation exhibited by high frequencies. In this work, we investigate acoustic attacks using lower frequencies in the audible range (<20 kHz), and we further expand our analysis to include how various image and object features are affected by the attacks. Specifically, we performed physical experiments to demonstrate the viability of our attacks on an off-the-shelf object detection model (YOLO11) by resonating a commercially available camera with various frequencies. Based on our results, we provide insights into several factors that make an AI CV system more vulnerable to these attacks, which could help inform the development of future mitigation strategies.

Nicole Villavicencio-Garduño, Maksim Ekin Eren, Milo Prisbrey +2

Carnegie Mellon University, Pittsburgh, USA · Los Alamos National Laboratory, Los Alamos, USA

Computer Vision Robotic Ultrasound Motion

Jun 9, 2026cs.CV

FSS-Net: Frequency-Spatial Synergy Network with Wavelet Attention for Carotid Artery Ultrasound Segmentation

Accurate segmentation of carotid arteries in ultrasound imaging is critical for stroke risk assessment. However, speckle noise, low contrast, and blurred boundaries remain major challenges. In this paper, we propose a Frequency-Spatial Synergy Network (FSS-Net) to achieve noise-robust and high-precision carotid artery segmentation. The network integrates wavelet transform, multi-domain attention, and edge enhancement into a unified encoder-decoder architecture. Specifically, a Channel-Spatial-Wavelet Attention (CSWA) module is designed to suppress noise and purify semantic features in the frequency domain. A Wavelet-Enhanced Bottleneck (WEB) module is introduced to capture long-range global dependencies efficiently. Furthermore, a Laplacian-Guided Adaptive Edge Fusion (LAEF) module compensates high-frequency details and maintains boundary continuity. Extensive experiments on carotid ultrasound datasets show that FSS-Net achieves a Dice score (DSC) of 96.46% and strong robustness under low SNR conditions, outperforming several state-of-the-art methods. This method realizes accurate segmentation of carotid artery in ultrasonic imaging, effectively identifies carotid atherosclerotic plaque, and is verified by other task (such as segmentation of breast cancer), suggesting that it has good clinical application potential in identifying abnormal tissue masses in ultrasonic images.

Jiawei Liu, Zhijiang Wan, Junhua Hu +5

Ji luan Academy, Nanchang University, Nanchang 330031, China · School of Information Engineering, Nanchang University, Nanchang 330031, China · State Key Laboratory of Water Cycle and Water Security, China Institute of Water Resources and Hydropower Research, Beijing 100038, China +4

Vessel Segmentation Clinical Ultrasound Dataset Coronary Angiography Robotic Ultrasound Wavelets

Jun 4, 2026cs.RO

Visuotactile and Explicitly Force-Controlled Robotic Ultrasound for Abdominal Volumetric Reconstruction

In this paper, we present a robotic ultrasound acquisition system that integrates stereo vision, touch-based feedback, and expert-informed strategies to perform autonomous and adaptive abdominal scans. The system records freehand motion and force data from expert radiologists, creating a framework to capture transducer motion, applied forces, and anatomical scanning strategies. This expert data is replayed to replicate characteristic scans with the robot, forming a foundation for further autonomous capabilities. Using stereo vision, the system generates three-dimensional topography maps of the patient's abdomen, which are refined through stiffness measurements at key points to delineate the rib cage boundary. These combined techniques enable the robot to execute two distinct scanning paths: an upward-angled sweep beneath the rib cage to visualize structures near the upper abdomen and a perpendicular sweep across soft tissue regions. A compliant, torque-controlled seven degree-of-freedom robotic manipulator is controlled to maintain consistent probe contact through closed-loop force control over the varied anatomical surfaces. Physical experiments demonstrate that the system achieves high-quality imaging comparable to expert scans while dynamically adapting to patient-specific topographies. Furthermore, the robotic system surpasses expert capabilities by enabling three-dimensional volume acquisition, which enhances diagnostic potential and provides volumetric data for advanced analyses. This work highlights the integration of expert knowledge into autonomous robotic systems and underscores the potential of combining perception-based autonomy with physical reasoning for enhanced diagnostic performance.

Adrian Piedra, R Brooke Jeffrey, Oussama Khatib

Stanford Robotics Laboratory, Computer Science Department, Stanford University, Stanford, CA 94305 USA · Department of Radiology, School of Medicine, Stanford University, Stanford, CA 94305 USA

Robotic Ultrasound Cone-Beam Computed Tomography Tactile Radiology Report Generation

Apr 28, 2026cs.CV

SAMe: A Semantic Anatomy Mapping Engine for Robotic Ultrasound

Robotic ultrasound has advanced local image-driven control, contact regulation, and view optimization, yet current systems lack the anatomical understanding needed to determine what to scan, where to begin, and how to adapt to individual patient anatomy. These gaps make systems still reliant on expert intervention to initiate scanning. Here we present SAMe, a semantic anatomy mapping engine that provides robotic ultrasound with an explicit anatomical prior layer. SAMe addresses scan initiation as a target-to-anatomy-to-action process: it grounds under-specified clinical complaints into structured target organs, instantiates a patient-specific anatomical representation for the grounded targets from a single external body image, and translates this representation into control-facing 6-DoF probe initialization states without any additional registration using preoperative CT or MRI. The anatomical representation maintained by SAMe is explicit, lightweight (single-organ inference in 0.08s), and compatible with downstream control by design. Across semantic grounding, anatomical instantiation, and real-robot evaluation, SAMe shows strong performance across the full initialization pipeline. In real-robot experiments, centroid-based SAMe initialization outperformed the body-keypoint-based heuristic baseline under a budget-matched single-target setting for both liver (86.7% versus 46.7%) and kidney (80.0% versus 73.3%) initialization. Furthermore, The trial-level organ-hit rate reached 97.3% for liver and 83.3% for kidney when multiple candidate targets were available. These results establish an explicit anatomical prior layer that addresses scan initialization and is designed to support broader downstream autonomous scanning pipelines, providing the anatomical foundation for complaint-driven, anatomically informed robotic ultrasonography.

Jing Zhang, Duojie Chen, Wentao Jiang +7

School of Computer Science, Wuhan University, China · Hubei Center for Applied Mathematics, Wuhan University, China · Department of Ultrasound, The Central Hospital of Wuhan, China +4

Robotic Ultrasound Anatomy Semantic Mapping

Apr 22, 2026cs.RO

A Vision-Language-Action Model for Adaptive Ultrasound-Guided Needle Insertion and Needle Tracking

Ultrasound (US)-guided needle insertion is a critical yet challenging procedure due to dynamic imaging conditions and difficulties in needle visualization. Many methods have been proposed for automated needle insertion, but they often rely on hand-crafted pipelines with modular controllers, whose performance degrades in challenging cases. In this paper, a Vision-Language-Action (VLA) model is proposed for adaptive and automated US-guided needle insertion and tracking on a robotic ultrasound (RUS) system. This framework provides a unified approach to needle tracking and needle insertion control, enabling real-time, dynamically adaptive adjustment of insertion based on the obtained needle position and environment awareness. To achieve real-time and end-to-end tracking, a Cross-Depth Fusion (CDF) tracking head is proposed, integrating shallow positional and deep semantic features from the large-scale vision backbone. To adapt the pretrained vision backbone for tracking tasks, a Tracking-Conditioning (TraCon) register is introduced for parameter-efficient feature conditioning. After needle tracking, an uncertainty-aware control policy and an asynchronous VLA pipeline are presented for adaptive needle insertion control, ensuring timely decision-making for improved safety and outcomes. Extensive experiments on both needle tracking and insertion show that our method consistently outperforms state-of-the-art trackers and manual operation, achieving higher tracking accuracy, improved insertion success rates, and reduced procedure time, highlighting promising directions for RUS-based intelligent intervention.

Yuelin Zhang, Qingpeng Ding, Longxiang Tang +2

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · Shenzhen International Graduate School, Tsinghua University, China +1

Robotic Ultrasound Visual Servoing Diffusion-Based Vision-Language-Actions Geometric Guidance