cs.CVFeb 17, 2026

Scaling Ultrasound Volumetric Reconstruction via Mobile Augmented Reality

Authors: Kian Wei Ng, Yujia Gao, Deborah Khoo, Ying Zhen Tan, Chengzheng Mao, Haojie Cheng, Andrew Makmur, Kee Yuan Ngiam, 

Organizations: National University Health System, Singapore · College of Design and Engineering, National University of Singapore, Singapore

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Accurate volumetric characterization of lesions is essential for oncologic diagnosis, risk stratification, and treatment planning. While imaging modalities such as Computed Tomography provide high-quality 3D data, 2D ultrasound (2D-US) remains the preferred first-line modality for breast and thyroid imaging due to cost, portability, and safety factors. However, volume estimates derived from 2D-US suffer from high inter-user variability even among experienced clinicians. Existing 3D ultrasound (3D-US) solutions use specialized probes or external tracking hardware, but such configurations increase costs and diminish portability, constraining widespread clinical use. To address these limitations, we present Mobile Augmented Reality Volumetric Ultrasound (MARVUS), a resource-efficient system designed to increase accessibility to accurate and reproducible volumetric assessment. MARVUS is interoperable with conventional ultrasound (US) systems, using a foundation model to enhance cross-specialty generalization while minimizing hardware requirements relative to current 3D-US solutions. In a user study involving experienced clinicians performing measurements on breast phantoms, MARVUS yielded a substantial improvement in volume estimation accuracy (mean difference: 0.469 cm3) with reduced inter-user variability (mean difference: 0.417 cm3). Additionally, we prove that augmented reality (AR) visualizations enhance objective performance metrics and clinician-reported usability. Collectively, our findings suggests that MARVUS can enhance US-based cancer screening, diagnostic workflows, and treatment planning in a scalable, cost-conscious, and resource-efficient manner. Usage video demonstration available (https://youtu.be/m4llYcZpqmM).

Figures & tables

 UltrasoundTrackingAugmentedSegmentation
Ref.ProbeSensorReality DeviceAlgorithm
[ 24 ]Standard (2D)Robotic Arm-Specialized
[ 32 ]Standard (2D)Depth Camera-Specialized
[ 25 ]Standard (2D)Stereo CameraHeadsetManual
[ 10 ]Specialized (3D)Stereo CameraHeadsetSpecialized
OursStandard (2D)PhonePhonePrompt-based
Table 1: System comparison for related 3D-US and Augmented Reality works
Figure 1: The MARVUS workflow. Calibration – US probe with strapped ArUco markers (a) is tracked via a mobile phone (b). Calibration is done via a simple phantom (c), with region-of-interest and keypoints detected from a raw screen capture (d). Reconstruction – A freehand sweep imaging a nodule is done with accompanying augmented reality visualizations (e). Nodule segmentation is obtained in a semi-automated manner (f) and compounded into 3D for volume estimation. Verification – The 3D mesh is verified in augmented reality (g) by comparing the mesh-plane intersection with a live ultrasound feed (green denotes intersection between the mesh and the live US image).
Figure 2: US Calibration Phantom . A single-piece 3D-printed phantom is used for single-frame spatial calibration using US-visible "ledges" (a) with intersections enabling calibration. The "ledges" (L1,L2,L3)(L_{1},L_{2},L_{3}) are brightest when viewed directly in-plane due to US physics. (b) Visible markers are used for extrinsics calibration (right).
Figure 3: Nodule Reconstruction Steps . (a) Segmentation of tracked US sequence produces a point cloud. (b) Voxelization resamples and fills sparsely sampled locations. (c) A meshing algorithm is applied to smoothen noise-induced highly irregular surfaces.
 Image AcquisitionVolume EstimationNodule Assessment
ControlManualEllipsoid FormulaN/A
ReconSweepAutomatedMesh
Recon+ARSweep (AR)AutomatedMesh (AR)
Table 2: User Study: Volume Assessment Methods
Figure 4: Nodule Volume Estimation . Our AR 3D-US setup improves accuracy and reduces operator variability (left) , notably for complex non-spherical nodules (right) .
TrackerPhantom TypeCR (mm)
Optotrak Certus (NDI) [ 12 ]Double-N Wire0.938±0.1180.938\pm 0.118
Vicra (NDI) [ 30 ]Arbitrary Wire0.896±0.0750.896\pm 0.075
Polaris Spectra (NDI) [ 8 ]Triple-N Wire0.690±0.3100.690\pm 0.310
Mobile Phone (Ours)3D-printed Ledge0.826±0.4470.826\pm 0.447
Table 3: US Calibration: Calibration Reproducibility (CR) against Related Works
More in this topic

Ultrasound Imaging

Ultrasound imaging uses sound waves to visualize anatomy and function, with AI improving image reconstruction, diagnosis, measurement, and scan guidance.

See all 150 papers

Explore similar work

CardsList
Mar 1, 2026

MLRecon: Robust Markerless Freehand 3D Ultrasound Reconstruction via Coarse-to-Fine Pose Estimation

Freehand 3D ultrasound (US) reconstruction promises volumetric imaging with the flexibility of standard 2D probes, yet existing tracking paradigms face a restrictive trilemma: marker-based systems demand prohibitive costs, inside-out methods require intrusive sensor attachment, and sensorless approaches suffer from severe cumulative drift. To overcome these limitations, we present MLRecon, a robust markerless 3D US reconstruction framework delivering drift-resilient 6D probe pose tracking using a single commodity RGB-D camera. Leveraging the generalization power of vision foundation models, our pipeline enables continuous markerless tracking of the probe, augmented by a vision-guided divergence detector that autonomously monitors tracking integrity and triggers failure recovery to ensure uninterrupted scanning. Crucially, we further propose a dual-stage pose refinement network that explicitly disentangles high-frequency jitter from low-frequency bias, effectively denoising the trajectory while maintaining the kinematic fidelity of operator maneuvers. Experiments demonstrate that MLRecon significantly outperforms competing sensorless and sensor-aided methods, achieving average position errors as low as 0.88 mm on complex trajectories and yielding high-quality 3D reconstructions with sub-millimeter mean surface accuracy. This establishes a new benchmark for low-cost, accessible volumetric US imaging in resource-limited clinical settings.

Yi Zhang, Puxun Tu, Kun Wang +3

Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China · Department of Ultrasound in Medicine, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China · Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

Ultrasound ImagingObject Pose Estimation
AskPDF
Aug 17, 2026

US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina

Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.

Cheng Zhang, Xingzheng Wu, Guihao Yan +4

Innovation School of Artificial Intelligence, Hefei University of Technology Hefei, China

Robotic UltrasoundVision-Language-Action ModelsUltrasound ImagingVLMs for Robotics
AskPDF
Jul 2, 2026

Towards Real-World Ultrasound Understanding: Large Vision-Language Models from Multi-Image Examinations with Long-Form Reports

Large vision-language models (LVLMs) have achieved strong performance across many medical imaging tasks, yet their application to ultrasound remains limited due to its inherent complexity and variability. In this work, we revisit what is truly needed to enable real-world ultrasound understanding. Instead of introducing complex architectures or elaborate training strategies, we show that data scale and clinically faithful data alignment are the key factors. We construct a large-scale dataset of 1.5M real-world ultrasound examinations, containing 17.7M images, multi-organ coverage, and paired uncurated clinical reports. Crucially, we organize the data at the examination level, aligning multiple images with their corresponding reports to reflect real clinical workflows. We then fine-tune a standard LVLM using low-rank adaptation (LoRA) on this dataset without task-specific modifications. Surprisingly, this simple recipe already leads to strong performance across diverse ultrasound understanding tasks, outperforming prior methods designed with more complex pipelines. Beyond these results, we present model and data scaling analyses that provide insights into the role of scale in ultrasound LVLMs.

Bingcong Yan, Chunlei Li, Jingliang Hu +3

1MedAI Technology (Wuxi) Co. Ltd. · 2Technical University of Munich

VLM AdaptationUltrasound ImagingMedical VLMs
AskPDF