Organizations: Chair for Computer-Aided Medical Procedures and Augmented Reality, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany · The University of Hong Kong, Hong Kong, China
Intraoperative Cone Beam Computed Tomography (CBCT) provides a reliable 3D anatomical context essential for interventional planning. However, its static nature fails to provide continuous monitoring of soft-tissue deformations induced by respiration, probe pressure, and surgical manipulation, leading to navigation discrepancies. We propose a deformation-aware CBCT updating framework that leverages robotic ultrasound as a dynamic proxy to infer tissue motion and update static CBCT slices in real time. Starting from calibration-initialized alignment with linear correlation of linear combination (LC2)-based rigid refinement, our method establishes accurate multimodal correspondence. To capture intraoperative dynamics, we introduce the ultrasound correlation UNet (USCorUNet), a lightweight network trained with optical flow-guided supervision to learn deformation-aware correlation representations, enabling accurate, real-time dense deformation field estimation from ultrasound streams. The inferred deformation is spatially regularized and transferred to the CBCT reference to produce deformation-consistent visualizations without repeated radiation exposure. We validate the proposed approach through deformation estimation and ultrasound-guided CBCT updating experiments. Results demonstrate real-time end-to-end CBCT slice updating and physically plausible deformation estimation, enabling dynamic refinement of static CBCT guidance during robotic ultrasound-assisted interventions. The source code is publicly available at https://github.com/anonymous-codebase/us-cbct-demo.
Figures & tables
Figure 1: System overview. (a) Rigid calibration between robotic ultrasound and CBCT. (b) Image-based registration refinement. (c) Deformation estimation with USCorUNet. (d) Deformation transfer for CBCT slice updating. Ultrasound examples from the in vivo arm dataset (c) and the CT-mapped phantom dataset (b,d) illustrate cross-domain applicability. Conf. map denotes confidence map.
Figure 2: Architecture of USCorUNet.
Figure 3: Bidirectional deformation estimation results. (a,b) In vivo arm examples (Dataset A); (c,d) probe-induced motion (Datasets B and D); (e) externally induced motion (Dataset B). Orange/yellow dashed lines indicate visual alignment guides for I0 , I1 . The color bar indicates flow direction (x/y).
Method
MAE ↓
NCC ↑
FB ↓
Fold(% ) ↓
Dice(% ) ↑
Baseline
RAFT
0.05±0.02
0.85±0.10
1.81±1.22
0.24±0.14
90.31±4.14
Ours
USCorUNet
0.05±0.02
0.85±0.09
0.85±0.57
0.13±0.10
90.62±3.76
Ablation Study
w/o Corr.
0.08±0.03
0.61±0.18
1.32±3.14
0.56±0.51
89.10±3.33
Table 1: Testset performance of the base model on Datasets A, averaged over both deformation directions ( I0→I1 , I1→I0 ) and reported as mean ± standard deviation. ↓/↑ indicate lower/higher is better. Best results are in bold.
Method
1 N
2 N
3 N
4 N
5 N
6 N
DefCor-Net
95.9±3.3
92.4±4.8
91.1±7.5
87.8±7.1
87.8±12.2
82.6±12.1
USCorUNet
95.4±2.6
94.1±2.4
94.0±3.6
93.2±1.8
89.8±1.5
87.5±1.2
Table 2: Bone-mask Dice under the force-stratified protocol of DefCor-Net [ 9 ] on Dataset A. Dice is reported for the I1→I0 direction.
Motion
Method
MAE ↓
NCC ↑
FB ↓
Fold(% ) ↓
Probe- induced
RAFT
0.03±0.01
0.94±0.03
1.03±2.27
0.23±0.14
Base model
0.04±0.01
0.84±0.08
0.84±0.65
0.15±0.11
Probe-adapted
0.03±0.01
0.93±0.03
0.33±0.28
0.09±0.10
Externally induced
RAFT
0.03±0.01
0.93±0.05
0.91±1.35
0.14±0.13
Base model
0.05±0.02
0.71±0.17
1.28±1.95
0.45±0.55
External-adapted
0.03±0.01
0.92±0.05
0.23±0.24
0.07±0.07
Table 3: Test-set performance of fine-tuned models on Datasets B–D (probe-induced) and Dataset B (externally induced), averaged over both directions.
Figure 4: CT1′ update results on two representative abdominal phantom cases (Dataset D) using USCorUNet, RAFT, and LC2-FFD ( CT0 : source; CT1 : target). Red/orange boxes indicate structural artifacts/severe deformation regions.
Model
MAE ↓
SSIM ↑
Dice (%) ↑
Timing (ms) ↓
USCorUNet
0.33 ± 0.04
0.22 ± 0.02
82.22 ± 10.54
11.25 ± 0.44
RAFT
0.34 ± 0.04
0.21 ± 0.03
79.86 ± 12.31
56.24 ± 1.78
LC2-FFD
0.33 ± 0.05
0.22 ± 0.03
58.91 ± 8.77
5764.26 ± 510.63
Table 4: Quantitative results for CBCT updating on Dataset D using ultrasound-based deformation estimation, with all three methods evaluated under identical conditions and runtime reported end-to-end.
Slice-to-volume registration between ultrasound (US) and preoperative computed tomography (CT) imaging would enhance many minimally invasive interventions, for example by locating soft tissue structures intra-operatively that are discernible in CT. While optical tracking enables initial rigid registration, contact from the probe induces soft tissue deformations that inhibit accurate alignment. In this work, we introduce a deformable CT-ultrasound registration framework that incorporates anatomical priors derived from CT to improve registration under deformation. Rigid registration is first established using a robot-assisted optical tracking system, after which a deformable transformation is estimated using a sinusoidal implicit neural representation (SIREN) optimized per frame. Tissue stiffness is approximated from CT-based HU values and used as spatially varying regularization, suppressing deformation in rigid structures such as bone while allowing more flexibility in soft tissue. Two additional constraints capture the physics of probe contact: a contact-zone displacement prior that drives the displacement field to compress tissue below the probe face, and a fan-geometry regularization term based on beam direction and convex transducer field of view. Model parameters are optimized with a normalized gradient field (NGF). The proposed approach improves alignment over rigid initialisation by 17% and outperforms classical deformable baselines while maintaining near-zero topological folding.
Agnieszka Lach, Magdalena Wysocki, Feng Li +7
medPhoton GmbH, Salzburg, Austria · Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany
Robotic ultrasound commonly targets standardized views, predefined scanning protocols, or expert-scan reproduction. We target a different role: while a clinician performs the primary procedure, a robotic assistant maintains a clinician-selected view so that changing tissue remains observable. We present contact-adaptive robotic ultrasound monitoring derived from robot-free demonstrations. Sonologger clamps a six-axis inertial sensor to the clinician-operated probe and records synchronized ultrasound and probe-frame relative rotations without a robot, camera, or tracker. Rather than imitating trajectories, the recordings measure which probe axis and sign the clinician uses to correct an image offset. A measured image latency is removed before still-move-still segmentation yields relative-rotation labels; translation is excluded from control because of inertial drift. These measurements parameterize the Demonstration-Informed Image-Motion Controller (DIMC), an interpretable, non-learned controller combined with beam-axis force regulation and an independent safety supervisor with 5 N forced retreat. On a dynamic bladder phantom, DIMC acquired and held a task-relevant view in 7 of 7 blind, pose-paired trials versus 1 of 7 for a static hold (McNemar p = 0.031), and retained the view during syringe-driven hydro-distension while the supervisor bounded the force excursion. The framework offers an operational representation of ultrasound context for contact-adaptive intraoperative monitoring; clinical validation remains future work.
Seong Jeong, Minsung Kim, Dongho Yee +8
Department of Transdisciplinary Medicine, Seoul National University Hospital · Rosota Inc., Seoul, Republic of Korea · Department of Medicine, Seoul National University College of Medicine +6
Computed tomography (CT) is indispensable for clinical diagnosis and image-guided interventions but exposes patients to ionizing radiation, motivating the development of safer imaging alternatives. Ultrasound (US) is non-ionizing and widely accessible; however, it is highly operator dependent and lacks quantitative tissue characterization, often leading to diagnostic uncertainty and unnecessary CT examinations. This work presents a 3D ultrasound-derived pseudo-CT (UD-pCT) framework that generates CT-like anatomical reference volumes inferred from US, without aiming to reproduce physically accurate Hounsfield Units. Paired 3D kidney US and CT volumes from the TRUSTED dataset are first spatially aligned using a landmark-based multimodal registration pipeline, creating high-quality paired inputs for supervised training of an adversarial framework. The proposed Bottleneck Transformer Residual U-Net3D (BT-ResUNet3D) model employs a 3D residual encoder-decoder generator augmented with a transformer bottleneck, enabling effective modeling of fine-grained local anatomical structures as well as long-range volumetric dependencies, while a 3D Conditional PatchGAN discriminator enforces local structural realism in the synthesized pseudo-CT volumes. Quantitative evaluation using PSNR and SSIM demonstrates that the proposed method outperforms established baselines in structural fidelity and perceptual image quality. The UD-pCT volumes provide real-time anatomical reference for operator guidance, potentially reducing acquisition variability and unnecessary CT use. A limitation of this study is the relatively small paired dataset, which may limit the generalizability of the proposed model.
Sapna Sachan, Amulya Kumar Mahto
Indian Institute of Technology Guwahati, Assam, PIN 781039 India