PA-SFM: Tracker-free differentiable acoustic radiation for freehand 3D photoacoustic imaging
Authors: Shuang Li, Jian Gao, Chulhong Kim, Seongwook Choi, Qian Chen, Yibing Wang, Shuang Wu, Yu Zhang, +5 more
Organizations: Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing, China. · School of Intelligence Science and Technology, Nanjing University, Suzhou, China. · Department of Electrical Engineering, Convergence IT Engineering, Mechanical Engineering, and Medical Science and Engineering, Medical Device Innovation Center, Pohang University of Science and Technology, Pohang, Republic of Korea. · National Biomedical Imaging Center, Peking University, Beijing, China.
Three-dimensional (3D) handheld photoacoustic tomography typically relies on bulky and expensive external positioning sensors to correct motion artifacts, which severely limits its clinical flexibility and accessibility. To address this challenge, we present PA-SFM, a tracker-free framework that leverages exclusively single-modality photoacoustic data for both sensor pose recovery and high-fidelity 3D reconstruction via differentiable acoustic radiation modeling. Unlike traditional structure-from-motion (SFM) methods based on visual features, PA-SFM integrates the acoustic wave equation into a differentiable programming pipeline. By leveraging a high-performance, GPU-accelerated acoustic radiation kernel, the framework simultaneously optimizes the 3D photoacoustic source distribution and the sensor array pose via gradient descent. To ensure robust convergence in freehand scenarios, we introduce a coarse-to-fine optimization strategy that incorporates geometric consistency checks and rigid-body constraints to eliminate motion outliers. We validated the proposed method through both numerical simulations and in-vivo rat experiments. The results demonstrate that PA-SFM achieves sub-millimeter positioning accuracy and restores high-resolution 3D vascular structures comparable to ground-truth benchmarks, offering a low-cost, software-defined solution for clinical freehand photoacoustic imaging. The source code is publicly available at https://github.com/JaegerCQ/PA-SFM.
Figures & tables
Figure 1: The overview framework of PA-SFM algorithm for freehand multi-view 3D PAI. (a) The freehand multi-view 3D PAI. (b) The PA-SFM pipeline. (c) The differentiable single probe localization of PA-SFM. (d) The global differentiable pose fine-tuning of PA-SFM.
Figure 2: The reconstruction results of multi-view simulation data with PA-SFM algorithm. (a) The single view reconstruction, multi-view coarse pose reconstruction, multi-view fine-tuned pose reconstruction and ground truth of the phantom liver (Scale: 2 mm). (b) The 3D results of single view, coarse multi-view and fine multi-view reconstruction. (c) The multi-view setup of the simulation experiment.
Figure 3: The PA signal of the target sensor at different stages of the PA-SFM algorithm. (a) Differentiable single sensor location optimization. (b) The PA signal of the target sensor at the initial location. (c) The distance error curve of the single sensor location optimization. (d) The PA signal of the target sensor at the final location. (e) Global differentiable pose fine-tuning of the new pose. (f) (b) The PA signal of the target sensor before and after the global differentiable pose fine-tuning.
Figure 4: The reconstruction results of multi-view in-vivo data with PA-SFM algorithm. (a) The freehand multi-view 3D PAI for in-vivo rat kidney imaging. (b) The rat kidney imaging setup. (c) The single view reconstruction, multi-view coarse pose reconstruction, multi-view fine-tuned pose reconstruction and ground truth of the in-vivo rat kidney (Scale: 2 mm). (d) The 3D results of single view, coarse multi-view and fine multi-view reconstruction.
MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China. · National Biomedical Imaging Center, College of Future Technology, Peking University, Beijing, China. · Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China. +2
Department of Mathematics, University of Innsbruck Technikerstrasse 13, 6020 Innsbruck, Austria · Department of Computer Science, University of Innsbruck Technikerstrasse 21a, 6020 Innsbruck, Austria · Department of Mathematics, Yeungnam University 280 Daehak-Ro, Gyeongsan, Gyeongbuk 38541, South Korea