PA-SFM: Tracker-free differentiable acoustic radiation for freehand 3D photoacoustic imaging
Authors: Shuang Li, Jian Gao, Chulhong Kim, Seongwook Choi, Qian Chen, Yibing Wang, Shuang Wu, Yu Zhang, +5 more
Organizations: Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing, China. · School of Intelligence Science and Technology, Nanjing University, Suzhou, China. · Department of Electrical Engineering, Convergence IT Engineering, Mechanical Engineering, and Medical Science and Engineering, Medical Device Innovation Center, Pohang University of Science and Technology, Pohang, Republic of Korea. · National Biomedical Imaging Center, Peking University, Beijing, China.
Three-dimensional (3D) handheld photoacoustic tomography typically relies on bulky and expensive external positioning sensors to correct motion artifacts, which severely limits its clinical flexibility and accessibility. To address this challenge, we present PA-SFM, a tracker-free framework that leverages exclusively single-modality photoacoustic data for both sensor pose recovery and high-fidelity 3D reconstruction via differentiable acoustic radiation modeling. Unlike traditional structure-from-motion (SFM) methods based on visual features, PA-SFM integrates the acoustic wave equation into a differentiable programming pipeline. By leveraging a high-performance, GPU-accelerated acoustic radiation kernel, the framework simultaneously optimizes the 3D photoacoustic source distribution and the sensor array pose via gradient descent. To ensure robust convergence in freehand scenarios, we introduce a coarse-to-fine optimization strategy that incorporates geometric consistency checks and rigid-body constraints to eliminate motion outliers. We validated the proposed method through both numerical simulations and in-vivo rat experiments. The results demonstrate that PA-SFM achieves sub-millimeter positioning accuracy and restores high-resolution 3D vascular structures comparable to ground-truth benchmarks, offering a low-cost, software-defined solution for clinical freehand photoacoustic imaging. The source code is publicly available at https://github.com/JaegerCQ/PA-SFM.
Figures & tables
Figure 1: The overview framework of PA-SFM algorithm for freehand multi-view 3D PAI. (a) The freehand multi-view 3D PAI. (b) The PA-SFM pipeline. (c) The differentiable single probe localization of PA-SFM. (d) The global differentiable pose fine-tuning of PA-SFM.
Figure 2: The reconstruction results of multi-view simulation data with PA-SFM algorithm. (a) The single view reconstruction, multi-view coarse pose reconstruction, multi-view fine-tuned pose reconstruction and ground truth of the phantom liver (Scale: 2 mm). (b) The 3D results of single view, coarse multi-view and fine multi-view reconstruction. (c) The multi-view setup of the simulation experiment.
Figure 3: The PA signal of the target sensor at different stages of the PA-SFM algorithm. (a) Differentiable single sensor location optimization. (b) The PA signal of the target sensor at the initial location. (c) The distance error curve of the single sensor location optimization. (d) The PA signal of the target sensor at the final location. (e) Global differentiable pose fine-tuning of the new pose. (f) (b) The PA signal of the target sensor before and after the global differentiable pose fine-tuning.
Figure 4: The reconstruction results of multi-view in-vivo data with PA-SFM algorithm. (a) The freehand multi-view 3D PAI for in-vivo rat kidney imaging. (b) The rat kidney imaging setup. (c) The single view reconstruction, multi-view coarse pose reconstruction, multi-view fine-tuned pose reconstruction and ground truth of the in-vivo rat kidney (Scale: 2 mm). (d) The 3D results of single view, coarse multi-view and fine multi-view reconstruction.
Photoacoustic computed tomography (PACT) combines optical absorption contrast with acoustic detection for high-resolution deep-tissue imaging. A persistent challenge is that unknown speed-of-sound (SoS) heterogeneity changes acoustic time-of-flight, causing defocusing artifacts when reconstruction assumes a uniform SoS. Existing SoS-adaptive methods either rely on calibrated acoustic priors or optimize dense physical medium models, which becomes expensive and difficult to scale in 3D. We propose PAGS, a differentiable framework for blind autofocusing PACT via speed-of-sound-adaptive Gaussian splatting. PAGS represents the initial pressure field with sparse Gaussian photoacoustic (PA) sources and replaces explicit medium recovery with a compact anisotropic path-averaged SoS (ASoS) field parameterized by spherical harmonic probes. This latent propagation field directly controls source-to-transducer arrival-time alignment, while an analytic Gaussian acoustic projection maps the source representation to transducer signals efficiently. The resulting closed-loop signal-domain optimization jointly updates the Gaussian PA source parameters and the ASoS field from measured data, without calibrated SoS priors. Experiments on simulated and physical phantom data demonstrate improved reconstruction sharpness under heterogeneous acoustic media, robustness to sparse-view sampling, and computational benefits from the analytic Gaussian projection.
Jiarui Ge, Jintao Ma, Bangxu Fan +4
MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China. · National Biomedical Imaging Center, College of Future Technology, Peking University, Beijing, China. · Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China. +2
Photoacoustic tomography (PAT) is an emerging imaging modality that combines the complementary strengths of optical contrast and ultrasonic resolution. A central task is image reconstruction, where measured acoustic signals are used to recover the initial pressure distribution. For ideal point-like or line-like detectors, several efficient and fast reconstruction algorithms exist, including Fourier methods, filtered backprojection, and time reversal. However, when applied to data acquired with finite-size detectors, these methods yield systematically blurred images. Although sharper images can be obtained by compensating for finite-detector effects, supervised learning approaches typically require ground-truth images that may not be available in practice. We propose a self-supervised reconstruction method based on Noisier2Inverse that addresses finite-size detector effects without requiring ground-truth data. Our approach operates directly on noisy measurements and learns to recover high-quality PAT images in a ground-truth-free manner. Its key components are: (i) PAT-specific modeling that recasts the problem as angular deblurring; (ii) a Noisier2Inverse formulation in the polar domain that leverages the known angular point-spread function; and (iii) a novel, statistically grounded early-stopping rule. In experiments, the proposed method consistently outperforms alternative approaches that do not use supervised data and achieves performance close to supervised benchmarks, while remaining practical for real acquisitions with finite-size detectors.
Markus Haltmeier, Nadja Gruber, Gyeongha Hwang
Department of Mathematics, University of Innsbruck Technikerstrasse 13, 6020 Innsbruck, Austria · Department of Computer Science, University of Innsbruck Technikerstrasse 21a, 6020 Innsbruck, Austria · Department of Mathematics, Yeungnam University 280 Daehak-Ro, Gyeongsan, Gyeongbuk 38541, South Korea
Photoacoustic tomography (PAT) combines the optical absorption contrast of biological tissue with the spatial resolution of ultrasound, yet recovering the initial pressure distribution from sparse-view sensor measurements remains an ill-posed inverse problem. Iterative compressive-sensing solvers and unrolled deep networks both retain a dependence on the system matrix at inference, which leaves real-time clinical reconstruction computationally expensive. This paper proposes the Sensor Attention Network (SAN), a Transformer-based architecture that treats the full time series of each sensor as a token and maps raw measurements directly to the reconstructed image without invoking the system matrix at inference. For training and benchmarking, an analytical k-space H-matrix is constructed and validated against the k-Wave pseudo-spectral solver under matched geometry, achieving a mean per-sensor Pearson correlation of 0.919 +/- 0.049, with k-space apodization and Gaussian temporal damping acting synergistically to reduce the energy-normalized mismatch by 49%. Trained with a vessel-weighted loss on 488 augmented samples and evaluated on 46 held-out samples against ISTA, split-Bregman total variation (SBTV), and learned ISTA (LISTA), SAN attains the highest mean SSIM (0.522) and PSNR (22.09 dB) and the lowest NMSE (0.233). Paired t-tests and Wilcoxon signed-rank tests confirm the superiority of SAN over LISTA on PSNR, NMSE, and Pearson correlation at p < 1e-8, and over ISTA and SBTV on all fidelity metrics. By bypassing the H-matrix at inference, SAN reduces reconstruction time by at least an order of magnitude, supporting real-time PAT reconstruction.
Mary John, Shibili Said, Imad Barhumi +2
Abu Dhabi Polytechnic, Abu Dhabi, United Arab Emirates · United Arab Emirates University, Al Ain, United Arab Emirates