Optical Inversion and Spectral Unmixing of Spectroscopic Photoacoustic Images with Physics-Informed Neural Networks
Authors: Sarkis Ter Martirosyan, Xinyue Huang, David Qin, Anthony Yu, Stanislav Emelianov
Organizations: School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States · Institute of Biological and Medical Imaging, Helmholtz Zentrum Munich, 85764 Neuherberg, Germany · Chair of Biological Imaging, Technische Universität München, 80333 München, Germany · The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University School of Medicine, Atlanta, GA 30332, United States
Accurate estimation of the relative concentrations of chromophores in a spectroscopic photoacoustic (sPA) image can reveal immense structural, functional, and molecular information about physiological processes. However, due to nonlinearities and ill-posedness inherent to sPA imaging, concentration estimation is intractable. The Spectroscopic Photoacoustic Optical Inversion Autoencoder (SPOI-AE) aims to address the sPA optical inversion and spectral unmixing problems without assuming linearity. Herein, SPOI-AE was trained and tested on in vivo mouse lymph node sPA images with unknown ground truth chromophore concentrations. SPOI-AE better reconstructs input sPA pixels than conventional algorithms while providing biologically coherent estimates for optical parameters, chromophore concentrations, and the percent oxygen saturation of tissue. SPOI-AE's unmixing accuracy was validated using a simulated mouse lymph node phantom ground truth.
Figures & tables
Fig. 1: The complete architecture of SPOI-AE. For each sPA pixel numbered i , the inputs are pi and ∣ri∣ , the input pixel and the pixel’s distance from the origin, respectively. The reconstructed output pixel is denoted with pi . The FCNNs μs′ -Net and μa -Net generate estimates for the absorption coefficient μa,i and the reduced scattering coefficient μs,i′ . The spectral unmixing block produces chromophore concentration estimates ci from μa,i . A low-rank reconstruction of μa,i is recreated from the chromophore concentration estimates denoted as μa,i . Finally, the decoding stage implementing the photoacoustic optical forward problem described in section II-A , reconstructs pi from μa and μs′ .
Fig. 2: Pixel intensity histograms for the training dataset (TR) and testing dataset (TS) . Pixel counts are binned as a function of normalized intensity of the photoacoustic signal across all imaging wavelengths used. In other words, each point represents a single p(ri,λl) from either the training set or testing set.
Fig. 3: Example highlighting segmentation of a sPA image using Chan-Vese active contours. (US) Ultrasound image of a mouse lymph node. (Hand) Initial hand-drawn segmentation of the associated sPA image before the Chan-Vese active contour method was applied. (C-V) Evolved segmentation of the sPA image using the Chan-Vese active contour method initiated from the (Hand) segmentation. Note the exclusion of the background pixels and the inclusion of more of the biologically relevant foreground.
Fig. 4: Ground-truth phantom oxygenated hemoglobin and deoxygenated hemoglobin concentration images ( (HbO2) and (HHb) , respectively), with oxygenation of each inclusion annotated. Blood inclusions are embedded one centimeter deep into the simulation volume. Blood concentrations are ultimately mapped to initial pressure images, and two example wavelengths ( λ =760nm and λ =850nm) are plotted in P . Scalebars represent five millimeters.
Algorithm
MSE
MSAD
Lit. NLS
0.0270
0.194
NMF
0.0259
0.198
SPOI-AE Variants
Adjusted E
β=5
0.0114
0.096
Adjusted E
β=0
0.0124
0.116
Fixed E
β=5
0.0126
0.100
TABLE I: Testing Set Performance
Fig. 5: R2 results for the training set (TR) and the testing set (TS) as a function of imaging wavelength in nm.
Algorithm
E[R2]
Var[R2]
Lit. NLS
0.741
0.1559
NMF
0.750
0.156
SPOI-AE Variants
Adjusted E
β=5
0.897
0.0331
Adjusted E
β=0
0.889
0.0443
Fixed E
β=5
0.887
0.0412
TABLE II: Testing Set Wavelength Dependent GoF
Fig. 6: Average sPA pixel spectrum reconstruction versus average input sPA spectrum. SPOI-AE (red) better reconstructs the input than NMF (blue) and Lit. NLS (green) for the average training pixel (TR) . This performance translates to the average testing pixel (TS) . For both plots, normalized spectra is drawn as a function of imaging wavelength in nm.
Fig. 7: Optical parameters for the in vivo mouse lymph node image introduced in Fig. 3 at imaging wavelengths λ =680 nm, λ =760 nm, and λ =970 nm. (P) Input photoacoustic images captured at the indicated laser wavelength in normalized magnitude. ( μa ) Reconstructed absorption coefficient image estimated using SPOI-AE for the indicated laser wavelength estimated using ( 13 ). ( μs′ ) Reduced scattering coefficient image estimated using SPOI-AE at the indicated laser wavelength found using ( 11 ). Scalebars indicate five millimeters.
Fig. 8: Relative absorption spectra for oxygenated hemoglobin (HbO2) and deoxygenated hemoglobin (HHb) as a function of imaging wavelength in nm.
Fig. 9: Relative HbO2 and HHb concentration maps as well as SO2 percentage maps calculated using (A) SPOI-AE with adjusted absorption spectra, (B) SPOI-AE with fixed absorption spectra from literature, (C) NMF, and (D) Lit. NLS. Scalebars indicate five millimeters.
Fig. 10: SO2 distributions for the training (TR) and testing (TS) datasets as well as for each unmixing method investigated herein, namely: (A) SPOI-AE with adjusted absorption spectra, (B) SPOI-AE with fixed absorption spectra from literature, (C) NMF, and (D) NLS with literature spectra. The SO2 values provided by NMF and Lit. NLS are consistently lower than the healthy range.
Fig. 11: Relative HbO2 and HHb concentration maps and SO2 percentage maps of the ground truth simulation phantom (GT) calculated using (A) SPOI-AE with adjusted absorption spectra, (B) SPOI-AE with fixed absorption spectra from literature, (C) NMF, and (D) Lit. NLS. Scalebars indicate five millimeters.