Data-driven Synthesis of Magnetic Resonance Spectroscopy Data using a Variational Autoencoder
Authors: Dennis M. J. van de Sande, Julian P. Merkofer, Sina Amirrajab, Mitko Veta, Gerhard S. Drenthen, Jacobus F. A. Jansen, Marcel Breeuwer
Organizations: Department of Biomedical Engineering Eindhoven University of Technology Eindhoven, The Netherlands · Department of Electrical Engineering Eindhoven University of Technology Eindhoven, The Netherlands · Department of Radiology & Nuclear Medicine Maastricht University Medical Center Maastricht, The Netherlands
The development of deep learning methods for magnetic resonance spectroscopy (MRS) is often hindered by limited availability of large, high-quality training datasets. While physics-based simulations are commonly used to mitigate this limitation, accurately modeling all in-vivo signal components remains challenging. In this work, we propose a data-driven framework for synthesizing in-vivo MRS data using a variational autoencoder (VAE) trained exclusively on measured single-voxel spectroscopy data. The model learns a low-dimensional latent representation of complex-valued spectra and enables generation of new samples through latent-space sampling and interpolation. The generative performance of the proposed approach is evaluated using a comprehensive set of complementary analyses, including reconstruction quality, feature-level similarity using low-dimensional embeddings, application-based signal quality metrics, and metabolite quantification agreement. The results demonstrate that the VAE accurately reconstructs dominant spectral patterns and generates synthetic spectra that occupy the same feature space as in-vivo data. In an example application targeting GABA-edited spectroscopy, augmenting limited subsets of transients with synthetic spectra improves signal quality metrics such as signal-to-noise ratio, linewidth, and shape scores. However, the results also reveal limitations of the generative approach, including under-representation of stochastic noise and reduced accuracy in absolute metabolite quantification, particularly for applications sensitive to concentration estimates. These findings highlight both potential and limitations of data-driven MRS synthesis. Beyond the proposed model, this study introduces a structured evaluation framework for generative MRS methods, emphasizing the importance of application-aware validation when synthetic data are used for downstream analysis.
Figures & tables
Figure 1: Overview of the used model architecture. Real and imaginary parts of the spectrum are concatenated and used as input for a fully connected nn
Figure 2: Overview of the data generation pipeline. Real, in vivo spectra are first encoded into latent space. New samples are then generated using one of three methods: random sampling, interpolation, or hybrid sampling. Finally, all generated samples are decoded back into spectra.
Figure 3: Overview of the application-based evaluation method. gt spectra from a single subject are used to select a subset of n transients. These transients are encoded using the encoder nn and used for the generation of synthetic samples. The decoder nn generates synthetic transients to end up with the same number of transients as in the gt spectra. For the evaluation all three datasets are processed and fitted using Osprey.
Figure 4: Representative in-vivo spectra (blue) and corresponding reconstructions (orange) with residuals (black) for two subjects. Top panels show OFF and ON spectra with small, flat residuals, indicating accurate reconstruction. Bottom panels show another subject with slightly larger residuals, primarily around the water signal, while the overall spectral shapes are preserved.
Figure 5: umap visualizations comparing gt in-vivo spectra with synthetic spectra generated using three different data generation methods: random sampling (RS) (green), interpolation (red), and hybrid sampling (blue). The umap embedding is learned using the gt data, after which synthetic spectra generated from a single subset of transients are projected into the same low-dimensional space. Each point represents an individual spectrum, with markers indicating ON and OFF acquisitions. The shaded regions represent kernel density estimates of the embedded spectral distributions for both gt and synthetic data, illustrating the overall spatial extent and degree of overlap between datasets. Clusters corresponding to individual subjects are visible across all panels.
Figure 6: Comparison of OFF, ON, and difference (OFF–ON) spectra across different data generation methods for two representative subjects. The top row (a-c) shows spectra from subject C008 (subset transient #6), and the bottom row (d-f) shows spectra from subject C039 (subset transient #6). For each subject, OFF spectra, ON spectra, and the corresponding difference spectra are shown from left to right.
Figure 7: Overview of quantitative metrics for the selected windows (gray) and the different data generation methods: random sampling (green), interpolation (red), and hybrid generation (blue). Metrics are computed from the difference spectra. For the snr and linewidth metrics, gt values are shown as reference. Statistically significant differences are indicated (*** p<0.001 , ** p<0.01 , * p<0.05 ). Boxplots with fewer than the nominal 16×10=160 datapoints are explicitly marked.
Figure 8: Heatmaps showing the relative concentration error with respect to the gt test spectra for the OFF spectra. For each subject, the relative error is shown per window for the different reconstruction methods. The adjacent column summarizes the mare across all windows for each subject. Values printed in bold indicate the method yielding the lowest mare for that subject. The color scale is shared across methods. The ▲ symbols indicate values that exceed the displayed range.
Figure 9: Heatmaps showing the relative concentration error with respect to the gt test spectra for the difference spectra. For each subject, the relative error is shown per subset of transients for the different reconstruction methods. The adjacent column summarizes the mare across all subsets for each subject. Values printed in bold indicate the method yielding the lowest mare for that subject. The color scale is shared across methods. The ▲ symbols indicate values that exceed the displayed range.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Header / SubHeader
Values
Site (name or number)
1. Hardware
a. Field strength
3T
b. Manufacturer
Philips
c. Model (software version if available)
Achieva TX
d. RF coils: nuclei (transmit/receive), number of channels, type, body part
1H, 32-element, head coil
Appendix
Table 1: Minimum Reporting Standards for in-vivo Magnetic Resonance Spectroscopy (MRSinMRS) [ 22 ] for used dataset. Details are retrieved from the data and the work from van Bussel et al. [ 17 ]
Magnetic resonance spectroscopy (MRS) is used to quantify metabolites in vivo and estimate biomarkers for conditions ranging from neurological disorders to cancers. Quantifying low-concentration metabolites such as GABA (γ-aminobutyric acid) is challenging due to low signal-to-noise ratio (SNR) and spectral overlap. We investigate and validate deep learning for quantifying complex, low-SNR, overlapping signals from MEGA-PRESS spectra, devise a convolutional neural network (CNN) and a Y-shaped autoencoder (YAE), and select the best models via Bayesian optimisation on 10,000 simulated spectra from slice-profile-aware MEGA-PRESS simulations. The selected models are trained on 100,000 simulated spectra. We validate their performance on 144 spectra from 112 experimental phantoms containing five metabolites of interest (GABA, Glu, Gln, NAA, Cr) with known ground truth concentrations across solution and gel series acquired at 3 T under varied bandwidths and implementations. These models are further assessed against the widely used LCModel quantification tool. On simulations, both models achieve near-perfect agreement (small MAEs; regression slopes ≈1.00, R2≈1.00). On experimental phantom data, errors initially increased substantially. However, modelling variable linewidths in the training data significantly reduced this gap. The best augmented deep learning models achieved a mean MAE for GABA over all phantom spectra of 0.151 (YAE) and 0.160 (FCNN) in max-normalised relative concentrations, outperforming the conventional baseline LCModel (0.220). A sim-to-real gap remains, but physics-informed data augmentation substantially reduced it. Phantom ground truth is needed to judge whether a method will perform reliably on real data.
Zien Ma, S. M. Shermer, Oktay Karakuş +1
School of Computer Science and Informatics, Cardiff University, Cardiff, CF24 4AG, United Kingdom · Faculty of Science and Engineering, Swansea University, Swansea, SA2 8PP, United Kingdom
Magnetic resonance spectroscopy (MRS) is a non-invasive technique to measure the metabolic composition of tissues, offering valuable insights into neurological disorders, tumor detection, and other metabolic dysfunctions. However, accurate metabolite quantification is hindered by challenges such as spectral overlap, low signal-to-noise ratio, and various artifacts. Traditional methods like linear-combination modeling are susceptible to ambiguities and commonly only provide a theoretical lower bound on estimation accuracy in the form of the Cramér-Rao bound. This work introduces a Bayesian inference framework using Sylvester normalizing flows (SNFs) to approximate posterior distributions over metabolite concentrations, enhancing quantification reliability. A physics-based decoder incorporates prior knowledge of MRS signal formation, ensuring realistic distribution representations. We validate the method on simulated 7T proton MRS data, demonstrating accurate metabolite quantification, well-calibrated uncertainties, and insights into parameter correlations and multi-modal distributions.
Julian P. Merkofer, Dennis M. J. van de Sande, Alex A. Bhogal +1
Eindhoven University of Technology, Eindhoven, The Netherlands · University Medical Center Utrecht, Utrecht, The Netherlands
Objective. Standard Magnetic Resonance Imaging (MRI) reconstruction pipelines discard phase information captured during acquisition, despite evidence that it encodes tissue properties relevant to tumor diagnosis. Current machine learning approaches inherit this limitation by operating exclusively on reconstructed magnitude images. The aim of this study is to build a generative framework which is capable of jointly modeling magnitude and phase information of complex-valued MRI scans. Approach. The proposed generative framework combines a conditional variational autoencoder, which compresses complex-valued MRI scans into compact latent representations while preserving phase coherence, with a flow-matching-based generative model. Synthetic sample quality is assessed via a real-versus-synthetic classifier and by training downstream classifiers on synthetic data for abnormal tissue detection. Main results. The autoencoder preserves phase coherence above 0.997. Real-versus-synthetic classification yields low AUROC values between 0.50 and 0.66 across all acquisition sequences, indicating generated samples are nearly indistinguishable from real data. In downstream normal-versus-abnormal classification, classifiers trained entirely on synthetic data achieve an AUROC of 0.880, surpassing the real-data baseline of 0.842 on a publicly available dataset (fastMRI). This advantage persists on an independent external test set from a different institution with biopsy-confirmed labels. Significance. The proposed framework demonstrates the feasibility of jointly modeling magnitude and phase information for normal and abnormal complex-valued brain MRI data. Beyond synthetic data generation, it establishes a foundation for the usage of complete brain MRI information in future diagnostic applications and enables systematic investigation of how magnitude and phase jointly encode pathology-specific features.
Marco Schlimbach, Moritz Rempe, Jessica Mnischek +4
Department of Physics, Technical University Dortmund, Otto-Hahn-Straße 4a, 44227 Dortmund, Germany · Institute for AI in Medicine (IKIM), University Hospital Essen, Girardetstraße 2, 45131 Essen, Germany · 3Cancer Research Center Cologne Essen (CCCE), University Medicine Essen, Hufelandstraße 55, 45147 Essen, Germany +4