Data-driven Synthesis of Magnetic Resonance Spectroscopy Data using a Variational Autoencoder

Dennis M. J. van de Sande; Julian P. Merkofer; Sina Amirrajab; Mitko Veta; Gerhard S. Drenthen; Jacobus F. A. Jansen; Marcel Breeuwer

physics.med-phFeb 28, 2026

Data-driven Synthesis of Magnetic Resonance Spectroscopy Data using a Variational Autoencoder

Authors: Dennis M. J. van de Sande, Julian P. Merkofer, Sina Amirrajab, Mitko Veta, Gerhard S. Drenthen, Jacobus F. A. Jansen, Marcel Breeuwer

Organizations: Department of Biomedical Engineering Eindhoven University of Technology Eindhoven, The Netherlands · Department of Electrical Engineering Eindhoven University of Technology Eindhoven, The Netherlands · Department of Radiology & Nuclear Medicine Maastricht University Medical Center Maastricht, The Netherlands

Abstract

The development of deep learning methods for magnetic resonance spectroscopy (MRS) is often hindered by limited availability of large, high-quality training datasets. While physics-based simulations are commonly used to mitigate this limitation, accurately modeling all in-vivo signal components remains challenging. In this work, we propose a data-driven framework for synthesizing in-vivo MRS data using a variational autoencoder (VAE) trained exclusively on measured single-voxel spectroscopy data. The model learns a low-dimensional latent representation of complex-valued spectra and enables generation of new samples through latent-space sampling and interpolation. The generative performance of the proposed approach is evaluated using a comprehensive set of complementary analyses, including reconstruction quality, feature-level similarity using low-dimensional embeddings, application-based signal quality metrics, and metabolite quantification agreement. The results demonstrate that the VAE accurately reconstructs dominant spectral patterns and generates synthetic spectra that occupy the same feature space as in-vivo data. In an example application targeting GABA-edited spectroscopy, augmenting limited subsets of transients with synthetic spectra improves signal quality metrics such as signal-to-noise ratio, linewidth, and shape scores. However, the results also reveal limitations of the generative approach, including under-representation of stochastic noise and reduced accuracy in absolute metabolite quantification, particularly for applications sensitive to concentration estimates. These findings highlight both potential and limitations of data-driven MRS synthesis. Beyond the proposed model, this study introduces a structured evaluation framework for generative MRS methods, emphasizing the importance of application-aware validation when synthetic data are used for downstream analysis.

Figures & tables

Figure 1: Overview of the used model architecture. Real and imaginary parts of the spectrum are concatenated and used as input for a fully connected nn

Figure 2: Overview of the data generation pipeline. Real, in vivo spectra are first encoded into latent space. New samples are then generated using one of three methods: random sampling, interpolation, or hybrid sampling. Finally, all generated samples are decoded back into spectra.

Figure 3: Overview of the application-based evaluation method. gt spectra from a single subject are used to select a subset of $n$ transients. These transients are encoded using the encoder nn and used for the generation of synthetic samples. The decoder nn generates synthetic transients to end up with the same number of transients as in the gt spectra. For the evaluation all three datasets are processed and fitted using Osprey.

Panel of Figure 4 — Figure 4: Representative in-vivo spectra (blue) and corresponding reconstructions (orange) with residuals (black) for two subjects. Top panels show OFF and ON spectra with small, flat residuals, indicating accurate reconstruction. Bottom panels show another subject with slightly larger residuals, primarily around the water signal, while the overall spectral shapes are preserved.

Figure 5: umap visualizations comparing gt in-vivo spectra with synthetic spectra generated using three different data generation methods: random sampling (RS) (green), interpolation (red), and hybrid sampling (blue). The umap embedding is learned using the gt data, after which synthetic spectra generated from a single subset of transients are projected into the same low-dimensional space. Each point represents an individual spectrum, with markers indicating ON and OFF acquisitions. The shaded regions represent kernel density estimates of the embedded spectral distributions for both gt and synthetic data, illustrating the overall spatial extent and degree of overlap between datasets. Clusters corresponding to individual subjects are visible across all panels.

Panel of Figure 6 — Figure 6: Comparison of OFF, ON, and difference (OFF–ON) spectra across different data generation methods for two representative subjects. The top row (a-c) shows spectra from subject C008 (subset transient #6), and the bottom row (d-f) shows spectra from subject C039 (subset transient #6). For each subject, OFF spectra, ON spectra, and the corresponding difference spectra are shown from left to right.

Figure 7: Overview of quantitative metrics for the selected windows (gray) and the different data generation methods: random sampling (green), interpolation (red), and hybrid generation (blue). Metrics are computed from the difference spectra. For the snr and linewidth metrics, gt values are shown as reference. Statistically significant differences are indicated (*** $p<0.001$ , ** $p<0.01$ , * $p<0.05$ ). Boxplots with fewer than the nominal $16\times 10=160$ datapoints are explicitly marked.

Panel of Figure 8 — Figure 8: Heatmaps showing the relative concentration error with respect to the gt test spectra for the OFF spectra. For each subject, the relative error is shown per window for the different reconstruction methods. The adjacent column summarizes the mare across all windows for each subject. Values printed in bold indicate the method yielding the lowest mare for that subject. The color scale is shared across methods. The $\blacktriangle$ symbols indicate values that exceed the displayed range.

Panel of Figure 9 — Figure 9: Heatmaps showing the relative concentration error with respect to the gt test spectra for the difference spectra. For each subject, the relative error is shown per subset of transients for the different reconstruction methods. The adjacent column summarizes the mare across all subsets for each subject. Values printed in bold indicate the method yielding the lowest mare for that subject. The color scale is shared across methods. The $\blacktriangle$ symbols indicate values that exceed the displayed range.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Header / SubHeader	Values
Site (name or number)
1. Hardware
a. Field strength	3T
b. Manufacturer	Philips
c. Model (software version if available)	Achieva TX
d. RF coils: nuclei (transmit/receive), number of channels, type, body part	1H, 32-element, head coil

Appendix

Table 1: Minimum Reporting Standards for in-vivo Magnetic Resonance Spectroscopy (MRSinMRS) [ 22 ] for used dataset. Details are retrieved from the data and the work from van Bussel et al. [ 17 ]

Explore similar work

The Sim-to-Real Gap in MRS Quantification: A Systematic Deep Learning Validation for GABA
Feb 23, 2026Zien Ma, S. M. Shermer, Oktay Karakuş +1Sim-to-Real Transfer Convolutional Neural Networks
School of Computer Science and Informatics, Cardiff University, Cardiff, CF24 4AG, United Kingdom · Faculty of Science and Engineering, Swansea University, Swansea, SA2 8PP, United Kingdom
Physics-Informed Sylvester Normalizing Flows for Bayesian Inference in Magnetic Resonance Spectroscopy
May 6, 2025Julian P. Merkofer, Dennis M. J. van de Sande, Alex A. Bhogal +1Normalizing Flows Bayesian Inference
Eindhoven University of Technology, Eindhoven, The Netherlands · University Medical Center Utrecht, Utrecht, The Netherlands
Generative Modeling of Complex-Valued Brain MRI Data
Apr 16, 2026Marco Schlimbach, Moritz Rempe, Jessica Mnischek +4Medical Image Synthesis Brain MRI
Department of Physics, Technical University Dortmund, Otto-Hahn-Straße 4a, 44227 Dortmund, Germany · Institute for AI in Medicine (IKIM), University Hospital Essen, Girardetstraße 2, 45131 Essen, Germany · 3Cancer Research Center Cologne Essen (CCCE), University Medicine Essen, Hufelandstraße 55, 45147 Essen, Germany +4

Feb 23, 2026J/K move · Enter open · S save

The Sim-to-Real Gap in MRS Quantification: A Systematic Deep Learning Validation for GABA

Zien Ma, S. M. Shermer, Oktay Karakuş, Frank C. Langbein

School of Computer Science and Informatics, Cardiff University, Cardiff, CF24 4AG, United Kingdom · Faculty of Science and Engineering, Swansea University, Swansea, SA2 8PP, United Kingdom

Sim-to-Real Transfer Convolutional Neural Networks Magnetic Resonance Imaging Autoencoders

Open paper AskPDF arXiv

Magnetic resonance spectroscopy (MRS) is used to quantify metabolites in vivo and estimate biomarkers for conditions ranging from neurological disorders to cancers. Quantifying low-concentration metabolites such as GABA (

γ

-aminobutyric acid) is challenging due to low signal-to-noise ratio (SNR) and spectral overlap. We investigate and validate deep learning for quantifying complex, low-SNR, overlapping signals from MEGA-PRESS spectra, devise a convolutional neural network (CNN) and a Y-shaped autoencoder (YAE), and select the best models via Bayesian optimisation on 10,000 simulated spectra from slice-profile-aware MEGA-PRESS simulations. The selected models are trained on 100,000 simulated spectra. We validate their performance on 144 spectra from 112 experimental phantoms containing five metabolites of interest (GABA, Glu, Gln, NAA, Cr) with known ground truth concentrations across solution and gel series acquired at 3 T under varied bandwidths and implementations. These models are further assessed against the widely used LCModel quantification tool. On simulations, both models achieve near-perfect agreement (small MAEs; regression slopes

\approx 1.00

R^2 \approx 1.00

). On experimental phantom data, errors initially increased substantially. However, modelling variable linewidths in the training data significantly reduced this gap. The best augmented deep learning models achieved a mean MAE for GABA over all phantom spectra of 0.151 (YAE) and 0.160 (FCNN) in max-normalised relative concentrations, outperforming the conventional baseline LCModel (0.220). A sim-to-real gap remains, but physics-informed data augmentation substantially reduced it. Phantom ground truth is needed to judge whether a method will perform reliably on real data.

Abstract

Explore similar work

The Sim-to-Real Gap in MRS Quantification: A Systematic Deep Learning Validation for GABA

Physics-Informed Sylvester Normalizing Flows for Bayesian Inference in Magnetic Resonance Spectroscopy

Generative Modeling of Complex-Valued Brain MRI Data