Bayesian classification of astronomical spectra with class uncertainties

Simon Barton; Martin Sahlén; Andreas Korn; Christian Glaser

astro-ph.IMSep 18, 2026

Bayesian classification of astronomical spectra with class uncertainties

Authors: Simon Barton, Martin Sahlén, Andreas Korn, Christian Glaser

Organizations: Theoretical Astrophysics, Division of Astronomy and Space Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden. · Observational Astrophysics, Division of Astronomy and Space Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden · Division of High Energy Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden · Dept. of Physics, TU Dortmund University, D-44221 Dortmund, Germany

Abstract

Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.

Figures & tables

Figure 1: Selected example spectra of the SDSS dataset with the highest S/N of the respective class.

Figure 2: S/N distribution of each class for training and validation sets (bottom). Galactic (blue) and extragalactic (red) targets dominate the low- and high-noise regimes, respectively. Expected random-guessing accuracy based on the class ratio (top). For low S/N, the labels become easier to predict.

Panel of Figure 3 — Figure 3: Demonstration of the entropies of a predictive probability distribution in the three-simplex. Left: Large Shannon entropy $H$ and large differential entropy $h$ . The model assigns a high aleatoric ambiguity between galaxy and QSO to the input spectrum. Centre: Large $H$ and small $h$ . The model assigns low aleatoric uncertainty to the input spectrum, but is epistemically uncertain about its own prediction. Right: Small $H$ and $h$ . The model is certain about the spectrum and its prediction.

Figure 4: Overview scheme for this study. The arrows indicate flow of data.

Figure 5: Baseline CNN architecture, underlying all models. The hatched layers are enabled depending on the concrete model. The last block uses adaptive pooling to make the model compatible with different input spectra sizes.

Figure 6: Influence of the hyperparameters dropout rate and Adam weight decay on MCD accuracy (SDSS dataset).

Results (SDSS dataset)
Model	Accuracy [ $\%$ ]	AUC $\times 1000$	ECE [ $\%$ ]	NLL	Brier score $\times 1000$	Training time [s]	Inference time [ms]
Baseline CNN	$91.52\pm 0.02$	$997.4\pm 0.2$	$\bm{1.78\pm 0.05}$	N/A	N/A	$\bm{95}$	$\bm{0.012}$
Dirichlet	$87.9\pm 0.4$	$988\pm 1$	$5.47\pm 0.53$	$0.76\pm 0.07$	$31\pm 3$	$\bm{223}$	$0.029$
MCD100 ( $p=0.125$ )	$\bm{92.6\pm 0.1}$	$\bm{998.07\pm 0.01}$	$\bm{1.80\pm 0.02}$	$\bm{0.210\pm 0.003}$	$10.34\pm 0.03$	$\bm{436}$	$0.852$
MCD1000 ( $p=0.125$ )	$\bm{92.58\pm 0.07}$	$997.98\pm 0.01$	$\bm{1.80\pm 0.02}$	$\bm{0.206\pm 0.009}$	$\bm{9.79\pm 0.02}$	$\bm{436}$	$8.479$
VI ( $f=0$ , $\sigma=0.3$ )	$91.35\pm 0.07$	$997.35\pm 0.04$	$2.21\pm 0.01$	$0.230\pm 0.001$	$11.34\pm 0.03$	$1365$	$0.609$

Table 1: Results of all models used for the SDSS dataset. 4 4 4 Notes. Shown are the means and standard deviations of three independent training experiments with the same data splits, but different initialization weights. The highlighted values are the best in the column, within their uncertainty.

p

f

, and

\sigma

denote MC dropout rate, the rank of the VI covariance, and the standard deviation of its parameter prior, respectively. Training times are given until early stopping; inference times are given per spectrum. Both were taken on an Nvidia A5000 GPU.

Figure 7: Confusion matrix for a Monte Carlo dropout model on the SDSS dataset. The class names were abbreviated. The colours indicate percentages. The black lines separate the classes of different 4MOST coarse labels: star, quasar, galaxy. The percentages may not sum to exactly 100% due to rounding.

Figure 8: Examples for some of the most common confusion pairs on the SDSS dataset. Left: Two highest S/N spectra. The label format is ( $\text{actual}\rightarrow\text{prediction}$ ). Right: 100 samples from the MCD predictive posterior, where all other classes were marginalized into ‘other’. The crosses show the expectation values.

Panel of Figure 9 — Figure 9: Calibration diagrams (also known as reliability diagrams) for three models: Dirichlet (left), MCD (centre), and VI with rank $f=1$ and fixed prior (right). For each input $(x,y)$ and each class $c$ , the blue bars indicate how often $c$ coincides with $y$ , given the binned predicted probability $p(y|x)$ . The perfect values are shown in red, and take the binning effect into account. The ECE is the expected calibration error and measures the deviation between the blue bars and the red line, weighted by the respective prediction rate (black). A lower ECE is better. Blue bars above and below the red line correspond to under- and overconfidence of the model, respectively. These plots average all the probabilities and the error over all $c$ .

Figure 10: Effect of S/N on accuracy for all models. The error bars indicate standard deviations from three independent training experiments (SDSS dataset).

Figure 11: Shannon entropy vs S/N on the mock dataset. The Dirichlet model predictions have much higher entropy. The correlation matrix shows the relation of S/N (cause), accuracy (effect), Brier score (measure), and Shannon entropy (indicating uncertainty) of MCD predictions.

Results (4MOST mock dataset)
Model	Accuracy [ $\%$ ]	AUC $\times 1000$	ECE [ $\%$ ]	NLL	Brier score $\times 1000$	Training time [s]	Inference time [ms]
Baseline CNN	$92.80\pm 0.08$	$995.15\pm 0.04$	$\bm{1.82\pm 0.01}$	N/A	N/A	$\bm{229}$	$\bm{0.015}$
Dirichlet	$87.37\pm 0.09$	$986.95\pm 0.72$	$6.01\pm 0.16$	$0.620\pm 0.013$	$38.71\pm 0.99$	$968$	$0.037$
MCD50 ( $p=0.125$ )	$\bm{93.94\pm 0.07}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.59\pm 0.05}$	$576$	$0.581$
MCD100 ( $p=0.125$ )	$\bm{93.95\pm 0.08}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.60\pm 0.05}$	$576$	$1.168$
MCD200 ( $p=0.125$ )	$\bm{93.95\pm 0.05}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.60\pm 0.05}$	$576$	$2.336$

Table 2: Results for the 4MOST mock dataset. 5 5 5 Notes. All values, as in Footnote 4 .

Figure 12: Class confusion of the MCD model for the 4MOST mock dataset, as in Fig. 7 . The black lines separate blocks of extragalactic and galactic classes.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 13: Selected example spectra of the mock dataset with the highest S/N of the respective class.

Figure 14: S/N distribution (bottom) and resulting expected random-guessing accuracy (top) as in Fig. 2 for the mock dataset.

Figure 15: Receiver operator curve (ROC) of the MCD model on the SDSS dataset. In brackets

\text{AUC}\times 1000

Figure 16: Training curves for all models, showing the accuracies as a function of the number of trained epochs, for the three training runs on the 4MOST mock dataset. Loss functions are omitted for clarity. A strong separation between training and validation curves would indicate overfitting. A difference between runs would indicate unstable convergence and local loss minima. The stars indicate the training state with the lowest validation loss that were used for the results above.

Panel of Figure 17 — Figure 17: Confusion matrix. Top labels are the maximum S/N value for every actual–prediction pair. The bottom labels and colouring are as in Fig. 7 . Left: SDSS dataset. Right: 4MOST mock dataset.

Figure 18: Comparison of different metrics of uncertainty on the SDSS dataset. Abscissa and ordinate labels apply to full columns and rows, respectively. Correct and incorrect histogram bars are independently normalized to one.

Explore similar work

Classification of Astronomical Spectra Using PCA-Compressed Flux and Inverse-Variance Features
Jun 11, 2026Bruno Santos Meneses Barreto, Marcio Eisencraft
Departamento de Engenharia de Telecomunicações e Controle, Universidade de São Paulo, São Paulo-SP
Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantification
Jul 6, 2026Raphaël Bonnet-Guerrini, Bruno Sanchez, Dominique Fouchez +5Weakly Supervised Learning Learning with Noisy Labels
Università degli Studi di Milano, Department of Computer Science, Milan 20133, Italy · Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France · Université Côte d’Azur, INRIA, CNRS, Laboratoire J.A.Dieudonné, Maasai team, Nice 06000, France +2
AstroSpecLM: A Spectrum-Language Model for Evidence-Grounded Astronomical Spectral Analysis
Sep 7, 2026Jinghang Shi, Yanxia Zhang, Ali Luo +2Language Modeling Scientific QA

Jun 11, 2026J/K move · Enter open · S save

Classification of Astronomical Spectra Using PCA-Compressed Flux and Inverse-Variance Features

Bruno Santos Meneses Barreto, Marcio Eisencraft

Departamento de Engenharia de Telecomunicações e Controle, Universidade de São Paulo, São Paulo-SP

Open paper AskPDF arXiv

This paper evaluates a signal-processing and supervised-learning pipeline for classifying SDSS DR17 astronomical spectra into stars, galaxies, and quasars. Each spectrum is represented by its measured flux and inverse-variance information, combining spectral shape with a wavelength-dependent reliability profile. After resampling onto a common logarithmic wavelength grid, the flux and inverse-variance vectors are standardized and separately compressed using principal component analysis. The resulting components are concatenated and used to train several classifiers. The best performance was obtained with the LightGBM gradient-boosting classifier, reaching

94.6\%

accuracy and

92.1\%

balanced accuracy on the test set.

Abstract

Explore similar work

Classification of Astronomical Spectra Using PCA-Compressed Flux and Inverse-Variance Features

Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantification

AstroSpecLM: A Spectrum-Language Model for Evidence-Grounded Astronomical Spectral Analysis