Bayesian classification of astronomical spectra with class uncertainties

Simon Barton; Martin Sahlén; Andreas Korn; Christian Glaser

astro-ph.IMSep 18, 2026

Bayesian classification of astronomical spectra with class uncertainties

Authors: Simon Barton, Martin Sahlén, Andreas Korn, Christian Glaser

Organizations: Theoretical Astrophysics, Division of Astronomy and Space Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden. · Observational Astrophysics, Division of Astronomy and Space Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden · Division of High Energy Physics, Department of Physics and Astronomy, Uppsala University, Box 524, 751 20 Uppsala, Sweden · Dept. of Physics, TU Dortmund University, D-44221 Dortmund, Germany

Abstract

Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.

Figures & tables

Figure 1: Selected example spectra of the SDSS dataset with the highest S/N of the respective class.

Figure 2: S/N distribution of each class for training and validation sets (bottom). Galactic (blue) and extragalactic (red) targets dominate the low- and high-noise regimes, respectively. Expected random-guessing accuracy based on the class ratio (top). For low S/N, the labels become easier to predict.

Panel of Figure 3 — Figure 3: Demonstration of the entropies of a predictive probability distribution in the three-simplex. Left: Large Shannon entropy $H$ and large differential entropy $h$ . The model assigns a high aleatoric ambiguity between galaxy and QSO to the input spectrum. Centre: Large $H$ and small $h$ . The model assigns low aleatoric uncertainty to the input spectrum, but is epistemically uncertain about its own prediction. Right: Small $H$ and $h$ . The model is certain about the spectrum and its prediction.

Figure 4: Overview scheme for this study. The arrows indicate flow of data.

Figure 5: Baseline CNN architecture, underlying all models. The hatched layers are enabled depending on the concrete model. The last block uses adaptive pooling to make the model compatible with different input spectra sizes.

Figure 6: Influence of the hyperparameters dropout rate and Adam weight decay on MCD accuracy (SDSS dataset).

Results (SDSS dataset)
Model	Accuracy [ $\%$ ]	AUC $\times 1000$	ECE [ $\%$ ]	NLL	Brier score $\times 1000$	Training time [s]	Inference time [ms]
Baseline CNN	$91.52\pm 0.02$	$997.4\pm 0.2$	$\bm{1.78\pm 0.05}$	N/A	N/A	$\bm{95}$	$\bm{0.012}$
Dirichlet	$87.9\pm 0.4$	$988\pm 1$	$5.47\pm 0.53$	$0.76\pm 0.07$	$31\pm 3$	$\bm{223}$	$0.029$
MCD100 ( $p=0.125$ )	$\bm{92.6\pm 0.1}$	$\bm{998.07\pm 0.01}$	$\bm{1.80\pm 0.02}$	$\bm{0.210\pm 0.003}$	$10.34\pm 0.03$	$\bm{436}$	$0.852$
MCD1000 ( $p=0.125$ )	$\bm{92.58\pm 0.07}$	$997.98\pm 0.01$	$\bm{1.80\pm 0.02}$	$\bm{0.206\pm 0.009}$	$\bm{9.79\pm 0.02}$	$\bm{436}$	$8.479$
VI ( $f=0$ , $\sigma=0.3$ )	$91.35\pm 0.07$	$997.35\pm 0.04$	$2.21\pm 0.01$	$0.230\pm 0.001$	$11.34\pm 0.03$	$1365$	$0.609$

Table 1: Results of all models used for the SDSS dataset. 4 4 4 Notes. Shown are the means and standard deviations of three independent training experiments with the same data splits, but different initialization weights. The highlighted values are the best in the column, within their uncertainty.

p

f

, and

\sigma

denote MC dropout rate, the rank of the VI covariance, and the standard deviation of its parameter prior, respectively. Training times are given until early stopping; inference times are given per spectrum. Both were taken on an Nvidia A5000 GPU.

Figure 7: Confusion matrix for a Monte Carlo dropout model on the SDSS dataset. The class names were abbreviated. The colours indicate percentages. The black lines separate the classes of different 4MOST coarse labels: star, quasar, galaxy. The percentages may not sum to exactly 100% due to rounding.

Figure 8: Examples for some of the most common confusion pairs on the SDSS dataset. Left: Two highest S/N spectra. The label format is ( $\text{actual}\rightarrow\text{prediction}$ ). Right: 100 samples from the MCD predictive posterior, where all other classes were marginalized into ‘other’. The crosses show the expectation values.

Panel of Figure 9 — Figure 9: Calibration diagrams (also known as reliability diagrams) for three models: Dirichlet (left), MCD (centre), and VI with rank $f=1$ and fixed prior (right). For each input $(x,y)$ and each class $c$ , the blue bars indicate how often $c$ coincides with $y$ , given the binned predicted probability $p(y|x)$ . The perfect values are shown in red, and take the binning effect into account. The ECE is the expected calibration error and measures the deviation between the blue bars and the red line, weighted by the respective prediction rate (black). A lower ECE is better. Blue bars above and below the red line correspond to under- and overconfidence of the model, respectively. These plots average all the probabilities and the error over all $c$ .

Figure 10: Effect of S/N on accuracy for all models. The error bars indicate standard deviations from three independent training experiments (SDSS dataset).

Figure 11: Shannon entropy vs S/N on the mock dataset. The Dirichlet model predictions have much higher entropy. The correlation matrix shows the relation of S/N (cause), accuracy (effect), Brier score (measure), and Shannon entropy (indicating uncertainty) of MCD predictions.

Results (4MOST mock dataset)
Model	Accuracy [ $\%$ ]	AUC $\times 1000$	ECE [ $\%$ ]	NLL	Brier score $\times 1000$	Training time [s]	Inference time [ms]
Baseline CNN	$92.80\pm 0.08$	$995.15\pm 0.04$	$\bm{1.82\pm 0.01}$	N/A	N/A	$\bm{229}$	$\bm{0.015}$
Dirichlet	$87.37\pm 0.09$	$986.95\pm 0.72$	$6.01\pm 0.16$	$0.620\pm 0.013$	$38.71\pm 0.99$	$968$	$0.037$
MCD50 ( $p=0.125$ )	$\bm{93.94\pm 0.07}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.59\pm 0.05}$	$576$	$0.581$
MCD100 ( $p=0.125$ )	$\bm{93.95\pm 0.08}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.60\pm 0.05}$	$576$	$1.168$
MCD200 ( $p=0.125$ )	$\bm{93.95\pm 0.05}$	$\bm{996.25\pm 0.02}$	$1.87\pm 0.01$	$\bm{0.142\pm 0.001}$	$\bm{9.60\pm 0.05}$	$576$	$2.336$

Table 2: Results for the 4MOST mock dataset. 5 5 5 Notes. All values, as in Footnote 4 .

Figure 12: Class confusion of the MCD model for the 4MOST mock dataset, as in Fig. 7 . The black lines separate blocks of extragalactic and galactic classes.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 13: Selected example spectra of the mock dataset with the highest S/N of the respective class.

Figure 14: S/N distribution (bottom) and resulting expected random-guessing accuracy (top) as in Fig. 2 for the mock dataset.

Figure 15: Receiver operator curve (ROC) of the MCD model on the SDSS dataset. In brackets

\text{AUC}\times 1000

Figure 16: Training curves for all models, showing the accuracies as a function of the number of trained epochs, for the three training runs on the 4MOST mock dataset. Loss functions are omitted for clarity. A strong separation between training and validation curves would indicate overfitting. A difference between runs would indicate unstable convergence and local loss minima. The stars indicate the training state with the lowest validation loss that were used for the results above.

Panel of Figure 17 — Figure 17: Confusion matrix. Top labels are the maximum S/N value for every actual–prediction pair. The bottom labels and colouring are as in Fig. 7 . Left: SDSS dataset. Right: 4MOST mock dataset.

Figure 18: Comparison of different metrics of uncertainty on the SDSS dataset. Abscissa and ordinate labels apply to full columns and rows, respectively. Correct and incorrect histogram bars are independently normalized to one.

Explore similar work

Jun 11, 2026

Classification of Astronomical Spectra Using PCA-Compressed Flux and Inverse-Variance Features

This paper evaluates a signal-processing and supervised-learning pipeline for classifying SDSS DR17 astronomical spectra into stars, galaxies, and quasars. Each spectrum is represented by its measured flux and inverse-variance information, combining spectral shape with a wavelength-dependent reliability profile. After resampling onto a common logarithmic wavelength grid, the flux and inverse-variance vectors are standardized and separately compressed using principal component analysis. The resulting components are concatenated and used to train several classifiers. The best performance was obtained with the LightGBM gradient-boosting classifier, reaching

94.6\%

accuracy and

92.1\%

balanced accuracy on the test set.

Bruno Santos Meneses Barreto, Marcio Eisencraft

Departamento de Engenharia de Telecomunicações e Controle, Universidade de São Paulo, São Paulo-SP

Topic pending

Ask PDF

Jul 6, 2026

Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantification

Time-domain surveys generate many transient candidates, making Real-Bogus classification a critical step in automated discovery pipelines. Reliable labels are costly, while community labels can be noisy and survey-dependent. We aim to develop a Real-Bogus classification framework that can be trained without human-labeled data using injected transients and bogus-dominated survey data, remains robust under strong class contamination, and provides calibrated uncertainty quantification. We combine simulated transient injections with a contaminated survey class and train a dual-network model using asymmetric co-teaching for classes with different label-noise levels. We evaluate performance on a benchmark subset and analyze the learned representation with latent-space visualization tools. For uncertainty quantification (UQ), we compare MC dropout and deep ensembles and propose a low-cost hybrid strategy that exploits the dual-network setting to improve calibration. We extend the evaluation to the light-curve domain to assess recovery of light-curve classes. The method achieves strong Real-Bogus performance on the labeled subset and remains stable under severe class contamination. It recovers transient light-curve classes with high fidelity, while single-source identification is limited by ambiguity in light-curve-derived labels. Our hybrid UQ approach achieves competitive calibration relative to more expensive ensemble baselines. Latent-space analyses indicate that uncertainty aligns with the decision boundary and reveal subclasses within the bogus population. Our results show that injection-driven, weakly supervised training can enable scalable and consistent Real-Bogus classification without human-labeled training data while providing calibrated uncertainties. The method is suited for transfer to forthcoming surveys by re-running the injection-based training pipeline.

Raphaël Bonnet-Guerrini, Bruno Sanchez, Dominique Fouchez +5

Università degli Studi di Milano, Department of Computer Science, Milan 20133, Italy · Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France · Université Côte d’Azur, INRIA, CNRS, Laboratoire J.A.Dieudonné, Maasai team, Nice 06000, France +2

Weakly Supervised Learning Learning with Noisy Labels Uncertainty Calibration

Ask PDF

Sep 7, 2026

AstroSpecLM: A Spectrum-Language Model for Evidence-Grounded Astronomical Spectral Analysis

Astronomical spectra encode rich physical information, but drawing scientific conclusions from spectral features typically requires expert interpretation. This paper presents AstroSpecLM, a spectrum-language model that connects one-dimensional DESI spectra with Qwen3-4B to answer questions and provide explanations grounded in spectral evidence. Instead of generating question-answer pairs directly from templates or raw catalog fields, we first distill each spectrum into a compact set of catalog- and spectrum-derived facts, then use these facts as references to generate instruction-following conversations. The resulting model is competitive with specialist supervised baselines on classification and redshift estimation, while additionally producing natural-language explanations that reference specific spectral features. Our results indicate that grounding a language model in one-dimensional scientific spectra is feasible, and that fact-mediated instruction data yields a model capable of both prediction and explanation.

Jinghang Shi, Yanxia Zhang, Ali Luo +2

Language Modeling Scientific QA LLM Grounding

Ask PDF