Characterization of Residual Morphological Substructure Using Supervised and Unsupervised Deep Learning

Kameswara Bharadwaj Mantha; Daniel H. McIntosh; Cody Ciaschi; Rubyet Evan; Luther Landry; Henry C. Ferguson; Camilla Pacifici; Joel Primack

astro-ph.GAFeb 21, 2026

Characterization of Residual Morphological Substructure Using Supervised and Unsupervised Deep Learning

Authors: Kameswara Bharadwaj Mantha, Daniel H. McIntosh, Cody Ciaschi, Rubyet Evan, Luther Landry, Henry C. Ferguson, Camilla Pacifici, Joel Primack, +4 more

Organizations: Department of Physics & Astronomy, University of Missouri Kansas City, Kansas City, MO, 64110, USA. · Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218, USA. · Department of Physics, University of California, Santa Cruz, CA 95064, USA. · Department of Physics and Astronomy, University of Missouri, Columbia, MO 65211, USA.

Abstract

Automated characterization of galactic substructure is an essential step in understanding the transformative physical processes driving galaxy evolution. In this study, we investigate the application of deep learning (DL) frameworks to characterize different galactic substructures hosted within parametric light-profile subtracted residual'' images of a large sample galaxies from the CANDELS survey. We develop a supervised Convolutional Neural Network (CNN) and unsupervised Convolutional Variational Autoencoder (CvAE) and train it on the single-Sérsic profile fitting based residual images of $10,046$ bright and massive galaxies ($H<24.5\,{\rm mag}$ and $M_{\rm stellar} \geq 10^{9.5}\,M_{\odot}$) spanning $1<z<3$, in conjunction with their visual-based classification labels indicating the nature of residual substructures hosted within them. Using our unique data preprocessing approach, we prepare our residual images such that the inputs to our DL networks comprise only galaxy of interest'', and augment them such that our sample span uniformly across different residual characteristics. We assess the latent space of the CNN and CvAE using Principle Component Analysis (PCA) along with independently quantified metrics of residual strength (significant pixel flux $SPF$ , Bumpiness, and Residual Flux Fraction). We also employ an unsupervised Gaussian Mixture Modeling (GMM) based clustering scheme with Support Vector Classification (SVC) to identify groupings in PCA space that correspond to similar residual substructure. We find that our supervised CNN latent features in PCA space correlate with the $SPF$ values and distinguish between qualitatively strong and weak residual substructures. While our unsupervised CvAE latent space also correlates with visual and quantitative residual characteristics, but lacks clear discriminatory power when characterizing different residual substructures.

Figures & tables

Figure 1: Visualization of the H -band original images (left in each column pair) and their corresponding residual images (right panel in each column pair) for example CANDELS galaxies from our main sample, generated by performing single-Sérsic light-profile fitting using GALFIT ( Peng et al., 2002 ) by van der Wel et al. (2012) .

Residual Characteristic	Definition	Sample Size	Percentage
(1)	(2)	(3)	(4)
Clean	The residual has no apparent features or structures pertaining to the galaxy and is consistent with the background noise.	$3815$	$38.0\%$
General	The residual hosts strong left-over light with either a symmetric disk or spiral-like structure, or clumpy under-fit substructure associated to underlying plausible disk or spiral galaxy morphology.	$608$	$6.0\%$
Core	The residual contains one or more bright point-like grouping of pixels near to the central region (by eye) of the galaxy, and no excess away from the galaxy’s central region.	$1958$	$19.5\%$
Asymmetric	The residual contains concentrated or diffuse, asymmetric excess light outside the central region of the galaxy, with or without the presence of central excess light.	$3043$	$30.3\%$
Peculiar	The residual contains strong asymmetric excess light with or without multiple central point excess that aligns with the overall disturbed morphology of the galaxy.	$512$	$5.1\%$

Table 1: Breakdown of the five residual characteristic classes based on the visual inspection of our

10,046

massive galaxy sample from the CANDELS survey. Columns: (1) Name of the residual characteristic; (2) Adopted definition of the residual characteristic for the visual inspection; (3) Number of galaxies in the sample with the corresponding residual characteristic; (4) Percentage contribution of the residual characteristic out of

10,046

galaxies.

Figure 3: Visual illustration of our preprocessing step to prepare our residual images for the DL framework. We show the original image, source extraction based segmentation map highlighting the detected sources, residual image, region of interest corresponding the central galaxy in the image, and the GOI region of the residual image with a $50\,{\rm pix}\times 50\,{\rm pix}$ postage stamp size for reference.

Figure 4: The histograms of our five residual characteristic classes among our parent sample of $\sim 10,000$ galaxies (black), training sub-sample (red), testing sub-sample (blue), and augmented training data sets (magenta). For each class, we also show its fractional contribution to the data set with respective colored text.

Figure 5: Example visualization of our data augmentation step (see § 3.2 ). For three example residual images from our sample, we show five random horizontal flip and $45\deg$ rotation steps.

Figure 6: Visual illustration of our Significant Pixel Flux ( SPF ) calculation using an example galaxy image (top-left) from our sample. We show the source-masked image (top-right) and the Region of Interest ( ROI ; bottom-left) highlighting the Galaxy of Interest ( GOI ). We show the histograms of pixel fluxes (bottom-right) in the ROI (green), the sky background distribution (dashed black) fit with a Gaussian (solid black) and their lower and upper $3\sigma$ bounds with blue and red vertical lines, respectively. We highlight those pixels that are above and below the $3\sigma$ bounds in bottom-left panel with their respective colors. For this example, SPF $=11.28$ .

Figure 7: Overview of our supervised CNN (top) and unsupervised Convolutional Variational Autoencoder (bottom) frameworks used to analyze the galactic residual substructures (see § 4.1 & § 4.2 ). For both networks, we also visually illustrate our analysis of the latent space features using Principle Component Analysis and unsupervised clustering (see § 5 ).

Figure 8: Example visualization of our unsupervised Convolutional Variational Autoencoder network’s input (top) and reconstruction (bottom) images.

Figure 9: Visualization of principle component axes of the supervised latent space color coded with five residual class labels (left panel). In the right panel, we show the PC1 vs. PC2 only for the Clean, Peculiar, and General Classes.

Figure 10: The supervised CNN based latent space visualized in the PCA eigen axes for a subset of classes (Clean, Peculiar, and General; shown in Figure 9 ) and color coded by different residual quantitative metrics – $\log_{10}(SPF)$ (left panel), Bumpiness ( $B$ ; middle panel), and Residual Flux Fraction ( $RFF$ ; right panel).

Figure 11: Visualization of the unsupervised CvAE latent space in the PCA eigen axes (PC1 vs. PC2), where each data point is color coded by its visual residual class label (left panel). In the right panel, we only show the Clean, Peculiar, and General classes.

Figure 12: Visualization of the unsupervised CvAE latent space in PCA eigen axes along with residual quantitative metrics $SPF$ , $B$ , and $RFF$ following a similar layout as in Figure 10 .

Figure 13: Left panel : The sum of squared distances between the data points using $k$ -means clustering technique as a function of number of permissible clusters, indicating a transition from a sharp to gradual fall off at an optimal number of $6$ clusters. Right panel : We show unsupervised Gaussian Mixture Modeling based clustering of the PCA embedding of the CNN latent features, where the GMM identified clusters are labelled with unique color coding (see color bar).

Figure 14: Visualization of the residual images sampled randomly from the GMM-based clusters identified in the supervised CNN-based PCA space (in Figure 13 ). We outline the row pairs with borders corresponding to the color coding used in Figure 13 , except for the second and third rows, which are random samplings from multiple clusters (GMM labels $=0,1,5$ ). On each residual image, we indicate the GMM cluster label.

Figure 15: Same as Figure 13 , but for our unsupervised CvAE latent features in PCA space.

Figure 16: Following the similar layout as in Figure 14 , we show the residual images sampled randomly from the GMM-based clusters identified in the unsupervised CvAE based PCA space.

Explore similar work

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs
Jun 7, 2026Lanz Anthonee A. Lagman, Prospero C. Naval, Reinabelle C. ReyesConvolutional Neural Networks
Data Science Program, College of Science, University of the Philippines - Diliman, Philippines · Department of Computer Science, College of Engineering, University of the Philippines - Diliman, Philippines · National Institute of Physics, College of Science, University of the Philippines - Diliman, Philippines
Spatio-Spectroscopic Representation Learning using Unsupervised Convolutional Long-Short Term Memory Networks
Feb 20, 2026Kameswara Bharadwaj Mantha, Lucy Fortson, Ramanakumar Sankar +8Unsupervised Representation Learning Convolutional Autoencoder
Department of Physics and Astronomy, University of Minnesota Twin Cities, Minneapolis, USA · Company Name, Location, Country · European Space Agency (ESA), European Space Astronomy Centre (ESAC), Camino Bajo del Castillo s/n 28692 Villanueva de la Cañada, Madrid, Spain +4
Deep learning from the crowd Fundamentals of morphological galaxy classification
Sep 6, 2026Luis Enrique Sucar, Carlos del Burgo, Jonathan Serrano-PérezHierarchical Classification Convolutional Neural Networks
Instituto de Astrofísica de Canarias, Vía Láctea S/N, La Laguna 38200, Tenerife, Spain · Instituto Nacional de Astrofísica, Óptica y Electrónica, Luis Enrique Erro #1, CP 72840, Tonantzintla, Puebla, Mexico · Departamento de Astrofísica, Universidad de la Laguna, La Laguna 38200, Tenerife, Spain +1

Jun 7, 2026J/K move · Enter open · S save

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs

Lanz Anthonee A. Lagman, Prospero C. Naval, Reinabelle C. Reyes

Data Science Program, College of Science, University of the Philippines - Diliman, Philippines · Department of Computer Science, College of Engineering, University of the Philippines - Diliman, Philippines · National Institute of Physics, College of Science, University of the Philippines - Diliman, Philippines

Convolutional Neural Networks

Open paper AskPDF arXiv

Image data regarding galactic morphology is expected to increase both in quantity and quality for the next foreseeable years; thus it is important to explore which deep learning architectures adapted for image classification tasks are cost-effective. Residual and Inception networks are ideal for exploring classification convolutional neural networks (CNNs) due to their computational efficiency, achieved through techniques such as residual connections and parallelized inception modules, enabling deeper networks without excessively increasing computational complexity. In this work, we analyze the performance of ResNet101 and InceptionV4 on a spatially-augmented Galaxy10 DECals dataset. Retaining the ten-class classification of galaxies, we modify the image count of each class. We find that ResNet101 and InceptionV4 models achieved accuracies of

\sim

90%, comparable with reported performance in the literature. In terms of performance metrics, ResNet101 is superior to InceptionV4. Our results indicate that either of these CNN architectures could serve as a robust foundation for specialized pipelines for classification of galaxy images from upcoming surveys.

Abstract

Explore similar work

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs

Spatio-Spectroscopic Representation Learning using Unsupervised Convolutional Long-Short Term Memory Networks

Deep learning from the crowd Fundamentals of morphological galaxy classification