Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts

Eraldo Pereira Marinho

astro-ph.IMNov 4, 2026

Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts

Authors: Eraldo Pereira Marinho

Organizations: Institute of Geoscience and Exact Sciences, São Paulo State University (UNESP), Av. 24 A, 1515, Rio Claro, 13506-900, São Paulo, Brazil

Abstract

Reported performance in small astronomical image datasets can reflect object reuse, field-of-view differences, acquisition settings, and catalogue-derived properties as well as astronomical morphology. We designed \dataset{} to measure these effects in a four-class task comprising galaxies, globular clusters, nebulae, and open clusters. The dataset contains 370 catalogue-supported physical objects, each represented by three Digitised Sky Survey FITS bands, and uses a group-constrained split of 222 training, 75 validation, and 73 test objects. We compared a custom CNN--Transformer trained from scratch with ImageNet-pretrained ResNet-50 and ViT-B/16 under an adaptive full-field representation, a common angular field, and two central-region-masked controls, using three fixed seeds. In the one-pass object-independent test, mean balanced accuracy on the adaptive view was $0.773\pm0.014$ , $0.874\pm0.054$ , and $0.866\pm0.023$ for the custom model, ResNet-50, and ViT-B/16, respectively. No primary architecture contrast rejected its null after Holm correction; ResNet-50 and ViT-B/16 differed by 0.007 balanced-accuracy points (95% cluster-bootstrap interval $[-0.048,0.062]$ ). Controls based on acquisition and catalogue metadata reached 0.721--0.735, and central-region-masked adaptive views reached 0.731--0.800. These results demonstrate generalisation to unseen physical objects, but also show that image context, acquisition settings, and catalogue-derived object properties carry substantial class information. In this regime, high accuracy cannot be interpreted as evidence that a classifier learned target morphology, and the comparison does not isolate architecture from pretraining.

Figures & tables

Class	Objects	Minimum	First quartile	Median	Third quartile	Maximum
Galaxies	98	5	5	5	5	23
Globular clusters	92	5	5	5	6.14	15.64
Nebulae	85	5	5	15	15	120
Open clusters	95	5	15	27.2	71.5	120

Table 1: Composition of the eligible pool and five-number summary of source FOV in arcmin. Values are computed per physical object before splitting.

Split	Galaxies	Globular	Nebulae	Open	Total
Training	59	55	51	57	222
Validation	20	19	17	19	75
Test	19	18	17	19	73

Table 2: Final object counts by class and split.

Figure 1: Examples of the four input representations. Rows show one deterministically selected validation object from each class; columns show A, B, $A_{\mathrm{outer}}$ , and $B_{\mathrm{outer}}$ . Stored blue/red/NIR channels are displayed as NIR/red/blue pseudo-colour for visualization only. Black regions are the prescribed central masks. The source FOV shown at left applies to A and $A_{\mathrm{outer}}$ ; B uses the common 4-arcmin field for every object.

Family	Initialization	Optimizer	Base LR	Weight decay
Custom CNN–Transformer	From scratch	Adam	$9\times 10^{-5}$	$5\times 10^{-7}$
ResNet-50	ImageNet-1K	AdamW	$1\times 10^{-4}$	$1\times 10^{-4}$
ViT-B/16	ImageNet-1K	AdamW	$3\times 10^{-5}$	$5\times 10^{-2}$

Table 3: Training settings fixed before test evaluation. Every family uses batch size 16, at most 100 epochs, class-weighted cross-entropy, gradient-norm cap 3, ReduceLROnPlateau (factor 0.1, patience 5), and early stopping with patience 20.

Family	Balanced accuracy	Macro-F1
Custom CNN–Transformer	$0.764\pm 0.016$	0.752
ResNet-50	$0.860\pm 0.005$	0.859
ViT-B/16	$0.858\pm 0.045$	0.856

Table 4: Validation results on A; mean

\pm

sample SD over three seeds.

Family	Representation	Balanced accuracy	Macro-F1
Custom CNN–Transformer	B	$0.688\pm 0.020$	$0.676\pm 0.020$
	$A_{\mathrm{outer}}$	$0.728\pm 0.024$	$0.707\pm 0.039$
	$B_{\mathrm{outer}}$	$0.586\pm 0.029$	$0.575\pm 0.033$
ResNet-50	B	$0.720\pm 0.046$	$0.719\pm 0.046$
	$A_{\mathrm{outer}}$	$0.712\pm 0.107$	$0.703\pm 0.113$
	$B_{\mathrm{outer}}$	$0.660\pm 0.081$	$0.659\pm 0.079$

Table 5: Validation controls; mean

\pm

sample SD over three fixed seeds.

Family	Representation	Balanced accuracy	Macro-F1	Accuracy
Custom CNN–Transformer	A	$0.773\pm 0.014$	$0.748\pm 0.003$	$0.785\pm 0.016$
	B	$0.704\pm 0.050$	$0.698\pm 0.047$	$0.699\pm 0.049$
	$A_{\mathrm{outer}}$	$0.731\pm 0.019$	$0.694\pm 0.045$	$0.744\pm 0.016$
	$B_{\mathrm{outer}}$	$0.625\pm 0.016$	$0.624\pm 0.014$	$0.626\pm 0.016$
ResNet-50	A	$0.874\pm 0.054$	$0.873\pm 0.056$	$0.877\pm 0.055$
	B	$0.737\pm 0.030$	$0.733\pm 0.029$	$0.735\pm 0.029$

Table 6: Object-independent test results; mean

\pm

sample SD over three fixed seeds. A and B are the adaptive and common-field views; “outer” denotes the corresponding central-mask control.

Figure 2: Row-normalized confusion matrices for representation A. Each cell is the mean over the three fixed seeds and is labelled as a percentage; rows are true classes and columns are predictions. BA denotes mean balanced accuracy. Rounded entries may not sum to exactly 100.

Figure 3: Test balanced accuracy for each model family and image representation. Points show the mean and bars show sample SD over the three fixed seeds. Dashed lines are the two single-fit metadata-only controls. They are reference levels, not uncertainty intervals.

First	Second	Metric	Difference	95% interval	Raw $p$	Holm $p$
Custom	ResNet-50	Accuracy	$-0.091$	$[-0.169,-0.014]$	0.032	0.191
Custom	ResNet-50	Balanced accuracy	$-0.101$	$[-0.156,-0.043]$	0.016	0.130
Custom	ResNet-50	Macro-F1	$-0.126$	$[-0.201,-0.054]$	0.009	0.077
Custom	ViT-B/16	Accuracy	$-0.082$	$[-0.174,0.009]$	0.101	0.404
Custom	ViT-B/16	Balanced accuracy	$-0.093$	$[-0.159,-0.029]$	0.057	0.285
Custom	ViT-B/16	Macro-F1	$-0.118$	$[-0.206,-0.033]$	0.023	0.159

Table 7: Paired A-view architecture comparisons. Differences are first minus second family, averaged over the three seeds. Intervals use the paired assignment-unit bootstrap;

p

values use paired unit-level randomization. Holm adjustment covers the nine displayed tests.

Image Classification

Image classification assigns semantic labels to images, enabling recognition of objects, scenes, conditions, and fine-grained categories.

See all 412 papers

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs
Jun 7, 2026Lanz Anthonee A. Lagman, Prospero C. Naval, Reinabelle C. ReyesConvolutional Neural Networks
Data Science Program, College of Science, University of the Philippines - Diliman, Philippines · Department of Computer Science, College of Engineering, University of the Philippines - Diliman, Philippines · National Institute of Physics, College of Science, University of the Philippines - Diliman, Philippines
Accuracy Is Not Enough: An Efficiency- and Statistics-Aware Benchmark of ResNet50 and Vision Transformer for Explainable Multi-Class Gastrointestinal Diseases Classification
Oct 25, 2026Akwasi AsareMedical Image Classification Medical Image Benchmarks
Department of Research and Training, Centre for Medical and Agro Computing (C-MAC), Accra, Ghana · Department of Computer Science, Ghana Communication Technology University, Accra, Ghana · PerceptronCare Lab.
Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasets
Sep 13, 2026Abhilekha Dalal, Michael Okonoda, Eder Martinez +1Image Classification Convolutional Neural Networks

Jun 7, 2026J/K move · Enter open · S save

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs

Lanz Anthonee A. Lagman, Prospero C. Naval, Reinabelle C. Reyes

Data Science Program, College of Science, University of the Philippines - Diliman, Philippines · Department of Computer Science, College of Engineering, University of the Philippines - Diliman, Philippines · National Institute of Physics, College of Science, University of the Philippines - Diliman, Philippines

Convolutional Neural Networks

Open paper AskPDF arXiv

Image data regarding galactic morphology is expected to increase both in quantity and quality for the next foreseeable years; thus it is important to explore which deep learning architectures adapted for image classification tasks are cost-effective. Residual and Inception networks are ideal for exploring classification convolutional neural networks (CNNs) due to their computational efficiency, achieved through techniques such as residual connections and parallelized inception modules, enabling deeper networks without excessively increasing computational complexity. In this work, we analyze the performance of ResNet101 and InceptionV4 on a spatially-augmented Galaxy10 DECals dataset. Retaining the ten-class classification of galaxies, we modify the image count of each class. We find that ResNet101 and InceptionV4 models achieved accuracies of

\sim

90%, comparable with reported performance in the literature. In terms of performance metrics, ResNet101 is superior to InceptionV4. Our results indicate that either of these CNN architectures could serve as a robust foundation for specialized pipelines for classification of galaxy images from upcoming surveys.

Abstract

Image Classification

Explore similar work

Classifying galaxies in the Galaxy10 DECals dataset using Inception and Residual CNNs

Accuracy Is Not Enough: An Efficiency- and Statistics-Aware Benchmark of ResNet50 and Vision Transformer for Explainable Multi-Class Gastrointestinal Diseases Classification

Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasets