Contrastive Astromer: Two-Stage Contrastive Representation Learning for Astronomical Light Curve Classification

Justin Minggan Wei; Pavlos Protopapas; Guillermo Cabrera-Vives

astro-ph.IMOct 30, 2026

Contrastive Astromer: Two-Stage Contrastive Representation Learning for Astronomical Light Curve Classification

Authors: Justin Minggan Wei, Pavlos Protopapas, Guillermo Cabrera-Vives

Organizations: School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, 02138, USA · Department of Computer Science, Universidad de Concepción, Edmundo Larenas 219, Concepción, Chile

Abstract

Large astronomical surveys collect abundant light curves, but reliable class labels remain scarce. ASTROMER 2 learns from unlabeled light curves by reconstructing masked observations, yet this objective does not explicitly encourage class separation in a compact representation of each light curve. We introduce Contrastive Astromer, a two-stage approach that makes these representations more useful for classification without changing the ASTROMER 2 encoder architecture. First, unsupervised contrastive learning on single-band MACHO light curves brings together representations of augmented views of the same object. Supervised contrastive learning, supported by an auxiliary classification loss, then uses a separate labeled Alcock sample to strengthen class discrimination. We evaluate the resulting representations by freezing the encoder and training a separate classifier. On a held-out set of 600 objects, this classifier achieves a macro-F1 of 0.7943 with 500 labeled examples per class, exceeding our reproduction of the ASTROMER 2 finetune-and-classify pipeline by 5.10 percentage points. With downstream classifier training budgets of 20 and 100 examples per class, macro-F1 reaches 0.7078 and 0.7704, respectively, also surpassing this baseline. Embedding-space analyses further show that supervised adaptation improves local class consistency and global class separation. These results show that contrastive adaptation makes pretrained light-curve representations more effective for classification and more informative about the structure of variability classes.

Figures & tables

Panel of Figure 1 — Figure 1: Positive and negative views for one anchor in a three-object minibatch. For each object, one sampled light-curve window supplies two independently augmented views, represented here by their projected embeddings (subscripts 1 and 2). Objects A and B share a class; C belongs to a different class. In Stage 1, class labels are not used: only the companion view $A_{2}$ is positive, while both views of B are negatives because B is a different object, despite sharing A’s class. In Stage 2, class labels define the positive set, so both views of B become additional positives. The anchor itself is excluded from both sets. Only comparisons involving anchor $A_{1}$ are drawn; every view serves as an anchor in turn. The objects illustrate the pairing rules, not shared training data between stages.

Figure 2: Contrastive Astromer methodology. (a) Each augmented view passes separately through the shared ASTROMER 2 encoder

f_{\theta}

. Mean pooling over retained observations converts the token vectors into a 256-dimensional light-curve representation

h

. The projection head

g_{\phi}

maps

h

to the normalized 128-dimensional embedding

z

used for contrastive learning. The auxiliary linear classifier operates directly on

h

and is used only in Stage 2. (b) Stage 1 learns from unlabeled MACHO light curves using InfoNCE. Its encoder and projection-head weights initialize Stage 2, which uses labeled Alcock light curves with supervised contrastive learning and auxiliary cross-entropy (CE). (c) After Stage 2, both training heads are discarded. A separate downstream classifier is trained on representations from unaugmented windows, with the encoder frozen. Teal denotes retained encoder and pooling components; orange denotes temporary training heads; gray denotes data and representations. Light-curve thumbnails show real observations; the network, token, and vector graphics are schematic.

Figure 3: Augmented views of a real MACHO light-curve window. Panel (a) shows 200 consecutive observations of the long-period variable MACHO 23.3302.11. Panels (b)–(c) show two independently sampled augmented views, using the Stage 1 probabilities and strengths in Table 1 . Panels (d)–(i) illustrate each augmentation applied individually. Open gray circles show the original observations for reference, colored points show retained augmented observations, and crosses mark excluded observations. Temporal cropping retains a contiguous interval of observation indices; masking and cropping change visibility without introducing zero-valued measurements. All main panels share the same axes, original time origin, and original-window mean magnitude, which is subtracted for display only. Insets magnify the small changes from amplitude scaling and Gaussian noise without changing their strength.

	Stage 1		Stage 2
Augmentation operator	Probability	Strength / parameter	Probability	Strength / parameter
Time jitter	$0.5$	Maximum shift: $0.1$ times time range	$0.2$	Maximum shift: $0.03$ times time range
Amplitude scaling	$0.5$	Scale range: $[0.95,1.05]$	$0.25$	Scale range: $[0.98,1.02]$
Amplitude shift	$0.3$	Shift range: $[-0.1,0.1]$	$0.1$	Shift range: $[-0.03,0.03]$
Gaussian magnitude noise	$0.8$	Noise level: $0.01$	$0.35$	Noise level: $0.005$
Random observation masking	$0.3$	Drop probability: $0.1$	$0.1$	Drop probability: $0.05$

Table 1: Augmentation probabilities and strengths for Stage 1 and Stage 2.

Partition	Objects	Per class	Role
Stage 2 pool	17,294	class-imbalanced	representation learning
Downstream CV pool	3,000	500	five-fold selection
Final holdout	600	100	one-time testing
Total	20,894	—	—

Table 2: Object-disjoint Alcock partitions used in the experiments. Counts denote unique objects.

Stage	Dataset and split	Labels used?	Encoder updated?	Projection/ classifier head	Purpose
Stage 1	MACHO; no objects overlapping with Alcock	No	Yes	Projection head trained	Unsupervised contrastive representation learning
Stage 2	17,294-object Alcock representation pool	Yes	Yes	Projection head and auxiliary classifier trained	Supervised contrastive learning
CV and selection	3,000-object pool; five folds; Alcock20/100/500	Yes	No	Pooled- $h$ MLP head trained per run	Select one Stage 1 checkpoint using validation only
Final evaluation	Setting-specific training pool; 600-object holdout	Yes	No	Final MLP head retrained	Report holdout performance

Table 3: Data roles in the Contrastive Astromer workflow.

Setting	Stage 1 (unsupervised contrastive)	Stage 2 (supervised contrastive)	Downstream classifier head
Data split	MACHO; no objects overlapping with Alcock	17,294-object Alcock pool	3,000-object CV pool; 600-object holdout
Input window	200 observations	200 observations	central 200-observation window
Encoder	updated	updated	frozen
Objective	InfoNCE / NT-Xent	supervised contrastive + auxiliary CE	sparse categorical cross-entropy
Optimizer	Adam	Adam	Adam
Learning rate	$10^{-4}$	$10^{-4}$	$10^{-4}$

Table 4: Training and evaluation settings for the two encoder-construction stages and the downstream classifier head.

Figure 4: Downstream validation macro-F1 across all 100 Stage 1 checkpoints for standalone Stage 1 and the full two-stage pipeline. The horizontal axis identifies the Stage 1 checkpoint: the standalone curve evaluates that encoder directly, while the two-stage curve evaluates the encoder obtained after a further 15 epochs of Stage 2 training. Each point averages scores across three Alcock settings and five folds; shaded bands denote the sample standard deviation across these 15 runs. Validation selects Stage 1 epoch 47 for standalone classification and epoch 33 as the initialization for the full pipeline.

Model / Pipeline	Alcock20	Alcock100	Alcock500
Official ASTROMER 2 finetune-and-classify	$0.6933$	$0.7313$	$0.7433$
Original ASTROMER 2 encoder pooled- $h$	$0.1735$	$0.2700$	$0.3599$
Stage 1, epoch 47 (independently selected)	$0.6248$	$0.6801$	$0.7289$
Stage 1, epoch 33 (matched initialization)	$0.6056$	$0.6703$	$0.7429$
Stage 2 (from Stage 1, epoch 33)	$\mathbf{0.7078}$	$\mathbf{0.7704}$	$\mathbf{0.7943}$

Table 5: Final-refit test macro-F1 after five-fold cross-validation model and training-duration selection. Each final model is refit on the complete development pool and evaluated once on the fixed holdout.

Class	Precision	Recall	F1	Support
Cep_0	0.8977	0.7900	0.8404	100
Cep_1	0.6484	0.8300	0.7281	100
EC	0.7912	0.7200	0.7539	100
LPV	0.8353	0.7100	0.7676	100
RRab	0.8367	0.8200	0.8283	100
RRc	0.8091	0.8900	0.8476	100

Table 6: Per-class Alcock500 performance for the final-refit classifier built on the globally selected Stage 2 encoder.

Figure 5: Alcock500 confusion matrix for the final-refit classifier built on the globally selected Stage 2 encoder. Rows are true classes and columns are predictions on the fixed 600-object holdout; each row contains 100 objects.

Figure 6: Two-dimensional views of the 256-dimensional pooled- $h$ representations from the original ASTROMER 2, Stage 1 epoch 33, and Stage 2 from epoch 33 encoders (columns), using PCA, t-SNE, and UMAP (rows). Every panel contains the same class-balanced sample of 3,000 objects, with 500 objects per class, drawn from the complete 20,894-object Alcock pool. This combined sample is used only for qualitative visualization and is not treated as held-out evaluation evidence. Projection coordinates have no direct physical interpretation and should not be compared numerically across panels.

Encoder	k-NN macro-F1	Inter/intra ratio	Silhouette coefficient
Original ASTROMER 2	0.4926	1.5267	-0.1008
Stage 1, epoch 33	0.7133	1.2038	0.0101
Stage 2 from epoch 33	0.7462	1.7577	0.0926

Table 7: Direct embedding-space diagnostics evaluated on the fixed 600-object locked holdout. The k-NN classifier uses the 3,000-object development pool as its labeled reference set. Higher values are better for all three metrics.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Method	Configuration
Sampling	500 objects per class sampled without replacement from 20,894 unique Alcock objects; 3,000 objects total; random seed 42.
Preprocessing	StandardScaler fitted independently to each encoder’s 3,000 sampled embeddings.
PCA	Two components; random seed 42.
t-SNE	Two components; perplexity 30; PCA initialization; automatic learning rate; random seed 42.
UMAP	Two components; 15 nearest neighbors; minimum distance 0.1; Euclidean distance; random seed 42.

Appendix

Table A.1: Sampling, preprocessing, and dimensionality-reduction settings used to generate Figure 6 . Parameters not listed use the defaults of the recorded software versions.

Contrastive Learning

Contrastive learning trains representations by bringing related examples together and separating unrelated ones for recognition, retrieval, and transfer.

See all 335 papers

Domain-Informed Multi-View Self-Distillation for Astronomical Light-Curve Representation Learning with JEPA
Jun 26, 2026Yicheng RuiIrregular Time-Series Modeling Self-Supervised Time Series Representation Learning
Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai 201210, China
Feature-Based and Deep Learning Methods for Astronomical Light Curve Classification
Nov 3, 2026V. Baymakanov, Kh. Khamitov, A. Raimova +7Time Series Classification Astrophysics
HSE University · Southern Federal University · Lomonosov Moscow State University +3
Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts
Nov 4, 2026Eraldo Pereira MarinhoImage Classification Shortcut Learning
Institute of Geoscience and Exact Sciences, São Paulo State University (UNESP), Av. 24 A, 1515, Rio Claro, 13506-900, São Paulo, Brazil

Jun 26, 2026J/K move · Enter open · S save

Domain-Informed Multi-View Self-Distillation for Astronomical Light-Curve Representation Learning with JEPA

Yicheng Rui

Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai 201210, China

Irregular Time-Series Modeling Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture Multi-View Learning Time Series Representation Learning

Open paper AskPDF arXiv

Light curves describe temporal variations in the brightness of celestial objects. Learning robust representations of light curves is essential for large-scale automatic discovery in the dynamic universe, but existing time-series foundation models often struggle with the uneven sampling, complex noise, and wide range of physical timescales that characterize astronomical observations. We propose a domain-informed representation learning framework for irregular astronomical time series with Joint-Embedding predictive architecture (JEPA), combining semantics-preserving views, uncertainty-aware tokenization, and multi-view self-distillation. The encoders are trained with multi-view self-distillation using LeJEPA regularization on the LEAVES dataset and evaluated on the StarEmbed classification benchmark. On StarEmbed, our model outperforms hand-crafted features on 15 of 16 classification metrics. In few-shot linear probing, it achieves macro-F1 scores of 42.56

\pm

7.21 with one sample per class and 63.58

\pm

1.20 with 100 samples per class, consistently improving over hand-crafted features. Beyond variable-star classification, the learned representation supports similarity search, parameter estimation, and photometric zero-point drift detection. We further evaluate cross-domain adaptation on 12 heterogeneous irregular time-series datasets from PYRREGULAR, where the adapted variant matches or exceeds previous state-of-the-art performance on 5 datasets, compared with at most 3 wins by any single prior baseline. These results demonstrate that domain-informed multi-view self-distillation is an effective strategy for learning representations of irregular time series, while also highlighting that successful time-series representation learning requires domain-specific inductive biases rather than a universally optimal architecture.

Abstract

Contrastive Learning

Explore similar work

Domain-Informed Multi-View Self-Distillation for Astronomical Light-Curve Representation Learning with JEPA

Feature-Based and Deep Learning Methods for Astronomical Light Curve Classification

Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts