TEX-MEX: Similarity-Guided Temporal Contrastive Learning for Time Series

Aditya Radhakrishnan; Maxwell A. Xu; Wanting Mao; Santosh Kumar; Harish Haresamudram; James M. Rehg

cs.LGOct 14, 2026

TEX-MEX: Similarity-Guided Temporal Contrastive Learning for Time Series

Authors: Aditya Radhakrishnan, Maxwell A. Xu, Wanting Mao, Santosh Kumar, Harish Haresamudram, James M. Rehg

Organizations: Health Care Engineering Systems Center, University of Illinois Urbana-Champaign · Google · Department of Computer Science, University of Memphis

Abstract

Contrastive learning for time series depends on deciding which pairs of inputs should be treated as similar and dissimilar. Because few augmentations reliably preserve meaning in time series, prior methods have relied on two disjoint notions of similarity between windows: structural similarity, which relates windows with similar morphology, and temporal similarity, which relates windows that occur close together in time. We introduce TEX-MEX, to our knowledge the first approach to unify the two into a single relationship. Given a random batch of windows drawn across the dataset, TEX-MEX uses the encoder to identify which windows are structurally similar or dissimilar to each other. It then pulls each window toward the temporally shifted versions of similar windows and pushes it away from those of dissimilar ones, learning what typically accompanies similar windows in time. Across PPG, EEG, ECG, accelerometry, and continuous glucose monitoring, TEX-MEX broadly achieves state-of-the-art performance against contrastive, reconstructive, JEPA-based, and modality-specific approaches, establishing it as a strong general SSL objective. We further train a 560K-parameter PPG foundation model that achieves state-of-the-art performance across a broad range of downstream benchmarks while being roughly 50x smaller than the next-best unimodally pretrained PPG foundation model in our evaluation. We will release this foundation model open-source and open-weight to the community to support future work and practical applications.

Figures & tables

Figure 1: Overview of TEX-MEX. (1) Windows of sensor data are randomly sampled from the dataset; (2) The encoder scores each window’s similarity to an anchor (here,

x_{1}

) using cosine similarity; (3) Each window is offset in time (forward or backward) by a random

\delta

(here,

+3

windows). The anchor is pulled toward the temporal shifts of similar windows and away from those of dissimilar ones, in proportion to the similarity, composing structural and temporal notions.

Figure 2: Components of the TEX-MEX loss , shown for anchor $x_{1}$ with a batch of five windows and $\delta=+3$ . ( $w$ ) The anchor’s cosine similarity to each window is turned by two softmaxes into positive weights $w^{+}$ , which favor similar windows, and negative weights $w^{-}$ , which favor dissimilar ones. ( $P$ ) The shifted windows, weighted by $w^{+}$ , are attracted to the anchor. ( $N$ ) The same shifted windows, weighted by $w^{-}$ , are repelled. Every shifted window therefore counts as partly positive and partly negative, according to how related it is. ( $H$ ) The unshifted versions of the other windows, weighted by $w^{+}$ , are also repelled. This stops the encoder from ignoring time by mapping a window and its shift to the same point. The loss is $-\log\bigl(P/(P+N+H)\bigr)$ .

Figure 3: Comparison of signal distance measures by mutual class membership. On the held-out set, we sample 100 batches of 64 windows per domain and, for each anchor, retrieve its

k\in\{1,3,5\}

nearest neighbors according to each distance measure, reporting the proportion sharing the anchor’s ground-truth class. Bars show the unweighted mean across PPG, EEG, ECG, and HAR, with colored points indicating individual domains. TEX-MEX is evaluated at 1 epoch, 50% of training, and after training, and compared with REBAR and raw-signal DTW and

L_{2}

distances. Higher values indicate that a distance measure more reliably identifies semantically similar signals.

		AUROC $\uparrow$						PRAUC $\uparrow$
		Home		Venous		Ven. $\rightarrow$ Home		Home		Venous		Ven. $\rightarrow$ Home
	Method	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR
Muhammad et al. (2026)	PCA ( Shlens, 2014 )	.925	.821	.790	.744	.927	.827	.909	.842	.784	.770	.911	.847
	GluFormer ( Lutsker et al., 2025 )	.857	.889	.742	.709	.801	.530	.809	.880	.703	.758	.756	.555
	TS2Vec ( Yue et al., 2022 )	.854	.848	.735	.733	.805	.713	.880	.868	.716	.741	.754	.720
	MOMENT S ( Goswami et al., 2024 )	.849	.819	.712	.674	.736	.830	.834	.829	.668	.671	.751	.852

Table 2: Linear probe evaluation for CGM metabolic subphenotyping across three regimes and two endpoints. All results are mean over 20 seeds.

	AUROC $\uparrow$						PRAUC $\uparrow$
	Home		Venous		Ven. $\rightarrow$ Home		Home		Venous		Ven. $\rightarrow$ Home
Variant	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR	$\beta$ -cell	IR
TEX-MEX	.980	.952	.855	.796	.978	.952	.972	.946	.882	.848	.967	.943
w/o graded weighting	$-$ .042	$-$ .097	$-$ .049	$+$ .001	$-$ .171	$-$ .142	$-$ .046	$-$ .090	$-$ .118	$-$ .008	$-$ .159	$-$ .125
w/o frame separation $H$	$-$ .018	$-$ .066	$+$ .018	$+$ .004	$-$ .012	$-$ .057	$-$ .020	$-$ .044	$-$ .010	$-$ .015	$-$ .010	$-$ .033
w/o stop-gradient	$-$ .037	$-$ .079	$+$ .019	$-$ .010	$-$ .025	$-$ .069	$-$ .034	$-$ .052	$-$ .009	$-$ .019	$-$ .025	$-$ .047

Table 3: Ablation of TEX-MEX components on CGM metabolic subphenotyping. The top row reports absolute performance; all other rows report the change relative to it. Values are means over 20 seeds.

			General TSFMs			PPG FMs
			MOMENT	Chronos	Chronos-2	Pulse-PPG	PaPaGei-S	PaPaGei-P	AnyPPG	TEX-MEX
	Params (M)		341.25	201.37	119.48	28.50	5.79	4.99	4.04	0.56
	Embedding dim		1024	768	768	512	512	512	512	96
	Pretraining data		Gen. TS	Gen. TS	Gen. TS	PPG	PPG	PPG	PPG+ECG	PPG
PPG-BP	Hypertension	AUROC $\uparrow$	.6749	.6353	.6659	.6487	.6373	.6357	.6739	.7347
	Hypertension	PRAUC $\uparrow$	.3951	.3642	.4103	.3848	.3573	.3567	.4250	.4960

Table 4: Downstream performance of the TEX-MEX PPG foundation model against general time-series and PPG-specific foundation models. All results are mean over 15 seeds of 5-fold cross-validation. Bold is best, underline second best.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Hyperparameter	PPG	ECG	HAR	EEG
TEX-MEX	$\sigma$	1	8	8	2
	$\tau$	0.03	0.04	0.03	0.04
	$\tau_{w}$	0.06	0.14	0.10	0.10
SoftCLT	$\tau_{I}$	10	4	20	5
SoftCLT	$\tau_{T}$	0.5	1.0	2.0	2.5
TNC	$w$	0	0	0.1	0.05

Appendix

Table 5: Final method-specific hyperparameters selected on the validation set for each domain.

Method	Hyperparameter	PPG	ECG	HAR	EEG
TEX-MEX	Epochs	100	200	50	100
	Learning rate	$10^{-4}$	$10^{-4}$	$10^{-4}$	$10^{-4}$
	Weight decay	$10^{-5}$	$10^{-5}$	$10^{-5}$	$10^{-4}$
SoftCLT	Epochs	200	100	200	200
	Learning rate	$5{\times}10^{-4}$	$10^{-4}$	$5{\times}10^{-4}$	$10^{-3}$
	Weight decay	$10^{-4}$	$10^{-4}$	$10^{-4}$	$10^{-4}$

Appendix

Table 6: Selected shared training hyperparameters. Following Chen et al. (2026) , all three methods draw from the same grid (epochs

\in\{50,100,200\}

, learning rate

\in\{10^{-3},5{\times}10^{-4},10^{-4}\}

, weight decay

\in\{10^{-3},10^{-4},10^{-5}\}

, batch size

\in\{64,128\}

). A batch size of 64 was selected for all methods and domains, except TEX-MEX on HAR, which selected 128.

Encoder
Architecture	Stem conv + 6 residual blocks
Channels	$48\to 96$
Kernel size	11
Downsampling	Stride 2 every second block ( $8\times$ total)
Dropout	0.5
Pooling	Global average

Appendix

Table 7: PPG foundation model architecture and pretraining hyperparameters.

Self-Supervised Time Series Representation Learning

Self-supervised time series representation learning uses unlabeled sequences to learn features that transfer to forecasting, classification, and anomaly detection.

See all 143 papers

UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification
Mar 2, 2026Yessin Moakher, Youssef Attia El Hili, Vasilii FeofanovSelf-Supervised Time Series Representation Learning Time-Series Foundation Models
Ecole Polytechnique, France · Huawei Noah’s Ark Lab
Learning by Shifting: Temporal View Construction for Time Series Contrastive Learning
Jun 20, 2026Abdul-Kazeem Shamba, Kerstin Bach, Gavin TaylorContrastive Learning Time Series Classification
Norwegian University of Science and Technology, Norway · United States Naval Academy, USA
CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding
Jul 10, 2026Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian +2Contrastive Learning Hybrid CNN-Transformer Architectures
Sigma Nova Paris, France · Goethe University Frankfurt Frankfurt, Germany

Mar 2, 2026J/K move · Enter open · S save

UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification

Yessin Moakher, Youssef Attia El Hili, Vasilii Feofanov

Ecole Polytechnique, France · Huawei Noah’s Ark Lab

Self-Supervised Time Series Representation Learning Time-Series Foundation Models Time Series Classification Self-Supervised Learning Self-Distillation

Open paper AskPDF arXiv

Self-supervised foundation models have achieved remarkable success across domains, including time series. However, the potential of non-contrastive methods, a paradigm that has driven significant advances in computer vision, remains underexplored for time series. In this work, we adapt DINOv2-style self-distillation to pretrain a time series foundation model, building on the Mantis tokenizer and transformer encoder architecture as our backbone. Through a student-teacher framework, our method Utica learns representations that capture both temporal invariance via augmented crops and fine-grained local structure via patch masking. Our approach achieves state-of-the-art classification performance on both UCR and UEA benchmarks. These results suggest that non-contrastive methods are a promising and complementary pretraining strategy for time series foundation models.

Abstract

Self-Supervised Time Series Representation Learning

Explore similar work

UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification

Learning by Shifting: Temporal View Construction for Time Series Contrastive Learning

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding