DISC: Domain-Integrated Similarity Cost for Time-Series Dataset

Yen-Ku Liu; Hongjie Chen; Franck Dernoncourt; Ryan Rossi

cs.LGOct 22, 2026

DISC: Domain-Integrated Similarity Cost for Time-Series Dataset

Authors: Yen-Ku Liu, Hongjie Chen, Franck Dernoncourt, Ryan Rossi

Organizations: National Yang Ming Chiao Tung University · Dolby Laboratories · Adobe Research

Abstract

The growth of time-series datasets and foundation models has made dataset comparison a central part of time-series workflows. Assessing source-target transfer before model adaptation requires similarity estimates that can summarize transfer-related dataset relationships without exhaustive source-target evaluation over every candidate pair. Many metrics have been proposed for measuring similarity between time-series datasets, but they capture different dataset relationships and vary in how well they align with downstream utility across tasks. We evaluate this gap by measuring the correlation between source-target similarity scores and downstream forecasting errors in out-of-distribution and transfer-learning settings. We propose Domain-Integrated Similarity Cost (DISC), a learnable aggregation framework that uses calibration transfer observations to select, weight, and combine similarity metrics for different downstream tasks. By integrating complementary metric values, DISC better distinguishes source-target dataset differences that matter for downstream behavior. After calibration, DISC scores source-target pairs using only their similarity-metric values and yields higher alignment than single-metric references on the reported split. These results show that time-series dataset similarity is task-dependent, and that learned metric integration provides a stronger proxy for downstream transfer behavior than manually selecting one similarity metric.

Figures & tables

Fig. 1: Pairwise dataset distances under four similarity metrics. Darker cells indicate smaller distance and therefore higher metric-implied similarity. The panels illustrate why single-metric similarity is insufficient: WSD captures a broad related group, MAD sharpens only part of that relation, DBA+LCSS reveals additional temporal-shape similarities, and MMD rejects a spurious group that the other metrics rank as close. This motivates learning an integrated similarity cost that combines complementary evidence while avoiding over-reliance on any single metric.

Fig. 2: Overview of Domain-Integrated Similarity Cost (DISC). Given candidate similarity metrics

\{s_{m}(X,Y)\}_{m\in\mathcal{M}}

, DISC learns a task-specific base mixture

B(X,Y)

from standardized metric values

z_{m}(X,Y)

and a prior mixture

P(X,Y)

from source-wise metric ranks

r_{m}(X,Y)

. A learned gate

G(X,Y)

uses the prior mixture to modulate the base mixture and produce the final score

S_{\theta}(X,Y)=B(X,Y)G(X,Y)

. The base and prior mixtures are detailed in Section IV-B , and the gated integration is detailed in Section IV-C .

Hyperparameter	Value	Hyperparameter	Value
Optimizer	Adam	Learning rate	0.03
Epochs	500	Patience	100
Restarts	8	Softmax temperature $\tau$	0.55
Max gate modulation	3.0	Gate threshold range	$[0,1]$
Gate sharpness range	$[1,55]$	Global-correlation weight $\lambda$	0.10
Entropy weight $\eta$	0.006	Overlap weight $\mu$	0.02

TABLE I: DISC training hyperparameters used in the main experiment.

Task	Correlation (gain over OT)
Task	OT	WSD	MMD	MAD	DBA-DTW	DISC (ours)
OOD-TM	-0.007 (ref.)	0.445 ( $+0.453\uparrow$ )	-0.057 ( $-0.050\downarrow$ )	0.580 ( $+0.588\uparrow$ )	-0.016 ( $-0.008\downarrow$ )	0.601 ( $+0.609\uparrow$ )
OOD-LL	0.125 (ref.)	0.361 ( $+0.235\uparrow$ )	0.295 ( $+0.169\uparrow$ )	0.231 ( $+0.105\uparrow$ )	0.618 ( $+0.492\uparrow$ )	0.697 ( $+0.571\uparrow$ )
TL-TM	0.213 (ref.)	0.516 ( $+0.303\uparrow$ )	0.303 ( $+0.091\uparrow$ )	0.505 ( $+0.292\uparrow$ )	0.134 ( $-0.079\downarrow$ )	0.617 ( $+0.404\uparrow$ )
TL-LL	0.080 (ref.)	0.011 ( $-0.069\downarrow$ )	0.435 ( $+0.355\uparrow$ )	0.146 ( $+0.066\uparrow$ )	0.302 ( $+0.222\uparrow$ )	0.479 ( $+0.399\uparrow$ )
Avg	0.103 (ref.)	0.333 ( $+0.231\uparrow$ )	0.244 ( $+0.141\uparrow$ )	0.366 ( $+0.263\uparrow$ )	0.259 ( $+0.157\uparrow$ )	0.598 ( $+0.496\uparrow$ )

TABLE II: Correlation between similarity value and downstream task error for single metrics and DISC. OT is the reference; parentheses show gains over OT, bold marks the best value, and italics mark the second-best single metric.

Task	Correlation (gain over OT)
Task	OT	DBA-ED	DBA-LCSS	PCA-DTW	PCA-ED	PCA-LCSS	DISC (ours)
OOD-TM	-0.007 (ref.)	-0.016 ( $-0.008\downarrow$ )	0.000 ( $+0.007\uparrow$ )	0.381 ( $+0.389\uparrow$ )	0.416 ( $+0.423\uparrow$ )	0.048 ( $+0.055\uparrow$ )	0.601 ( $+0.609\uparrow$ )
OOD-LL	0.125 (ref.)	0.618 ( $+0.492\uparrow$ )	0.052 ( $-0.074\downarrow$ )	0.577 ( $+0.452\uparrow$ )	0.580 ( $+0.454\uparrow$ )	0.461 ( $+0.336\uparrow$ )	0.697 ( $+0.571\uparrow$ )
TL-TM	0.213 (ref.)	0.134 ( $-0.079\downarrow$ )	0.026 ( $-0.187\downarrow$ )	0.482 ( $+0.269\uparrow$ )	0.537 ( $+0.325\uparrow$ )	-0.025 ( $-0.238\downarrow$ )	0.617 ( $+0.404\uparrow$ )
TL-LL	0.080 (ref.)	0.302 ( $+0.222\uparrow$ )	0.000 ( $-0.080\downarrow$ )	0.113 ( $+0.033\uparrow$ )	0.078 ( $-0.002\downarrow$ )	0.107 ( $+0.027\uparrow$ )	0.479 ( $+0.399\uparrow$ )
Avg	0.103 (ref.)	0.259 ( $+0.157\uparrow$ )	0.019 ( $-0.083\downarrow$ )	0.388 ( $+0.286\uparrow$ )	0.403 ( $+0.300\uparrow$ )	0.148 ( $+0.045\uparrow$ )	0.598 ( $+0.496\uparrow$ )

TABLE III: Correlation between similarity value and downstream task error for single metrics and DISC (cont.). OT is the reference; parentheses show gains over OT, bold marks the best value, and italics mark the second-best single metric.

Fig. 3: Source-wise DISC examples for RQ2. Each panel fixes the test source dataset with the highest DISC source-wise Pearson correlation for one downstream task: (a) m3_monthly , (b) electricity_weekly , (c) wiki2000_nips , and (d) fred_md . The x-axis is the learned DISC similarity score and the y-axis is downstream forecasting error. Upward trends show that datasets assigned larger DISC scores tend to produce larger errors.

Task		Best single	Single R $\uparrow$	DISC R $\uparrow$	Relative change $\uparrow$
Protocol	Model	Best single	Single R $\uparrow$	DISC R $\uparrow$	Relative change $\uparrow$
Out-of-distribution	Time-MoE	MAD	0.580	0.601	+3.55%
Out-of-distribution	Lag-Llama	DBA-DTW	0.618	0.697	+12.84%
Transfer learning	Time-MoE	PCA-ED	0.537	0.617	+14.75%
Transfer learning	Lag-Llama	MMD	0.435	0.479	+10.15%
Average		–	0.543	0.598	+10.29%

TABLE IV: DISC compared with the best single metric for each downstream task on the reported split. R denotes Pearson correlation between similarity score and downstream task error; higher values indicate stronger alignment.

Task		Base metrics	Prior metrics	Prior modulation
Protocol	Model
Out-of-distribution	Time-MoE	DBA-LCSS (.549), MAD (.402) Wasserstein (.022)	PCA-ED (.717), Wasserstein (.173) DBA-ED (.045), DBA-DTW (.037), PCA-LCSS (.012)	2.26 $\times$ [1.00, 3.35]
	Lag-Llama	PCA-DTW (.612) PCA-LCSS (.383)	PCA-LCSS (.935) DBA-LCSS (.062)	1.21 $\times$ [1.00, 3.79]
Transfer learning	Time-MoE	MAD (.916) Wasserstein (.067)	PCA-ED (.719), DBA-ED (.180) DBA-DTW (.076), MMD (.017)	1.17 $\times$ [1.00, 3.81]
	Lag-Llama	MMD (.833) DBA-ED (.159)	DBA-DTW (.362), MAD (.324) PCA-ED (.312)	2.89 $\times$ [1.00, 3.95]

TABLE V: Learned DISC aggregation for each downstream task. Base and prior columns list selected metrics with learned softmax weights in parentheses; metrics below 0.01 are omitted. Prior modulation reports the mean and range of the positive gate factor multiplied with the base score on test-split pairs.

Task	Manual R	DISC R	Gain
OOD-TM	-0.066	0.601	N/A
OOD-LL	0.571	0.697	+21.98%
TL-TM	0.316	0.617	+95.13%
TL-LL	0.344	0.479	+39.32%
Avg	0.291	0.598	+105.38%

TABLE VI: Task-wise ablation against non-learned manual search. R denotes Pearson correlation between similarity distance and downstream task error. Manual R and DISC R are test-split correlations after parameter selection on training and validation datasets; Gain is DISC’s relative improvement, with N/A used for a negative manual baseline.

Group	Lag-1	Zero	Diversity
Low corr. datasets	0.841	0.191	3.935
High corr. datasets	0.556	0.061	5.115

TABLE VII: Sample-window diagnostics for low- and high-correlation groups. Lag-1 is adjacent-step autocorrelation, zero is the zero-valued sample fraction, and diversity is mean pairwise distance between z-normalized windows.

Task Transferability Estimation

Task transferability estimation predicts which models, datasets, tasks, or skills will help on a new target before costly adaptation.

See all 38 papers

Aug 8, 2026

TSDS-Toolbox: A Toolbox for Measuring Time-Series Dataset Similarity

The rapid advancement of artificial intelligence (AI) has significantly accelerated research in time-series analysis, particularly in forecasting, classification, and generation tasks. Recent models, especially foundation models, benefit from time-series dataset similarity due to its significant role in source dataset selection for fine-tuning. However, many existing implementations for benchmarking time-series dataset similarity methods are fragmented and difficult to extend. To address this, we present a unified framework, the Time-Series Dataset Similarity Toolbox (TSDS-Toolbox). Our work enables (1) systematic and reproducible comparisons of time-series dataset similarity methods; (2) flexible extensibility for users to add customized datasets, similarity methods, and downstream time-series tasks; and (3) consistent evaluation of both dataset-level and series-level similarity methods through integrated time-series dataset reducers. The effectiveness of TSDS-Toolbox is validated through comprehensive experiments under diverse experimental settings. Our toolbox is publicly available.

Yen-Ku Liu, Hongjie Chen, Ryan A. Rossi +1

National Yang Ming Chiao Tung University Hsinchu, Taiwan · Dolby Laboratories Atlanta, Georgia, USA · Adobe Research San Jose, California, USA +1

Training Data Selection

Ask PDF

Sep 30, 2026

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

Source-Free Universal Domain Adaptation (SF-UniDA) extends Universal Domain Adaptation by removing access to source data at adaptation time while still handling label-set mismatches between domains. Despite growing interest in this setting for image data, no benchmark exists for time series, which are more challenging. We present the first SF-UniDA benchmark on time series. In addition, we provide the first study of pretrained foundation models as feature extractors for time series domain adaptation. In this context, we identify a critical and previously underexplored limitation of all existing SF-UniDA methods: the inference threshold for unknown-sample rejection is highly sensitive. We address this by proposing a plug-in auto-thresholding module that can be integrated into any SF-UniDA method. Experiments on three well-known time series datasets confirm the suitability of this module. They also highlight that foundation models do not systematically outperform classical backbones and that SF-UniDA tailored for time series is yet to be developed.

Romain Mussard, Fannia Pacheco, Maxime Berar +2

Univ Rouen Normandie, INSA Rouen Normandie, Universit´e Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France

Source-Free Domain Adaptation Domain Adaptation Time Series Representation Learning Time-Series Foundation Models

Ask PDF

May 21, 2026

RADAR: Relative Angular Divergence Across Representations

Machine learning methods rely on data. However, gathering suitable data can be challenging due to availability constraints, cost, or the need for domain expertise. Expanding datasets with additional sources is a common response to limited data, yet this practice does not always improve downstream performance and can sometimes lead to a loss of performance, known as negative transfer. We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models. RADAR analyzes the layer-wise evolution of representations by measuring angular alignments and relative changes in distance along layer-to-layer displacement trajectories, and by comparing empirical distributions of within-domain and cross-domain dynamics. We hypothesize that domain transferability is related to the divergence between these trajectory distributions. We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text embedding models and cross-domain image classification with foundation vision models. Across several settings, RADAR provides competitive predictive performance relative to existing transferability metrics on several vision and text benchmarks, with particularly strong results when domain transitions are smooth or cleanly separated. Our ablations further suggest that the effectiveness of transferability estimation depends on the geometry of the model's internal representation space, with different modalities favoring different topological formulations.

Xavier Cadet, Mateusz Nowak, Peter Chin

Dartmouth College

Representation Geometry Task Transferability Estimation Cross-Domain Transfer Learning Transfer Learning

Ask PDF

Abstract

Task Transferability Estimation

Explore similar work

TSDS-Toolbox: A Toolbox for Measuring Time-Series Dataset Similarity

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

RADAR: Relative Angular Divergence Across Representations