DISC: Domain-Integrated Similarity Cost for Time-Series Dataset

Yen-Ku Liu; Hongjie Chen; Franck Dernoncourt; Ryan Rossi

cs.LGOct 22, 2026

DISC: Domain-Integrated Similarity Cost for Time-Series Dataset

Authors: Yen-Ku Liu, Hongjie Chen, Franck Dernoncourt, Ryan Rossi

Organizations: National Yang Ming Chiao Tung University · Dolby Laboratories · Adobe Research

Abstract

The growth of time-series datasets and foundation models has made dataset comparison a central part of time-series workflows. Assessing source-target transfer before model adaptation requires similarity estimates that can summarize transfer-related dataset relationships without exhaustive source-target evaluation over every candidate pair. Many metrics have been proposed for measuring similarity between time-series datasets, but they capture different dataset relationships and vary in how well they align with downstream utility across tasks. We evaluate this gap by measuring the correlation between source-target similarity scores and downstream forecasting errors in out-of-distribution and transfer-learning settings. We propose Domain-Integrated Similarity Cost (DISC), a learnable aggregation framework that uses calibration transfer observations to select, weight, and combine similarity metrics for different downstream tasks. By integrating complementary metric values, DISC better distinguishes source-target dataset differences that matter for downstream behavior. After calibration, DISC scores source-target pairs using only their similarity-metric values and yields higher alignment than single-metric references on the reported split. These results show that time-series dataset similarity is task-dependent, and that learned metric integration provides a stronger proxy for downstream transfer behavior than manually selecting one similarity metric.

Figures & tables

Fig. 1: Pairwise dataset distances under four similarity metrics. Darker cells indicate smaller distance and therefore higher metric-implied similarity. The panels illustrate why single-metric similarity is insufficient: WSD captures a broad related group, MAD sharpens only part of that relation, DBA+LCSS reveals additional temporal-shape similarities, and MMD rejects a spurious group that the other metrics rank as close. This motivates learning an integrated similarity cost that combines complementary evidence while avoiding over-reliance on any single metric.

Fig. 2: Overview of Domain-Integrated Similarity Cost (DISC). Given candidate similarity metrics

\{s_{m}(X,Y)\}_{m\in\mathcal{M}}

, DISC learns a task-specific base mixture

B(X,Y)

from standardized metric values

z_{m}(X,Y)

and a prior mixture

P(X,Y)

from source-wise metric ranks

r_{m}(X,Y)

. A learned gate

G(X,Y)

uses the prior mixture to modulate the base mixture and produce the final score

S_{\theta}(X,Y)=B(X,Y)G(X,Y)

. The base and prior mixtures are detailed in Section IV-B , and the gated integration is detailed in Section IV-C .

Hyperparameter	Value	Hyperparameter	Value
Optimizer	Adam	Learning rate	0.03
Epochs	500	Patience	100
Restarts	8	Softmax temperature $\tau$	0.55
Max gate modulation	3.0	Gate threshold range	$[0,1]$
Gate sharpness range	$[1,55]$	Global-correlation weight $\lambda$	0.10
Entropy weight $\eta$	0.006	Overlap weight $\mu$	0.02

TABLE I: DISC training hyperparameters used in the main experiment.

Task	Correlation (gain over OT)
Task	OT	WSD	MMD	MAD	DBA-DTW	DISC (ours)
OOD-TM	-0.007 (ref.)	0.445 ( $+0.453\uparrow$ )	-0.057 ( $-0.050\downarrow$ )	0.580 ( $+0.588\uparrow$ )	-0.016 ( $-0.008\downarrow$ )	0.601 ( $+0.609\uparrow$ )
OOD-LL	0.125 (ref.)	0.361 ( $+0.235\uparrow$ )	0.295 ( $+0.169\uparrow$ )	0.231 ( $+0.105\uparrow$ )	0.618 ( $+0.492\uparrow$ )	0.697 ( $+0.571\uparrow$ )
TL-TM	0.213 (ref.)	0.516 ( $+0.303\uparrow$ )	0.303 ( $+0.091\uparrow$ )	0.505 ( $+0.292\uparrow$ )	0.134 ( $-0.079\downarrow$ )	0.617 ( $+0.404\uparrow$ )
TL-LL	0.080 (ref.)	0.011 ( $-0.069\downarrow$ )	0.435 ( $+0.355\uparrow$ )	0.146 ( $+0.066\uparrow$ )	0.302 ( $+0.222\uparrow$ )	0.479 ( $+0.399\uparrow$ )
Avg	0.103 (ref.)	0.333 ( $+0.231\uparrow$ )	0.244 ( $+0.141\uparrow$ )	0.366 ( $+0.263\uparrow$ )	0.259 ( $+0.157\uparrow$ )	0.598 ( $+0.496\uparrow$ )

TABLE II: Correlation between similarity value and downstream task error for single metrics and DISC. OT is the reference; parentheses show gains over OT, bold marks the best value, and italics mark the second-best single metric.

Task	Correlation (gain over OT)
Task	OT	DBA-ED	DBA-LCSS	PCA-DTW	PCA-ED	PCA-LCSS	DISC (ours)
OOD-TM	-0.007 (ref.)	-0.016 ( $-0.008\downarrow$ )	0.000 ( $+0.007\uparrow$ )	0.381 ( $+0.389\uparrow$ )	0.416 ( $+0.423\uparrow$ )	0.048 ( $+0.055\uparrow$ )	0.601 ( $+0.609\uparrow$ )
OOD-LL	0.125 (ref.)	0.618 ( $+0.492\uparrow$ )	0.052 ( $-0.074\downarrow$ )	0.577 ( $+0.452\uparrow$ )	0.580 ( $+0.454\uparrow$ )	0.461 ( $+0.336\uparrow$ )	0.697 ( $+0.571\uparrow$ )
TL-TM	0.213 (ref.)	0.134 ( $-0.079\downarrow$ )	0.026 ( $-0.187\downarrow$ )	0.482 ( $+0.269\uparrow$ )	0.537 ( $+0.325\uparrow$ )	-0.025 ( $-0.238\downarrow$ )	0.617 ( $+0.404\uparrow$ )
TL-LL	0.080 (ref.)	0.302 ( $+0.222\uparrow$ )	0.000 ( $-0.080\downarrow$ )	0.113 ( $+0.033\uparrow$ )	0.078 ( $-0.002\downarrow$ )	0.107 ( $+0.027\uparrow$ )	0.479 ( $+0.399\uparrow$ )
Avg	0.103 (ref.)	0.259 ( $+0.157\uparrow$ )	0.019 ( $-0.083\downarrow$ )	0.388 ( $+0.286\uparrow$ )	0.403 ( $+0.300\uparrow$ )	0.148 ( $+0.045\uparrow$ )	0.598 ( $+0.496\uparrow$ )

TABLE III: Correlation between similarity value and downstream task error for single metrics and DISC (cont.). OT is the reference; parentheses show gains over OT, bold marks the best value, and italics mark the second-best single metric.

Fig. 3: Source-wise DISC examples for RQ2. Each panel fixes the test source dataset with the highest DISC source-wise Pearson correlation for one downstream task: (a) m3_monthly , (b) electricity_weekly , (c) wiki2000_nips , and (d) fred_md . The x-axis is the learned DISC similarity score and the y-axis is downstream forecasting error. Upward trends show that datasets assigned larger DISC scores tend to produce larger errors.

Task		Best single	Single R $\uparrow$	DISC R $\uparrow$	Relative change $\uparrow$
Protocol	Model	Best single	Single R $\uparrow$	DISC R $\uparrow$	Relative change $\uparrow$
Out-of-distribution	Time-MoE	MAD	0.580	0.601	+3.55%
Out-of-distribution	Lag-Llama	DBA-DTW	0.618	0.697	+12.84%
Transfer learning	Time-MoE	PCA-ED	0.537	0.617	+14.75%
Transfer learning	Lag-Llama	MMD	0.435	0.479	+10.15%
Average		–	0.543	0.598	+10.29%

TABLE IV: DISC compared with the best single metric for each downstream task on the reported split. R denotes Pearson correlation between similarity score and downstream task error; higher values indicate stronger alignment.

Task		Base metrics	Prior metrics	Prior modulation
Protocol	Model
Out-of-distribution	Time-MoE	DBA-LCSS (.549), MAD (.402) Wasserstein (.022)	PCA-ED (.717), Wasserstein (.173) DBA-ED (.045), DBA-DTW (.037), PCA-LCSS (.012)	2.26 $\times$ [1.00, 3.35]
	Lag-Llama	PCA-DTW (.612) PCA-LCSS (.383)	PCA-LCSS (.935) DBA-LCSS (.062)	1.21 $\times$ [1.00, 3.79]
Transfer learning	Time-MoE	MAD (.916) Wasserstein (.067)	PCA-ED (.719), DBA-ED (.180) DBA-DTW (.076), MMD (.017)	1.17 $\times$ [1.00, 3.81]
	Lag-Llama	MMD (.833) DBA-ED (.159)	DBA-DTW (.362), MAD (.324) PCA-ED (.312)	2.89 $\times$ [1.00, 3.95]

TABLE V: Learned DISC aggregation for each downstream task. Base and prior columns list selected metrics with learned softmax weights in parentheses; metrics below 0.01 are omitted. Prior modulation reports the mean and range of the positive gate factor multiplied with the base score on test-split pairs.

Task	Manual R	DISC R	Gain
OOD-TM	-0.066	0.601	N/A
OOD-LL	0.571	0.697	+21.98%
TL-TM	0.316	0.617	+95.13%
TL-LL	0.344	0.479	+39.32%
Avg	0.291	0.598	+105.38%

TABLE VI: Task-wise ablation against non-learned manual search. R denotes Pearson correlation between similarity distance and downstream task error. Manual R and DISC R are test-split correlations after parameter selection on training and validation datasets; Gain is DISC’s relative improvement, with N/A used for a negative manual baseline.

Group	Lag-1	Zero	Diversity
Low corr. datasets	0.841	0.191	3.935
High corr. datasets	0.556	0.061	5.115

TABLE VII: Sample-window diagnostics for low- and high-correlation groups. Lag-1 is adjacent-step autocorrelation, zero is the zero-valued sample fraction, and diversity is mean pairwise distance between z-normalized windows.

Task Transferability Estimation

Task transferability estimation predicts which models, datasets, tasks, or skills will help on a new target before costly adaptation.

See all 38 papers

TSDS-Toolbox: A Toolbox for Measuring Time-Series Dataset Similarity
Aug 8, 2026Yen-Ku Liu, Hongjie Chen, Ryan A. Rossi +1Training Data Selection
National Yang Ming Chiao Tung University Hsinchu, Taiwan · Dolby Laboratories Atlanta, Georgia, USA · Adobe Research San Jose, California, USA +1
A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations
Sep 30, 2026Romain Mussard, Fannia Pacheco, Maxime Berar +2Source-Free Domain Adaptation Domain Adaptation
Univ Rouen Normandie, INSA Rouen Normandie, Universit´e Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France
RADAR: Relative Angular Divergence Across Representations
May 21, 2026Xavier Cadet, Mateusz Nowak, Peter ChinRepresentation Geometry Task Transferability Estimation
Dartmouth College

Aug 8, 2026J/K move · Enter open · S save

TSDS-Toolbox: A Toolbox for Measuring Time-Series Dataset Similarity

Yen-Ku Liu, Hongjie Chen, Ryan A. Rossi, Franck Dernoncourt

National Yang Ming Chiao Tung University Hsinchu, Taiwan · Dolby Laboratories Atlanta, Georgia, USA · Adobe Research San Jose, California, USA+1

Training Data Selection

Open paper AskPDF arXiv

The rapid advancement of artificial intelligence (AI) has significantly accelerated research in time-series analysis, particularly in forecasting, classification, and generation tasks. Recent models, especially foundation models, benefit from time-series dataset similarity due to its significant role in source dataset selection for fine-tuning. However, many existing implementations for benchmarking time-series dataset similarity methods are fragmented and difficult to extend. To address this, we present a unified framework, the Time-Series Dataset Similarity Toolbox (TSDS-Toolbox). Our work enables (1) systematic and reproducible comparisons of time-series dataset similarity methods; (2) flexible extensibility for users to add customized datasets, similarity methods, and downstream time-series tasks; and (3) consistent evaluation of both dataset-level and series-level similarity methods through integrated time-series dataset reducers. The effectiveness of TSDS-Toolbox is validated through comprehensive experiments under diverse experimental settings. Our toolbox is publicly available.

Abstract

Task Transferability Estimation

Explore similar work

TSDS-Toolbox: A Toolbox for Measuring Time-Series Dataset Similarity

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

RADAR: Relative Angular Divergence Across Representations