Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series
Organizations: Engineering Product Development (EPD), SUTD, Singapore · Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore
Abstract
Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.
Figures & tables
| Domain | Dataset | Task | Signals | Samples | Median len | Sampling |
|---|---|---|---|---|---|---|
| Battery | NASA [ 17 ] | RUL reg. | voltage, current | 499 | 0.1 Hz | |
| temperature | ||||||
| Semiconductor | CMP [ 14 ] | MRR reg. | pressure, flowrate | 2321 | 1 Hz | |
| rotation rate, usage | ||||||
| Machining | CNC [ 15 ] | tool wear reg. | vibration (tri-axis), | 225 | 25.6 kHz | |
| sound |
| NASA | CMP | CNC | PU | # Params | |
| Model | (RMSE) | (RMSE) | (RMSE) | (macro-F1) | (M) |
| MiniRocket [ 6 ] | 29.44 1.01 | 4.02 0.05 | 28.97 3.82 | 0.4962 0.01 | — |
| PatchTST [ 13 ] | 34.84 4.37 | 8.39 0.36 | 12.87 2.51 | 0.1851 0.00 | 0.135 |
| MOMENT-small [ 10 ] | 40.53 5.61 | 5.34 0.03 | 12.95 1.10 | 0.5824 0.04 | 37.9 |
| FISHER-small [ 9 ] | — | — | 14.54 2.52 | 0.4860 0.04 | 21.4 |
| TS-JEPA (adapted) [ 8 ] | 19.40 1.96 | 4.09 0.07 | 31.02 19.84 | 0.6147 0.03 | 0.363 |
| Model | Dataset | ||||
|---|---|---|---|---|---|
| AC | spec | NASA | CMP | CNC | PU |
| 10.78 | 4.05 | 10.59 | 0.7578 | ||
| ✓ | 18.07 | 4.01 | 10.08 | 0.6819 | |
| ✓ | 7.35 | 3.92 | 10.55 | 0.6077 | |
| ✓ | ✓ | 6.65 | 3.86 | 10.47 | 0.7793 |
Self-Supervised Time Series Representation Learning
Self-supervised time series representation learning uses unlabeled sequences to learn features that transfer to forecasting, classification, and anomaly detection.
See all 161 papers