W-JEPA: Wavelet Joint-Embedding Predictive Architecture for Wearable Activity Recognition

Yogesh Prabhu; Loki Natarajan; Jingjing Zou

cs.LGOct 14, 2026

W-JEPA: Wavelet Joint-Embedding Predictive Architecture for Wearable Activity Recognition

Authors: Yogesh Prabhu, Loki Natarajan, Jingjing Zou

Organizations: Department of Computer Science and Engineering, University of California San Diego (UCSD). · Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego (UCSD). · UCSD Shiley-Marcos Alzheimer’s Disease Research Center. · UCSD Moores Cancer Center.

Abstract

A major challenge in wearable device activity recognition is the heterogeneity across datasets in sensor placement, channel count, sampling rate and subjects. Moreover, the motion patterns that distinguish activities often appear as subtle, scale-specific variations within otherwise similar waveforms. As a result, raw signals are often ambiguous, and models trained on them tend to rely on cues specific to a particular sensor setup or participant, which limits both cross-dataset transfer and generalization to unseen participants. Addressing this requires representations that preserve such activity-relevant, scale-specific variation. We introduce W-JEPA, a self-supervised framework that exploits the multiscale structure of inertial motion and leverages a joint-embedding predictive architecture, by organizing windows into a fixed grid of wavelet tokens indexed by scale, sensor axis, and time. These wavelet tokens help expose activity-specific detail within broadly similar raw signals, and serve as the units of masked latent prediction. W-JEPA predicts masked token representations supplied by an exponential-moving-average (EMA) target encoder. Because recovering a masked token requires relating motion at one scale and axis to motion at other scales and axes, the model learns the cross-scale, cross-axis structure of human movement. Across six wearable activity benchmarks, W-JEPA outperforms state-of-the-art methods under random-window splits and remains the strongest when subjects are held out, improving average macro-F1 by 8.7 percentage points over the strongest published baseline. These results indicate that physically grounded multiscale tokenization, combined together with latent predictive learning, provides a promising foundation for transferable representations of heterogeneous wearable inertial data.

Figures & tables

Panel of Figure 1 — Figure 1: Wavelet tokens reveal differences between similar activity signals. (a) Sample window of horizontal and vertical exercise-bike cycling from a DSADS participant’s left leg gyroscope $z$ -axis exhibit similar raw waveforms. (b) The reconstructed db4 $D_{3}$ component reveals larger pronounced oscillations during horizontal cycling in this pair. (c) Mean RMS of the second $D_{3}$ token across 59 windows per activity and participant, with higher values for horizontal cycling in all eight participants. More details appear in Appendix D.1 .

Panel of Figure 2 — Figure 2: W-JEPA. (a) Wavelet tokenization and masking. Rows are sensor channels and columns are wavelet bands $A_{4}$ – $D_{1}$ , ordered from coarse to fine. Each channel and wavelet band combination produces two temporal tokens, shown as a single cell for clarity and masked together; black cells indicate masked token pairs. (b) Masked latent prediction. The predictor estimates representations at masked positions from visible-token context, with targets supplied by an EMA target encoder.

Method	Metric	PAMAP2	DSADS	MHealth	RealWorld	UCI-HAR	USC-HAD
BYOL + perm_jit	Acc.	$65.65\pm 3.68$	$64.66\pm 5.59$	$57.70\pm 19.68$	$67.59\pm 0.53$	$74.49\pm 2.78$	$57.81\pm 3.15$
BYOL + perm_jit	F1	$65.33\pm 4.65$	$63.99\pm 6.12$	$51.32\pm 20.57$	$62.99\pm 0.45$	$71.43\pm 2.98$	$53.71\pm 2.69$
BYOL + lfc	Acc.	$72.01\pm 3.32$	$65.13\pm 3.27$	$73.10\pm 1.25$	$65.42\pm 1.37$	$77.36\pm 3.48$	$59.48\pm 3.70$
BYOL + lfc	F1	$72.16\pm 3.56$	$63.79\pm 4.64$	$73.46\pm 0.51$	$59.87\pm 1.51$	$76.21\pm 3.76$	$57.63\pm 2.71$
BYOL + Mixup	Acc.	$72.94\pm 2.00$	$65.74\pm 5.73$	$74.13\pm 3.41$	$64.31\pm 1.74$	$75.82\pm 4.68$	$58.09\pm 1.45$
BYOL + Mixup	F1	$73.30\pm 2.08$	$64.25\pm 6.91$	$73.76\pm 3.77$	$58.92\pm 1.99$	$72.89\pm 5.86$	$56.43\pm 1.09$

Table 1: Subject-disjoint comparison across six wearable activity recognition datasets. Results are averaged over five random seeds under the unified protocol of § 4 .

Method	Metric	PAMAP2	DSADS	MHealth	RealWorld	UCI-HAR	USC-HAD
BYOL + perm_jit	Acc.	$84.06\pm 8.26$	$89.18\pm 5.02$	$85.49\pm 7.66$	$95.52\pm 0.27$	$77.65\pm 3.80$	$78.45\pm 5.40$
BYOL + perm_jit	F1	$80.53\pm 13.74$	$88.73\pm 5.45$	$79.57\pm 11.37$	$95.60\pm 0.40$	$75.94\pm 4.48$	$73.43\pm 6.08$
BYOL + lfc	Acc.	$90.82\pm 1.20$	$96.71\pm 0.14$	$92.28\pm 1.60$	$96.11\pm 0.34$	$81.52\pm 4.04$	$82.85\pm 1.20$
BYOL + lfc	F1	$90.66\pm 1.45$	$96.71\pm 0.14$	$91.41\pm 1.76$	$96.14\pm 0.46$	$80.43\pm 4.76$	$77.57\pm 1.30$
BYOL + Mixup	Acc.	$89.76\pm 0.89$	$97.00\pm 0.52$	$92.09\pm 1.36$	$96.00\pm 0.19$	$81.66\pm 3.67$	$81.28\pm 2.44$
BYOL + Mixup	F1	$89.53\pm 1.07$	$96.97\pm 0.52$	$91.75\pm 1.52$	$96.00\pm 0.20$	$80.54\pm 4.56$	$75.88\pm 3.34$

Table 2: Random-window comparison across six wearable activity recognition datasets. Results are averaged over five random seeds under the unified protocol of § 4 .

Figure 3: Subject-disjoint macro-F1 with wavelet tokens and learned temporal patches as training participants are added. A fixed 20% participant split is held out for testing, and training participants are drawn from the remaining pool.

Panel of Figure 4 — Figure 4: Each panel compares W-JEPA without and with wavelets. (a) Confusion matrices for cycling on a horizontal exercise bike and cycling on a vertical exercise bike, using percentages within each true class; predictions of other activities are “Other classes.” (b) 2D PCA projections of learned representations for the same two classes.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Train samples	Test samples	Train participants	Test participants
PAMAP2	835	2,998	2	7
DSADS	2,242	6,726	2	6
MHealth	267	1,068	2	8
RealWorld	2,551	10,209	3	12
UCI-HAR	486	1,946	6	24
USC-HAD	1,069	4,284	3	11

Appendix

Table 5: Dataset statistics for the subject-disjoint train/test splits. The number of samples corresponds to the number of segmented sensor windows used for training and testing.

Variant	Metric	PAMAP2	DSADS	MHealth	RealWorld	UCI-HAR	USC-HAD
W-JEPA (SIGReg)	Acc.	$93.62\pm 0.07$	$99.02\pm 0.09$	$99.23\pm 0.27$	$96.92\pm 0.06$	$89.74\pm 0.30$	$87.76\pm 0.45$
W-JEPA (SIGReg)	F1	$93.70\pm 0.04$	$99.01\pm 0.10$	$99.22\pm 0.24$	$97.13\pm 0.07$	$89.39\pm 0.29$	$84.42\pm 0.37$
W-JEPA (VICReg)	Acc.	$\mathbf{93.77\pm 0.20}$	$99.11\pm 0.08$	$98.97\pm 0.57$	$\mathbf{97.13\pm 0.18}$	$90.66\pm 0.87$	$87.97\pm 0.71$
W-JEPA (VICReg)	F1	$\mathbf{93.82\pm 0.22}$	$99.10\pm 0.08$	$99.04\pm 0.53$	$97.33\pm 0.14$	$90.45\pm 0.88$	$84.75\pm 0.62$
W-JEPA	Acc.	$93.75\pm 0.22$	$\bm{99.22\pm 0.13}$	$\bm{99.28\pm 0.35}$	$\bm{97.13\pm 0.14}$	$\bm{92.91\pm 1.23}$	$\bm{88.54\pm 0.35}$
W-JEPA	F1	$93.79\pm 0.21$	$\bm{99.22\pm 0.13}$	$99.18\pm 0.19$	$\bm{97.36\pm 0.12}$	$\bm{92.59\pm 1.26}$	$\bm{86.52\pm 0.33}$

Appendix

Table 6: W-JEPA regularizer random-window split ablations, summarizes five seeds mean and standard deviation across the six benchmarks. Bold indicates the highest mean per metric, including ties.

Figure 5: One-factor-at-a-time sensitivity study on UCI-HAR. From left to right, the panels vary the fraction of token positions masked during pretraining, the number of temporal bins used to summarize each wavelet band (shown as the nominal input samples per bin), the input-window length, and the requested fraction of labeled fine-tuning windows. The shared baseline uses a 65% mask ratio, two bins per band, and 500-sample windows. Points report top-1 test accuracy from one run; nominal bin sizes are not literal wavelet receptive fields.

Panel of Figure 6 — Figure 6: Wavelet-scale attention on held-out DSADS participants. Left: relative attention change across noise doses across all activities; Middle: clean attention by activity relative to uniform allocation. Right: relative attention change by activity at 30% noise. All panels use the final encoder layer.

	Held-out participants		Random windows
Noise (%)	Accuracy (%)	Macro-F1 (%)	Accuracy (%)	Macro-F1 (%)
0	81.73	81.77	99.28	99.28
5	$81.65\pm 0.02$	$81.58\pm 0.02$	$99.22\pm 0.01$	$99.21\pm 0.01$
10	$81.87\pm 0.03$	$81.67\pm 0.02$	$99.05\pm 0.04$	$99.04\pm 0.04$
15	$81.14\pm 0.07$	$80.77\pm 0.11$	$98.90\pm 0.02$	$98.88\pm 0.02$
20	$81.00\pm 0.05$	$80.73\pm 0.08$	$98.66\pm 0.04$	$98.63\pm 0.04$

Appendix

Table 7: DSADS full-test performance under fixed input noise on a single seed. Nonzero doses report mean

\pm

sample standard deviation across three noise realizations; clean inputs were evaluated once.

Figure 7: PCA and UMAP projections of W-JEPA’s final-block representations across six human activity recognition datasets. Each point represents a test window, colored by activity class within each dataset.

Explore similar work

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition
Jul 17, 2026Mohd Halim Mohd Noor, Abdulrahman M. A. BarakaHuman Activity Recognition Self-Supervised Time Series Representation Learning
School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine
Bio-Inspired Self-Supervised Learning for Wrist-worn Accelerometer Data
Mar 11, 2026Prithviraj Tarale, Kiet Chu, Abhishek Varghese +4Human Activity Recognition Self-Supervised Time Series Representation Learning
College of Information and Computer Sciences, University of Massachusetts, Amherst, United States · Stevens Institute of Technology, Hoboken, United States · Google Health, Seattle, United States +1
Inertia-1: An Open Exploration of Wearable Motion Foundation Models
Jul 7, 2026Zongzhe Xu, Aakarsh Anand, Sarah Jiang +4Human Activity Recognition Wearable Sensing
University of California, Los Angeles · Duke University · John Hopkins University

Jul 17, 2026J/K move · Enter open · S save

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

Mohd Halim Mohd Noor, Abdulrahman M. A. Baraka

School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine

Human Activity Recognition Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture Invariant Representation Learning

Open paper AskPDF arXiv

Sensor-based human activity recognition (HAR) has achieved significant progressed in fully supervised learning settings. However, these supervised learning models rely on large amount of labeled data, which require labor-intensive collection and meticulous annotation. To address these challenges, this paper proposes a Joint Embedding Predictive Architecture framework tailored for sensor-based HAR, designed to learn robust and generalizable representations from unlabeled datasets. The proposed framework features an encoder designed to explicitly model both the fine-grained local temporal representations within individual window and the long-term temporal sequence of adjacent windows. Furthermore, we introduce an improved Variance-Invariance-Covariance Regularization (VICReg) objective function that incorporates computationally lightweight norm term to stabilize the JEPA pre-training phase. This term balances variance, invariance and covariance constraints to prevent representation collapse. The proposed HAR-JEPA framework is evaluated using two benchmark continuously performed activity datasets. The results show that high-quality representations are successfully learned by the proposed framework. Furthermore, the representations learned by HAR-JEPA demonstrates superior generalization on minority, high variance transitional activities such as sit-to-stand and sit-to-lie where supervised learning tend to overfit due to limited support.

Abstract

Explore similar work

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

Bio-Inspired Self-Supervised Learning for Wrist-worn Accelerometer Data

Inertia-1: An Open Exploration of Wearable Motion Foundation Models