Human Activity Recognition (HAR) using inertial measurement units (IMUs) enables a wide range of applications, yet the field still lacks a unified model that can generalize across diverse subjects, devices, and activities. Training such a model is difficult due to two key challenges: sensing heterogeneity -- differences in sampling rates, channel configurations, and sensor placements -- and poor generalization to unseen activities and label vocabularies. We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model that addresses both challenges through a two-stage training framework. Stage 1 pretrains the IMU encoder with heterogeneity-aware self-supervised learning, including adaptive-pooling tokenization, channel-independent feature extraction, and contextualized sensor conditioning that injects natural-language sensor descriptions into each channel embedding. Stage 2 aligns this IMU encoder with text embeddings via synonym-aware soft contrastive learning, enabling open-set recognition via cosine-similarity retrieval without per-dataset classifiers. Trained on 10 public HAR datasets and evaluated on 7 held-out datasets, HALO outperforms five state-of-the-art baselines on all 8 aggregate metrics, and still leads on 3 of 4 settings under baseline-matched inputs. Despite using only ~35M trainable parameters -- 10x fewer than the latest foundation model MOMENT (341.2M) -- HALO improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points. On two further datasets with severe distribution shift, every model including HALO collapses zero-shot. A video demonstration of HALO's performance in real world is available at https://youtu.be/rooVKragtFU
Figures & tables
Figure 1. HALO is a domain-specific IMU foundation model that can tackle various sensing heterogeneity for open-set human activity recognition.
Figure 2. Dataset heterogeneity across 14 HAR datasets in our benchmark. Training datasets (blue circles) and test datasets (red triangles) span 20–100 Hz in sampling rates, 3–45 channels, and 4–19 activity labels.
Figure 3. System overview of HALO. The sensing heterogeneity is addressed through heterogeneity-aware pretraining of the IMU encoder (Stage 1), followed by synonym-aware soft contrastive learning to align sensor features with text semantics (Stage 2), enabling generalizable and open-set activity recognition. Zero-shot inference is achieved by swapping label banks at runtime without retraining.
Figure 4. Heterogeneity-aware Pretraining. Each row shows a challenge (left) and the corresponding design in HALO (right): adaptive-pooling tokenization for variable sampling rates; channel-independent feature extraction for variable channels; and contextualized sensor conditioning for variable sensor placements.
Figure 5. IMU-to-text alignment and zero-shot inference. Top: During training, the IMU encoder is aligned with a frozen text encoder in a shared embedding space via synonym-aware contrastive learning, so that semantically related activities are clustered together. Bottom: At inference, the IMU segment is encoded and matched against an open label library via cosine similarity-enabling zero-shot recognition without retraining or dataset-specific classifiers.
Dataset
Hz
#Subj
#Act
#Ch
Description
Cov.
Diff.
UCI-HAR ( Anguita et al., 2013 )
50
30
6
9
Smartphone IMU
HHAR ( Stisen et al., 2015 )
∼ 50
9
6
6
Phone + watch
UniMiB ( Micucci et al., 2017 )
50
30
17
3
Phone acc, ADLs
MHEALTH ( Banos et al., 2014 )
50
10
12
6+
Body IMU suite
PAMAP2 ( Reiss and Stricker, 2012 )
100
9
18
27
Multi-IMU
WISDM ( Kwapisz et al., 2011 )
20
51
18
6
Phone + watch
Table 1. Dataset overview. Top: 10 training datasets. Bottom: 7 held-out test datasets. † Main test datasets. ‡ Severe OOD datasets; reported separately. Cov. : Fraction of test activities that have an equivalent in the training set. Diff. : Difficulty level determined by distribution shift and label novelty.
Model
Paradigm
Dim
Train.
Infer.
LiMU-BERT ( Xu et al., 2021 )
Masked recon.
72
72.7K
72.7K
MOMENT ( Goswami et al., 2024 )
General TS
6144
341.2M
341.2M
CrossHAR ( Chang et al., 2024 )
Self-sup HAR
72
531.4K
531.4K
LanHAR ( Li et al., 2024 )
Text-aligned
768
13.0M ∗
122.9M
LLaSA ( Imran et al., 2025 )
LLM generative
—
∼ 6.7B
∼ 6.7B
HALO (ours)
Text-aligned
384
∼ 35M
∼ 58M
Table 2. Baseline overview. Train. : parameters updated during pretraining and alignment. Infer. : Total parameters used at inference, including frozen modules. HALO’s inference total includes a single frozen MiniLM (22.7M) shared for both contrastive alignment and channel conditioning; all text embeddings are precomputable. ∗ LanHAR freezes SciBERT (109.9M) during stage-2 and unfreezes it for fine-tuning.
ZS-O
ZS-C
1%
10%
Model
Acc
F1
Acc
F1
Acc
F1
Acc
F1
HALO (ours)
42.0
21.0
53.1
37.2
76.5
70.0
85.7
81.7
MOMENT
28.3
7.2
44.7
33.2
73.8
69.8
81.3
76.8
CrossHAR
21.2
5.5
38.2
31.7
61.1
55.2
78.6
75.2
LiMU-BERT
21.7
7.5
33.1
23.1
35.7
22.7
51.3
43.6
LanHAR
15.8
6.2
28.4
21.7
43.3
35.2
56.2
51.5
Table 3. Overall performance in average accuracy (%) and macro F1 (%) across 5 main test datasets. Bold : best result for each metric. ZS-O/ZS-C: zero-shot open-set/closed-set settings. HALO and LanHAR are evaluated at each dataset’s native sampling rate; other baselines use data resampled to 20 Hz. † LLaSA: published 7B model, zero-shot only (no supervised)
Accuracy (%)
Macro F1 (%)
Dataset
ZS-O
ZS-C
1%
10%
ZS-O
ZS-C
1%
10%
MotionSense
49.3
64.1
88.6
95.0
30.3
49.3
87.1
94.6
RealWorld
48.0
48.0
75.1
85.6
33.3
37.9
74.0
86.4
MobiAct
42.2
50.0
65.3
72.9
16.3
12.8
35.2
51.7
Shoaib
49.2
54.1
81.6
95.9
17.6
51.5
81.2
95.7
Opportunity
21.1
49.3
72.0
79.3
7.4
34.2
72.6
80.0
Table 4. Per-dataset results of HALO on the five main test datasets. ZS-O/ZS-C: zero-shot open/closed-set. All datasets are held out from training.
HARTH
VTT-ConIoT
ZS-O
ZS-C
1%
10%
ZS-O
ZS-C
1%
10%
HALO
2.0
2.5
64.7
78.3
1.3
2.2
1.9
16.9
MOMENT
0.4
1.9
66.7
65.4
1.6
5.2
18.4
34.8
CrossHAR
0.3
0.9
42.4
71.9
0.7
5.0
10.6
30.9
LiMU-BERT
0.0
20.2
54.4
19.1
3.4
7.1
4.3
15.0
LanHAR
0.9
1.1
3.4
70.0
8.3
6.9
7.7
7.7
Table 5. Accuracy on severe out-of-domain datasets (%). ZS-O/ZS-C: zero-shot open/closed-set. All models collapse on HARTH zero-shot (sensor distribution shift); VTT-ConIoT is capped at ∼ 50% by novel activities. † LLaSA: zero-shot only, subsampled.
Configuration
ZS-O
ZS-C
1%
10%
Full model
44.9
51.9
74.7
85.4
− Channel-text fusion
25.3 (–19.6)
43.8 (–8.1)
72.9 (–1.8)
82.5 (–2.9)
− Signal augmentation
45.3 (+0.4)
53.0 (+1.1)
70.9 (–3.8)
83.5 (–1.9)
− Text augmentation
42.7 (–2.2)
49.9 (–2.0)
67.7 (–7.0)
84.2 (–1.2)
Table 6. Ablation study of design components. Each row removes one component while keeping all others fixed. Results report average accuracy (%) across five main test datasets, with performance deltas relative to the full model shown in parentheses.
Configuration
ZS-O
ZS-C
1%
10%
HALO (native Hz, rich desc.)
42.0
53.1
76.5
85.7
HALO (20 Hz, generic desc.)
30.6
49.4
71.7
84.7
Best baseline (MOMENT)
28.3
44.7
73.8
81.3
Table 7. Analysis of native sampling rate and evaluation fairness with average accuracy (%) across five main test datasets. In Row 2, HALO is evaluated using the same 20Hz, 6 - channel inputs with generic textual descriptions as four of the five baselines (LanHAR uses 50Hz data). Even under these conservative and standardized inputs, HALO outperforms baselines in three of the four evaluation settings.
Figure 6. Analysis of embedding space. UMAP projection of learned IMU and text embeddings at epoch 100. Circles: per-activity IMU centroids; diamonds: SBERT text centroids; lines connect matched IMU–text pairs.
Figure 7. Patch size sensitivity. Zero-shot closed-set accuracy (%) across candidate patch durations. The dashed vertical line indicates the selected patch size of 1.0 s. Original datasets are evaluated at 20 Hz, while Shoaib and Opportunity are evaluated at their native sampling rates.
Model
Params
ZS-O
ZS-C
1%
10%
Tiny
6M
32.8
52.1
72.6
86.4
Small
35M
46.0
53.4
77.5
85.3
Medium
63M
37.8
47.7
72.6
84.5
Table 8. Scaling of model size. Average accuracy (%) on 3 main test datasets at 20 Hz. The Small model (35M) achieves the best performance, while the Medium model regresses on all zero-shot metrics despite having twice as many parameters.
Model
Params
Avg (ms)
p95 (ms)
Peak Mem. (MB)
CPU Wt/ms
CrossHAR
110K
0.29
0.30
1.20
224.0K
LiMU-BERT
110K
0.31
0.36
0.27
188.0K
LLaSA
105K
0.33
0.46
0.02
187.2K
LanHAR
8.8M
0.77
0.89
0.05
71.8K
MOMENT
24.0M
37.69
37.98
25.05
1.8K
HALO (ours)
34.0M
7.16
7.37
32.81
2414.4K
Table 9. Inference overhead on an iPhone 16 Pro. All models are benchmarked with same setting: identical 6-channel IMU inputs using Core ML, with 10 warm-up iterations followed by 100 timed iterations. Params denotes the profiled model size. CPU Wt/ms reports cycle-weighted CPU activity normalized by benchmark duration.
Figure 8. Deployment setting of smartphone-based IMU HAR in the real-world case study.
Figure 9. Performance continuous IMU-based HAR on the smartphone.
School of Computer Science and Engineering, Sun Yat-sen University · Engineering Research Centre of Applied Technology on Machine Translation and Artificial Intelligence, Macao Polytechnic University
Korea University Seoul, Republic of Korea · Massachusetts Institute of Technology Cambridge, Massachusetts, USA · Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea