Deep learning has improved inertial measurement unit (IMU) sensing for mobile and wearable applications. However, an IMU model trained in one domain often becomes unreliable when it is used with a new user, device, or body position. Existing methods usually treat this problem as a static model-design task: they pretrain a stronger representation, add data augmentation, or select one adaptation method before deployment. In practice, the target domain is only gradually observed, labels are scarce, and different domain shifts require different sensing actions. This paper presents SenseAgent, an LLM-guided sensing agent for cross-domain IMU activity recognition. Instead of asking an LLM to classify raw IMU signals, SenseAgent uses the LLM as a runtime planner over sensing tools, source-domain experience memory, online target memory, and verifiers. The agent builds a label-free diagnosis report from the target stream and uses it to decide whether to keep raw inference or invoke specialized tools, including gravity-aware sensing, prototype transfer, and style normalization. Verifiers check source calibration, target-memory reliability, and no-harm criteria before accepting high-risk tool decisions. SenseAgent also supports scarce feedback without retraining the backbone or replacing the label-free route. This design converts cross-domain IMU sensing from a fixed inference pipeline into a closed-loop sensing process that diagnoses target shifts, selects suitable sensing actions, and rejects unsafe adaptations. We evaluate SenseAgent across multiple IMU datasets and deployment shifts. Results show that its verified route selection improves cross-domain sensing, especially under harder placement and compound shifts, and further benefits from limited user feedback.
Figures & tables
Figure 1. OOD difficulty is not uniform across target domains. (a) Raw PatchTST macro-F1 on TNDA varies substantially across users, positions, and compound OOD settings. (b) Representative target domains show that the same OOD type can include both easy and hard cases.
Figure 2. Adaptation and physical cues are target-dependent on TNDA. (a) A 1-shot prototypical baseline can either improve or degrade target domains. (b) Dynamic windows provide a stable gravity reference, whereas static windows show larger dispersion. Static dispersion has a weak negative association with raw static-class F1.
Figure 3. Overview of SenseAgent.
Dataset
Classes
Source
User OOD
Pos./Dev. OOD
Compound OOD
HHAR
6
25 / 13.0K
14 / 7.2K
21 / 13.9K
7 / 4.3K
TNDA
8
60 / 21.9K
40 / 15.0K
90 / 32.8K
60 / 22.5K
PAMAP2
12
10 / 9.6K
6 / 6.1K
5 / 5.7K
3 / 3.4K
Feng
12
120 / 412.8K
80 / 285.8K
135 / 464.4K
90 / 321.5K
Table 1. Dataset partitions used in our experiments. Each split entry reports the number of domains and windows.
Method ↓
Feng
PAMAP2
TNDA
HHAR
Average
User
Pos.
Comp.
User
Pos.
Comp.
User
Pos.
Comp.
User
Dev.
Comp.
User
Pos./Dev.
Comp.
GRU
0.8041
0.7613
0.7203
0.4634
0.0999
0.1193
0.8262
0.3928
0.3571
0.5810
0.7543
0.5390
0.7735
0.6154
0.5649
PatchTST
0.7751
0.8002
0.7327
0.4973
0.1531
0.1910
0.8219
0.4473
0.4392
0.7430
0.7416
0.7285
0.7734
0.6559
0.6123
T3A
0.6904
0.7177
0.6757
0.3954
0.1756
0.2478
0.7385
0.4397
0.4433
0.5159
0.5437
0.5220
0.6741
0.5927
0.5738
SimCLR
0.7605
0.7829
0.7158
0.4450
0.1847
0.2100
0.7869
0.4100
0.3953
0.6601
0.6895
0.6933
0.7445
0.6295
0.5851
CrossHAR-III
0.7487
0.7374
0.6780
0.3743
0.1745
0.1729
0.7337
0.2996
0.2801
0.6479
0.6350
0.6391
0.7183
0.5606
0.5176
Table 2. Cross-domain macro-F1 comparison across four IMU sensing datasets. Dataset-specific columns report the mean across target domains. Avg. columns report macro-F1 averaged over all target domains in the corresponding OOD setting. The best result in each column is highlighted in bold.
Method ↓
Feng
PAMAP2
TNDA
HHAR
Average
User
Pos.
Comp.
User
Pos.
Comp.
User
Pos.
Comp.
User
Dev.
Comp.
User
Pos./Dev.
Comp.
T3A+FB ( N=1 )
0.6872
0.7162
0.6736
0.3956
0.1876
0.2517
0.7429
0.4630
0.4684
0.5191
0.5490
0.5335
0.6738
0.6009
0.5826
T3A+FB ( N=8 )
0.6866
0.7149
0.6770
0.3819
0.2837
0.3138
0.7710
0.5777
0.5793
0.5638
0.5930
0.5805
0.6854
0.6469
0.6293
Linear Probe ( N=8 )
0.2709
0.3013
0.2651
0.1665
0.2204
0.1898
0.4134
0.4067
0.4050
0.4245
0.4606
0.4613
0.3225
0.3508
0.3247
PTN 1-shot †
0.6374
0.6902
0.6102
0.3704
0.4669
0.4359
0.7207
0.6926
0.7107
0.5556
0.5778
0.6668
0.6416
0.6772
0.6471
SenseAgent ( N=1 )
0.7655
0.8016
0.7263
0.5276
0.3128
0.4031
0.8184
0.5397
0.5613
0.7447
0.7451
0.7296
0.7683
0.6932
0.6585
Table 3. Scarce-feedback macro-F1 across OOD target domains. Rows with N=1 or N=8 use randomly sampled labeled anchors per target domain. † PTN-1shot uses one labeled support example per class and is therefore a stronger class-balanced few-shot reference. The best result in each column is highlighted in bold and the second-best result is underlined.
Figure 4. Effect of feedback budget on OOD adaptation. Solid lines show SenseAgent; dashed lines show PTN 5-shot. HHAR is evaluated up to N=64 due to limited target samples.
Dataset
Final Route (%) Raw / Safe / Grav.
Verifier Override (%)
Gravity Harm (%)
Final No-Harm (%)
Feng
0.5 / 97.8 / 1.6
93.7
98.1
93.2
HHAR
28.6 / 71.4 / 0.0
71.4
61.9
100.0
PAMAP2
0.0 / 21.4 / 78.6
0.0
7.1
92.9
TNDA
0.0 / 8.8 / 91.2
4.2
28.6
75.3
Table 4. Runtime route selection and no-harm diagnostics under zero feedback. Verifier override measures interventions on unsupported gravity actions. Gravity harm and final no-harm are post-hoc evaluation diagnostics.
Router
Feng
HHAR
PAMAP2
TNDA
Overall
Prompt-Rule
+0.0017
+0.0002
+0.0291
+0.0051
+0.0034
SenseAgent
-0.0021
+0.0003
+0.1040
+0.0724
+0.0265
Oracle
+0.0029
+0.0650
+0.1203
+0.0919
+0.0413
Table 5. No-LLM planner baseline under zero feedback. Values are macro-F1 differences relative to raw model. Prompt-Rule uses the same reports, tools, memory, and verifier as SenseAgent but replaces the LLM planner with deterministic rules. Oracle is a post-hoc upper bound.
Dataset
N
Δ SenseAgent
Δ Feedback-Adaptive
Δ Verifier Override
TNDA
0
+0.0027
+0.0027
-0.0298
1
+0.0035
+0.0034
-0.0544
8
+0.0003
+0.0009
-0.0526
HHAR
0
-0.0002
-0.0002
-0.4524
1
0.0000
0.0000
-0.5079
8
-0.0002
+0.0001
-0.5794
Table 6. Difference between GPT-5.4 and Qwen3.5-9B planners on TNDA and HHAR. Values are weighted by the number of domains. Positive values indicate GPT-5.4 is higher.
Figure 5. Route ablation under zero-feedback and scarce-feedback settings. SenseAgent uses verified routing to keep the beneficial route and suppress harmful adaptation.
Data
N
Gravity LoRA
Gravity Proto.
Gravity Verifier
Target Memory
Source Reliab.
Static Geometry
Feng
0
0.000
+0.002
-0.190
-0.022
+0.002
+0.004
HHAR
0
0.000
0.000
-0.031
+0.006
+0.001
0.000
PAMAP2
0
-0.107
-0.104
-0.071
-0.046
-0.016
-0.051
TNDA
0
+0.004
-0.074
-0.008
-0.056
+0.018
-0.042
Feng
1
0.000
+0.005
-0.132
-0.014
+0.001
+0.005
HHAR
1
0.000
0.000
-0.013
+0.005
0.000
0.000
Table 7. Component ablation results. Values are macro-F1 changes relative to the full SenseAgent, where negative values indicate removing the component hurts performance.
Raw Route / Window
Gravity Route / Window
Report E2E
Cold LLM Generation
In/Out Tokens
7.82 ms
14.91 ms
1.76 s
10.52 s
2.87K / 60.3
Table 8. Deployment overhead under zero feedback. Report E2E is measured per target report; cold LLM cost is measured per generated action.
Methods
Sensor
LLM
Runtime
Target
Adapt
Evidence
Reason
Plan
Memory
Verifier
Semantic HAR Alignment ( Yan et al., 2025 )
✓
✓
✗
✗
✗
LLM-IMU Sensing ( Li et al., 2025 ; Hong et al., 2025 ; Zhang et al., 2026 )
✓
✓
✗
✗
✗
IMU Reasoning ( Asif Imran Shouborno et al., 2025 ; Zhang et al., 2025 )
✓
✓
✗
✗
✗
IoT-LLM ( An et al., 2026 )
✓
✓
✗
✗
✗
AutoIoT ( Shen et al., 2025 )
✗
✓
✓
✗
✗
Table 9. Comparison with LLM-based sensing systems.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Template content
Role
You are SenseAgent, an action planner for cross-domain IMU activity recognition. You do not classify raw IMU windows directly; you propose a prediction route, a feedback-use policy, and a memory update from a structured diagnosis report.
Input constraints
You only see a JSON diagnosis report containing label-free target evidence, source-calibrated route/tool summaries, memory statistics, gravity/prototype/style reliability, and optional user-provided feedback anchors. Do not use dataset name, target domain identity, user identity, device identity, body position, OOD type, or hidden target labels.
Action space
Choose a route instantiated by tool actions from raw_patchtst , safe_groupwise , gravity_proto_tool , raw_proto_tool , and style_normalization , with gravity variant auto , lora , or base ; choose feedback use from conservative freeze, local adjustment, propagation, fusion, or fusion-propagation; and choose memory update from append , append_filtered , and freeze .
Decision rule
Use candidate utilities, raw stability, shift diagnostics, gravity reliability, prototype support, style reliability, and feedback evidence as weak evidence rather than fixed rules. Prefer conservative actions when raw inference is stable or support for specialized routes is weak. Proposed actions are checked by downstream verifiers and converted into final executable routes only if accepted.
Table 10. Prompt template for the SenseAgent planner.
τs
τr
Macro-F1
No-Harm
Override
0.35
0.65
0.6657
0.734
0.458
0.35
0.75
0.6655
0.733
0.457
0.35
0.85
0.6607
0.684
0.390
0.45
0.65
0.6935
0.854
0.601
0.45
0.75
0.6933
0.851
0.598
0.45
0.85
0.6882
0.799
0.517
Appendix
Table 11. Zero-feedback sensitivity of the route verifier thresholds. No-Harm reports the fraction of target domains whose replayed route is no worse than raw PatchTST.
Wi-Fi channel state information (CSI) has enabled device-free sensing applications such as human activity recognition. However, CSI sensing models remain brittle in cross-domain deployment, where changes in users or environments can produce incorrect predictions. Existing solutions usually treat this problem as an offline model-design problem, by pretraining a stronger representation or applying one fixed adaptation method to the entire target domain. In practice, labeled target data are scarce and different classes may fail in different ways under the same domain shift. To address this, we propose CSI-Agent, an evidence-seeking LLM agent that reformulates cross-domain CSI adaptation as a deployment-time decision-making problem. Rather than processing raw CSI or making sample-level predictions, CSI-Agent summarizes target-domain behavior into sensing-grounded class-level evidence. It establishes a strong target-adaptive default from complementary CSI views and uses an LLM planner to determine whether each class should retain the default or invoke a specialized action. Deterministic verification and bounded execution further reduce unreliable interventions. We evaluate CSI-Agent on four public datasets using five cross-domain splits covering device, user, environment, and compositional shifts. Under 1-shot adaptation, CSI-Agent achieves the best target-domain performance across all splits and improves the average Macro-F1 by about 16% compared to the strongest baseline method.
Tianya Zhao, Chuan Liu, Xuyu Wang
Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, FL 33199, US
Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is a cornerstone of mobile health, smart environments, and human-computer interaction. However, current deep learning-based HAR models often struggle with heavy reliance on labeled data, position-specific ambiguity, and a lack of transparent reasoning. Inspired by the advanced agents framework, which emulates a collaborative agent using Large Language Models (LLMs), we propose SensingAgents, a novel multi-agent system for robust IMU activity recognition. SensingAgents organizes LLM-powered agents into specialized roles: a group of Analyst Agents for position-specific sensor analysis (arm, wrist, belt, pocket), a pair of Advocate Agents that resolves sensor conflicts through dynamic and static dialectical debates, and a Decision Agent that ensures reliability under sensor drift or failure. Evaluation on the Shoaib dataset demonstrates that SensingAgents significantly outperforms state-of-the-art single-agent and multi-agent LLM models, achieving an accuracy of 79.5% in a zero setting--29% higher than existing agent models and 9.4% higher than deep learning baselines--particularly in complex scenarios where multi-sensor data is conflicting or noisy. Our work highlights the potential of multi-agent collaborative reasoning for advancing the robustness and interpretability of ubiquitous sensing systems.
Naiyu Zheng, Tianlong Yu, Haochen Yin +3
City University of Hong Kong Hong Kong · Hubei University China · The Chinese University of Hong Kong Hong Kong +1
Human Activity Recognition (HAR) using inertial measurement units (IMUs) enables a wide range of applications, yet the field still lacks a unified model that can generalize across diverse subjects, devices, and activities. Training such a model is difficult due to two key challenges: sensing heterogeneity -- differences in sampling rates, channel configurations, and sensor placements -- and poor generalization to unseen activities and label vocabularies. We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model that addresses both challenges through a two-stage training framework. Stage 1 pretrains the IMU encoder with heterogeneity-aware self-supervised learning, including adaptive-pooling tokenization, channel-independent feature extraction, and contextualized sensor conditioning that injects natural-language sensor descriptions into each channel embedding. Stage 2 aligns this IMU encoder with text embeddings via synonym-aware soft contrastive learning, enabling open-set recognition via cosine-similarity retrieval without per-dataset classifiers. Trained on 10 public HAR datasets and evaluated on 7 held-out datasets, HALO outperforms five state-of-the-art baselines on all 8 aggregate metrics, and still leads on 3 of 4 settings under baseline-matched inputs. Despite using only ~35M trainable parameters -- 10x fewer than the latest foundation model MOMENT (341.2M) -- HALO improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points. On two further datasets with severe distribution shift, every model including HALO collapses zero-shot. A video demonstration of HALO's performance in real world is available at https://youtu.be/rooVKragtFU
Zihan Ding, Liyu Zhang, Xiaomin Ouyang
Hong Kong University of Science and Technology Hong Kong, China