SenseAgent: An LLM Agent for Adaptive Cross-Domain IMU Sensing

Tianya Zhao; Chuan Liu; Xuyu Wang

cs.AISep 25, 2026

SenseAgent: An LLM Agent for Adaptive Cross-Domain IMU Sensing

Authors: Tianya Zhao, Chuan Liu, Xuyu Wang

Organizations: Florida International University Miami, Florida, USA

Abstract

Deep learning has improved inertial measurement unit (IMU) sensing for mobile and wearable applications. However, an IMU model trained in one domain often becomes unreliable when it is used with a new user, device, or body position. Existing methods usually treat this problem as a static model-design task: they pretrain a stronger representation, add data augmentation, or select one adaptation method before deployment. In practice, the target domain is only gradually observed, labels are scarce, and different domain shifts require different sensing actions. This paper presents SenseAgent, an LLM-guided sensing agent for cross-domain IMU activity recognition. Instead of asking an LLM to classify raw IMU signals, SenseAgent uses the LLM as a runtime planner over sensing tools, source-domain experience memory, online target memory, and verifiers. The agent builds a label-free diagnosis report from the target stream and uses it to decide whether to keep raw inference or invoke specialized tools, including gravity-aware sensing, prototype transfer, and style normalization. Verifiers check source calibration, target-memory reliability, and no-harm criteria before accepting high-risk tool decisions. SenseAgent also supports scarce feedback without retraining the backbone or replacing the label-free route. This design converts cross-domain IMU sensing from a fixed inference pipeline into a closed-loop sensing process that diagnoses target shifts, selects suitable sensing actions, and rejects unsafe adaptations. We evaluate SenseAgent across multiple IMU datasets and deployment shifts. Results show that its verified route selection improves cross-domain sensing, especially under harder placement and compound shifts, and further benefits from limited user feedback.

Figures & tables

Panel of Figure 1 — Figure 1. OOD difficulty is not uniform across target domains. (a) Raw PatchTST macro-F1 on TNDA varies substantially across users, positions, and compound OOD settings. (b) Representative target domains show that the same OOD type can include both easy and hard cases.

Panel of Figure 2 — Figure 2. Adaptation and physical cues are target-dependent on TNDA. (a) A 1-shot prototypical baseline can either improve or degrade target domains. (b) Dynamic windows provide a stable gravity reference, whereas static windows show larger dispersion. Static dispersion has a weak negative association with raw static-class F1.

Dataset	Classes	Source	User OOD	Pos./Dev. OOD	Compound OOD
HHAR	6	25 / 13.0K	14 / 7.2K	21 / 13.9K	7 / 4.3K
TNDA	8	60 / 21.9K	40 / 15.0K	90 / 32.8K	60 / 22.5K
PAMAP2	12	10 / 9.6K	6 / 6.1K	5 / 5.7K	3 / 3.4K
Feng	12	120 / 412.8K	80 / 285.8K	135 / 464.4K	90 / 321.5K

Table 1. Dataset partitions used in our experiments. Each split entry reports the number of domains and windows.

Method $\downarrow$	Feng			PAMAP2			TNDA			HHAR			Average
Method $\downarrow$	User	Pos.	Comp.	User	Pos.	Comp.	User	Pos.	Comp.	User	Dev.	Comp.	User	Pos./Dev.	Comp.
GRU	0.8041	0.7613	0.7203	0.4634	0.0999	0.1193	0.8262	0.3928	0.3571	0.5810	0.7543	0.5390	0.7735	0.6154	0.5649
PatchTST	0.7751	0.8002	0.7327	0.4973	0.1531	0.1910	0.8219	0.4473	0.4392	0.7430	0.7416	0.7285	0.7734	0.6559	0.6123
T3A	0.6904	0.7177	0.6757	0.3954	0.1756	0.2478	0.7385	0.4397	0.4433	0.5159	0.5437	0.5220	0.6741	0.5927	0.5738
SimCLR	0.7605	0.7829	0.7158	0.4450	0.1847	0.2100	0.7869	0.4100	0.3953	0.6601	0.6895	0.6933	0.7445	0.6295	0.5851
CrossHAR-III	0.7487	0.7374	0.6780	0.3743	0.1745	0.1729	0.7337	0.2996	0.2801	0.6479	0.6350	0.6391	0.7183	0.5606	0.5176

Table 2. Cross-domain macro-F1 comparison across four IMU sensing datasets. Dataset-specific columns report the mean across target domains. Avg. columns report macro-F1 averaged over all target domains in the corresponding OOD setting. The best result in each column is highlighted in bold.

Method $\downarrow$	Feng			PAMAP2			TNDA			HHAR			Average
Method $\downarrow$	User	Pos.	Comp.	User	Pos.	Comp.	User	Pos.	Comp.	User	Dev.	Comp.	User	Pos./Dev.	Comp.
T3A+FB ( $N=1$ )	0.6872	0.7162	0.6736	0.3956	0.1876	0.2517	0.7429	0.4630	0.4684	0.5191	0.5490	0.5335	0.6738	0.6009	0.5826
T3A+FB ( $N=8$ )	0.6866	0.7149	0.6770	0.3819	0.2837	0.3138	0.7710	0.5777	0.5793	0.5638	0.5930	0.5805	0.6854	0.6469	0.6293
Linear Probe ( $N=8$ )	0.2709	0.3013	0.2651	0.1665	0.2204	0.1898	0.4134	0.4067	0.4050	0.4245	0.4606	0.4613	0.3225	0.3508	0.3247
PTN 1-shot †	0.6374	0.6902	0.6102	0.3704	0.4669	0.4359	0.7207	0.6926	0.7107	0.5556	0.5778	0.6668	0.6416	0.6772	0.6471
SenseAgent ( $N=1$ )	0.7655	0.8016	0.7263	0.5276	0.3128	0.4031	0.8184	0.5397	0.5613	0.7447	0.7451	0.7296	0.7683	0.6932	0.6585

Table 3. Scarce-feedback macro-F1 across OOD target domains. Rows with

N=1

N=8

use randomly sampled labeled anchors per target domain.

\dagger

PTN-1shot uses one labeled support example per class and is therefore a stronger class-balanced few-shot reference. The best result in each column is highlighted in bold and the second-best result is underlined.

Panel of Figure 4 — Figure 4. Effect of feedback budget on OOD adaptation. Solid lines show SenseAgent; dashed lines show PTN 5-shot. HHAR is evaluated up to $N=64$ due to limited target samples.

Dataset	Final Route (%) Raw / Safe / Grav.	Verifier Override (%)	Gravity Harm (%)	Final No-Harm (%)
Feng	0.5 / 97.8 / 1.6	93.7	98.1	93.2
HHAR	28.6 / 71.4 / 0.0	71.4	61.9	100.0
PAMAP2	0.0 / 21.4 / 78.6	0.0	7.1	92.9
TNDA	0.0 / 8.8 / 91.2	4.2	28.6	75.3

Table 4. Runtime route selection and no-harm diagnostics under zero feedback. Verifier override measures interventions on unsupported gravity actions. Gravity harm and final no-harm are post-hoc evaluation diagnostics.

Router	Feng	HHAR	PAMAP2	TNDA	Overall
Prompt-Rule	+0.0017	+0.0002	+0.0291	+0.0051	+0.0034
SenseAgent	-0.0021	+0.0003	+0.1040	+0.0724	+0.0265
Oracle	+0.0029	+0.0650	+0.1203	+0.0919	+0.0413

Table 5. No-LLM planner baseline under zero feedback. Values are macro-F1 differences relative to raw model. Prompt-Rule uses the same reports, tools, memory, and verifier as SenseAgent but replaces the LLM planner with deterministic rules. Oracle is a post-hoc upper bound.

Dataset	$N$	$\Delta$ SenseAgent	$\Delta$ Feedback-Adaptive	$\Delta$ Verifier Override
TNDA	0	+0.0027	+0.0027	-0.0298
	1	+0.0035	+0.0034	-0.0544
	8	+0.0003	+0.0009	-0.0526
HHAR	0	-0.0002	-0.0002	-0.4524
	1	0.0000	0.0000	-0.5079
	8	-0.0002	+0.0001	-0.5794

Table 6. Difference between GPT-5.4 and Qwen3.5-9B planners on TNDA and HHAR. Values are weighted by the number of domains. Positive values indicate GPT-5.4 is higher.

Panel of Figure 5 — Figure 5. Route ablation under zero-feedback and scarce-feedback settings. SenseAgent uses verified routing to keep the beneficial route and suppress harmful adaptation.

Data	N	Gravity LoRA	Gravity Proto.	Gravity Verifier	Target Memory	Source Reliab.	Static Geometry
Feng	0	0.000	+0.002	-0.190	-0.022	+0.002	+0.004
HHAR	0	0.000	0.000	-0.031	+0.006	+0.001	0.000
PAMAP2	0	-0.107	-0.104	-0.071	-0.046	-0.016	-0.051
TNDA	0	+0.004	-0.074	-0.008	-0.056	+0.018	-0.042
Feng	1	0.000	+0.005	-0.132	-0.014	+0.001	+0.005
HHAR	1	0.000	0.000	-0.013	+0.005	0.000	0.000

Table 7. Component ablation results. Values are macro-F1 changes relative to the full SenseAgent, where negative values indicate removing the component hurts performance.

Raw Route / Window	Gravity Route / Window	Report E2E	Cold LLM Generation	In/Out Tokens
7.82 ms	14.91 ms	1.76 s	10.52 s	2.87K / 60.3

Table 8. Deployment overhead under zero feedback. Report E2E is measured per target report; cold LLM cost is measured per generated action.

Methods	Sensor	LLM	Runtime	Target	Adapt
	Evidence	Reason	Plan	Memory	Verifier
Semantic HAR Alignment ( Yan et al., 2025 )	✓	✓	✗	✗	✗
LLM-IMU Sensing ( Li et al., 2025 ; Hong et al., 2025 ; Zhang et al., 2026 )	✓	✓	✗	✗	✗
IMU Reasoning ( Asif Imran Shouborno et al., 2025 ; Zhang et al., 2025 )	✓	✓	✗	✗	✗
IoT-LLM ( An et al., 2026 )	✓	✓	✗	✗	✗
AutoIoT ( Shen et al., 2025 )	✗	✓	✓	✗	✗

Table 9. Comparison with LLM-based sensing systems.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Template content
Role	You are SenseAgent, an action planner for cross-domain IMU activity recognition. You do not classify raw IMU windows directly; you propose a prediction route, a feedback-use policy, and a memory update from a structured diagnosis report.
Input constraints	You only see a JSON diagnosis report containing label-free target evidence, source-calibrated route/tool summaries, memory statistics, gravity/prototype/style reliability, and optional user-provided feedback anchors. Do not use dataset name, target domain identity, user identity, device identity, body position, OOD type, or hidden target labels.
Action space	Choose a route instantiated by tool actions from raw_patchtst , safe_groupwise , gravity_proto_tool , raw_proto_tool , and style_normalization , with gravity variant auto , lora , or base ; choose feedback use from conservative freeze, local adjustment, propagation, fusion, or fusion-propagation; and choose memory update from append , append_filtered , and freeze .
Decision rule	Use candidate utilities, raw stability, shift diagnostics, gravity reliability, prototype support, style reliability, and feedback evidence as weak evidence rather than fixed rules. Prefer conservative actions when raw inference is stable or support for specialized routes is weak. Proposed actions are checked by downstream verifiers and converted into final executable routes only if accepted.
Output format	Return JSON only: {"route":..., "feedback_use":..., "memory_update":..., "rationale":...} .

Appendix

Table 10. Prompt template for the SenseAgent planner.

$\tau_{s}$	$\tau_{r}$	Macro-F1	No-Harm	Override
0.35	0.65	0.6657	0.734	0.458
0.35	0.75	0.6655	0.733	0.457
0.35	0.85	0.6607	0.684	0.390
0.45	0.65	0.6935	0.854	0.601
0.45	0.75	0.6933	0.851	0.598
0.45	0.85	0.6882	0.799	0.517

Appendix

Table 11. Zero-feedback sensitivity of the route verifier thresholds. No-Harm reports the fraction of target domains whose replayed route is no worse than raw PatchTST.

Explore similar work

CSI-Agent: LLM-Assisted Few-Shot Adaptation for Cross-Domain Wi-Fi CSI Sensing
Sep 25, 2026Tianya Zhao, Chuan Liu, Xuyu WangFew-Shot Domain Adaptation Wi-Fi Sensing
Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, FL 33199, US
SensingAgents: A Multi-Agent Collaborative Framework for Robust IMU Activity Recognition
May 6, 2026Naiyu Zheng, Tianlong Yu, Haochen Yin +3Multi-Agent LLM Systems Human Activity Recognition
City University of Hong Kong Hong Kong · Hubei University China · The Chinese University of Hong Kong Hong Kong +1
HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition
Aug 27, 2026Zihan Ding, Liyu Zhang, Xiaomin OuyangHuman Activity Recognition Multimodal Foundation Models
Hong Kong University of Science and Technology Hong Kong, China

Sep 25, 2026J/K move · Enter open · S save

CSI-Agent: LLM-Assisted Few-Shot Adaptation for Cross-Domain Wi-Fi CSI Sensing

Tianya Zhao, Chuan Liu, Xuyu Wang

Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, FL 33199, US

Few-Shot Domain Adaptation Wi-Fi Sensing

Open paper AskPDF arXiv

Wi-Fi channel state information (CSI) has enabled device-free sensing applications such as human activity recognition. However, CSI sensing models remain brittle in cross-domain deployment, where changes in users or environments can produce incorrect predictions. Existing solutions usually treat this problem as an offline model-design problem, by pretraining a stronger representation or applying one fixed adaptation method to the entire target domain. In practice, labeled target data are scarce and different classes may fail in different ways under the same domain shift. To address this, we propose CSI-Agent, an evidence-seeking LLM agent that reformulates cross-domain CSI adaptation as a deployment-time decision-making problem. Rather than processing raw CSI or making sample-level predictions, CSI-Agent summarizes target-domain behavior into sensing-grounded class-level evidence. It establishes a strong target-adaptive default from complementary CSI views and uses an LLM planner to determine whether each class should retain the default or invoke a specialized action. Deterministic verification and bounded execution further reduce unreliable interventions. We evaluate CSI-Agent on four public datasets using five cross-domain splits covering device, user, environment, and compositional shifts. Under 1-shot adaptation, CSI-Agent achieves the best target-domain performance across all splits and improves the average Macro-F1 by about 16% compared to the strongest baseline method.

Abstract

Explore similar work

CSI-Agent: LLM-Assisted Few-Shot Adaptation for Cross-Domain Wi-Fi CSI Sensing

SensingAgents: A Multi-Agent Collaborative Framework for Robust IMU Activity Recognition

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition