Indoor context deciphered by ambient sensing underpins many applications, from assisted living to smart buildings, yet remains weakly connected to the language-aligned paradigm of modern multimodal systems. Through billions of IoT devices, ambient WiFi Channel State Information (CSI) captures the indoor context of human activity and environmental dynamics passively via ubiquitous, always-on infrastructure. However, unlike vision or audio, CSI lacks naturally paired text like captions or transcripts, limiting its integration with language-driven AI systems. We present CSI-TextBench, a large-scale dataset and benchmark for aligning CSI data with natural language, unifying 3 sensing benchmarks into 285,790 samples across 7 tasks and pairing them with 2,291 hierarchically structured, label-derived descriptions via many-to-many assignment to construct 2.0 million CSI-text training tuples. Benchmarking 5 alignment methods from 4 paradigm families, we show that contrastive CSI-text alignment matches supervised classifiers without class labels, adapts to unseen classes with few samples, and responds to open-vocabulary natural-language prompts. A single unified model supports compositional scene understanding, answering joint queries over activity, proximity, and identity, achieving 86.9% top-1 accuracy on 140 composite scenes, outperforming task-specific classifier ensembles. CSI-TextBench enables ubiquitous ambient sensing as a language-indexed perception layer, enabling indoor contextual awareness in language-driven AI systems.
Figures & tables
Figure 1: CSI-TextBench overview. CSI-TextBench pairs WiFi CSI recordings with natural-language descriptions across seven ambient sensing tasks. The benchmark supports language-grounded sensing, including QA, scene understanding, and open-vocabulary recognition.
Work
Signal
Open Source
Alignment
#Task
#Class
#Sample
#Pair
WiFi2Cap [ 30 ]
WiFi CSI
✗
✓
1
100
N/A.
N/A.
Wi-Chat [ 37 ]
WiFi CSI
✗
✗
1
4
393
393
Wi-CLIP [ 38 ]
WiFi CSI
✗
✓
1
6
N/A
N/A
mmCLIP [ 4 ]
mmWave radar
✓
✓
1
> 200
N/A
N/A
RadarLLM [ 14 ]
mmWave radar
✗
✓
1
N/A
13,308
13,308
TRACE [ 18 ]
FMCW radar
✗
✓
1
126
25,830
25,830
Table 1: Sensing-language datasets/benchmarks. N/A = not reported.
Task
Dataset
Classes
CSI Samples
Unique Texts
Texts/Sample
Total Pairs
HAR
CSI-Bench
5
55,684
221
9
501,156
Proximity
CSI-Bench
4
55,684
216
9
501,156
User ID
CSI-Bench
7
55,684
224
8
445,472
Fall Detection
CSI-Bench
2
6,700
10
5
33,500
Motion Source
CSI-Bench
4
60,858
20
5
304,290
Sign Recognition
SignFi
276
8,280
1,380
5
41,400
Table 2: Dataset statistics for CSI-TextBench. The three multitask CSI-Bench tasks (HAR, Proximity, User ID) share the same 55,684 underlying CSI recordings; each appears separately. The 285,790 task-level instances correspond to 174,422 unique recordings.
Method
HAR
Proximity
User ID
Fall
MotSrc
SignFi
XRF55
CLIP InfoNCE
97.8 ± 0.6
94.6 ± 0.4
100.0 ± 0.0
94.2 ± 0.0
99.5 ± 0.2
97.0 ± 1.0
77.5 ± 0.4
SigLIP
95.7 ± 2.8
84.4 ± 3.8
100.0 ± 0.0
95.5 ± 0.5
99.5 ± 0.3
71.8 ± 32.2
74.5 ± 1.2
LanguageBind
95.1 ± 2.8
76.0 ± 3.3
100.0 ± 0.0
95.3 ± 0.3
99.4 ± 0.1
75.7 ± 29.8
77.6 ± 0.4
Barlow Twins
92.6 ± 1.1
88.8 ± 5.6
98.8 ± 1.3
59.8 ± 6.8
63.2 ± 1.4
61.3 ± 27.4
65.1 ± 3.8
Masked Pred.
94.2 ± 2.8
92.1 ± 2.7
99.2 ± 0.6
94.1 ± 0.6
96.1 ± 1.8
95.7 ± 6.0
78.3 ± 2.0
Supervised
97.9
96.3
100.0
95.5
99.8
99.4
82.9
Table 3: Zero-shot classification accuracy (%, mean ± std over 3 seeds).
CSI → Text R@1
R@5
R@10
Method
HAR
Proximity
User ID
Fall
MotSrc
SignFi
XRF55
SignFi
XRF55
SignFi
CLIP InfoNCE
90.2
94.9
99.9
94.2
48.3
95.8
78.0
99.1
95.4
99.7
SigLIP
91.9
84.7
100.0
95.9
48.0
26.4
73.0
57.9
93.9
72.3
LanguageBind
91.2
79.6
100.0
95.6
48.3
33.6
77.6
57.5
94.0
69.5
Barlow Twins
94.1
80.9
99.8
61.6
20.3
22.7
66.4
55.3
81.4
71.0
Masked Pred.
94.6
95.5
98.9
93.7
47.6
87.3
75.4
97.7
93.2
99.6
Table 4: Cross-modal retrieval on R@1(%). R@5 and R@10 for large-class tasks.
Zero-shot Accuracy
Retrieval (held-out)
Few-shot Accuracy
Seen (200)
Held-out (76)
R@5
R@10
Med. Rank
1-shot
5-shot
10-shot
Performance (%)
96.4
11.3
29.7
46.1
12
54.0 ± 3.0
64.7 ± 1.5
74.6 ± 1.1
Table 5: Unseen-class generalization on SignFi. Retrieval metrics (R@ K , Med. Rank) are on held-out classes only.
Train template
Layperson
Expert (raw)
Expert (anch.)
Sensor (raw)
Sensor (anch.)
Accuracy (%)
95.9
96.0
59.1
96.2
35.4
77.1
Table 6: Open-vocabulary recognition accuracy on HAR (zero-shot).
Training text
Train template
Layperson
Expert (raw)
Sensor (raw)
Sensor (ens.)
Held-out free-form
Template
95.9
96.0
59.1
35.4
76.4
95.0
Free-form
94.5
93.9
87.0
80.2
84.0
94.0
Table 7: HAR zero-shot accuracy (%) of template- and free-form-supervised models. Ens. averages five unseen prompts per class; held-out free-form is the mean over held-out human-written prompt families.
HAR
Proximity
User ID
Fall
MotSrc
SignFi
XRF55
Accuracy (%)
96.0
84.5
100.0
95.9
98.2
96.1
79.2
Table 8: Unified multitask model: zero-shot accuracy across 7 tasks, no task-specific heads. F1 within 1 pp throughout.
HAR
Proximity
User ID
Joint / Top-1
Top-3
Top-5
MRR
L1: Per-attribute
94.6
82.3
88.3
70.4
–
–
–
L2: Composite
96.6
89.2
99.9
86.9
96.9
98.2
0.919
Table 9: Ambient scene QA and composite-scene retrieval accuracy (%) on the co-labeled CSI-Bench subset.
Ablation
Variant
Zero-shot Accuracy
F1
LP Accuracy
CSI Encoder (HAR)
ResNet-18 1D
98.7%
98.7%
99.0%
ViT-1D
95.5%
95.6%
96.3%
PatchTST
94.1%
94.2%
95.8%
Text Encoder (HAR)
CLIP ViT-B/32
98.7%
98.7%
99.0%
MiniLM-L6-v2
90.2%
90.5%
95.7%
Text Granularity (HAR / Prox)
Level 1
98.0% / 94.6%
98.1% / 94.7%
98.7% / –
Table 10: Design ablations. Underlined : default. LP = linear probe on frozen embeddings.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Attribute
Question
Candidate Answers
Activity
“What is the person doing?”
{walking, running, jumping, sitting and breathing, waving}
Table 11: Ambient QA question templates and candidate answers.
Text → CSI R@1
R@5
R@10
Method
HAR
Proximity
User ID
Fall
MotSrc
SignFi
XRF55
SignFi
XRF55
SignFi
CLIP InfoNCE
100.0
100.0
100.0
100.0
100.0
100.0
80.0
99.3
82.5
98.2
SigLIP
100.0
100.0
100.0
100.0
100.0
46.4
92.7
32.0
93.1
35.0
LanguageBind
80.0
25.0
83.3
50.0
100.0
35.5
52.7
23.7
52.0
24.2
Barlow Twins
100.0
100.0
100.0
0.0
100.0
31.9
78.2
26.0
81.5
28.9
Masked Pred.
100.0
100.0
100.0
100.0
100.0
98.2
89.1
93.8
88.7
93.6
Appendix
Table 12: Text-to-CSI retrieval. Left: R@1 (%) across all tasks. Right: R@5 and R@10 for large-class tasks. Bold : best per column.
Relevance
Hit@1
mAP
R-Precision
Strict
0.970
0.953
0.924
Class-level
0.990
0.804
0.668
Appendix
Table 13: CSI → Text retrieval over the full 221-caption HAR gallery (CLIP InfoNCE).
Training setting
Open-set (140-way)
Closed-set (12-way)
Full-data reference
58.5
99.2
Joint caption withheld
49.5
89.8
Scene withheld
13.9
63.5
Appendix
Table 14: Level 2 top-1 accuracy (%) on the 1,306 test samples of the 12 held-out combinations.
Shift
Supervised
Zero-shot aligned
1-shot
5-shot
10-shot
20-shot
Cross-environment
36.9
31.9
36.6 ± 4.8
36.9 ± 4.4
40.7 ± 3.3
45.4 ± 5.3
Cross-device
39.6
39.9
43.1 ± 6.0
46.7 ± 2.3
48.8 ± 7.0
55.0 ± 3.5
Appendix
Table 15: HAR accuracy (%) under cross-environment and cross-device shift. K -shot results are mean ± std over 5 trials.
Task
Source
Published
Ours (Sup.)
Ours (Zero-Shot)
HAR
CSI-Bench [ 41 ]
75.4
97.9
97.8
Proximity
CSI-Bench [ 41 ]
77.5
96.3
94.6
User ID
CSI-Bench [ 41 ]
99.5
100.0
100.0
Fall Detection
CSI-Bench [ 41 ]
94.9
95.5
94.2
Motion Source
CSI-Bench [ 41 ]
99.6
99.8
99.5
SignFi
SignFi [ 16 ]
94.8
99.4
97.0
Appendix
Table 16: CSI-TextBench results in context with published source-dataset numbers. All evaluations are in-domain.
Figure 2: t-SNE visualization of aligned CSI and text embeddings for XRF55 action recognition (55 classes, CLIP-InfoNCE). Circles denote CSI sample embeddings; stars denote text prototype embeddings. Matching colors indicate the same action class.
WiFi sensing based on Channel State Information (CSI) promises ubiquitous, device-free perception, yet current research remains trapped in a Tower of Babel - fragmented into isolated silos where models are tailored to specific hardware dialects, fixed environments, and narrow tasks. The primary bottleneck is the Heterogeneity Gap: the disparity in signal dimensions, sampling rates, and semantic labels that prevents cross-system understanding. To bridge this gap, we propose a foundation-model framework that treats CSI not merely as raw signals but as a structured language with a learnable universal grammar. We first curate and standardize a large collection of heterogeneous real-world CSI datasets, establishing a unified infrastructure that allows incompatible signal formats to be treated as a single corpus. Second, we introduce a modular architecture that acts as a universal translator where lightweight dataset-specific adapters tokenize diverse signal inputs into a shared latent vocabulary, while a shared self-supervised Transformer backbone learns the temporal syntax of human motion and environmental dynamics. This design decouples sensing semantics from hardware syntax. Extensive evaluations show that by mastering this universal language, our approach consistently outperforms task-specific baselines and exhibits strong generalization capability in new environments, achieving superior efficiency in few-shot scenarios. By effectively absorbing heterogeneity, the framework offers a path toward robust, general-purpose wireless sensing, mirroring the linguistic generalization observed in Large Language Models. The code implementation is available at: https://github.com/cjychenjiayi/WiLLM.
Jiayi Chen, Weiting Ou, Guangxu Zhu
Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute (SLAI)
Privacy-preserving semantic understanding of human activities is important for indoor sensing, yet existing Wi-Fi CSI-based systems mainly focus on pose estimation or predefined action classification rather than fine-grained language generation. Mapping CSI to natural-language descriptions remains challenging because of the semantic gap between wireless signals and language and direction-sensitive ambiguities such as left/right limb confusion. We propose WiFi2Cap, a three-stage framework for generating action captions directly from Wi-Fi CSI. A vision-language teacher learns transferable supervision from synchronized video-text pairs, and a CSI student is aligned to the teacher's visual space and text embeddings. To improve direction-sensitive captioning, we introduce a Mirror-Consistency Loss that reduces mirrored-action and left-right ambiguities during cross-modal alignment. A prefix-tuned language model then generates action descriptions from CSI embeddings. We also introduce the WiFi2Cap Dataset, a synchronized CSI-RGB-sentence benchmark for semantic captioning from Wi-Fi signals. Experimental results show that WiFi2Cap consistently outperforms baseline methods on BLEU-4, METEOR, ROUGE-L, CIDEr, and SPICE, demonstrating effective privacy-friendly semantic sensing.
Tzu-Ti Wei, Chu-Yu Huang, Yu-Chee Tseng +1
PAIRLab, College of AI, National Yang Ming Chiao Tung University, Taiwan, R.O.C
Wi-Fi channel state information (CSI) has enabled device-free sensing applications such as human activity recognition. However, CSI sensing models remain brittle in cross-domain deployment, where changes in users or environments can produce incorrect predictions. Existing solutions usually treat this problem as an offline model-design problem, by pretraining a stronger representation or applying one fixed adaptation method to the entire target domain. In practice, labeled target data are scarce and different classes may fail in different ways under the same domain shift. To address this, we propose CSI-Agent, an evidence-seeking LLM agent that reformulates cross-domain CSI adaptation as a deployment-time decision-making problem. Rather than processing raw CSI or making sample-level predictions, CSI-Agent summarizes target-domain behavior into sensing-grounded class-level evidence. It establishes a strong target-adaptive default from complementary CSI views and uses an LLM planner to determine whether each class should retain the default or invoke a specialized action. Deterministic verification and bounded execution further reduce unreliable interventions. We evaluate CSI-Agent on four public datasets using five cross-domain splits covering device, user, environment, and compositional shifts. Under 1-shot adaptation, CSI-Agent achieves the best target-domain performance across all splits and improves the average Macro-F1 by about 16% compared to the strongest baseline method.
Tianya Zhao, Chuan Liu, Xuyu Wang
Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, FL 33199, US