KOALA: Koopman Operator Learning for WiFi-Based Anticipatory Hum

Quang-Anh N. D.; Duc Pham Minh; Thao Phuong Pham; Minh Anh Nguyen; Huan X. Nguyen; Tuan Dang

cs.LGAug 16, 2026

KOALA: Koopman Operator Learning for WiFi-Based Anticipatory Hum

Authors: Quang-Anh N. D., Duc Pham Minh, Thao Phuong Pham, Minh Anh Nguyen, Huan X. Nguyen, Tuan Dang

Organizations: London Digital Twin Research Centre Middesex University · University of Arkansas · Agora Intelligence Lab

Abstract

WiFi Channel State Information (CSI) has emerged as a privacy-preserving alternative to cameras for human pose estimation. However, existing approaches treat pose inference as an instantaneous regression problem and do not model temporal dynamics, making future motion prediction infeasible. Naively applying vision-based prediction methods compounds the estimation noise already present in CSI-derived poses, as autoregressive rollouts amplify errors at every step. We propose KOALA, the framework for human motion prediction directly from WiFi CSI, by lifting noisy CSI-derived pose sequences into a learned Koopman latent space where nonlinear dynamics become linear, enabling multi-horizon prediction via simple matrix-vector products without autoregressive iteration or error accumulation. A residual CSI-conditioned operator resolves the identity attractor problem inherent from Koopman formulations, and an anchor-delta prediction head eliminates the degenerate shortcut of copying the current pose across all horizons. To regularise the lifting and operator jointly, we introduce a Koopman Anchored Latent (KAL) loss that operates in the temporal-encoder feature space, enforcing dynamical consistency across prediction horizons without requiring contrastive, spectral, or auxiliary losses. Experiments on MM-Fi and WiPose show that KOALA achieves robust, consistent performance across both short- and long-term prediction horizons, outperforming all baselines by a substantial margin.

Figures & tables

Figure 1: KOALA predicts future human motion from WiFi CSI and observed skeleton poses, these are lifted into a Koopman latent space, where a single CSI-conditioned linear operator propagates the state to multiple future horizons without autoregressive iteration.

Figure 2: Overview of the KOALA framework. WiFi CSI is encoded by a Mamba CSI encoder whose features are passed to the HPE module and skeleton encoder. The HPE module estimates poses via temporal refinement, joint token construction, per-joint Mamba, and skeleton-biased attention; estimated poses are gradient-detached before entering the skeleton encoder. Dual-stream fusion and a temporal encoder generate contextual motion features, which are lifted into the Koopman latent space. A residual CSI-conditioned operator propagates latent states across prediction horizons without autoregression. Inverse lifting and a delta head decode pose deltas, which are added to an anchor pose for final predictions.

Methods	Metrics	100ms	300ms	500ms	1000ms
MetaFi++ ( Zhou et al., 2023 )	MPJPE $\downarrow$	42.74	41.78	40.62	38.64
MetaFi++ ( Zhou et al., 2023 )	PA-MPJPE $\downarrow$	26.62	26.21	26.22	25.74
HPE-Li ( D. Gian et al., 2024 )	MPJPE $\downarrow$	40.00	39.64	39.98	39.69
HPE-Li ( D. Gian et al., 2024 )	PA-MPJPE $\downarrow$	26.69	26.60	26.92	27.01
GraphPose-Fi ( Chen et al., 2026 )	MPJPE $\downarrow$	31.91	31.36	31.03	31.07
GraphPose-Fi ( Chen et al., 2026 )	PA-MPJPE $\downarrow$	23.48	23.27	23.13	23.43

Table 1: Performance comparison across prediction horizons on the WiPose dataset.

Setting	Metric	Protocol 1						Protocol 2						Protocol 3
Setting	Metric	100	300	500	1000	1500	2000	100	300	500	1000	1500	2000	100	300	500	1000	1500	2000
S1	MPJPE $\downarrow$	54.5	58.2	62.5	66.7	65.0	63.6	53.2	56.9	60.8	64.0	62.1	61.0	55.2	62.3	69.2	76.2	73.3	69.8
	PA-MPJPE $\downarrow$	45.5	48.0	51.2	54.1	53.4	52.6	44.3	46.6	49.1	51.0	50.2	49.8	46.5	52.6	58.3	61.3	63.2	60.1
	PCK@20 $\uparrow$	91.0	89.5	87.6	85.6	86.5	87.3	91.3	89.9	88.2	86.6	87.5	88.3	90.6	87.8	85.0	82.3	83.7	85.3
	PCK@10 $\uparrow$	72.5	69.9	67.3	64.7	65.9	67.1	73.1	70.6	68.1	66.0	67.2	68.3	72.0	68.3	65.1	62.3	64.1	65.8
S2	MPJPE $\downarrow$	55.7	62.0	68.4	74.7	71.9	68.7	52.8	58.2	63.5	68.0	65.4	63.5	53.1	59.0	64.5	68.9	66.4	64.4

Table 2: Evaluation performance across protocols and settings at different horizons (ms) on the MM-Fi.

Methods	Metrics	100ms	300ms	500ms	1000ms	1500ms	2000ms
HPE-Li ( D. Gian et al., 2024 )	MPJPE $\downarrow$	375.2	383.4	351.6	351.2	339.8	368.5
	PA-MPJPE $\downarrow$	120.7	122.3	119.4	118.4	119.5	122.8
	PCK@20 $\uparrow$	3.8	4.3	7.0	8.6	10.0	6.3
	PCK@10 $\uparrow$	0.3	0.2	0.8	1.0	1.1	0.5
MetaFi++ ( Zhou et al., 2023 )	MPJPE $\downarrow$	311.7	308.4	311.2	312.8	316.9	312.0
	PA-MPJPE $\downarrow$	105.5	105.4	104.6	104.7	105.1	105.1

Table 3: Performance comparison across prediction horizons on the MM-Fi dataset.

Figure 3: Qualitative results between predicted poses and Ground Truth (GT) for different time steps.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Phase	Epochs	$\alpha$	Skeleton encoder input
Warmup	1-6	$1.0$	Ground-truth $p_{t}$
Decay	7-12	$1-(e-6)/6\;\in(1,0)$	Interpolation
End-to-end	13-20	$0.0$	$\mathrm{sg}(\tilde{p}_{t})$

Appendix

Table 4: Scheduled settings.

Figure 4: Additional qualitative results between predicted poses and GT for different time steps.

Metric	MM-Fi	WiPose
Total Parameters	4.8M	4.5M
FLOPs	0.083G	0.083G
Inference Latency	3.4 ms	3.1 ms
GPU Memory (FP32)	19 MB	18 MB
GPU Memory (INT8)	5 MB	5 MB
FPS	$\sim$ 292	$\sim$ 324

Appendix

Table 5: KOALA efficiency metrics on MM-Fi and WiPose settings.

Table 6: Per-joint MPJPE (mm) of KOALA on MM-Fi and WiPose. Lower is better.

	100ms	300ms	500ms	1000ms	1500ms	2000ms
$\epsilon_{h}$	10.6 $\pm$ 1.7	40.3 $\pm$ 5.4	113.7 $\pm$ 17.6	1218.7 $\pm$ 250.1	12120.0 $\pm$ 3076.8	115888.3 $\pm$ 34070.9
rel. $\epsilon_{h}$	0.68 $\pm$ 0.11	2.59 $\pm$ 0.36	7.30 $\pm$ 1.15	78.3 $\pm$ 16.2	778.7 $\pm$ 199.8	7444.4 $\pm$ 2206.6

Appendix

Table 7: Empirical Koopman invariance defect (absolute

\epsilon_{h}

, mean

\pm

std, and relative to target norm) across prediction horizons.

Variant	100ms	300ms	500ms	1000ms	1500ms	2000ms
w/o Residual Operator	55.8	62.4	67.8	72.1	69.7	67.4
w/o Delta Prediction	67.7	64.8	64.6	65.2	65.7	66.5
w/o Dual Stream Fusion	52.7	67.2	80.2	92.8	90.1	78.8
w/o HPE Pretraining	105.4	106.1	106.7	107.7	107.3	106.7
KOALA (Ours)	52.1	57.0	61.9	65.7	63.4	61.9

Appendix

Table 8: Ablation study results of KOALA across prediction horizons. Evaluated on MPJPE (mm), lower values indicate better performance.

Variant	Metric	100ms	300ms	500ms	1000ms	Avg
Oracle	MPJPE $\downarrow$	8.9	13.2	16.4	22.1	15.2
Oracle	PA-MPJPE $\downarrow$	10.3	13.8	16.1	19.6	15.0
KOALA	MPJPE $\downarrow$	26.1	26.0	26.5	27.3	26.5
KOALA	PA-MPJPE $\downarrow$	20.8	20.8	21.0	21.3	21.0
$\Delta$ (Oracle $-$ Ours)	MPJPE $\downarrow$	17.2	12.8	9.6	5.2	11.2
$\Delta$ (Oracle $-$ Ours)	PA-MPJPE $\downarrow$	10.5	7.0	5.1	1.7	6.1

Appendix

Table 9: Oracle upper bound comparison on WiPose dataset.

Variant	Metric	100ms	300ms	500ms	1000ms	1500ms	2000ms	Avg
Oracle	MPJPE $\downarrow$	25.3	42.9	52.5	58.4	56.3	53.9	48.2
Oracle	PA-MPJPE $\downarrow$	22.0	37.0	44.5	47.1	46.2	45.1	40.3
KOALA	MPJPE $\downarrow$	52.1	57.0	61.9	65.7	63.4	61.9	60.3
KOALA	PA-MPJPE $\downarrow$	43.5	47.0	50.5	52.8	51.8	51.1	49.5
$\Delta$ (Oracle $-$ Ours)	MPJPE $\downarrow$	26.8	14.1	9.4	7.3	7.1	8.0	12.1
$\Delta$ (Oracle $-$ Ours)	PA-MPJPE $\downarrow$	21.5	10.0	6.0	5.7	5.6	6.0	9.2

Appendix

Table 10: Oracle upper bound comparison on MM-Fi dataset.

(A) Model dimension $d$
Variant	100ms	300ms	500ms	1000ms	1500ms	2000ms
$d=64$	54.16 / 45.75	58.74 / 48.94	63.32 / 52.19	67.55 / 54.72	65.28 / 53.82	63.59 / 52.94
$d=256$	52.95 / 44.12	56.54 / 46.48	60.49 / 49.30	64.24 / 51.82	62.41 / 50.84	61.31 / 50.40
(B) Latent dimension $D_{z}$
$D_{z}=64$	53.75 / 45.31	61.14 / 51.39	67.85 / 56.91	74.60 / 62.82	72.32 / 62.44	68.60 / 59.23
$D_{z}=128$	54.72 / 45.94	59.86 / 49.56	64.83 / 52.93	69.13 / 55.67	66.83 / 54.80	64.89 / 53.78

Appendix

Table 11: Ablation study of KOALA under different hyperparameter settings. MPJPE / PA-MPJPE (mm), lower is better.

Variant	100ms	300ms	500ms	1000ms	1500ms	2000ms
w/o $\mathcal{L}_{\text{est}}$	132.7 / 116.3	132.8 / 116.1	132.8 / 116.0	132.8 / 116.0	132.8 / 116.0	132.9 / 116.1
w/o $\mathcal{L}_{\text{pred}}$	141.0 / 94.2	150.3 / 102.6	159.1 / 110.1	168.0 / 117.0	166.4 / 117.5	158.0 / 110.0
KOALA (Ours)	52.1 / 43.5	57.0 / 47.0	61.9 / 50.5	65.7 / 52.8	63.4 / 51.8	61.9 / 51.1

Appendix

Table 12: Loss ablation study of KOALA. MPJPE / PA-MPJPE (mm), lower is better.

Variant	100ms	300ms	500ms	1000ms	1500ms	2000ms	Spread
Zero-Velocity	56.3	68.9	80.4	91.9	89.6	78.9	22.6
KOALA (Ours)	52.1	57.0	61.9	65.7	63.4	61.9	9.8

Appendix

Table 13: Zero-velocity baseline (

\hat{\mathbf{p}}_{T+h}=\bar{\mathbf{p}}

, no learned delta) vs. KOALA.

Variant	Metric	100 ms	300 ms	500 ms	1000 ms	1500 ms	2000 ms
DMD ( Schmid, 2010 )	MPJPE $\downarrow$	61.1	72.3	84.4	97.2	95.6	86.8
	PA-MPJPE $\downarrow$	51.6	62.2	72.7	83.3	83.9	75.7
	PCK@20 $\uparrow$	88.4	84.4	80.7	77.2	77.9	80.0
	PCK@10 $\uparrow$	68.9	65.4	62.2	59.3	59.9	61.0
LRU ( Orvieto et al., 2023 )	MPJPE $\downarrow$	54.1	58.4	60.8	61.3	59.2	59.0
	PA-MPJPE $\downarrow$	45.4	49.4	51.5	51.3	49.6	49.6

Appendix

Table 14: Comparison across predictor formulations.

Human Motion Prediction

Human motion prediction forecasts how people will move, supporting applications from animation and activity understanding to safe robot interaction.

See all 53 papers

WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation
Jul 13, 2026Doeon Kim, Jungyoon Lee, Seongsin Kim +1Self-Supervised Learning Joint-Embedding Predictive Architecture
Department of Intelligent Semiconductors, Soongsil University, Republic of Korea · Department of AI Convergence Security, Soongsil University, Republic of Korea · School of AI Software, Soongsil University, Republic of Korea
C-MambaPose: Phase-Preserving Complex Mamba for Cross-Environment Wi-Fi Human Pose Estimation
Jun 5, 2026Phuc H. Nguyen, Van-Dinh NguyenDomain Generalization Complex-Valued Neural Networks
VinUniversity, Hanoi, Vietnam
RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation
Jul 3, 2026Zhangcheng Hou, Tomoaki OhtsukiRelative Pose Estimation Domain Generalization
Faculty of Science and Technology, Keio University, Yokohama 223-8522, Japan

Jul 13, 2026J/K move · Enter open · S save

WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation

Doeon Kim, Jungyoon Lee, Seongsin Kim, Seong-heum Kim

Department of Intelligent Semiconductors, Soongsil University, Republic of Korea · Department of AI Convergence Security, Soongsil University, Republic of Korea · School of AI Software, Soongsil University, Republic of Korea

Self-Supervised Learning Joint-Embedding Predictive Architecture 3D Human Pose Estimation Human Pose Estimation Wi-Fi Sensing

Open paper AskPDF arXiv

WiFi Channel State Information (CSI) enables privacy-preserving human pose sensing in camera-denied environments, but existing WiFi-based pose estimators often fail under environment shifts and rely on costly camera-based annotation pipelines that limit scale. We propose WiFi-JEPA, a self-supervised framework that learns CSI-native representations by predicting masked latent embeddings instead of reconstructing raw CSI signals that may contain hardware-specific artifacts. WiFi-JEPA makes three contributions: (i) CSI-specific tokenization and link masking tailored to the CSI tensor over channel, time, and link (C,T,L); masking entire Tx-Rx antenna links forces the model to predict one spatial link view from others, capturing cross-link correlations informative of 3D spatial structure. (ii) A ray-tracing CSI simulation pipeline that generates diverse unlabeled CSI from randomized geometric primitives, providing scalable pre-training data without pose annotations. (iii) State-of-the-art results on Person-in-WiFi-3D: WiFi-JEPA outperforms prior WiFi-CSI baselines on both single- and multi-person 3D pose estimation under the same evaluation protocol. We also show that simulated CSI provides complementary pre-training signal to real CSI, and that four vision-native SSL objectives degrade performance below training from scratch, whereas WiFi-JEPA consistently improves downstream pose estimation.

Abstract

Human Motion Prediction

Explore similar work

WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation

C-MambaPose: Phase-Preserving Complex Mamba for Cross-Environment Wi-Fi Human Pose Estimation

RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation