TSFFormer: A Frequency-Domain Spike Transformer for Green Human Activity Recognition
Organizations: Xidian University, Xi’an 710071, China
Abstract
Wearable Inertial Measurement Units (IMUs) enable pervasive Human Activity Recognition (HAR), but traditional Deep Neural Networks (DNNs) suffer from an "arithmetic-latency deadlock" due to dense floating-point MAC operations. While Spiking Neural Networks (SNNs) offer ultra-low power, existing ones face spatial blind spots, rigid neurons, and gradient degradation. To address these, we propose TSFFormer, a purely spiking Fourier Transformer for Green HAR. It introduces two key innovations: (1) a MultiHeadFourier attention that performs frequency-domain global mixing, reducing complexity from to , eliminating spatial blind spots, and saving 50% GPU memory; and (2) a dual-compartment Temporal Spiking LIF (TSLIFNode) that utilizes dynamic thresholds to adapt to non-stationary rhythms and mitigate gradient degradation. Extensive experiments on 8 public IMU benchmarks demonstrate that TSFFormer surpasses state-of-the-art SNNs and even the classic DeepConvLSTM, achieving a 3 faster training speed with an energy footprint of 0.1 pJ. Ablation studies confirm the synergistic effect of our modules, providing a highly accurate, ultra-low-power event-driven paradigm for wearable sensing.
Figures & tables
| Dataset | Split | Subjects | Classes | Window Size( ) | Stride | Model input | Sensor Nodes( ) |
|---|---|---|---|---|---|---|---|
| Daily-Sports | [0.70,0.15,0.15] | 8 | 19 | 125 | 50 | 5 | |
| HAR70 | [0.70,0.15,0.15] | 18 | 7 | 100 | 50 | 2 | |
| HuGaDB | [0.70,0.15,0.15] | 18 | 12 | 128 | 64 | 6 | |
| Opportunity | [0.70,0.15,0.15] | 4 | 4 | 60 | 30 | 1 | |
| PAMAP2 | [0.70,0.15,0.15] | 9 | 12 | 200 | 100 | 3 | |
| TNDA | [0.70,0.15,0.15] | 50 | 8 | 200 | 100 | 6 |
| Type | Model | Daily-sports | HAR70 | HuGaDB | Opportunity | PAMAP2 | TNDA | USC-HAD | WISDM |
|---|---|---|---|---|---|---|---|---|---|
| ANN | DeepConvLSTM | 86.22 | 90.60 | 89.73 | 94.10 | 81.16 | 92.28 | 75.72 | 71.23 |
| SNN | LMUformer | 65.78 | 79.47 | 85.62 | 91.06 | 58.01 | 80.36 | 57.99 | 27.69 |
| QKformer | 68.46 | 85.35 | 86.17 | 88.99 | 64.56 | 81.75 | 61.81 | 41.63 | |
| Simple-SNN | 52.54 | 77.22 | 81.47 | 88.75 | 57.40 | 71.84 | 51.26 | 36.62 | |
| SpikeGRU | 6.57 | 48.50 | 29.01 | 45.43 | 12.35 | 14.36 | 11.37 | 12.07 | |
| SpikeRNN | 64.07 | 86.91 | 89.55 | 90.45 | 64.60 | 79.78 | 53.93 | 20.07 |
| Model | Daily-Sports | HAR70 | HuGaDB | Opportunity | PAMAP2 | TNDA | USC-HAD | WISDM |
|---|---|---|---|---|---|---|---|---|
| TSFFormer | 87.89 | 90.54 | 90.42 | 93.79 | 83.33 | 98.25 | 78.65 | 86.35 |
| w/o MultiHeadFourier | 86.09 | 88.95 | 88.50 | 92.25 | 80.85 | 96.89 | 78.94 | 81.18 |
| w/o TSLIFNode | 80.13 | 88.03 | 90.14 | 91.85 | 79.76 | 93.12 | 71.39 | 63.85 |
| w/o Both | 64.03 | 83.94 | 86.86 | 87.96 | 61.48 | 77.36 | 54.14 | 26.07 |
| Model configuration | Epochs to 95% | Mean epoch time (s) | Peak GPU memory (MB) | Best validation (%) | Time to best validation (h) |
|---|---|---|---|---|---|
| TSFFormer | 22 | 188.5 | 10905 | 96.60 | 2.6 |
| w/o MultiHeadFourier | 39 | 567.2 | 20713 | 95.45 | 12.4 |
| w/o TSLIFNode | Not Achieved | 112.2 | 7968 | 92.71 | 1.34 |
| w/o Both | Not Achieved | 294.91 | 10334 | 78.19 | 7.13 |
Spiking Transformers
Spiking Transformers combine attention-based architectures with event-driven spiking neurons to process information using sparse, temporally coded computation.
See all 44 papers