Cross-Layer Co-Optimized LSTM Accelerator for Real-Time Gait Analysis

Mohammad Hasan Ahmadilivani; Levent Aksoy; Mohammad Eslami; Jaan Raik; Alar Kuusik

cs.ARApr 15, 2026

Cross-Layer Co-Optimized LSTM Accelerator for Real-Time Gait Analysis

Authors: Mohammad Hasan Ahmadilivani, Levent Aksoy, Mohammad Eslami, Jaan Raik, Alar Kuusik

Organizations: Tallinn University of Technology, Tallinn, Estonia

Abstract

Long Short-Term Memory (LSTM) neural networks have penetrated healthcare applications where real-time requirements and edge computing capabilities are essential. Gait analysis that detects abnormal steps to prevent patients from falling is a prominent problem for such applications. Given the extremely stringent design requirements in performance, power dissipation, and area, an Application-Specific Integrated Circuit (ASIC) enables an efficient real-time exploitation of LSTMs for gait analysis, achieving high accuracy. To the best of our knowledge, this work presents the first cross-layer co-optimized LSTM accelerator for real-time gait analysis, targeting an ASIC design. We conduct a comprehensive design space exploration from software down to layout design. We carry out a bit-width optimization at the software level with hardware-aware quantization to reduce the hardware complexity, explore various designs at the register-transfer level, and generate alternative layouts to find efficient realizations of the LSTM accelerator in terms of hardware complexity and accuracy. The physical synthesis results show that, using the 65 nm technology, the die size of the accelerator's layout optimized for the highest accuracy is 0.325 mm^2, while the alternative design optimized for hardware complexity with a slightly lower accuracy occupies 15.4% smaller area. Moreover, the designed accelerators achieve accurate gait abnormality detection 4.05x faster than the given application requirement.

Figures & tables

Fig. 1: (a) LSTM NN structure; (b) LSTM cell.

Parameters	$U^{f,i,g,o}$	$W^{f,i,g,o}$	$B^{f,i,g,o}$	$W^{FC1}$	$B^{FC1}$	$W^{FC2}$	$B^{FC2}$
# of cells/neurons	20	20	20	20	20	2	2
# of gates	4	4	4	-	-	-	-
# of parameters in a gate/neuron	20	4	1	20	1	20	1
Total	1600	320	80	400	20	40	2

TABLE I: Number of parameters in all layers of the LSTM NN.

Metrics	Accuracy	F1-score
Ataxia	87.53%	72.28%
Diplegia	81.48%	74.74%
Hemiplegia	87.11%	67.47%
Parkinson’s	82.08%	72.50%

TABLE II: Accuracy and F1-score of the LSTM NN.

Fig. 2: Steps in the design of an LSTM NN hardware accelerator.

Fig. 3: LSTM accelerator design in hardware.

Fig. 4: Exploration of degradation in accuracy (left) and F1-score (right) over different bit-widths of parameters and operations. Green color represents lower degradation, yellow, orange, and red colors represent higher degradation, respectively.

Configuration	#1	#2	#3	#4	#5	#6	#7
Parameter bit-width	(10,8)	(10,8)	(10,8)	(9,7)	(9,7)	(9,7)	(8,6)
Operation bit-width	(13,8)	(13,9)	(12,8)	(13,8)	(13,9)	(12,8)	(13,9)

TABLE III: Selected fixed-point configurations.

Parameter	Operation	Config.	Area ( ${\mu m}^{2}$ )	Delay ( $ns$ )	Power ( $nW$ )
FxP(10,8)	FxP(13,8)	#1	104633	15.6	720963
	FxP(13,9)	#2	104487	14.7	722755
	FxP(12,8)	#3	96345	14.5	686553
FxP(9,7)	FxP(13,8)	#4	100283	15.5	670316
	FxP(13,9)	#5	100153	15.1	662930
	FxP(12,8)	#6	92152	14.6	474603

TABLE IV: Gate-level synthesis results of accelerator designs.

Design	Area ( ${\mu m}^{2}$ )	Delay ( $ns$ )	Power ( $nW$ )
1	89996	15.2	659818
2	93161	7.4	3330029
3	93696	6.9	3604827
4	95448	6.4	3954104
5	98255	5.9	4649098
6	100113	5.4	5328803

TABLE V: Gate-level synthesis results of the design with the configuration #7 under strict delay constraints.

Target output	Max error	Average error
i, f, g, o gates	0.01562	0.00143
tanh, sigmoid	0.00390	0.00025
C, H	0.05078	0.01566
Neurons in FC	0.01562	0.00104
NN full simulation	0.03125	0.01249

TABLE VI: Error in the components implemented in hardware compared to their software implementations for configuration #4.

#Configuration	Worst accuracy degradation from software	Worst F1-score degradation from software	Worst accuracy degradation from full-precision	Worst F1-score degradation from full-precision
#1	0.41%	0.49%	0.89%	1.34%
#2	0.18%	0.05%	1.01%	1.15%
#3	0.11%	0.14%	0.80%	1.28%
#4	0.25%	0.22%	0.53%	0.71%
#5	-0.03%	0.13%	0.50%	0.49%
#6	0.06%	0.001%	0.50%	0.72%

TABLE VII: Worst-case accuracy and F1-score degradation in hardware across all datasets for different bit-width configurations.

Panel of Figure 12 — Fig. 5: Layouts with routed view of the selected designs: (a) configuration #7; (b) configuration #5.

Panel of Figure 13 — Fig. 6: Layouts with placement view of the selected designs: (a) configuration #7; (b) configuration #5.

Design	configuration #7	configuration #5	Gain
Total area ( ${\mathrm{\SIUnitSymbolMicro m}}^{2}$ )	152,369	174,537	12.70%
Slack ( $\mathrm{ns}$ )	32.224	31.372	NA
Internal Power ( $\mathrm{mW}$ )	1.233	1.372	10.13%
Switching Power ( $\mathrm{mW}$ )	0.588	0.659	10.77%
Leakage Power ( $\mathrm{mW}$ )	0.006	0.007	14.27%
Total Power ( $\mathrm{mW}$ )	1.827	2.038	10.35%

TABLE VIII: Physical synthesis results.

Design	Ours	[ 12 ]	[ 13 ]	[ 14 ]	[ 15 ]
Application	Gait analysis	Speech recognition	Language processing	Language modeling	Speech recognition
Technology (nm)	65	65	40	65	65
Area (mm 2 )	0.152	1.57	0.45	2.62	7.74
Power (mW)	1.827	1.24	6.16	20.4	67.3
On-Chip Memory (KB)	2.704	82	88.5	106	297
Voltage (V)	1.2	0.75	1.1	1.1	1.1

TABLE IX: Comparison with the state-of-the-art LSTM NNs.

AI Accelerator Inference

AI accelerator inference studies hardware and software for running neural networks faster and more efficiently across data-center, edge, and mobile devices.

See all 103 papers

Mar 14, 2026

Exploiting temporal parallelism for LSTM Autoencoder acceleration on FPGA

Recurrent Neural Networks (RNNs) are vital for sequential data processing. Long Short-Term Memory Autoencoders (LSTM-AEs) are particularly effective for unsupervised anomaly detection in time-series data. However, inherent sequential dependencies limit parallel computation. While previous work has explored FPGA-based acceleration for LSTM networks, efforts have typically focused on optimizing a single LSTM layer at a time. We introduce a novel FPGA-based accelerator using a dataflow architecture that exploits temporal parallelism for concurrent multi-layer processing of different timesteps within sequences. Experimental evaluations on four representative LSTM-AE models with varying widths and depths, implemented on a Zynq UltraScale+ MPSoC FPGA, demonstrate significant advantages over CPU (Intel Xeon Gold 5218R) and GPU (NVIDIA V100) implementations. Our accelerator achieves latency speedups up to 79.6x vs. CPU and 18.2x vs. GPU, alongside energy-per-timestep reductions of up to 1722x vs. CPU and 59.3x vs. GPU. These results, including superior network depth scalability, highlight our approach's potential for high-performance, real-time, power-efficient LSTM-AE-based anomaly detection on FPGAs.

Aimilios Leftheriotis, Dimosthenis Masouros, Dimitrios Soudris +1

University of Patras, Patras, Greece · National Technical University of Athens, Athens, Greece

FPGA-Based Neural Network Acceleration Efficient Neural Network Inference Autoencoders

Ask PDF

Apr 27, 2026

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

Long contexts improve capabilities of large language models but pose serious hardware challenges: compute and memory footprints grow linearly with sequence length. Particularly, the decoding phase continuously accesses massive KV cache, dramatically increasing bandwidth and computing pressure. Existing accelerators are primarily designed and evaluated for short contexts. They suffer from significant performance degradation when processing long contexts. To bridge this gap, we identify the major bottleneck and present a hardware accelerator for long context attention decoding via hardware-software co-design. On the software side, we propose dual-compression dynamic sparse attention. It combines ultra-low-precision quantization with feature sparsity to minimize prediction overhead. A hardware-friendly approximate Top-K selection further reduces filter complexity from

O(n \log k)

O(n)

. On the hardware side, we deeply optimize compute and memory access to tackle bottlenecks from intricate interplay between sparse attention and long contexts, and establish a performance model to derive the optimal co-design scheme. The resulting hardware adopts a fully pipelined parallel architecture and achieves

O(n)

efficiency even for long sequences. Experiments show that our design delivers

3.82\times

speedup and

74.19\times

energy efficiency over A100. Compared to SOTA accelerators, this is the first ASIC accelerator that efficiently supports long context inference, with at least

3.5\times

higher throughput and

2.08\times

better energy efficiency.

Wang Fan, Wei Cao, Xi Zha +5

Fudan University

Self-Attention Long-Context Language Model Inference LLM Inference Acceleration Sparse Attention Attention Mechanisms

Ask PDF

Jun 12, 2026

On-Device Neural Architecture Search

This paper proposes a new approach to near-sensor computing, in which a lightweight Neural Architecture Search (NAS) is performed directly on the deployment device to find the best tiny neural architecture for analyzing the real-time data acquired through sensors. This new adaptation capability can be particularly useful in the case of human-machine interfaces for which the neural network analyzing the biometrical data can be re-designed each time the user changes, after a guided data collection procedure, fighting the typical data variations between individuals on a new level. To implement the proposed approach a new NAS has been designed and then validated on the Italian Sign Language dataset (ISL), a collection of surface electromyography (sEMG) signals of the signs of the Italian alphabet, using several embedded systems. Moreover, further validation on the Case Western Reserve University dataset (CWRU), a benchmark for intelligent fault diagnosis, is presented to suggest another possible application of the proposed approach. When run on a Raspberry Pi 4, the proposed NAS performs beyond the state of the art proposing a tiny neural architecture having 0.63 times less RAM occupancy and 5.96 percentage points of more accuracy in the case of the ISL dataset; and 0.44 times less RAM occupancy and 0.2 percentage points of more accuracy in the case of the CWRU dataset.

Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo +2

Department of Electrical, Electronic, Telecommunication Engineering and Naval Architecture, DITEN University of Genoa, Genoa 16145, Italy · Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Piazza Martiri della Libert`a 33, Pisa 56127, Italy · Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, Ghezzano, 56010 Pisa, Italy +1

Hardware-Aware NAS Edge ML Neural Architecture Search

Ask PDF

Abstract

AI Accelerator Inference

Explore similar work

Exploiting temporal parallelism for LSTM Autoencoder acceleration on FPGA

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

On-Device Neural Architecture Search