Cross-Layer Co-Optimized LSTM Accelerator for Real-Time Gait Analysis

Mohammad Hasan Ahmadilivani; Levent Aksoy; Mohammad Eslami; Jaan Raik; Alar Kuusik

cs.ARApr 15, 2026

Cross-Layer Co-Optimized LSTM Accelerator for Real-Time Gait Analysis

Authors: Mohammad Hasan Ahmadilivani, Levent Aksoy, Mohammad Eslami, Jaan Raik, Alar Kuusik

Organizations: Tallinn University of Technology, Tallinn, Estonia

Abstract

Long Short-Term Memory (LSTM) neural networks have penetrated healthcare applications where real-time requirements and edge computing capabilities are essential. Gait analysis that detects abnormal steps to prevent patients from falling is a prominent problem for such applications. Given the extremely stringent design requirements in performance, power dissipation, and area, an Application-Specific Integrated Circuit (ASIC) enables an efficient real-time exploitation of LSTMs for gait analysis, achieving high accuracy. To the best of our knowledge, this work presents the first cross-layer co-optimized LSTM accelerator for real-time gait analysis, targeting an ASIC design. We conduct a comprehensive design space exploration from software down to layout design. We carry out a bit-width optimization at the software level with hardware-aware quantization to reduce the hardware complexity, explore various designs at the register-transfer level, and generate alternative layouts to find efficient realizations of the LSTM accelerator in terms of hardware complexity and accuracy. The physical synthesis results show that, using the 65 nm technology, the die size of the accelerator's layout optimized for the highest accuracy is 0.325 mm^2, while the alternative design optimized for hardware complexity with a slightly lower accuracy occupies 15.4% smaller area. Moreover, the designed accelerators achieve accurate gait abnormality detection 4.05x faster than the given application requirement.

Figures & tables

Fig. 1: (a) LSTM NN structure; (b) LSTM cell.

Parameters	$U^{f,i,g,o}$	$W^{f,i,g,o}$	$B^{f,i,g,o}$	$W^{FC1}$	$B^{FC1}$	$W^{FC2}$	$B^{FC2}$
# of cells/neurons	20	20	20	20	20	2	2
# of gates	4	4	4	-	-	-	-
# of parameters in a gate/neuron	20	4	1	20	1	20	1
Total	1600	320	80	400	20	40	2

TABLE I: Number of parameters in all layers of the LSTM NN.

Metrics	Accuracy	F1-score
Ataxia	87.53%	72.28%
Diplegia	81.48%	74.74%
Hemiplegia	87.11%	67.47%
Parkinson’s	82.08%	72.50%

TABLE II: Accuracy and F1-score of the LSTM NN.

Fig. 2: Steps in the design of an LSTM NN hardware accelerator.

Fig. 3: LSTM accelerator design in hardware.

Fig. 4: Exploration of degradation in accuracy (left) and F1-score (right) over different bit-widths of parameters and operations. Green color represents lower degradation, yellow, orange, and red colors represent higher degradation, respectively.

Configuration	#1	#2	#3	#4	#5	#6	#7
Parameter bit-width	(10,8)	(10,8)	(10,8)	(9,7)	(9,7)	(9,7)	(8,6)
Operation bit-width	(13,8)	(13,9)	(12,8)	(13,8)	(13,9)	(12,8)	(13,9)

TABLE III: Selected fixed-point configurations.

Parameter	Operation	Config.	Area ( ${\mu m}^{2}$ )	Delay ( $ns$ )	Power ( $nW$ )
FxP(10,8)	FxP(13,8)	#1	104633	15.6	720963
	FxP(13,9)	#2	104487	14.7	722755
	FxP(12,8)	#3	96345	14.5	686553
FxP(9,7)	FxP(13,8)	#4	100283	15.5	670316
	FxP(13,9)	#5	100153	15.1	662930
	FxP(12,8)	#6	92152	14.6	474603

TABLE IV: Gate-level synthesis results of accelerator designs.

Design	Area ( ${\mu m}^{2}$ )	Delay ( $ns$ )	Power ( $nW$ )
1	89996	15.2	659818
2	93161	7.4	3330029
3	93696	6.9	3604827
4	95448	6.4	3954104
5	98255	5.9	4649098
6	100113	5.4	5328803

TABLE V: Gate-level synthesis results of the design with the configuration #7 under strict delay constraints.

Target output	Max error	Average error
i, f, g, o gates	0.01562	0.00143
tanh, sigmoid	0.00390	0.00025
C, H	0.05078	0.01566
Neurons in FC	0.01562	0.00104
NN full simulation	0.03125	0.01249

TABLE VI: Error in the components implemented in hardware compared to their software implementations for configuration #4.

#Configuration	Worst accuracy degradation from software	Worst F1-score degradation from software	Worst accuracy degradation from full-precision	Worst F1-score degradation from full-precision
#1	0.41%	0.49%	0.89%	1.34%
#2	0.18%	0.05%	1.01%	1.15%
#3	0.11%	0.14%	0.80%	1.28%
#4	0.25%	0.22%	0.53%	0.71%
#5	-0.03%	0.13%	0.50%	0.49%
#6	0.06%	0.001%	0.50%	0.72%

TABLE VII: Worst-case accuracy and F1-score degradation in hardware across all datasets for different bit-width configurations.

Panel of Figure 12 — Fig. 5: Layouts with routed view of the selected designs: (a) configuration #7; (b) configuration #5.

Panel of Figure 13 — Fig. 6: Layouts with placement view of the selected designs: (a) configuration #7; (b) configuration #5.

Design	configuration #7	configuration #5	Gain
Total area ( ${\mathrm{\SIUnitSymbolMicro m}}^{2}$ )	152,369	174,537	12.70%
Slack ( $\mathrm{ns}$ )	32.224	31.372	NA
Internal Power ( $\mathrm{mW}$ )	1.233	1.372	10.13%
Switching Power ( $\mathrm{mW}$ )	0.588	0.659	10.77%
Leakage Power ( $\mathrm{mW}$ )	0.006	0.007	14.27%
Total Power ( $\mathrm{mW}$ )	1.827	2.038	10.35%

TABLE VIII: Physical synthesis results.

Design	Ours	[ 12 ]	[ 13 ]	[ 14 ]	[ 15 ]
Application	Gait analysis	Speech recognition	Language processing	Language modeling	Speech recognition
Technology (nm)	65	65	40	65	65
Area (mm 2 )	0.152	1.57	0.45	2.62	7.74
Power (mW)	1.827	1.24	6.16	20.4	67.3
On-Chip Memory (KB)	2.704	82	88.5	106	297
Voltage (V)	1.2	0.75	1.1	1.1	1.1

TABLE IX: Comparison with the state-of-the-art LSTM NNs.

AI Accelerator Inference

AI accelerator inference studies hardware and software for running neural networks faster and more efficiently across data-center, edge, and mobile devices.

See all 103 papers

Exploiting temporal parallelism for LSTM Autoencoder acceleration on FPGA
Mar 14, 2026Aimilios Leftheriotis, Dimosthenis Masouros, Dimitrios Soudris +1FPGA-Based Neural Network Acceleration Efficient Neural Network Inference
University of Patras, Patras, Greece · National Technical University of Athens, Athens, Greece
Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding
Apr 27, 2026Wang Fan, Wei Cao, Xi Zha +5Self-Attention Long-Context Language Model Inference
Fudan University
On-Device Neural Architecture Search
Jun 12, 2026Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo +2Hardware-Aware NAS Edge ML
Department of Electrical, Electronic, Telecommunication Engineering and Naval Architecture, DITEN University of Genoa, Genoa 16145, Italy · Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Piazza Martiri della Libert`a 33, Pisa 56127, Italy · Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, Ghezzano, 56010 Pisa, Italy +1

Mar 14, 2026J/K move · Enter open · S save

Exploiting temporal parallelism for LSTM Autoencoder acceleration on FPGA

Aimilios Leftheriotis, Dimosthenis Masouros, Dimitrios Soudris, George Theodoridis

University of Patras, Patras, Greece · National Technical University of Athens, Athens, Greece

FPGA-Based Neural Network Acceleration Efficient Neural Network Inference Autoencoders

Open paper AskPDF arXiv

Recurrent Neural Networks (RNNs) are vital for sequential data processing. Long Short-Term Memory Autoencoders (LSTM-AEs) are particularly effective for unsupervised anomaly detection in time-series data. However, inherent sequential dependencies limit parallel computation. While previous work has explored FPGA-based acceleration for LSTM networks, efforts have typically focused on optimizing a single LSTM layer at a time. We introduce a novel FPGA-based accelerator using a dataflow architecture that exploits temporal parallelism for concurrent multi-layer processing of different timesteps within sequences. Experimental evaluations on four representative LSTM-AE models with varying widths and depths, implemented on a Zynq UltraScale+ MPSoC FPGA, demonstrate significant advantages over CPU (Intel Xeon Gold 5218R) and GPU (NVIDIA V100) implementations. Our accelerator achieves latency speedups up to 79.6x vs. CPU and 18.2x vs. GPU, alongside energy-per-timestep reductions of up to 1722x vs. CPU and 59.3x vs. GPU. These results, including superior network depth scalability, highlight our approach's potential for high-performance, real-time, power-efficient LSTM-AE-based anomaly detection on FPGAs.

Abstract

AI Accelerator Inference

Explore similar work

Exploiting temporal parallelism for LSTM Autoencoder acceleration on FPGA

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

On-Device Neural Architecture Search