FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Vision

Zekai Wu; Shuqi Fan; Mengyin Liu; Yuhua Luo; Xincheng Lin; Ming Yan; Junhao Wu; Xiuhong Lin

cs.CVMar 20, 2026

FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Vision

Authors: Zekai Wu, Shuqi Fan, Mengyin Liu, Yuhua Luo, Xincheng Lin, Ming Yan, Junhao Wu, Xiuhong Lin,

Organizations: Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University · National Institute for Data Science in Health and Medicine, Xiamen University · ShanghaiTech University

Abstract

Precise motion timing (PMT) is crucial for swift motion analysis. A millisecond difference may determine victory or defeat in sports competitions. Despite substantial progress in human pose estimation (HPE), PMT remains largely overlooked by the HPE community due to the limited availability of high-temporal-resolution labeled datasets. Today, PMT is achieved using high-speed RGB cameras in specialized scenarios such as the Olympic Games; however, their high costs, light sensitivity, bandwidth, and computational complexity limit their feasibility for daily use. We developed FlashCap, the first flashing LED-based MoCap system for PMT. With FlashCap, we collect a millisecond-resolution human motion dataset, FlashMotion, comprising the event, RGB, LiDAR, and IMU modalities, and demonstrate its high quality through rigorous validation. To evaluate the merits of FlashMotion, we perform two tasks: precise motion timing and high-temporal-resolution HPE. For these tasks, we propose ResPose, a simple yet effective baseline that learns residual poses based on events and RGBs. Experimental results show that ResPose reduces pose estimation errors by ~40% and achieves millisecond-level timing accuracy, enabling new research opportunities. The dataset and code will be shared with the community.

Figures & tables

Dataset	Sensor Modalities				Label FPS		Dur.	Label Frames (k)	# Seqs.	# Subjs.
Dataset	RGB	MoCap	LiDAR	Event	2D	3D	Dur.	Label Frames (k)	# Seqs.	# Subjs.
LiDARHuman26M [ 33 ]	✔	IMU	✔	-	-	10	5h	184	20	13
HSC4D [ 17 ]	-	IMU	✔	-	-	20	42min	50	8	1
SLOPER4D [ 18 ]	✔	IMU	✔	-	30	20	2.8h	300	15	12
CIMI4D [ 60 ]	✔	IMU	✔	-	-	20	2h	180	42	12
LIPD [ 43 ]	✔	IMU	✔	-	-	10	1.7h	62	10	6

Table 1: Comparisons with related human motion datasets. Datasets with Event camera data are grouped in the second block. Our FlashMotion provides the highest 2D label frame rate (1000Hz) and the largest number of labeled frames (7.15M) by a significant margin. Abbreviations: (Dur.) Duration, (# Seqs.) Number of Sequences, (# Subjs.) Number of Subjects.

Figure 2: The FlashCap Mocap outfit(left) and multi-modal capture device(right).

Figure 3: An example of the FlashCap data annotation pipeline: (0) Event Streams. (1) Identified Event Clusters. (2) Cluster Frequency Analysis. (3) Filtered Clusters After Noise Removal. (4) Matched LED-Cluster Pairs (Labels).

Figure 4: Examples of the FlashMotion dataset. (a) RGB modality, (b) LiDAR point clouds, (c) IMUs measurements, (d) events (50ms), (e) 2D Label/Annotation.

Figure 5: Interpolated vs. Ground-Truth 1000 Hz Poses. (Left) Event frames at selected timestamps. (Middle) Accumulated event stream. (Right) trajectory comparison. The red line indicates 20 Hz interpolation, and the green line indicates 1000 Hz ground truth.

Figure 6: Qualitative evaluation against a high-speed camera: Columns (a), (c), and (e) show that our labels (green spots) overlap with the images captured by high-speed cameras closely. This demonstrate the correctness of FlashMotion labels qualitatively. Columns (b), (d), and (f) show that the corresponding events overlaid on the RGB frames. The red/blue dots indicate events with positive/negative polarity.

Figure 7: Qualitative evaluation when ablating the annotation pipeline: (a) $w/o$ $d_{ji}^{t}$ in Sec. 3.3.4 (b) $w/o$ $d^{p}_{ji}$ in Sec. 3.3.4 . (c) $w/o$ Outlier Filtering (Sec. 3.3.3 ). (d) $w/o$ Tracking. (e) complete pipeline. The name of the identified joints are depicted in red near the location of identified joints. Each joints are connected by blue lines, and the yellow circles in (c) and (d) indicates missed labels due to failed detection and occlusion, respectively.

Figure 8: Architecture of ResPose. It obtain high-temporal resolution pose $P_{i}$ through $P_{i}=P_{\text{rgb}}+P_{i}^{\Delta}$ , where $P_{\text{rgb}}$ is the anchor pose from the RGB branch and $P_{i}^{\Delta}$ is the residual pose at time $i$

Method	Kicking		Crossing Hands
Method	Precision $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
$w/o$ $d_{ji}^{t}$	43.34%	97.80%	23.64%	98.54%
$w/o$ $d^{p}_{ji}$	69.70%	97.56%	64.00%	98.33%
$w/o$ Outlier Filtering	96.52%	95.69%	83.17%	98.50%
$w/o$ Tracking	98.38%	98.16%	84.29%	98.21%
Complete pipeline	99.99%	98.99%	99.99%	98.65%

Table 2: Quantitative evaluation against human annotated labels.

Method	Kicking	Punching	Jumping
ViTPose [ 59 ]	48.5	62.3	31.4
Hybrid ANN-SNN [ 3 ]	85.2	54.1	66.7
LEIR [ 61 ]	112.4	135.8	78.2
ResPose (Ours)	7.2	4.8	6.5

Table 3: The Mean Error of Estimated Time (PMT).Unit:

ms

Method	MPJPE $\downarrow$	PCK0.3 $\uparrow$	PCK0.5 $\uparrow$
ViTPose (linear)	10.06	0.96	0.98
ViTPose (spline)	10.13	0.96	0.98
Hybrid ANN-SNN [ 3 ]	22.48	0.82	0.91
EventPointPose [ 11 ]	51.61	0.48	0.74
LEIR [ 61 ]	59.02	0.35	0.65
GraphEnet [ 22 ]	43.01	0.65	0.83

Table 4: High Temporal Resolution HPE Task in FlashMotion.

Figure 9: Precise timing. (a) (d) (g) (j): Previous RGB frame just before crossing lines, (b) (e) (h) (i): RGB frame of corresponding events just after crossing the lines, (c) (f) (i) (l): The event and human pose at the moment when crossing the line. (a)-(f): Punching. (g)-(i): Kicking. (j)-(l): Jumping.

Figure 10: Qualitative comparison of trajectories for the High Temporal Resolution HPE task over a 50 ms interval. (a) ViTPose. (b) ViT (spline). (c) Hybrid ANN-SNN. (d) EventPointPose. (e) GraphEnet. (f) EvSharp2Blur. (g) LEIR. (h) ResPose (Ours).

Markerless Motion Capture

Markerless motion capture estimates 3D body movement from video or other sensors without attached tracking markers.

See all 67 papers

Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration
Aug 3, 2026Aoru Xue, Yujing Sun, Yiming Ren +3Markerless Motion Capture Multimodal Robustness
ShanghaiTech University, Shanghai, China · ShanghaiTech University, China · Digital Trust Centre, Nanyang Technological University, Singapore +1
Integrated Real-Time Motion Tracking and AI Analysis for Athletic Performance Optimization
Apr 23, 2026Parth Agrawal, Ronit, Sagar Kumar +1Markerless Motion Capture Sports Video Analysis
Department of Computer Science and Engineering, Chandigarh University Punjab, India
E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation
Apr 9, 2026Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin +23D Human Pose Estimation Egocentric Vision
MPI for Informatics, SIC · Saarland University, SIC · Bielefeld University

Aug 3, 2026J/K move · Enter open · S save

Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration

Aoru Xue, Yujing Sun, Yiming Ren, Kwok-Yan Lam+2

ShanghaiTech University, Shanghai, China · ShanghaiTech University, China · Digital Trust Centre, Nanyang Technological University, Singapore+1

Markerless Motion Capture Multimodal Robustness Camera-LiDAR Fusion 3D Human Pose Estimation

Open paper AskPDF arXiv

We propose Sen-Cap, a Sensor-Flexible and Noise-Resilient 3D human motion Capture framework that integrates multi-modal data from LiDAR and camera. While multi-modal sensors provide richer information than single-modal sensors, existing approaches still suffer from two core challenges. First, multi-modal alignment/matching across arbitrarily deployed sensors is typically handled by explicit calibration, which propagates errors under changing viewpoints and in turn constrains deployment to fixed, highly overlapped layouts. Second, prior methods degrade under severe noise or partial sensor failures, which are common in real-world environments. To address these challenges, Sen-Cap introduces a Unified Across-Sensor Motion Estimator that reconstructs local pose and shape in a human-centric space without calibrations between sensors, supporting a flexible number of sensors, as well as a Noise-Resistant Trajectory Tracker that maintains robustness under severe point cloud noise through iterative refinement. These sensor-flexible and noise-resilient features make Sen-Cap more practical in real-world deployment. Notably, operating in real time, Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion, as well as strong cross-domain performance on LiDARHuman26M and RELI11D. This combination of flexibility and robustness opens new opportunities for motion capture in real-world scenarios, e.g. sports analytics, field robotics, and large-scale immersive environments.

Abstract

Markerless Motion Capture

Explore similar work

Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration

Integrated Real-Time Motion Tracking and AI Analysis for Athletic Performance Optimization

E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation