KPM-Bench: A Kinematic Parsing Motion Benchmark for Fine-grained Motion-centric Video Understanding

Boda Lin; Yongjie Zhu; Xiaocheng Gong; Wenyu Qin; Meng Wang

cs.CVFeb 19, 2026

KPM-Bench: A Kinematic Parsing Motion Benchmark for Fine-grained Motion-centric Video Understanding

Authors: Boda Lin, Yongjie Zhu, Xiaocheng Gong, Wenyu Qin, Meng Wang

Organizations: Kuaishou Technology

Abstract

Despite recent advancements, video captioning models still face significant limitations in accurately describing fine-grained motion details and suffer from severe hallucination issues. These challenges become particularly prominent when generating captions for motion-centric videos, where precise depiction of intricate movements and limb dynamics is crucial yet often neglected. To alleviate this gap, we introduce an automated annotation pipeline that integrates kinematic-based motion computation with linguistic parsing, enabling detailed decomposition and description of complex human motions. Based on this pipeline, we construct and release the Kinematic Parsing Motion Benchmark (KPM-Bench), a novel open-source dataset designed to facilitate fine-grained motion understanding. KPM-Bench consists of (i) fine-grained video-caption pairs that comprehensively illustrate limb-level dynamics in complex actions, (ii) diverse and challenging question-answer pairs focusing specifically on motion understanding, and (iii) a meticulously curated evaluation set specifically designed to assess hallucination phenomena associated with motion descriptions. Furthermore, to address hallucination issues systematically, we propose the linguistically grounded Motion Parsing and Extraction (MoPE) algorithm, capable of accurately extracting motion-specific attributes directly from textual captions. Leveraging MoPE, we introduce a precise hallucination evaluation metric that functions independently of large-scale vision-language or language-only models. By integrating MoPE into the GRPO post-training framework, we effectively mitigate hallucination problems, significantly improving the reliability of motion-centric video captioning models.

Figures & tables

Figure 2: The statistics details of KPM-Bench. The KPM-Bench contains three subsets: KPM-Cap, KPM-QA, and KPM-HA. In addition to visualizing the cases and the category distribution of each subset, we also show the annotation density of the KPM-Cap, the option distribution statistics of the KPM-QA, and the motion type statistics of the KPM-HA.

Figure 3: The pipeline of dataset construction. First, the video frames are processed through 3D pose estimation and kinematic calculation to obtain kinematic-related numerical attributes. Then, these kinematic attributes and the video frames are fed into GPT to obtain PaMoR-Tuple format annotations, which are then further expanded into dense captions.

Figure 4: Comparison cases of caption annotation performance using the our method on MoVid [ 6 ] and Dream-1K [ 42 ] .

Model	Utilized-LLM	Size	B-1 $\uparrow$	B-4 $\uparrow$	R-L $\uparrow$	B-Score $\uparrow$	GPT-Score $\uparrow$	GPT-Acc $\uparrow$	GPT-Hall $\downarrow$	Mo-Hall $\uparrow$
Closed-source VLM-APIs
GPT-4.1 [ 2 ]	-	-	$38.78$	$9.99$	$29.68$	$93.73$	$3.23$	$62.41\%$	$2.32$	$0.603$
Gemini-2.5Pro [ 1 ]	-	-	$32.22$	$4.52$	$25.52$	$92.80$	$3.11$	$58.79\%$	$2.37$	$0.595$
Open-sourced Video-VLMs
InternVideo-2.5 [ 36 ]	InternLM2.5 [ 37 ]	8B	$20.46$	$3.05$	$19.45$	$86.56$	$2.35$	$30.70\%$	$2.48$	$0.573$
MiniCPM-V2.6 [ 41 ]	Qwen2 [ 39 ]	8B	$24.39$	$2.24$	$20.06$	$92.77$	$2.52$	$34.36\%$	$2.55$	$0.541$

Table 1: Comparison of KPM with other closed-source and open-sourced VLMs on KPM-Cap and KPM-HA. The B-1 means BLEU-1, B-4 means BLEU-4, R-L means Rouge-L, and B-Score means Bert-Score. GPT-hall and Mo-Hall are metrics for detecting the number of sentence hallucinations and action hallucinations in the captions of the KPM-HA, respectively.

Model	Utilized-LLM	Size	All $\uparrow$	Move $\uparrow$	Human $\uparrow$	Camera $\uparrow$	Emotion $\uparrow$	Environment $\uparrow$
Closed-source VLM-APIs
GPT-4.1 [ 2 ]	-	-	$84.41$	$75.34$	$77.62$	$72.88$	$88.13$	$95.79$
Gemini-2.5Pro [ 1 ]	-	-	$81.55$	$77.31$	$76.22$	$71.08$	$87.93$	$95.19$
Open-sourced Video-VLMs
InternVideo-2.5 [ 36 ]	InternLM2.5 [ 37 ]	8B	$85.95$	$83.70$	$75.74$	$77.42$	$92.21$	$94.22$
VideoLLaMA3 [ 44 ]	QWen2.5	7B	$58.69$	$57.25$	$47.85$	$54.39$	$83.85$	$61.26$

Table 2: Comparison of KPM with other closed-source and Open-sourced VLMs on KPM-QA. All metrics in the table use Accuracy.

Model	MVBench	FAVOR	MotionBench
Model	ALL	ALL	ALL	MR	LM	CM	MO	AO	RC
QWen2.5-VL-7B	$68.94$	$40.76$	$46.23$	45.11	$45.42$	42.28	$66.70$	36.21	33.08
KPM (w/o MoPE)	69.03	41.25	47.41	$45.09$	46.41	$42.07$	67.05	$35.22$	$32.98$
KPM	$69.01$	$41.02$	$46.57$	$45.06$	$46.33$	$42.05$	$67.02$	$35.15$	$32.90$

Table 3: The results of KPM-series on other motion-related benchmarks.

Figure 5: The human motion (subfigure

1

\rightarrow

subfigure

4

) can be decomposing into Position Translation (subfigure

1

\rightarrow

subfigure

2

) and Postural Transformation (subfigure

1

\rightarrow

subfigure

3

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The case of the joint angles calculation.

Joint Angle	Joint Point Combination (index, name)	Description
Left Shoulder (LShoulder)	(7, 5, 11) – Left Elbow, Left Shoulder, Left Hip	Position of the left upper arm relative to the torso.
Right Shoulder (RShoulder)	(8, 6, 12) – Right Elbow, Right Shoulder, Right Hip	Position of the right upper arm relative to the torso.
Left Elbow (LElbow)	(5, 7, 9) – Left Shoulder, Left Elbow, Left Wrist	Flexion/extension of the left elbow (forearm relative to upper arm).
Right Elbow (RElbow)	(6, 8, 10) – Right Shoulder, Right Elbow, Right Wrist	Flexion/extension of the right elbow (forearm relative to upper arm).
Left Hip (LHip)	(5, 11, 13) – Left Shoulder, Left Hip, Left Knee	Angle between the left thigh and the torso.
Right Hip (RHip)	(6, 12, 14) – Right Shoulder, Right Hip, Right Knee	Angle between the right thigh and the torso.

Appendix

Table 4: Definition of the 10 joint hinge angles used in KPM pipeline. Each angle is computed at the middle joint from its two adjacent segments.

Figure 7: An example of PaMoR. The motion process description is decomposed into seven core elements: MI (Motion Initiation), AE (Agent Entity), PE (Patient Entity), MM (Magnitude Modifier), DI (Direction Indicator), AQ (Agent Qualifier), and PQ (Patient Qualifier). These elements are expressed using three types of units.

Figure 8: Prompt for evaluating caption quality.

Figure 9: Prompt of PaMoR-Tuple.

Figure 10: Prompt of PaMoR-Caption.

Figure 11: Prompt for generating dense caption from GPT and Gemini.

Figure 12: Prompt of hallucination evaluation via GPT-4.1.

Figure 13: Prompt of Caption Comparison.

Dataset	Version	Better-GPT $\uparrow$	Overall $\uparrow$	Accuracy $\uparrow$	Motion Detail $\uparrow$	Temp Spatial $\uparrow$	Fluency $\uparrow$	Better-Human $\uparrow$
MotionBench	Our	$47.7\%$	$4.095$	$3.945$	$4.313$	$3.891$	$4.158$	$49.2\%$
MotionBench	Original	$52.3\%$	$4.421$	$4.012$	$4.132$	$3.924$	$4.203$	$50.8\%$
MoVid	Our	$71.6\%$	$4.285$	$4.462$	$4.278$	$4.351$	$4.372$	$57.9\%$
MoVid	Original	$28.4\%$	$3.692$	$3.984$	$3.675$	$3.928$	$4.195$	$42.1\%$
Dream-1K	Our	$69.8\%$	$4.267$	$4.438$	$4.261$	$4.337$	$4.358$	$55.7\%$
Dream-1K	Original	$30.2\%$	$3.714$	$3.995$	$3.492$	$3.945$	$4.211$	$44.3\%$

Appendix

Table 5: The comparison between KPM version caption and other dataset caption. The numerical value ranges from 1 to 5.

Figure 14: The visualization case of hallucination between KPM and KPM w / o MoPE. The red text corresponds to the hallucination content.

Version	Better $\uparrow$	Overall $\uparrow$	Accuracy $\uparrow$	Motion Detail $\uparrow$	Temp Spatial $\uparrow$	Fluency $\uparrow$
KPM-3D	$80.25\%$	$4.309$	$4.519$	$4.319$	$4.393$	$4.394$
KPM-2D	$19.75\%$	$3.736$	$4.019$	$3.699$	$3.962$	$4.226$

Appendix

Table 6: The comparison between 3D version and 2D version of KPM. The numerical value ranges from 1 to 5.

Vel $\uparrow$	Angle-Vel $\uparrow$	Vel-Score $\uparrow$	Angle-Score $\uparrow$	Frame-Angles $\uparrow$	Vel-FFT $\uparrow$	Angle-FFT $\uparrow$
$3.637$	$3.821$	$4.577$	$3.766$	$3.485$	$3.883$	$3.975$

Appendix

Table 7: Ablation study of kinematic calculation. The numerical value ranges from 1 to 5.

Figure 15: The visualization cases for dense and fine-grained motion-centric video caption.

Figure 16: The visualization cases for human interaction type question.

Figure 17: The visualization cases for motion description type question.

Figure 18: The visualization cases for motion emotion type question.

Figure 19: The visualization cases for movement interaction type question.

Video Captioning

Video captioning turns visual and audiovisual sequences into language that describes their events, entities, timing, and context.

See all 68 papers

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos
Jun 28, 2026Weisong Liu, Haochen Wang, Kuan Gao +8Video Captioning Video-Language Models
Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University · Peking University +3
MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning
Jun 22, 2026Weile Guo, Shenghong He, Danying Mo +3VLM Evaluation Spatiotemporal Reasoning
Sun Yat-Sen University · Shenzhen University
MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models
Jul 1, 2026Sihan Chen, Jiale Li, Jianghang Lin +1VLM Evaluation Hallucination Detection in VLMs
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, Shenzhen, China · South China University of Technology, Guangzhou, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China

Jun 28, 2026J/K move · Enter open · S save

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

Weisong Liu, Haochen Wang, Kuan Gao, Yuhao Wang+7

Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University · Peking University+3

Video Captioning Video-Language Models

Open paper AskPDF arXiv

We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

Abstract

Video Captioning

Explore similar work

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning

MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models