KPM-Bench: A Kinematic Parsing Motion Benchmark for Fine-grained Motion-centric Video Understanding

Boda Lin; Yongjie Zhu; Xiaocheng Gong; Wenyu Qin; Meng Wang

cs.CVFeb 19, 2026

KPM-Bench: A Kinematic Parsing Motion Benchmark for Fine-grained Motion-centric Video Understanding

Authors: Boda Lin, Yongjie Zhu, Xiaocheng Gong, Wenyu Qin, Meng Wang

Organizations: Kuaishou Technology

Abstract

Despite recent advancements, video captioning models still face significant limitations in accurately describing fine-grained motion details and suffer from severe hallucination issues. These challenges become particularly prominent when generating captions for motion-centric videos, where precise depiction of intricate movements and limb dynamics is crucial yet often neglected. To alleviate this gap, we introduce an automated annotation pipeline that integrates kinematic-based motion computation with linguistic parsing, enabling detailed decomposition and description of complex human motions. Based on this pipeline, we construct and release the Kinematic Parsing Motion Benchmark (KPM-Bench), a novel open-source dataset designed to facilitate fine-grained motion understanding. KPM-Bench consists of (i) fine-grained video-caption pairs that comprehensively illustrate limb-level dynamics in complex actions, (ii) diverse and challenging question-answer pairs focusing specifically on motion understanding, and (iii) a meticulously curated evaluation set specifically designed to assess hallucination phenomena associated with motion descriptions. Furthermore, to address hallucination issues systematically, we propose the linguistically grounded Motion Parsing and Extraction (MoPE) algorithm, capable of accurately extracting motion-specific attributes directly from textual captions. Leveraging MoPE, we introduce a precise hallucination evaluation metric that functions independently of large-scale vision-language or language-only models. By integrating MoPE into the GRPO post-training framework, we effectively mitigate hallucination problems, significantly improving the reliability of motion-centric video captioning models.

Figures & tables

Figure 2: The statistics details of KPM-Bench. The KPM-Bench contains three subsets: KPM-Cap, KPM-QA, and KPM-HA. In addition to visualizing the cases and the category distribution of each subset, we also show the annotation density of the KPM-Cap, the option distribution statistics of the KPM-QA, and the motion type statistics of the KPM-HA.

Figure 3: The pipeline of dataset construction. First, the video frames are processed through 3D pose estimation and kinematic calculation to obtain kinematic-related numerical attributes. Then, these kinematic attributes and the video frames are fed into GPT to obtain PaMoR-Tuple format annotations, which are then further expanded into dense captions.

Figure 4: Comparison cases of caption annotation performance using the our method on MoVid [ 6 ] and Dream-1K [ 42 ] .

Model	Utilized-LLM	Size	B-1 $\uparrow$	B-4 $\uparrow$	R-L $\uparrow$	B-Score $\uparrow$	GPT-Score $\uparrow$	GPT-Acc $\uparrow$	GPT-Hall $\downarrow$	Mo-Hall $\uparrow$
Closed-source VLM-APIs
GPT-4.1 [ 2 ]	-	-	$38.78$	$9.99$	$29.68$	$93.73$	$3.23$	$62.41\%$	$2.32$	$0.603$
Gemini-2.5Pro [ 1 ]	-	-	$32.22$	$4.52$	$25.52$	$92.80$	$3.11$	$58.79\%$	$2.37$	$0.595$
Open-sourced Video-VLMs
InternVideo-2.5 [ 36 ]	InternLM2.5 [ 37 ]	8B	$20.46$	$3.05$	$19.45$	$86.56$	$2.35$	$30.70\%$	$2.48$	$0.573$
MiniCPM-V2.6 [ 41 ]	Qwen2 [ 39 ]	8B	$24.39$	$2.24$	$20.06$	$92.77$	$2.52$	$34.36\%$	$2.55$	$0.541$

Table 1: Comparison of KPM with other closed-source and open-sourced VLMs on KPM-Cap and KPM-HA. The B-1 means BLEU-1, B-4 means BLEU-4, R-L means Rouge-L, and B-Score means Bert-Score. GPT-hall and Mo-Hall are metrics for detecting the number of sentence hallucinations and action hallucinations in the captions of the KPM-HA, respectively.

Model	Utilized-LLM	Size	All $\uparrow$	Move $\uparrow$	Human $\uparrow$	Camera $\uparrow$	Emotion $\uparrow$	Environment $\uparrow$
Closed-source VLM-APIs
GPT-4.1 [ 2 ]	-	-	$84.41$	$75.34$	$77.62$	$72.88$	$88.13$	$95.79$
Gemini-2.5Pro [ 1 ]	-	-	$81.55$	$77.31$	$76.22$	$71.08$	$87.93$	$95.19$
Open-sourced Video-VLMs
InternVideo-2.5 [ 36 ]	InternLM2.5 [ 37 ]	8B	$85.95$	$83.70$	$75.74$	$77.42$	$92.21$	$94.22$
VideoLLaMA3 [ 44 ]	QWen2.5	7B	$58.69$	$57.25$	$47.85$	$54.39$	$83.85$	$61.26$

Table 2: Comparison of KPM with other closed-source and Open-sourced VLMs on KPM-QA. All metrics in the table use Accuracy.

Model	MVBench	FAVOR	MotionBench
Model	ALL	ALL	ALL	MR	LM	CM	MO	AO	RC
QWen2.5-VL-7B	$68.94$	$40.76$	$46.23$	45.11	$45.42$	42.28	$66.70$	36.21	33.08
KPM (w/o MoPE)	69.03	41.25	47.41	$45.09$	46.41	$42.07$	67.05	$35.22$	$32.98$
KPM	$69.01$	$41.02$	$46.57$	$45.06$	$46.33$	$42.05$	$67.02$	$35.15$	$32.90$

Table 3: The results of KPM-series on other motion-related benchmarks.

Figure 5: The human motion (subfigure

1

\rightarrow

subfigure

4

) can be decomposing into Position Translation (subfigure

1

\rightarrow

subfigure

2

) and Postural Transformation (subfigure

1

\rightarrow

subfigure

3

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The case of the joint angles calculation.

Joint Angle	Joint Point Combination (index, name)	Description
Left Shoulder (LShoulder)	(7, 5, 11) – Left Elbow, Left Shoulder, Left Hip	Position of the left upper arm relative to the torso.
Right Shoulder (RShoulder)	(8, 6, 12) – Right Elbow, Right Shoulder, Right Hip	Position of the right upper arm relative to the torso.
Left Elbow (LElbow)	(5, 7, 9) – Left Shoulder, Left Elbow, Left Wrist	Flexion/extension of the left elbow (forearm relative to upper arm).
Right Elbow (RElbow)	(6, 8, 10) – Right Shoulder, Right Elbow, Right Wrist	Flexion/extension of the right elbow (forearm relative to upper arm).
Left Hip (LHip)	(5, 11, 13) – Left Shoulder, Left Hip, Left Knee	Angle between the left thigh and the torso.
Right Hip (RHip)	(6, 12, 14) – Right Shoulder, Right Hip, Right Knee	Angle between the right thigh and the torso.

Appendix

Table 4: Definition of the 10 joint hinge angles used in KPM pipeline. Each angle is computed at the middle joint from its two adjacent segments.

Figure 7: An example of PaMoR. The motion process description is decomposed into seven core elements: MI (Motion Initiation), AE (Agent Entity), PE (Patient Entity), MM (Magnitude Modifier), DI (Direction Indicator), AQ (Agent Qualifier), and PQ (Patient Qualifier). These elements are expressed using three types of units.

Figure 8: Prompt for evaluating caption quality.

Figure 9: Prompt of PaMoR-Tuple.

Figure 10: Prompt of PaMoR-Caption.

Figure 11: Prompt for generating dense caption from GPT and Gemini.

Figure 12: Prompt of hallucination evaluation via GPT-4.1.

Figure 13: Prompt of Caption Comparison.

Dataset	Version	Better-GPT $\uparrow$	Overall $\uparrow$	Accuracy $\uparrow$	Motion Detail $\uparrow$	Temp Spatial $\uparrow$	Fluency $\uparrow$	Better-Human $\uparrow$
MotionBench	Our	$47.7\%$	$4.095$	$3.945$	$4.313$	$3.891$	$4.158$	$49.2\%$
MotionBench	Original	$52.3\%$	$4.421$	$4.012$	$4.132$	$3.924$	$4.203$	$50.8\%$
MoVid	Our	$71.6\%$	$4.285$	$4.462$	$4.278$	$4.351$	$4.372$	$57.9\%$
MoVid	Original	$28.4\%$	$3.692$	$3.984$	$3.675$	$3.928$	$4.195$	$42.1\%$
Dream-1K	Our	$69.8\%$	$4.267$	$4.438$	$4.261$	$4.337$	$4.358$	$55.7\%$
Dream-1K	Original	$30.2\%$	$3.714$	$3.995$	$3.492$	$3.945$	$4.211$	$44.3\%$

Appendix

Table 5: The comparison between KPM version caption and other dataset caption. The numerical value ranges from 1 to 5.

Figure 14: The visualization case of hallucination between KPM and KPM w / o MoPE. The red text corresponds to the hallucination content.

Version	Better $\uparrow$	Overall $\uparrow$	Accuracy $\uparrow$	Motion Detail $\uparrow$	Temp Spatial $\uparrow$	Fluency $\uparrow$
KPM-3D	$80.25\%$	$4.309$	$4.519$	$4.319$	$4.393$	$4.394$
KPM-2D	$19.75\%$	$3.736$	$4.019$	$3.699$	$3.962$	$4.226$

Appendix

Table 6: The comparison between 3D version and 2D version of KPM. The numerical value ranges from 1 to 5.

Vel $\uparrow$	Angle-Vel $\uparrow$	Vel-Score $\uparrow$	Angle-Score $\uparrow$	Frame-Angles $\uparrow$	Vel-FFT $\uparrow$	Angle-FFT $\uparrow$
$3.637$	$3.821$	$4.577$	$3.766$	$3.485$	$3.883$	$3.975$

Appendix

Table 7: Ablation study of kinematic calculation. The numerical value ranges from 1 to 5.

Figure 15: The visualization cases for dense and fine-grained motion-centric video caption.

Figure 16: The visualization cases for human interaction type question.

Figure 17: The visualization cases for motion description type question.

Figure 18: The visualization cases for motion emotion type question.

Figure 19: The visualization cases for movement interaction type question.

Video Captioning

Video captioning turns visual and audiovisual sequences into language that describes their events, entities, timing, and context.

See all 68 papers

Jun 28, 2026

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

Weisong Liu, Haochen Wang, Kuan Gao +8

Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University · Peking University +3

Video Captioning Video-Language Models

Ask PDF

Jun 22, 2026

MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning

Motion instruction generation in cross-video comparison aims to produce corrective feedback that describes the differences between a query and a reference motion. However, existing models often generate instructions that exhibit motion hallucinations, failing to reflect actual kinematic differences between paired videos. To systematically investigate these hallucinations, we introduce MotionHalluc, a dedicated benchmark for evaluating motion hallucinations in paired-video comparison. MotionHalluc comprises 1540 fine-grained questions over 553 video pairs, evaluating hallucinations along three core dimensions: (1)directional hallucination, (2)attributional hallucination, and (3)temporal hallucination. Extensive evaluations of state-of-the-art large multimodal models demonstrate high susceptibility to these hallucinations. Furthermore, we provide Perceive-Parse-Verify (PPV) as a training-free measurements extraction and verification baseline that converts candidate instructions into executable measurement queries and supplies kinematic measurements at inference time. Our results show that this simple measurements injection yields an average 10.6% performance gain across models, suggesting that motion reasoning with explicit quantitative measurements is a key factor in reducing hallucinations in cross-video comparison. Our code and dataset are available at https://github.com/Leocoding0971/MotionHalluc.

Weile Guo, Shenghong He, Danying Mo +3

Sun Yat-Sen University · Shenzhen University

VLM Evaluation Spatiotemporal Reasoning Hallucination Detection

Ask PDF

Jul 1, 2026

MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models

Video Large Language Models (VideoLLMs) have shown strong progress in video understanding, yet they still suffer from hallucinations that are inconsistent with visual evidence. Existing benchmarks mainly focus on object hallucination or coarse action perception, leaving a key video-specific problem underexplored: motion hallucination, in which models infer human motions that are absent from the video. We present MoHallBench, a benchmark for diagnosing motion hallucination in VideoLLMs. MoHallBench systematically evaluates three major sources of hallucination: co-occurrence priors, sequential inference, and similarity confusion. It contains 11,306 video clips and 40,493 question-answer pairs, covering binary-choice, multiple-choice, and generative settings. We further introduce a bi-directional questioning protocol with bias-aware metrics to reduce affirmation bias in binary evaluation. Experiments on ten recent open-source VideoLLMs reveal a clear decoupling between action recognition and hallucination resistance, as models that perform well on positive action recognition often fail on adversarial negatives. Among all settings, sequential inference hallucination is the most severe, showing that current models tend to over-infer expected outcomes from partial motion cues. Our analyses further confirm that stronger priors and finer-grained similarity substantially amplify hallucination. We hope MoHallBench can facilitate future evaluation and mitigation of motion hallucination in VideoLLMs.

Sihan Chen, Jiale Li, Jianghang Lin +1

State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, Shenzhen, China · South China University of Technology, Guangzhou, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China

VLM Evaluation Hallucination Detection in VLMs Video-Language Models

Ask PDF

Abstract

Video Captioning

Explore similar work

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning

MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models