Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

Yanan Wang; Tingsong Li; Kaixun Jiang; Chongyang Zhong; Chenwei Xoe; Zhaohe Liao

cs.CVSep 29, 2026

Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

Authors: Yanan Wang, Tingsong Li, Kaixun Jiang, Chongyang Zhong, Chenwei Xoe, Zhaohe Liao

Organizations: Zhejiang University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Fudan University · Shanghai Jiao Tong University

Abstract

Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, particularly across camera cuts. Improving these details requires evaluation and training that distinguish missing information from incorrect assertions. We introduce FlexBench, a benchmark spanning 3,105 shots and 18,161 evaluation queries, with human-verified identities and systematic per-person coverage of fine-grained limb actions and states. Its reference-derived checklists support automated assessment of complete captions in their person and shot contexts. Our Graded Physical Alignment score (GPA) awards credit for correct content and deducts points for incorrect or fabricated actions, making these errors explicit in the aggregate score. Building on this rubric, we propose Graded Margin Direct Preference Optimization (GM-DPO), which assigns stronger preference margins and greater training weight to more severe action errors. Across three VLM backbones, GM-DPO achieves the highest substantive-action and GPA scores among the evaluated preference objectives, improving GPA over DPO by 2.02-3.40 points. On Qwen3-8B, it reduces the weighted hallucination rate by 21.3% relative to DPO. These gains accompany sustained long-form output, improved shot structure, and competitive performance on three additional multimodal benchmarks.

Figures & tables

Figure 1: Overview of our framework. FlexBench evaluates fine-grained limb motion for individual people across shots. GPA aggregates graded factual credit and explicit error penalties, while GM-DPO uses action-error severity to adjust preference margins and loss weights.

Figure 2: FlexBench curation pipeline: video filtering, shot boundary calibration, VLM pre-labeling, and multi-stage expert refinement for granular limb-action modeling.

Figure 3: Comprehensive overview of FlexBench: (A) Scenario coverage; (B) Dimensional comparison against existing video benchmarks; (C) Structural shot, actor, and duration distributions; (D) Kinematic action categories; (E) Action and temporal query counts.

Figure 4: The preference pair curation pipeline and perturbation spectrum. Ground-truth dense captions ( $y_{w},S=+1.0$ ) are perturbed via rule-based spatial editing and schema-constrained LLM rewriting to synthesize negative samples across five graded physical error regimes ( $\Delta S\in[0.5,2.0]$ ).

Model	Motion Accuracy				Motion Sequence				Output Statistics
	Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$	Len	SMR (%)
Proprietary Frontier APIs
Gemini-3.1-Pro	17.80	42.21	27.56	34.27	26.53	28.18	30.04	28.78	855.7	50.27
Gemini-3.5-Flash	16.70	43.61	27.46	33.98	27.03	26.88	28.77	27.86	709.4	56.22
Seed-2.1-Pro	32.24	44.38	37.10	44.92	47.00	44.14	47.31	46.30	782.3	68.31
Kimi-k2.6	18.47	42.13	27.93	36.04	30.35	30.26	32.94	31.62	683.4	53.57

Table 1: Zero-shot performance on FlexBench across SOTA VLMs. Best results are in bold .

Model Variant	Motion Accuracy				Motion Sequence				Output Statistics
	Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$	Len	SMR (%)
Qwen3-8B-SFT	6.92	42.31	21.08	29.88	13.30	15.54	15.81	15.23	1634	89.7
Qwen2.5-7B-Base	3.02	23.17	11.08	14.26	8.86	9.88	10.38	9.93	1131	49.0
Qwen2.5-7B-DPO	3.92	24.22	12.04	25.05	15.13	15.96	18.39	17.01	1420	94.0
Qwen2.5-7B-cDPO	3.05	25.80	12.15	25.20	15.25	14.55	17.65	16.24	1303	96.0
Qwen2.5-7B-IPO	2.94	27.14	12.62	26.40	17.83	18.42	19.22	18.70	1521	98.7

Table 2: Main alignment results on FlexBench. Base models in gray italics ; best aligned in bold .

Figure 5: The accuracy-hallucination trade-off across three VLM architectures on FlexBench, normalized relative to standard DPO as origin (0, 0).

Figure 6: Row-normalized agreement matrices between human consensus and LLM.

Model	$\omega$	$m$	GPA	$\mathbf{GPA}_{\mathrm{clip}}$	$\mathbf{TSA}_{\mathrm{w}}$
Qwen2.5	✗	✗	12.04	25.05	17.01
	✓	✗	12.29	25.00	16.52
	✗	✓	13.20	25.44	17.73
	✓	✓	14.06	25.82	18.46
Qwen3	✗	✗	19.66	28.86	19.78
	✓	✗	19.93	29.42	19.88

Table 3: Ablation across backbones.

Method Variant	V-MME	POPE		V-GPT
Method Variant	Acc	Acc	F1	Score
Q2.5-DPO	60.06	87.43	86.06	2.50
Q2.5-cDPO	59.58	87.46	86.06	2.50
Q2.5-IPO	60.45	87.44	86.09	2.46
Q2.5-SimPO	60.26	87.41	86.02	2.48
Q2.5-GM-DPO	60.62	87.56	86.15	2.51

Table 4: Cross-benchmark generalization across backbones.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Double-blind human identification study on preference pair naturalness. (a) The empirical distribution of correct identifications per sample closely matches the theoretical $\text{Binomial}(10,0.5)$ chance baseline. (b) Individual identification accuracies across all 10 human evaluators fluctuate tightly around the 50% chance threshold (overall accuracy = 49.60%, binomial $p=0.5813>0.05$ ).

Method	Preference Loss $\ell_{M}$	Hyperparameters
DPO ( Rafailov et al., 2023 )	$-\log\sigma(\Delta r_{\theta})$	$\beta=0.1$
cDPO ( Mitchell, 2023 )	$-(1-\varepsilon)\log\sigma(\Delta r_{\theta})-\varepsilon\log\sigma(-\Delta r_{\theta})$	$\beta=0.1,\ \varepsilon=0.1$
IPO ( Azar et al., 2024 )	$\left(\bar{h}_{\theta}-\frac{1}{2\beta}\right)^{2}$	$\beta=0.5$
SimPO ( Meng et al., 2024 )	$-\log\sigma\!\left(\beta[q_{\theta}(y_{w}\mid x)-q_{\theta}(y_{l}\mid x)]-\gamma_{\mathrm{S}}\right)$	$\beta=2.0,\ \gamma_{\mathrm{S}}=1.0$
GM-DPO (Ours)	$-(1+\alpha\Delta S)\log\sigma(\Delta r_{\theta}-\gamma\Delta S)$	$\beta=0.1,\ \gamma=0.2,\ \alpha=0.1$

Appendix

Table 5: Mathematical formulations and key hyperparameter configurations of benchmarked offline preference optimization objectives. Let

\Delta r_{\theta}=\beta\log\frac{\pi_{\theta}(y_{w}\mid x)}{\pi_{\text{ref}}(y_{w}\mid x)}-\beta\log\frac{\pi_{\theta}(y_{l}\mid x)}{\pi_{\text{ref}}(y_{l}\mid x)}

denote the implicit reward margin. All methods incorporate an auxiliary

+1.0\times\mathcal{L}_{\text{NLL}}

loss.

Qwen2.5-7B
Method	Motion Accuracy				Motion Sequence
	Subs.	Non-subs.	GPA	$\mathbf{GPA}_{\mathrm{clip}}$	$\mathbf{Seq}_{2}$	$\mathbf{Seq}_{3}$	$\mathbf{Seq}_{\mathrm{chain}}$	$\mathbf{TSA}_{\mathrm{w}}$
DPO	$3.92\pm 0.12$	$24.22\pm 0.29$	$12.04\pm 0.11$	$25.05\pm 0.29$	$15.13\pm 0.34$	$15.96\pm 0.13$	$18.39\pm 0.21$	$17.01\pm 0.18$
cDPO	$3.05\pm 0.33$	$25.80\pm 0.13$	$12.15\pm 0.25$	$25.20\pm 0.29$	$15.25\pm 0.45$	$14.55\pm 0.33$	$17.65\pm 0.25$	$16.24\pm 0.13$
IPO	$2.94\pm 0.33$	$27.14\pm 0.14$	$12.62\pm 0.18$	$\mathbf{26.40}\pm 0.27$	$\mathbf{17.83}\pm 0.39$	$\mathbf{18.42}\pm 0.33$	$19.22\pm 0.20$	$\mathbf{18.70}\pm 0.25$
SimPO	$1.39\pm 0.39$	$\mathbf{30.67}\pm 0.13$	$13.10\pm 0.20$	$25.42\pm 0.18$	$15.32\pm 0.25$	$15.32\pm 0.22$	$17.71\pm 0.42$	$16.52\pm 0.25$

Appendix

Table 6: FlexBench results across three training seeds, reported as mean

\pm

sample standard deviation. The highest mean within each backbone is highlighted in bold .

Backbone	GPA Gain			Mean	95% CI
	Run 1	Run 2	Run 3
Qwen2.5-7B	$+2.12$	$+1.83$	$+2.11$	$+2.02$	$[1.61,\,2.43]$
Qwen3-8B	$+3.41$	$+3.80$	$+2.99$	$+3.40$	$[2.39,\,4.41]$
LLaVA-2-8B	$+2.57$	$+2.67$	$+2.90$	$+2.71$	$[2.29,\,3.13]$

Appendix

Table 7: Paired GPA gains over DPO.

Model	Weight	Margin	Motion Accuracy (Penalty-Aware)				Motion Sequence (Non-Penalty)
			Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$
Qwen2.5-7B	✗	✗	3.92	24.22	12.04	25.05	15.13	15.96	18.39	17.01
	✓	✗	4.19	24.44	12.29	25.00	15.86	14.61	17.92	16.52
	✗	✓	5.66	24.51	13.20	25.44	16.16	15.89	19.47	17.73
	✓	✓	6.14	25.95	14.06	25.82	17.05	17.31	19.72	18.46
Qwen3-8B	✗	✗	7.16	38.40	19.66	28.86	17.69	19.94	20.52	19.78

Appendix

Table 8: Comprehensive ablation study of GM-DPO core components across all backbones on FlexBench. Omitting both components recovers the standard DPO baseline. Best performance within each model family is in bold .

Model Variant	Short	Medium	Long	Overall
Qwen2.5-7B-Base	68.89	59.11	50.11	59.37
Qwen2.5-7B-DPO	69.11	60.18	50.89	60.06
Qwen2.5-7B-cDPO	68.89	59.53	50.33	59.58
Qwen2.5-7B-IPO	69.56	60.18	51.61	60.45
Qwen2.5-7B-SimPO	69.40	60.05	51.34	60.26
Qwen2.5-7B-GM-DPO (Ours)	69.75	60.17	51.93	60.62

Appendix

Table 9: Generalization performance on Video-MME across video duration splits and overall scores. Reference base models are shown in gray italics ; within each backbone family, the best result among aligned models is in bold , and the second-best is underlined . The overall column is shaded in blue .

Model Variant	Adversarial	Popular	Random	Overall Acc	Yes%	F1
Qwen2.5-7B-Base	86.50	87.37	88.23	87.37	40.03	85.97
Qwen2.5-7B-DPO	86.53	87.43	88.33	87.43	40.14	86.06
Qwen2.5-7B-cDPO	86.57	87.47	88.33	87.46	40.12	86.06
Qwen2.5-7B-IPO	86.53	87.40	88.40	87.44	40.29	86.09
Qwen2.5-7B-SimPO	86.50	87.38	88.36	87.41	40.10	86.02
Qwen2.5-7B-GM-DPO (Ours)	86.67	87.53	88.47	87.56	40.12	86.15

Appendix

Table 10: Zero-shot object hallucination evaluation on POPE across adversarial, popular, and random splits. Best aligned results within each model group are highlighted in bold , and second-best are underlined ; Overall Acc is shaded in light blue , and F1 is shaded in light pink .

	Generic Understanding			Temporal	Consistency	Overall
Model Variant	Correctness	Detail	Context	Reasoning
Qwen2.5-7B-Base	2.33	2.23	2.63	2.16	2.91	2.45
Qwen2.5-7B-DPO	2.32	2.33	2.64	2.22	2.98	2.50
Qwen2.5-7B-cDPO	2.31	2.35	2.63	2.23	2.98	2.50
Qwen2.5-7B-IPO	2.30	2.29	2.58	2.19	2.92	2.46
Qwen2.5-7B-SimPO	2.31	2.32	2.60	2.23	2.96	2.48

Appendix

Table 11: Video-ChatGPT evaluation across five dimensions using Qwen-3.8-Max as the judge on a 1–5 scale. Overall is the unweighted mean of the five scores, rounded to two decimal places. Base models are shown in gray italics . Best and second-best aligned results within each backbone are bold and underlined , respectively; ties at the displayed precision share the same formatting.

Figure 8: Qualitative comparison of physical motion understanding on a representative multi-shot sequence from FlexBench. Predictions highlight salient limb manipulation and chronological interactions across 7 atomic checkpoints, with local premise scores annotated in parentheses ( $-1.0$ for confabulations, $-0.5$ for chirality/imprecise errors, $+0.5/+1.0$ for grounded dynamics). Each block header reports the unnormalized raw cumulative score ( $\text{Total Score}=\sum_{i=1}^{7}s_{i}$ ).

Explore similar work

NextMotionQA: Benchmarking and Judging Human Motion Understanding with Vision-Language Models
Jun 3, 2026Yong Cao, Chuqiao Li, Xianghui Xie +2Visual Question Answering Benchmarks Human Motion
University of Tübingen, Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany
Building a Precise Video Language with Human-AI Oversight
Apr 22, 2026Zhiqiu Lin, Chancharik Mitra, Siyuan Cen +13Video-Language Models Cinematic Ideal
1Carnegie Mellon University · 2Harvard University
MotionAtlas: Detailed Region Captioning for Motion-Centric Videos
Jun 28, 2026Weisong Liu, Haochen Wang, Kuan Gao +8Video Captioning Motion
Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University · Peking University +3

Jun 3, 2026, cs.CVJ/K move · Enter open · S save

NextMotionQA: Benchmarking and Judging Human Motion Understanding with Vision-Language Models

Yong Cao, Chuqiao Li, Xianghui Xie, Gerard Pons-Moll+1

University of Tübingen, Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany

Visual Question Answering Benchmarks Human Motion Text-To-Motion Generation

Open paper AskPDF arXiv

Reliable evaluation of human motion understanding is fundamental to advancing embodied AI, robotics, and animation. However, existing benchmarks suffer from coarse semantic granularity, undifferentiated difficulty, limited annotation quality, and pervasive answer ambiguity, leaving them unable to diagnose where current models fail. To bridge this gap, we introduce NextMotionQA, a comprehensive benchmark that leverages vision-language models (VLMs) for semi-automated, expert-verified dataset. NextMotionQA features three complementary tasks: multiple-choice question answering, video captioning, and fine-grained error correction. Each task is systematically structured across three core semantic axes and stratified into three task complexity levels. Our extensive evaluation of twelve representative VLMs uncovers critical capability gaps and weakness that remain invisible under conventional, single-task evaluations. In a complementary direction, recent work has begun using VLMs as judges for text-to-motion evaluation; we ask whether they show the same degradation under harder tasks. We find that VLMs align strongly with expert ratings on coarse criteria (Cohen's κ=0.70) but break down on fine-grained, part-level judgment (κ=0.10), validating the paradigm in its strong regime while clarifying its limits.

Abstract

Explore similar work

NextMotionQA: Benchmarking and Judging Human Motion Understanding with Vision-Language Models

Building a Precise Video Language with Human-AI Oversight

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos