Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

Yanan Wang; Tingsong Li; Kaixun Jiang; Chongyang Zhong; Chenwei Xoe; Zhaohe Liao

cs.CVSep 29, 2026

Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

Authors: Yanan Wang, Tingsong Li, Kaixun Jiang, Chongyang Zhong, Chenwei Xoe, Zhaohe Liao

Organizations: Zhejiang University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Fudan University · Shanghai Jiao Tong University

Abstract

Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, particularly across camera cuts. Improving these details requires evaluation and training that distinguish missing information from incorrect assertions. We introduce FlexBench, a benchmark spanning 3,105 shots and 18,161 evaluation queries, with human-verified identities and systematic per-person coverage of fine-grained limb actions and states. Its reference-derived checklists support automated assessment of complete captions in their person and shot contexts. Our Graded Physical Alignment score (GPA) awards credit for correct content and deducts points for incorrect or fabricated actions, making these errors explicit in the aggregate score. Building on this rubric, we propose Graded Margin Direct Preference Optimization (GM-DPO), which assigns stronger preference margins and greater training weight to more severe action errors. Across three VLM backbones, GM-DPO achieves the highest substantive-action and GPA scores among the evaluated preference objectives, improving GPA over DPO by 2.02-3.40 points. On Qwen3-8B, it reduces the weighted hallucination rate by 21.3% relative to DPO. These gains accompany sustained long-form output, improved shot structure, and competitive performance on three additional multimodal benchmarks.

Figures & tables

Figure 1: Overview of our framework. FlexBench evaluates fine-grained limb motion for individual people across shots. GPA aggregates graded factual credit and explicit error penalties, while GM-DPO uses action-error severity to adjust preference margins and loss weights.

Figure 2: FlexBench curation pipeline: video filtering, shot boundary calibration, VLM pre-labeling, and multi-stage expert refinement for granular limb-action modeling.

Figure 3: Comprehensive overview of FlexBench: (A) Scenario coverage; (B) Dimensional comparison against existing video benchmarks; (C) Structural shot, actor, and duration distributions; (D) Kinematic action categories; (E) Action and temporal query counts.

Figure 4: The preference pair curation pipeline and perturbation spectrum. Ground-truth dense captions ( $y_{w},S=+1.0$ ) are perturbed via rule-based spatial editing and schema-constrained LLM rewriting to synthesize negative samples across five graded physical error regimes ( $\Delta S\in[0.5,2.0]$ ).

Model	Motion Accuracy				Motion Sequence				Output Statistics
	Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$	Len	SMR (%)
Proprietary Frontier APIs
Gemini-3.1-Pro	17.80	42.21	27.56	34.27	26.53	28.18	30.04	28.78	855.7	50.27
Gemini-3.5-Flash	16.70	43.61	27.46	33.98	27.03	26.88	28.77	27.86	709.4	56.22
Seed-2.1-Pro	32.24	44.38	37.10	44.92	47.00	44.14	47.31	46.30	782.3	68.31
Kimi-k2.6	18.47	42.13	27.93	36.04	30.35	30.26	32.94	31.62	683.4	53.57

Table 1: Zero-shot performance on FlexBench across SOTA VLMs. Best results are in bold .

Model Variant	Motion Accuracy				Motion Sequence				Output Statistics
	Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$	Len	SMR (%)
Qwen3-8B-SFT	6.92	42.31	21.08	29.88	13.30	15.54	15.81	15.23	1634	89.7
Qwen2.5-7B-Base	3.02	23.17	11.08	14.26	8.86	9.88	10.38	9.93	1131	49.0
Qwen2.5-7B-DPO	3.92	24.22	12.04	25.05	15.13	15.96	18.39	17.01	1420	94.0
Qwen2.5-7B-cDPO	3.05	25.80	12.15	25.20	15.25	14.55	17.65	16.24	1303	96.0
Qwen2.5-7B-IPO	2.94	27.14	12.62	26.40	17.83	18.42	19.22	18.70	1521	98.7

Table 2: Main alignment results on FlexBench. Base models in gray italics ; best aligned in bold .

Figure 5: The accuracy-hallucination trade-off across three VLM architectures on FlexBench, normalized relative to standard DPO as origin (0, 0).

Figure 6: Row-normalized agreement matrices between human consensus and LLM.

Model	$\omega$	$m$	GPA	$\mathbf{GPA}_{\mathrm{clip}}$	$\mathbf{TSA}_{\mathrm{w}}$
Qwen2.5	✗	✗	12.04	25.05	17.01
	✓	✗	12.29	25.00	16.52
	✗	✓	13.20	25.44	17.73
	✓	✓	14.06	25.82	18.46
Qwen3	✗	✗	19.66	28.86	19.78
	✓	✗	19.93	29.42	19.88

Table 3: Ablation across backbones.

Method Variant	V-MME	POPE		V-GPT
Method Variant	Acc	Acc	F1	Score
Q2.5-DPO	60.06	87.43	86.06	2.50
Q2.5-cDPO	59.58	87.46	86.06	2.50
Q2.5-IPO	60.45	87.44	86.09	2.46
Q2.5-SimPO	60.26	87.41	86.02	2.48
Q2.5-GM-DPO	60.62	87.56	86.15	2.51

Table 4: Cross-benchmark generalization across backbones.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Double-blind human identification study on preference pair naturalness. (a) The empirical distribution of correct identifications per sample closely matches the theoretical $\text{Binomial}(10,0.5)$ chance baseline. (b) Individual identification accuracies across all 10 human evaluators fluctuate tightly around the 50% chance threshold (overall accuracy = 49.60%, binomial $p=0.5813>0.05$ ).

Method	Preference Loss $\ell_{M}$	Hyperparameters
DPO ( Rafailov et al., 2023 )	$-\log\sigma(\Delta r_{\theta})$	$\beta=0.1$
cDPO ( Mitchell, 2023 )	$-(1-\varepsilon)\log\sigma(\Delta r_{\theta})-\varepsilon\log\sigma(-\Delta r_{\theta})$	$\beta=0.1,\ \varepsilon=0.1$
IPO ( Azar et al., 2024 )	$\left(\bar{h}_{\theta}-\frac{1}{2\beta}\right)^{2}$	$\beta=0.5$
SimPO ( Meng et al., 2024 )	$-\log\sigma\!\left(\beta[q_{\theta}(y_{w}\mid x)-q_{\theta}(y_{l}\mid x)]-\gamma_{\mathrm{S}}\right)$	$\beta=2.0,\ \gamma_{\mathrm{S}}=1.0$
GM-DPO (Ours)	$-(1+\alpha\Delta S)\log\sigma(\Delta r_{\theta}-\gamma\Delta S)$	$\beta=0.1,\ \gamma=0.2,\ \alpha=0.1$

Appendix

Table 5: Mathematical formulations and key hyperparameter configurations of benchmarked offline preference optimization objectives. Let

\Delta r_{\theta}=\beta\log\frac{\pi_{\theta}(y_{w}\mid x)}{\pi_{\text{ref}}(y_{w}\mid x)}-\beta\log\frac{\pi_{\theta}(y_{l}\mid x)}{\pi_{\text{ref}}(y_{l}\mid x)}

denote the implicit reward margin. All methods incorporate an auxiliary

+1.0\times\mathcal{L}_{\text{NLL}}

loss.

Qwen2.5-7B
Method	Motion Accuracy				Motion Sequence
	Subs.	Non-subs.	GPA	$\mathbf{GPA}_{\mathrm{clip}}$	$\mathbf{Seq}_{2}$	$\mathbf{Seq}_{3}$	$\mathbf{Seq}_{\mathrm{chain}}$	$\mathbf{TSA}_{\mathrm{w}}$
DPO	$3.92\pm 0.12$	$24.22\pm 0.29$	$12.04\pm 0.11$	$25.05\pm 0.29$	$15.13\pm 0.34$	$15.96\pm 0.13$	$18.39\pm 0.21$	$17.01\pm 0.18$
cDPO	$3.05\pm 0.33$	$25.80\pm 0.13$	$12.15\pm 0.25$	$25.20\pm 0.29$	$15.25\pm 0.45$	$14.55\pm 0.33$	$17.65\pm 0.25$	$16.24\pm 0.13$
IPO	$2.94\pm 0.33$	$27.14\pm 0.14$	$12.62\pm 0.18$	$\mathbf{26.40}\pm 0.27$	$\mathbf{17.83}\pm 0.39$	$\mathbf{18.42}\pm 0.33$	$19.22\pm 0.20$	$\mathbf{18.70}\pm 0.25$
SimPO	$1.39\pm 0.39$	$\mathbf{30.67}\pm 0.13$	$13.10\pm 0.20$	$25.42\pm 0.18$	$15.32\pm 0.25$	$15.32\pm 0.22$	$17.71\pm 0.42$	$16.52\pm 0.25$

Appendix

Table 6: FlexBench results across three training seeds, reported as mean

\pm

sample standard deviation. The highest mean within each backbone is highlighted in bold .

Backbone	GPA Gain			Mean	95% CI
	Run 1	Run 2	Run 3
Qwen2.5-7B	$+2.12$	$+1.83$	$+2.11$	$+2.02$	$[1.61,\,2.43]$
Qwen3-8B	$+3.41$	$+3.80$	$+2.99$	$+3.40$	$[2.39,\,4.41]$
LLaVA-2-8B	$+2.57$	$+2.67$	$+2.90$	$+2.71$	$[2.29,\,3.13]$

Appendix

Table 7: Paired GPA gains over DPO.

Model	Weight	Margin	Motion Accuracy (Penalty-Aware)				Motion Sequence (Non-Penalty)
			Subs.	Non-subs.	GPA	$\bf GPA_{\text{clip}}$	$\bf Seq_{2}$	$\bf Seq_{3}$	$\bf Seq_{\text{chain}}$	$\bf TSA_{\text{w}}$
Qwen2.5-7B	✗	✗	3.92	24.22	12.04	25.05	15.13	15.96	18.39	17.01
	✓	✗	4.19	24.44	12.29	25.00	15.86	14.61	17.92	16.52
	✗	✓	5.66	24.51	13.20	25.44	16.16	15.89	19.47	17.73
	✓	✓	6.14	25.95	14.06	25.82	17.05	17.31	19.72	18.46
Qwen3-8B	✗	✗	7.16	38.40	19.66	28.86	17.69	19.94	20.52	19.78

Appendix

Table 8: Comprehensive ablation study of GM-DPO core components across all backbones on FlexBench. Omitting both components recovers the standard DPO baseline. Best performance within each model family is in bold .

Model Variant	Short	Medium	Long	Overall
Qwen2.5-7B-Base	68.89	59.11	50.11	59.37
Qwen2.5-7B-DPO	69.11	60.18	50.89	60.06
Qwen2.5-7B-cDPO	68.89	59.53	50.33	59.58
Qwen2.5-7B-IPO	69.56	60.18	51.61	60.45
Qwen2.5-7B-SimPO	69.40	60.05	51.34	60.26
Qwen2.5-7B-GM-DPO (Ours)	69.75	60.17	51.93	60.62

Appendix

Table 9: Generalization performance on Video-MME across video duration splits and overall scores. Reference base models are shown in gray italics ; within each backbone family, the best result among aligned models is in bold , and the second-best is underlined . The overall column is shaded in blue .

Model Variant	Adversarial	Popular	Random	Overall Acc	Yes%	F1
Qwen2.5-7B-Base	86.50	87.37	88.23	87.37	40.03	85.97
Qwen2.5-7B-DPO	86.53	87.43	88.33	87.43	40.14	86.06
Qwen2.5-7B-cDPO	86.57	87.47	88.33	87.46	40.12	86.06
Qwen2.5-7B-IPO	86.53	87.40	88.40	87.44	40.29	86.09
Qwen2.5-7B-SimPO	86.50	87.38	88.36	87.41	40.10	86.02
Qwen2.5-7B-GM-DPO (Ours)	86.67	87.53	88.47	87.56	40.12	86.15

Appendix

Table 10: Zero-shot object hallucination evaluation on POPE across adversarial, popular, and random splits. Best aligned results within each model group are highlighted in bold , and second-best are underlined ; Overall Acc is shaded in light blue , and F1 is shaded in light pink .

	Generic Understanding			Temporal	Consistency	Overall
Model Variant	Correctness	Detail	Context	Reasoning
Qwen2.5-7B-Base	2.33	2.23	2.63	2.16	2.91	2.45
Qwen2.5-7B-DPO	2.32	2.33	2.64	2.22	2.98	2.50
Qwen2.5-7B-cDPO	2.31	2.35	2.63	2.23	2.98	2.50
Qwen2.5-7B-IPO	2.30	2.29	2.58	2.19	2.92	2.46
Qwen2.5-7B-SimPO	2.31	2.32	2.60	2.23	2.96	2.48

Appendix

Table 11: Video-ChatGPT evaluation across five dimensions using Qwen-3.8-Max as the judge on a 1–5 scale. Overall is the unweighted mean of the five scores, rounded to two decimal places. Base models are shown in gray italics . Best and second-best aligned results within each backbone are bold and underlined , respectively; ties at the displayed precision share the same formatting.

Figure 8: Qualitative comparison of physical motion understanding on a representative multi-shot sequence from FlexBench. Predictions highlight salient limb manipulation and chronological interactions across 7 atomic checkpoints, with local premise scores annotated in parentheses ( $-1.0$ for confabulations, $-0.5$ for chirality/imprecise errors, $+0.5/+1.0$ for grounded dynamics). Each block header reports the unnormalized raw cumulative score ( $\text{Total Score}=\sum_{i=1}^{7}s_{i}$ ).

Explore similar work

Jun 3, 2026cs.CV

NextMotionQA: Benchmarking and Judging Human Motion Understanding with Vision-Language Models

Reliable evaluation of human motion understanding is fundamental to advancing embodied AI, robotics, and animation. However, existing benchmarks suffer from coarse semantic granularity, undifferentiated difficulty, limited annotation quality, and pervasive answer ambiguity, leaving them unable to diagnose where current models fail. To bridge this gap, we introduce NextMotionQA, a comprehensive benchmark that leverages vision-language models (VLMs) for semi-automated, expert-verified dataset. NextMotionQA features three complementary tasks: multiple-choice question answering, video captioning, and fine-grained error correction. Each task is systematically structured across three core semantic axes and stratified into three task complexity levels. Our extensive evaluation of twelve representative VLMs uncovers critical capability gaps and weakness that remain invisible under conventional, single-task evaluations. In a complementary direction, recent work has begun using VLMs as judges for text-to-motion evaluation; we ask whether they show the same degradation under harder tasks. We find that VLMs align strongly with expert ratings on coarse criteria (Cohen's κ=0.70) but break down on fine-grained, part-level judgment (κ=0.10), validating the paradigm in its strong regime while clarifying its limits.

Yong Cao, Chuqiao Li, Xianghui Xie +2

University of Tübingen, Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany

Visual Question Answering Benchmarks Human Motion Text-To-Motion Generation

Ask PDF

Apr 22, 2026cs.CV

Building a Precise Video Language with Human-AI Oversight

Video-language models (VLMs) learn to reason about the dynamic visual world through natural language. We introduce a suite of open datasets, benchmarks, and recipes for scalable oversight that enable precise video captioning. First, we define a structured specification for describing subjects, scenes, motion, spatial, and camera dynamics, grounded by hundreds of carefully defined visual primitives developed with professional video creators such as filmmakers. Next, to curate high-quality captions, we introduce CHAI (Critique-based Human-AI Oversight), a framework where trained experts critique and revise model-generated pre-captions into improved post-captions. This division of labor improves annotation accuracy and efficiency by offloading text generation to models, allowing humans to better focus on verification. Additionally, these critiques and preferences between pre- and post-captions provide rich supervision for improving open-source models (Qwen3-VL) on caption generation, reward modeling, and critique generation through SFT, DPO, and inference-time scaling. Our ablations show that critique quality in precision, recall, and constructiveness, ensured by our oversight framework, directly governs downstream performance. With modest expert supervision, the resulting model outperforms closed-source models such as Gemini-3.1-Pro. Finally, we apply our approach to re-caption large-scale professional videos (e.g., films, commercials, games) and fine-tune video generation models such as Wan to better follow detailed prompts of up to 400 words, achieving finer control over cinematography including camera motion, angle, lens, focus, point of view, and framing. Our results show that precise specification and human-AI oversight are key to professional-level video understanding and generation. Data and code are available on our project page: https://linzhiqiu.github.io/papers/chai/

Zhiqiu Lin, Chancharik Mitra, Siyuan Cen +13

1Carnegie Mellon University · 2Harvard University

Video-Language Models Cinematic Ideal

Ask PDF

Jun 28, 2026cs.CV

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

Weisong Liu, Haochen Wang, Kuan Gao +8

Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University · Peking University +3

Video Captioning Motion

Ask PDF