V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning

Shulin Tian; Minglun Li; Yuhao Dong; Hao Ding; Jiarui Yao; Haiwen Diao; Jingkang Yang; Hongyuan Zhu

cs.CVAug 26, 2026

V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning

Authors: Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu,

Organizations: S-Lab, Nanyang Technological University · A*STAR · Independent Researcher · UIUC

Abstract

Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.

Figures & tables

Figure 1: Rubric-grounded visual reasoning. A visual question answering example is decomposed into rubric items that check visual faithfulness, reasoning consistency, and instruction following.

Figure 2: Overview of Visual Rubrics-Based Reinforcement Learning. V-Rubrics 50K expands VQA to atomic VF/RC/IF rubric items, which provide fine-grained scores and prefix-localized credit for GRPO-based VLM post-training.

Figure 3: Data Distribution of V-Rubrics 50K. The inner ring shows the VF/RC/IF composition of 352,938 rubric items, while the thin outer ring shows the distribution of 50,248 examples across 17 canonical sources. Outer-ring arc length is proportional to each source’s percentage, with colors mapped to source names in the legend.

Model	SFT Data	RL Data	General VLM	Knowledge & Academic Reasoning				Overall Avg.
Model	SFT Data	RL Data	MMBench-Dev	MMMU Val	MMMU-Pro	MMMU-Pro V	Avg.	Overall Avg.
Closed-source models
GPT-4o ( OpenAI, 2024b )	–	–	88.4	69.1	54.0	49.7	57.60	65.30
GPT-4o mini ( OpenAI, 2024a )	–	–	83.8	59.4	39.9	35.2	44.83	54.58
Open-source Instruct models
LLaVA-OneVision-7B ( Li et al., 2024a )	4.8M	–	80.8	48.8	29.5	18.7	32.33	44.45

Table 1: Performance on general VLM and knowledge benchmarks. Within each model group, green and blue mark the best and second-best results; Knowledge Avg. is the mean of the three knowledge metrics, while Overall Avg. additionally includes MMBench-Dev.

Model	SFT Data	RL Data	Visual Math & Reasoning						Chart & Logic			Overall Avg.
Model	SFT Data	RL Data	MathVista mini	MathVision test	MathVerse V/O	DynaMath Worst	WeMath Loose	Math Avg.	Logic Vista	CharXiv Reas.	Chart Avg.	Overall Avg.
Closed-source models
GPT-4o ( OpenAI, 2024b )	–	–	63.8	31.1	40.6	34.5	62.8	46.56	64.4	–	–	–
GPT-4o mini ( OpenAI, 2024a )	–	–	55.1	27.3	30.0	31.6	48.8	38.56	41.4	34.1	37.75	38.33
Open-source Instruct models
LLaVA-OneVision-7B ( Li et al., 2024a )	4.8M	–	62.6	17.6	17.6	9.0	43.5	30.06	32.0	23.6	27.80	29.41

Table 2: Performance on visual math, chart, and logic benchmarks. Within each model group, green and blue mark the best and second-best results; Math Avg. is the mean of the five visual-math metrics, Chart Avg. the mean of LogicVista and CharXiv, and Overall Avg. the mean of all seven metrics.

Signal	Credit	Overall Avg.
Answer only	Sequence-level	66.25
Answer + rubrics	Sequence-level	67.74
Answer + rubrics	Component + prefix	68.04

Table 3: Reward and credit-assignment ablation.

Figure 4: Qualitative comparison between answer-only GRPO and rubric-based checkpoints. Top: rubric training improves reasoning consistency by making the intermediate inference explicit. Bottom: rubric training improves visual faithfulness by tracing the graph before answering.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Final examples	Share
AI2D	1,758	3.5%
ChartGalaxy	1,289	2.6%
ChartNet	681	1.4%
ChartQA	687	1.4%
ChartQA-X	1,314	2.6%
ChartX	1,203	2.4%

Appendix

Table 4: Canonical source inventory of V-Rubrics 50K. Counts are computed from the 50,248 records in the current release; source shares are rounded independently to one decimal place.

Setting	Configuration
Initialization and data
Backbone before SFT	Qwen3-VL-8B-Instruct
SFT corpus	OpenMMReasoner-SFT-874K
RL corpus	V-Rubrics 50K
Actor initialization	$\pi_{\theta^{(0)}}=\pi_{\mathrm{SFT}}$ ; $\pi_{\theta}$ is trainable
Reference policy	$\pi_{\mathrm{ref}}=\pi_{\mathrm{SFT}}$ at initialization and frozen thereafter

Appendix

Table 6: GRPO training configuration. Settings are shared by the rubric-based run and the answer-level baseline except where a row lists both variants separately.

Faithfulness in VLMs

Faithfulness in vision-language models (VLMs) concerns whether their answers and reasoning genuinely rely on visual evidence.

See all 84 papers

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
Jun 29, 2026Peng, Lee, Yin Zhang +9RL for VLMs Vision-Language Grounding
GMLab · Hong Kong Polytechnic University · XPENG Robotics
Rationale Matters: Learning Transferable Rubrics via Proxy-Guided Critique for VLM Reward Models
Mar 17, 2026Weijie Qiu, Dai Guan, Junxin Wang +6Multimodal Reward Modeling VLM Evaluation
Qwen Large Model Application Team, Alibaba · Institute of Automation, Chinese Academy of Sciences
RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning
Feb 25, 2026Yukun Chen, Jiaming Li, Longze Chen +10VLM Reasoning Rubric-Based RL
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Alibaba Group +1

Jun 29, 2026J/K move · Enter open · S save

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

Peng, Lee, Yin Zhang, Yanglin Zhang+8

GMLab · Hong Kong Polytechnic University · XPENG Robotics

RL for VLMs Vision-Language Grounding VLM Reasoning Visually Grounded Reasoning Vision-Language Alignment Faithfulness in VLMs+2

Open paper AskPDF arXiv

Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs). However, directly applying RL to rollout multimodal reasoning can lead to instability, due to the exploitation of language priors, the neglect of visual evidence, and the generation of reasoning traces that are fluent yet not visually grounded. The question arises: Can initially steer the policy toward visually faithful reasoning regime before applying reinforcement learning? To this end, we propose a Faithful Warm-Start (FWS) strategy that first curates samples with explicit vision-language causal relationships from six general VQA benchmarks to construct the FaithfulQA dataset, where each of the image-question pairs gains a certain degree of visual observations, question requirements, commonsense knowledge, domain knowledge, and the final answer. Subsequently, a VLM-based judge is employed to further purify the dataset, ensuring strong causal consistency and visual faithfulness. This warm-start stage equips the model with the capability to understand causally grounded vision-language patterns before subsequent RL optimization under sparse answer-level rewards. Experimental results show that such faithful supervision improves answer accuracy, stabilizes RL training, and reduces visually unsupported reasoning.

Abstract

Faithfulness in VLMs

Explore similar work

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

Rationale Matters: Learning Transferable Rubrics via Proxy-Guided Critique for VLM Reward Models

RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning